Binary woes.

Did you know if you mix binary reads with normal text reads that eventually the reads will fail?  I didn't know until today and it took a while to track down the root cause.  All of my IO up to this point has been simple fputs/fputc/fgetc which has worked fine, but with the level loading I wanted to serialized the tiles array (basically the level layout) as a binary chunk in the midst of a normal text file.  Saving that worked as expected, where I end up with something like the following...

object Level_0 type=Level
  TileBinaryData:{insert lots of random bits here}
  Width=50
  Height=50
  LevelName="TestLevel"
end

I added a couple hooks to my Reader/Writer classes that wrap around fread/fwrite which just takes a pointer and a size essentially.  Writing worked fine as I said, but about ~400 bytes into reading the tiles some internal buffer in the FILE structure would run out, and fread would start to fail from then on.  That led me to believe there's a fixed limit on how big text files will successfully read but that's not the case as my binary data actually reduces the file size quite a bit as I'm able to be more efficient with the storage.

At any rate, adding a little 'b' to the fopen calls when appropriate seems to have solved the issues just fine.  Not a big deal once I figured it out but it was a bit frustrating to come across it in this manner.  Looks like this isn't terribly uncommon either as a quick google search yielded a few hits...

http://stackoverflow.com/questions/474733/unexpected-output-copying-file-in-c

File IO seems like exactly the kind of API that is unforgiving, somewhat finicky to get working right, and once you're done you never touch it again.  I look forward to never having to touch it again some day.  As an upside level saving/loading is now significantly faster, and there's even further room to optimize as I add binary import/export options to the property system (currently only doing manual binary writes for the tiles).

Great success!

Level saving and loading works!  I need to fix up a few more things and optimize some obvious bits, but the core functionality is there!  I would post a screenshot to show the success but it really wouldn't look any different than a normal shot - oh well.

I'm currently exporting out all level objects in a text format (the same format I've built for loading data) which seems to work fairly well.  It's fairly slow to parse (string manipulation in C++ isn't really a pleasant process, and I'm sure my string class implementation isn't helping matters much) and it takes quite a bit of space (current 50x50 level with 15 creatures is taking about 185kb on disk), but the upside is that it's human readable/editable/debuggable.

The basic process for saving is as follows:

- Construct a new package object
- Add current level to the root set of objects for that package
- Copy that root set to a new set of ObjectsToSave
- Start traversing all objects in the ObjectsToSave set, adding any objects they reference to ObjectsToSave such that we recurse through the entire object "tree"
- Write out a "packageinfo" block, which is really just forward declarations of objects contained in the package (useful for loading which I'll point out below)
- Iterate through the ObjectsToSave set and write out any data that differs from the current set of defaults

Fortunately I'm able to leverage the object/property system pretty well for this, so at the highest logical point it's only 15 lines of code or so to save a set of objects.  Loading is a little more complicated at the top level since I have to do some extra parsing to know what data is incoming, but it's still pretty manageable at this point.  At any rate, the process for loading is currently:

- Construct a new package object
- Open the package, read the "packageinfo" which will create all of the objects contained in this package using the default values and creates them using the same names that they had on save.  This allows us to easily find references to other objects contained in the packages when loading without having to resort to a separate fix-up phase.
- Read in the objects' data
- Hand out some post-load notifications for specific objects to fix-up any data (currently only the level object does some munging)

There's also a little trickery of finding the loaded player, copying over it's values to the current player, and then deleting the loaded version.  All in all it's fairly straightforward and seems to work fairly well.  It took quite a while to find all the properties that I either wasn't saving out that I needed to, or all the other bits of data that get initialized on level creation and now needs to be handled separately.  For the most part this has helped me find older chunks of code that needed a slight refactoring anyways which is always a good thing (although I did wonder if it would ever end on several occasions).

Woot!

Good design reference

Listing this more so that I don't forget about it, but someone else may find it useful -

http://www.designersnotebook.com/Design_Resources/No_Twinkie_Database/no_twinkie_database.htm

Bit in the ass...

Ran into this the hard way, and the first hit on google clearly explains the issue:
http://www.artima.com/cppsource/nevercall.html.  I can't wait until I finish this project and move on to another language with a different (hopefully more obvious) set of quirks.

Hey, that ain't right.

Still plugging away, although some friends at work convinced me to play some WoW again to prepare for the impending goblin invasion, so progress has been a little stilted.  At any rate, I've managed to make some headway on a few important additions, the easiest and yet most work creating of which was writing a proper logging system.

In the past I only had in-game logging meant for game messages that I was hijacking for specific errors/warnings I cared about (syntax errors when loading data files mostly).  Given that I didn't want to flood the game UI with a bunch of other information it tended to be very specific and not terribly helpful - but now I've added a log which goes straight to disk, which has given me freedom to spread little Logf()'s all throughout my code base.  Adding some to a few key places exposed some interesting bugs that would have surely gone undetected for quite some time.

- All objects were doing duplicate work from a copy and paste error in my object definition macros (gg virtual destructors, you win).
- Level objects were being added to the tickable list twice which caused double game updates.  I don't have much in game animation yet, so it's not really noticeable at this stage but it would have been annoying once I started implementing projectiles and other effects.
- Object instance counts were incrementing on one type (correct) and decrementing on another (incorrect) for dynamic types.  Fortunately the right counter was being incremented, so I wasn't seeing any funky object naming (or name clashing) yet, but again this would inevitably cause pain at some point.

Aside from that fun I've written most of the framework I think I'll need to save level instances.  I've already added the logic to link up levels correctly to each other (stairs up match the stairs down on the other side for example) and left the hooks needed to handle other entities transitioning across levels once I get to that feature.  Hopefully in another 100 lines of code or so this game will finally be susceptible to save scumming!

Updates

I've made a few additions over the last week or so, figured it's worth a quick post to list some of the bigger points.

Zones now exist and act as collections for levels, storing the information needed to generate them as needed, along with the information to link them together.   I've worked through the basic functionality to create levels and transition between them via LevelLinks (aka stairs, hatches, etc) through player interaction.  Along with this came several fixes related to cleaning up Levels for deletion and it looks like there's definitely more work to be done here, but for now the basic framework is in place and working.  It was a slightly magical moment the first time I worked out the last kinks and was able to travel back and forth between levels repeatedly...

As I mentioned in the previous post, entering the data for the starting zone exposed some limitations of my data file parsing system.  So I ended up spending a fair bit of time shoring this up and cleaning up some annoyances that had been bothering me for a while.  First and foremost was simplifying the declaration of properties for serialization to make it less error prone and more readable.

The old way (with lots of snipping):

REGISTER_TYPE(ACreatureClass)
void ACreatureClass::DefineProperties(ObjectType *Type, Property *LastProperty)
{
ADD_PROPERTY(CREATE_PROPERTY_STRUCT(ACreatureClass,TextCell,Cell));
ADD_PROPERTY(CREATE_PROPERTY_STRUCT(ACreatureClass,Range_Int,BaseHealth));
ADD_PROPERTY(CREATE_PROPERTY(ACreatureClass,float,BaseEvasion));
ADD_PROPERTY(CREATE_PROPERTY(ACreatureClass,int,BaseMovementSpeed));
ADD_PROPERTY(CREATE_PROPERTY_CUSTOM(ACreatureClass,String,BrainTypeName));
ADD_PROPERTY(CREATE_PROPERTY(ACreatureClass,int,BaseSightRadius));
ADD_PROPERTY(CREATE_PROPERTY_OBJECT(ACreatureClass,AWeapon,StartingWeapon));
ADD_PROPERTY(CREATE_PROPERTY_ARRAY_EXT(ACreatureClass,ItemBodySlotDesc,BodySlots,CREATE_PROPERTY_STRUCT(ACreatureClass,ItemBodySlotDesc,BodySlots)));
}

And the new way:

REGISTER_TYPE(ACreatureClass)
CREATE_PROPERTY_STRUCT(TextCell,Cell)
CREATE_PROPERTY_STRUCT(Range_Int,BaseHealth)
CREATE_PROPERTY(float,BaseEvasion)
CREATE_PROPERTY(int,BaseMovementSpeed)
CREATE_PROPERTY_STRING(BrainTypeName)
CREATE_PROPERTY(int,BaseSightRadius)
CREATE_PROPERTY_OBJECT(AWeapon,StartingWeapon)
CREATE_PROPERTY_ARRAY_STRUCT(ItemBodySlotDesc,BodySlots)
END_REGISTER_TYPE(ACreatureClass)

I've also added a property type for referencing data types, which will make a safer replacement for the string lookups I've been using (BranTypeName, ClassName, etc).  Array properties also received some bolstering, making it possible to access properties by index and support for referencing inline other property definitions which comes in handy when you just need to touch one property of an object stored in base type's array.

At the moment I'm focusing on getting saving to disk working so I can have some sort of persistence in this game.  So far I've written a basic file writer and a debug command to test the property system; the remaining work doesn't seem that far off now.  Looks like the biggest problem to solve next is how to convert pointers into some sort of lookup table that I can safely save off and then rebuild on load when needed.

Working on the first zone...

I decided a good way to gain traction is to set a pretty clear (and attainable) goal - ideally something that helps progress the "game" side of this project rather than just more engine tweaks.  To that end I've started piecing together the basics of a multi-level dungeon with a simple progression of environment, creatures, and at some point loot.  Thus begins the first goblin caves adventure...

Of course I ran into a bit of a snag when I started writing the class definition for zones, specifically trying to lock down what a zone needs to know and how best to represent it.  Ultimately I decided to take the approach of writing the zone data in whatever way seemed natural, and then worked backwards to turn that into C++ definitions.  That of course exposed some limitations of my data file format/parsing that I'll definitely want to revisit soon, but for now it'll get the job done.