Project

General

Profile

Save games implementation

Added by quintus over 1 year ago

Dear all,

from the programming of TSC I recall that dealing with save games can be difficult if it is not properly thought through from the start. Notably, TSC effectively required to manually implement XML serialiser functions for every new object inserted into the game. That was quite a hazzle. For this project, I think we can do with a quite limited set of persistant information:

  • The achievments cleared;
  • Any equipment and spells collected, along with the item count if required (like number of arrows for a bow)
  • Player location;
  • Game wall time;
  • Maybe some miscellaneous data I cannot yet think of.

Things like exact position of monsters are not required, because I intend to follow the idea of savegames used in Zelda's OoT. That is, it is possible to save pretty much anywhere (with few exceptions, like boss fights), but on loading you are reset to a specific defined place, like a dungeon entry, not the exact position and exact environmental state that was active when the game was saved.

This approach allows to collect all data that is permanently saved into a single global object which can be saved on the hard disk. I have implemented a simple approach to this here and here with commit 0134eb41881d8d51190bc07e2d85190d09e99925. The GameState structure is a nested structure containing all information to be stored permanently. It provides save() and load() members which write this information to a file identified by a slot (which is internally translated to a file path, currently not working, because I have not yet implemented a path model; it simply uses a hardcoded path under /tmp as of now). When a game is started, the player is intended to select a slot, which will then be passed to these member functions.

I would like to have your opinion on the actual storage format. The current implementation simply writes a memory dump of the GameState structure into the save game file. I originally thought that this is bad, because amending the structure would break the save game files, but thinking about it: if the structure is changed, this probably means significant things have changed. If for example we decided to add a new item to be collected early in the game, what should happen to pre-existing save games which have been saved in a point of the game after the item is intended to be collected? There seems to be no reasonable answer to that question, hence I actually think it is not that bad if a save game cannot be loaded after the GameState structure was amended. Better do not publically release until the GameState structure is finished. Or do I overlook something here?

-quintus


Replies (10)

RE: Save games implementation - Added by sydney over 1 year ago

I don't have a full understanding of everything relating to this, but I do believe your conclusions seem correct. It would be nice to make the save file forward comparable, but if the player is half way into the game, what options are chosen at that point when a new feature/item is added? Not to make it a nuisance mentioning their project again, but Freedroid RPG has a similar issue; every time they seem to release a new version with a new map/features you have to restart from the beginning in the game. Not necessarily a bad thing, I don't see why that is an issue unless someone only plays the game very occasionally, and that gets stretched over multiple releases. Not sure a clean way to deal with it either way. Otherwise, yeah, I cannot think of anything else to add to the conversation at the moment.

RE: Save games implementation - Added by quintus over 1 year ago

Not to make it a nuisance mentioning their project again, but Freedroid RPG has a similar issue;

It actually is valid and important information. It shows me that other open-source projects have found a similar problem and their appears to be no accepted general solution to it. In TSC, there was lots of compatibility code in the save game loader that tried to find a sensible result for each thing added, but to be fair, this never worked well. It is too easy to overlook many cases, and it probably is better to outright reject the savegame than leaving the player with a half-broken game state.

Interestingly, this seems to be a problem specific to open-source games, because they typically evolve further after their release. Commercial games circumvent the problem by not revising the game in significant ways anymore after the release. Given that this game has a rather clear goal line, we might be able to approach this state of “finished” and be able to circumvent the problem as well. In that case, the save game incompatibility will only occur for developers (who can deal with it anyway by manually setting the GameState structure somewhere in the code) and beta testers (who know they are playing a beta version).

Not necessarily a bad thing, I don't see why that is an issue unless someone only plays the game very occasionally, and that gets stretched over multiple releases. Not sure a clean way to deal with it either way.

The player always has the option to (compile and) run an old version of the game. What I do think is that rather than crashing on loading old savegame files or creating something half-broken is that the game should display a sensible error message like ”this save game state was created by an old version of the game (version XYZ) and cannot be loaded with this version. Please use the old version to continue this save game.”.

RE: Save games implementation - Added by quintus over 1 year ago

Am 18. April 2021 um 14:16 Uhr +0200 schrieb xet7:

https://redmine.guelker.eu/boards/3/topics/417?r=424#message-424
xet7

About file format, my opinion here. YMMV.

https://en.wikipedia.org/wiki/INI_file

https://github.com/benhoyt/inih

INI is a nice format and inih a nice project (I used it several
times), but I worry whether it is complex enough for the GameState
object. If the complexity does not increase over the currently
implemented one, it could work, but if it does, we will need a
serialisation format that allows nesting, e.g., YAML.

Apart from its simplicity, do you favour INI because it is
human-readable? Do you think it is useful to have a save game file in
a human-readable format? I mean, it is an invitation to cheating.
I normally am all for human-readable configuration files, but a
savegame format is not a configuration file...

-quintus

RE: Save games implementation - Added by refi64 over 1 year ago

The current implementation simply writes a memory dump of the GameState structure into the save game file.

This actually has a subtle issue: ABI compatbility. The specifics of types aren't fixed across implementations, e.g.:

  • On x64 Windows, a long is 4 bytes, on x64 Linux it's 8 bytes.
  • Any STL types have completely different layouts and sizes (at the moment there are none, so I'm guessing you might already be aware of this, but it's still worth noting). This of course does include std::string.
  • Some types differ based on build flags.
  • Not sure if any desktop or mobile ARM platforms are potential targets, but if so, they have their own padding requirements, and I'd imagine you'd run into issues there.

If any of these are deal-breakers, I'd encourage checking out something like Flatbuffers, where you write a DSL file describing your structures, and it's automatically compiled with some C++ wrappers. Not perfect, of course, but still quite good for stuff like this IME.

RE: Save games implementation - Added by quintus over 1 year ago

Thank you for your reply, refi! You make some good points.

Am 20. April 2021 um 07:01 Uhr +0200 schrieb refi64:

https://redmine.guelker.eu/boards/3/topics/417?r=427#message-427
This actually has a subtle issue: ABI compatbility. The specifics of
types aren't fixed across implementations, e.g.:

I am aware of this, but what I thought was: it is rather uncommon to
copy a savegame file between systems. That is, normally the user will
use it on the system he has created the save game anyway. I found it
safe to assume that the type specifics do not change on a single
system.

  • Any STL types have completely different layouts and sizes (at the moment there are none, so I'm guessing you might already be aware of this, but it's still worth noting). This of course does include std::string.

std::string might turn out as a problem if we need it at some point
and plain old char arrays are insufficient. That the GameState
structure does not contain STL types has an additional reason: I
expect the respective header file to be included in pretty much every
cpp file in the game and hence want to keep it as lightweight as
possible in order to keep complexity and compilation times low.

STL types' implementations may change between compiler releases as
well, but I do not think this happens with regard to the basic types.

  • Some types differ based on build flags.

This is something I did not know and might actually hit us. Thanks for
pointing it out.

  • Not sure if any desktop or mobile ARM platforms are potential targets, but if so, they have their own padding requirements, and I'd imagine you'd run into issues there.

As for mobile targetting, I personally will not do it, but of course
if anyone wishes to do so I do not think the game's code should
effectively prohibit it. Desktop ARM is becoming more common these
days and I even think xet7 owns an ARM laptop, so it might be more of
a target. Still, if the memory dump is created on an ARM system and
loaded on the very same ARM system, I would expect it to properly
load. If this assumption is wrong, then I think we need to do away
with the memory dump approach.

If any of these are deal-breakers, I'd encourage checking out
something like Flatbuffers, where you write a DSL file describing
your structures, and it's automatically compiled with some C++
wrappers. Not perfect, of course, but still quite good for stuff
like this IME.

I will take a look. The current implementation has the appeal of being
refreshingly simple in its implementation and I hesitate to give this
up and add a dependency.

-quintus

RE: Save games implementation - Added by xet7 over 1 year ago

For me, every CPU is a target. I have access to x86, x64, arm, s390x, ppc, ppc64le. I'm trying to get access to RISC-V.

For TSC, it does build fine on those CPU's, because Linux and Windows compatibility exists, and dependencies are available.

BR,
xet7

RE: Save games implementation - Added by xet7 over 1 year ago

Except 680x0, I don't have enough RAM and CPU speed on my Amiga yet for TSC. I don't yet have newer Amiga hardware that have more RAM and CPU speed. I can emulate some of that hardware in UAE and Qemu though.

BR,
xet7

RE: Save games implementation - Added by xet7 over 1 year ago

quintus wrote in RE: Save games implementation:

Apart from its simplicity, do you favour INI because it is
human-readable? Do you think it is useful to have a save game file in
a human-readable format? I mean, it is an invitation to cheating.
I normally am all for human-readable configuration files, but a
savegame format is not a configuration file...

-quintus

Somehow it's easier for me to debug and edit text based
format, than binary format. I have failed to rescue
data from MySQL and MongoDB binary formats, that
they use to save raw data.

What Linux tools are there to edit binary format?
Like adding something in middle of file?

For nested structures, maybe YAML (or TOML) would
work better than INI.

I think it should be some file format where
read write code of that format already exists.

BR,
xet7

RE: Save games implementation - Added by quintus over 1 year ago

I have opened this ticket on the tracker for further discussion of the question and for tracking progress of whatever implementation we choose. Please continue the discussion over there. I close this thread.

-quintus

    (1-10/10)