How to save and load levels

Hello there!
This blog will be about one of the most important aspects of a game's architecture, or perhaps even any kind of system (beyond gaming): loading an managing data i.e. levels!

Preface

If you've ever programmed any kind of system which manages and loads data, you may have at some point tried to figure out how to load the data from somewhere that isn't the program. Say you are making a program where you are trying to find the price of a book, you have programmed functions for adding to cart, checking the price etc, but now you need to actually have the data there. The first thing you would do is fill the data by hard-coding a bunch of values and pushing them into something like a list and run the program to see if you can check things or add them to the cart. Then once this all works neatly you would probably wonder "Hmm... how can I add items without hard-coding them in to the program"

This may not seem related to gaming at first, but it's more similar than you think.

The magic of binary

I've been dabbling around with the 65c186 assembly which was used to program Super Nintendo games (sadly I've only gone as far to make a single coloured background as of writing this). I did wonder how they included level files, did they write some kind of asset packing program to store everything into the game? Was everything all hard-coded into the game? Did the game have to access resources from outside?

It turns out that the assembly language (and other assemblers too) had a secret weapon called ".incbin". This made the question of writing asset packing programs or loading things from outside completely moot. The reason why is that the assembler did the asset packing itself, specifically when it compiles everything into one read-only memory image (basically the whole game itself). It all gets packed into one space, which makes it easy to load everything into the game, since it already has pointers to certain bits of memory like levels.

The memory is stored as a bunch of characters which may mean gibberish on first glance, but if you know what you are doing, then you will know exactly what they mean. The SNES game "Legend of Zelda: A link to the past" has a clever way of loading dialogue: instead of loading each individual word, it checks a single character and writes a combination of words based on it, for example a character with the hexadecimal value DE is "with" and E3 "you". When combined together you could write "With you" with almost a quarter of the characters needed to even write these words!

Know your files

However in something like a higher level language like C# this is more difficult. For example in Unity, you can't just have all your game in a single .exe file. There are other things that will be alongside the exe file. These may be things that Unity require to run, the part that is most relevant to this post is a strange file named "assets", this pretty much stores all the game's objects and files. Thankfully this is done when you build the game so you don't need to worry too much about it. But the difficult thing about this is that you need to tell the program where the file is and how to convert it into something that can be read, again Unity does this for you so need not worry.

Actually saving the data

What should one save data into? There are multiple answers to this question, there is the aforementioned binary conversion. One way is making an interpreter that turns level data into a text file and making a custom parser, one game of mine I'm working on at the moment uses this method. For reasons explained later, the interpreter would write the text "level:", following a series of characters. As it goes through every single object in the level, it writes to a text file each character at a time. Each character represents a different object. I also store data for the timer which writes the string "data:" right before it shows the time to set the timer. Information about the custom parser will be explained later on in this post.
Or you could use a JSON converter (Unity has this built-in to the program) which does all this work for you. First you would need to write some data structures to tell which object/data is which so you can use them later on to load into a level. You would then need to gather all the data from all of the objects so they can fill in these data structures with things like positions, type and the such like. Unity would then get to work converting all of that into a single JSON text file. Unity compiles this JSON file into that 'assets' file I was mentioning about when you build the game (turn it into an EXE file) so the engine knows where to find the file.

Unloading the data cargo

So now you've come up with a way to save data, now we need to unload it. Firstly we would need to make a content loader which checks to see any level files we have saved. Then we unload the files from binary into something we can read. Perhaps it could be a JSON file, we could write a system to convert from JSON into whatever format it was. Like how Unity has a way to convert data structures into JSON files, it can do the reverse. So we convert a text file into data that can be loaded. Great. So now what?
We would need to read from the data itself and arrange everything together based on what it says. For example if there was a list of tile data from the JSON file, we would iterate through all of them and create new tiles based on these data structures.
Or if you are writing your own custom parser, it would be a bit trickier. You would need to write a state machine so the loader would know what it is doing. For example I mentioned writing "level:" and "data:" somewhere in the text file. This is not just for show, this is so that the parser can know what section they are doing, for example when it encounters the "data:" string, it realizes "I'm now going to read the next 3 characters and convert them into an integer" so it reads for instance "015" and converts that into an integer. Before that though it would encounter the "level:" string and then would know that it is now converting every character it comes across into a block depending on which character it is.

Conclusion

Data loading is not something you would think about when making something like a game, but it is absolutely necessary (if you want to make something that can be saved, loaded and modified. There are plenty of ways to save and load data which makes it a pretty interesting topic, there are also ways to crunch and compress the way that data is loaded like the Legend of Zelda example.

That's all from me!