I have a game framework which I've mutated throughout the years as I add and remove things or experiment with it. Its first iteration had OpenGL ES 2.0 and Direct3D 9 support. In its current form it's using Metal3 and D3D12. It's a great learning tool.

It has been sort of stuck in various modes of its latest incarnation for a few years, I typically don't have much time, or sometimes capacity, left to engage with it. However, as it's been nearly 10 years since the last game I released, I am attempting to produce a new one in my spare time (if that is at all possible), and I wanted to see if I could use my tiny framework to complete a product.

It's a small and fairly simple framework, and each time I iterate a bit on it, I attempt to make it even simpler where possible. One example of simplicity is the way I serialize resources: I like to keep game resources as read-only POD structs which can be deserialized trivially into their runtime representation. It may have its own tradeoffs, perhaps notably when serializing for different types of architectures, however, for my purposes it serves me well.

Serializing

I originally implemented this type of serialization as part of my offline asset baking tooling, where the serialization part was done manually with a utility class to ease its process. It was a manual process, nonetheless.

The idea is to store the top-level POD struct resource as-is, with the same alignment characteristics as your target architecture, or possibly even in a platform-agnostic manner.

The top-level struct is copied directly into a static memory section. Then, you have to find where the struct (and any nested structs) have fields which are pointers and "patch" them. Whenever you encounter such a field you have to serialize its data (recursively) into a dynamic memory section, which immediately follows the static section. You then update the pointer to have a relative address instead of an absolute one.

The serialized data would then look something like this: Serialized Struct

Deserializing

When deserializing, you do so as a single read of contiguous bytes of static and dynamic data. The header is small and of fixed size so it may be read directly into a temporary stack-allocated object. The temporary data section is meant to be discarded immediately after deserialization, if there is any. An example of temporary data would be a compiled shader IL blob to be used to create the runtime representation and then the blob can be discarded. Whereas you'd like to keep other important shader reflection data included in the top-level struct.

At this point you may choose how the pointers are interpreted at runtime. You can opt to patch the pointers after deserialization to make their addresses absolute. If you do so, you must also store a pointer-patching table in the temporary data section of file.

Otherwise, you may choose to keep the pointers relative even at runtime. If so, you could, perhaps, use a pointer wrapper object with pointer member access operator overload if using C++. You also could benefit from storing pointers as fixed-size across platforms should the maximum data size of a single resource be reasonable.

A drawback of keeping the pointers relative is that you can only access them through the pointer to the top-level struct. Or, if you want to pass around the pointer itself, you'll have to convert it to an absolute pointer before doing so.

Odin

Lately I've been playing around with Odin, which I've found to be a pleasure to work with, and it simplifies things even further.

All structs in Odin are POD and it also supports reflection, so we can serialize offline any kind of struct resource we use at runtime directly (that is the same structure is used in both direction), with no other representation or manual encoding needed.

After implementing this in Odin, all the serialization/deserialization code for any struct came out to about 340 lines of code. The utility class alone in the previous tool was around ~800 lines of code, not to mention usages of it to serialize different types — which is automatically handled in the Odin implementation.

Odin also includes built-in relative pointers with user-selected backing integer types. Therefore, the relative pointer implementation mentioned above can be used more naturally. Although this feature may be deprecated in favor of the package core:relative which uses explicit semantics instead of language magic.

In opted for using absolute pointers exclusively after experimenting with both types of pointers in Odin. The relative implementation, though it saves a step during deserialization, emerged as a bit more complicated when implementing serialization. The simple pointer-patch deserialization post-step seemed like a better tradeoff to me.