Simple scene system to ease getting started with 3d game dev

[zet23t]

When working on a game during the last raylib gamejam, I realized again how difficult it is to handle scene data and how much it would help to have a working solution to that problem.

It is right now pretty difficult to use 3D content for games: While 3d formats like OBJ / GLTF / etc can be loaded quite easily, a lot of highly useful information is lost:

• Names
• Cloned mesh instances in GLTF are right now copied and transformed during loading
• The meshes don't have a transform/pivot
• Hierarchy

A typical use case is the desire to export a (simple) level from Blender to use it as part of a game in raylib. The problem is now that there is no way to identify individual objects, and if there are lots of instances, the mesh copies lead to a lot of inefficiencies. Additionally, the model rendering doesn't do any culling. The current workaround is to load meshes individually and assemble them via code or own editors - but this is hardly convenient.

Scene loading + management is not a trivial topic, but there are a few features that wouldn't be complicated to implement and that would enable beginners to get started more easily.

In my experience, the core problems that need to be regularly solved are

• named objects
• finding named objects
• hierarchy
• pivots of objects (transforms)
• reusing meshes (not duplicating, like the model loader is doing right now)
• culling
• attaching userdata to objects
• adding / deleting / enabling / disabling scene objects

I don't think we need a one-solution-fits-it-all solution, just enough that it becomes easy to load / create scenes from external programs like blender in a game.

I have worked on such systems since the beginning of the year in C + raylib and think I could spend time on working something out that could be used as a drop-in-solution, but I would like to get other people opinion on this so I don't develop something that no one is really interested in using.

The API functions I would like to have would look something like this:

Scene LoadScene(const char *gltfFile);
void UnloadScene(Scene scene);
SceneObjectId FindSceneObject(Scene scene, const char *name);
int IsSceneObjectValid(Scene scene, SceneObjectId id);

// passing the camera for culling 
void DrawScene(Scene scene, Camera camera, Matrix rootMatrix);

// transformation
void SetSceneObjectPosition(Scene scene, SceneObjectId id, Vector3 localPosition);
void SetSceneObjectRotationtion(Scene scene, SceneObjectId id, Vector3 eulerAngles);
void SetSceneObjectScale(Scene scene, SceneObjectId id, Vector3 scale);

void SetSceneObjectParent(Scene scene, SceneObjectId id, SceneObjectId parent);
int GetSceneObjectChildCount(Scene scene, SceneObjectId id); 
SceneObjectId GetSceneObjectChildren(Scene scene, SceneObjectId id, int childIndex); 

void SetSceneObjectName(Scene scene, SceneObjectId id, const char *name);
const char GetSceneObjectName(Scene scene, SceneObjectId id);

void DeleteSceneObject(Scene scene, SceneObjectId id);

// associating a model that is to be rendered at the position of the scene object
void SetSceneObjectModel(Scene scene, SceneObjectId id, Model model);
Model GetSceneObjectModel(Scene scene, SceneObjectId id);
void ClearSceneObjectModel(Scene scene, SceneObjectId id);

I would find it important to not let the user deal with pointers to minimize memory handling errors. SceneObjectIds would be indices with a generation counter, so object deletion would never cause problems with memory associated errors. As long as a scene is not unloaded, using the API should be failsafe.

While this API is pretty basic, I think it would really help as a starting point for most users. Keeping the codebase short and easy to use would be the primary goals.

[Cotterzz]

So something like this:

graph LR
A{Scene} --> B(SceneObject0)
A{Scene} --> C[Camera]
B(SceneObject0) --> H[Model0]
B(SceneObject0) --> D(SceneObject1)
D(SceneObject1) --> I[Model1]
D(SceneObject1) --> J[Model2]
D(SceneObject1) --> E(SceneObject2)
D(SceneObject1) --> F(SceneObject3)

Loading

With the Scene and SceneObjects forming the hierarchy.
Many Models to a SceneObject, or just one?
Should there be a mandatory root SceneObject or does the Scene itself give us this? (So Scene acts as the level 0 SceneObject)

Do we need to add any functionality to the existing model in respect of this, eg. get the SceneObject it belongs to?
Obviously we want things to work as they do now without scenes.

In three.js, you add the Camera to the scene, (and you can add more than one), but then everything is scene-based there.
Do we need to do that here? Is there any reason for a Camera to belong to a Scene, if we still have to specify which Camera to draw from?

Why the need for rootMatrix in DrawScene? Won't that come from the Scene, Camera or root SceneObject?

[zet23t]

Yes, the graph showing well what the structure would be describing.

Any object without a parent object is considered to be a root object - so a scene could be a simple list of models that are to be rendered without any hierarchy.

I would try not to add more than necessary to the scene as built in data. If the user wants to use an object as anchor point for the camera (e.g. because it is the child of a player controlled scene object), then this could be done by storing the reference to this "camera" object, using then the transformation information to create a Camera3D struct, which would be used for rendering.

Using models on objects is optional.

The same goes for other concepts: lights for example could be represented as objects as well, but it's up to the user to translate the meaning (position / orientation) to the own use case.

The only help the scene format would provide here would be allowing to add custom metadata information to the objects.

The focus is to handle the hierarchy, model drawing and culling.

[MikiZX1]

[EDITED] I think the camera mentioned is the same camera3D we already use when drawing models, that in this case would also be used to draw the scene. This might be easier for us, the users, to integrate this new functionality in our workflow(?).
To me this all looks like a solid first implementation. The second iterration could look into transparent materials and eventually drawcall optimization.
one/two additional functions to propose might be GetSceneBoundingBox and possibly IsBoundingBoxInFrustum but this sounds more like an extra.
Would it make sense to have the initial implementation work only with instanced meshes(even when there is only one instance of a mesh in the scene)?
I feel these things(transparency, material optimizations, instancing) are for 2.0 but might be worth considering in design from the start? It is also possible to leave the transparency to the user to draw/deal with on their own.

[zet23t]

Yes, it is correct, I would not make cameras or lights part of the scene system. The primary focus of the scene handling would be to allow loading GLTF content and retaining hierarchy and transforms, so it becomes easier to load model data from blender and using it directly in the game and then doing something with this information.

For the rendering, I would still rely on the Model API provided by raylib. Sorting is a very valid point that I encountered this morning as well when fleshing out a bit of code. In my experience, having layer masks and sorting keys is often more than enough to implement a lot of effects, so I would try to make this part of the API.

Probably the render call would need to be extended to something like this:

void DrawScene(SceneId sceneId, Camera3D camera, Matrix transform, int layerMask, int sortMode);

The usage could then look like this:

DrawScene(scene, camera, identity, LAYER_DEFAULT, SORT_NONE);
DrawScene(scene, camera, identity, LAYER_FX, SORT_BACK_TO_FRONT);

Another option could be that there's a function to obtain a list of models and transforms for a certain camera perspective and the calling code would need to handle all drawing operations (batching, model drawing, shader mode toggling, etc). But I think that would be an auxilery function for advanced users.

[zet23t]

I started trying out to implement the API. I simplfied and renamed a few things:

• Nodes are SceneObjects; shorter and closer to what it's called in other systems
• The SceneNodeId contains the info which scene it belongs to; makes it easier to manipulate scene nodes

Implementing the GLTF file loading will however be a major pain, the loader for the model loading looks anything but fun. It probably makes sense to have the scene API and the GLTF loading API split into distinct files.

A very basic example looks like this right now:

    SceneId sceneId = LoadScene();
    SceneModelId firTreeId = AddModelToScene(sceneId, firTree, "fir tree", 1);

    for (int i = 0; i < 100; i++)
    {
        SceneNodeId firTreeNodeId = AcquireSceneNode(sceneId);
        SetSceneNodeModel(firTreeNodeId, firTreeId);
        float rx = GetRandomValue(-400, 400) * 0.01f;
        float rz = GetRandomValue(-400, 400) * 0.01f;
        SetSceneNodePosition(firTreeNodeId, rx, 0, rz);
    }
(...)
        BeginMode3D(camera);
        DrawScene(sceneId, camera, MatrixIdentity(), 0, SCENE_DRAW_SORT_NONE);
        EndMode3D();

[Not-Nik]

While this effort is commendable, I think it is way out of scope for raylib. It involves making major decisions that raylib shouldn't make as a low-level rendering library. LoadScene and DrawScene especially bother me, because for the former, choosing a single format is incredibly restricting (not to speak of the fact that glTF hasn't been the industry standard for scene representation in quite some time) and for the latter, making decisions on how the rendering works (forward/deferred, occlusion culling?, frustum colling?, if both how are they implemented) is adding opinions to the library that narrow its target audience. All in all this very much feels like something more suited in raylib-extras.

[zet23t]

It certainly is a module for raylib-extras and that would be my goal. As this is a high level feature set, it certainly has to make decisions about how things are implemented and which features are to be supported. What I would like to achieve is that this package could be a starting point for developers to get things running quickly and modifying it to their needs when needed. If it doesn't tick someone's requirements, it may still serve as a blueprint.

[Cotterzz]

It's a real showstopper for a lot of new users not having scene functionality.
Coming from three.js, I was surprised by how limited Raylib's 3D functionality was by not having a scene graph.
Having it would make adding features like lighting and shadow much easier, or at least how they're implemented would be based on the same scene model, rather than re-implementing it each time.
Also, being able to work with gltf is important, it is industry standard. usd would be more work, not less.

[raysan5]

As commented, this is a high-level system that I don't think raylib needs but it could be useful as a higher-level module for use or reference.