Better grass support

[AnyOldName3]

Instancing means using built-in OpenGL/DirectX functions to draw multiple copies of the same object with one draw call. It only works for objects sharing the same mesh, isn't as cheap as a non-instanced draw call, and still requires you to upload all the data that's unique to each object whenever it changes, which is still a significant cost. It won't help much in OpenMW as in many cases, this data is nearly the same size as the mesh itself, and despite looking identical, the meshes are subtly different. It's better for the GPU to be given ten tufts of grass at once than on their own, but still much worse than being given all the grass in the scene at once.

Batching is usually a catch-all term for anything that means drawing more things with fewer API calls, so even using texture atlases can count, even though the same number of actual draw calls happen. It can also mean specifically when multiple meshes are combined into one bigger mesh. If you do it right, and bake all the ancillary data into the combined mesh/texture, this is the most efficient way to use the GPU, as it gets all the data at once and can just chew on it at its own pace, instead of waiting for the CPU to drip-feed it. This is easy to do for static meshes that don't change at all between when they're first loaded and the last time they're ever seen. MGE XE's distant land feature does this, but has to pretend things are static when they aren't really, and this can cause problems.

When we say robust batching, we mean a hypothetical system where we can get as close to the best-case scenario for GPU utilisation as is reasonable given that Morrowind doesn't have a single truly static mesh. This isn't something built into OSG as anyone making software from scratch would have a much easier time preventing it being necessary than actually implementing it. It's solely because of how Morrowind was designed and our inability to completely replace its assets that we need a bespoke system.

We have a basic design for a system that should work perfectly. However, turning it into a reality is a big job, and no one's been available who has the skills, experience and time.

[Time4Tea]

@AnyOldName3: thanks for the information, I find this all really interesting.

One question though: there is no groundcover in the original Morrowind game, so doesn't that give some more flexibility for how it could be implemented in OpenMW (in terms of using a more 'ideal' solution)? I mean, compared to other assets that have to conform to how things were done in the original game.

When we say robust batching, we mean a hypothetical system where we can get as close to the best-case scenario for GPU utilisation as is reasonable given that Morrowind doesn't have a single truly static mesh. This isn't something built into OSG as anyone making software from scratch would have a much easier time preventing it being necessary than actually implementing it. It's solely because of how Morrowind was designed and our inability to completely replace its assets that we need a bespoke system.

Perhaps you were referring to other things besides just groundcover here, but since groundcover wasn't in MW, those assets have to be created from scratch anyway, don't they? So, couldn't those meshes be made 'truly static', if the OpenMW dev team decided that was the best way to go?

[AnyOldName3]

Groundcover isn't the only thing that would benefit from batching and any post-1.0 future-of-grass solution will only make new content that happens to specifically be grass go faster. We're going to need a robust batching system to make regular Morrowind fast and pretty at the same time and that'll end up making legacy grass faster, too.

As for a future grass-specific solution, there are a few things games do:

• Just place a bunch of grass meshes by hand like Morrowind. Hopefully, the game's engine and the grass assets are well-designed enough that this isn't a problem (e.g. they can be merged).
• Add a tool to the game's editor that places a bunch of grass meshes based on a brush tool or something. The end result of this is basically identical to the one above, but it's faster to make the game in the first place. Hopefully, the game's engine and the grass assets are well-designed enough that this isn't a problem (e.g. they can be merged).
• Use a terrain texture to specify the grass density, and treat it as a probability distribution function for where to spawn grass meshes. This is what Skyrim does. Hopefully, the game's engine and the grass assets are well-designed enough that this isn't a problem (e.g. they can be merged).
• One of the above three, but when you've got your collection of meshes for a region, combine it onto one bigger mesh (either at runtime or when creating the game. I think this is what Fallout 4 does (it definitely does it for other kinds of static mesh), but grass might not actually end up in precombines.
• Use a terrain texture to specify the grass density and use a geometry shader to turn the terrain mesh into a bazillion tiny blades of grass. Geometry shaders are the best way to turn the terrain mesh into a bazillion tiny blades of grass, but they're usually slower than having a single combined grass mesh for a cell with less grass but bigger, and can look worse as you're more likely to end up with one kind of grass for a whole area rather than a nice range. https://forum.unity.com/threads/release ... er.530919/ has some screenshots and videos of this kind of grass. It's pretty, but more like a lawn than wilderness.

[FiftyTifty]

Just out of interest, from someone who doesn't have a great deal of experience with rendering:

Is this 'robust batching' a feature that is already present in OSG, that 'just' needs to be taken advantage of in OpenMW, or is it something that OpenMW would have to create from scratch? Also, I'm kind of curious as to what these concepts of 'instancing' and 'batching' mean - is there a good reference anywhere?

I'm a big fan of groundcover and curious as to what would be involved to implement it in OpenMW.

There are two generic, standard types of batching. Static batching, and dynamic batching.

Stat batching is where meshes that aren't changed in any way at runtime, are merged into one big mesh prior to runtime. This is bad, as it has a very limited use scope, and will break if an object is used at runtime despite having been merged.

Dynamic batching is where meshes are merged into one big mesh, which is done repeatedly after each frame has been drawn. This is bad as it's constantly being re-processed, which is a huge performance drain in even just slightly dense scenes.

Robust batching is not a specific implementation, instead just kind of a nomenclature we've come up with. There is a way to get the best of both worlds, which is batching most meshes at runtime, but not re-processing them after every frame, even if many of the meshes are changed during gameplay. Boris Vorontsov, the creator of ENB, talked about how to do exactly that in this discussion I had with him: http://enbseries.enbdev.com/forum/viewt ... 869#p69747

Pack meshes in few big vertex buffers and add data to each inside it or in external stream which identify individual mesh, for example one float value. Use that value to index to any data, for example matrices for transformation, which will be selected in shader similar to matrices of bones. Works fine with dynamic objects and all you need is to remove deleted meshes or add new to large buffers. Can be done on hardware too via other tricks (don't ask that please).

It would be a good idea for the OpenMW devs to contact him about this.

Another approach would be to just switch to Vulkan, but OSG doesn't support that. There is VKSceneGraph, but that's still in a very alpha state at the moment.

[AnyOldName3]

What you called Dynamic Batching is what's actually known as Incredibly Stupid Batching, and no one ever uses it because it's worse than no batching at all. Actual dynamic batching just means that you're sorting out the batches at runtime and not that you're redoing it every frame. Lots of games sort out their batching when a level loads and then don't touch it again, but that's still dynamic batching. With Morrowind, we don't have enough information to do that, as even if the halting problem wasn't insoluble and we could determine what could might be modified by a script at runtime, the in-game console is still a thing, and you can't predict what people will type. That just means that we need a system that can react to changes.

In theory, this isn't actually all that complicated. All we need to do is keep track of which objects got put into each combined mesh, and then when one of those is changed, switch back to using the uncombined meshes while another thread regenerates a new combined mesh. To avoid having to keep doing this, there are various heuristics that can be applied, which can be as simple as never adding anything back into a combined mesh after it's been changed once and always using the uncombined version from then on. The difficulty comes from there being lots of places where the scene graph can be modified and lots of decisions to be made about when things can be combined versus when they need to stay separate (e.g. if objects use different textures, we can't just combine them without also building a texture atlas, and if we're building texture atlases at runtime, we need to deal with there being different formats and ways of sampling textures like wrapping, but Morrowind uses lots of tiny textures, so there's not much benefit to merging things if we can't merge things with different textures).

Vulkan isn't a magic bullet here either. It does make each draw call cheaper, but it doesn't make them cheap, especially when you need to change renderer state in between each one, as is the case for Morrowind. A draw call containing two triangles is always going to be wasting the massive parallelism of a GPU no matter what API it comes from, and sending the command to draw it over PCIe is always going to take longer than drawing it does.

[Time4Tea]

@MajinCry ok, thanks for the info. Interesting stuff.

So, does OSG not provide support for any kind of batching at all?

[akortunov]

So, does OSG not provide support for any kind of batching at all?

No, devs suggest to optimize resources themselves instead.

[AnyOldName3]

It's a perfectly reasonable stance for them to take, too. Most of the time, people creating a graphics application have the power to make the things they're drawing not really dumb. Even if they don't, they usually have the power to make the ways those things can be changed not dumb. There are various classes that can perform basic optimisation and joining of meshes, which is enough for probably every other OSG user ever, but nothing capable of dealing with the craziness Bethesda left us with. It's also not entirely unreasonable for Bethesda to have left us with this craziness. In the two decades since they made the game, GPUs have come a lot further than CPUs, so things that are dumb now aren't the same things as were dumb in the past.

[FiftyTifty]

It's a perfectly reasonable stance for them to take, too. Most of the time, people creating a graphics application have the power to make the things they're drawing not really dumb. Even if they don't, they usually have the power to make the ways those things can be changed not dumb. There are various classes that can perform basic optimisation and joining of meshes, which is enough for probably every other OSG user ever, but nothing capable of dealing with the craziness Bethesda left us with. It's also not entirely unreasonable for Bethesda to have left us with this craziness. In the two decades since they made the game, GPUs have come a lot further than CPUs, so things that are dumb now aren't the same things as were dumb in the past.

OSG Is more of a barebones API, by the looks of it. Something like BGFX, which is used by the NeoAxis game engine, would be closer to that. And yeah, it doesn't help that Bethesda has consistently messed up renderers and level design.

At least with Vulkan, you can throw more cores at the draw call processing stage, if parallelization is implemented. Can't really do that with OpenGL nor Direct3D. Coupled with robust batching, performance would be stellar. Batching would be better, because that would greatly reduce the draw calls issued by shadows and lights in the scene, and Morrowind has a very low polygon budget art design.

[Time4Tea]

At least with Vulkan, you can throw more cores at the draw call processing stage, if parallelization is implemented. Can't really do that with OpenGL nor Direct3D. Coupled with robust batching, performance would be stellar. Batching would be better, because that would greatly reduce the draw calls issued by shadows and lights in the scene, and Morrowind has a very low polygon budget art design.

Vulkan looks great. I get the impression OpenMW is quite CPU-limited, so anything that allows more extensive use of CPU parallelization would seem to be a good step. It looks like there is a 'sister' project to OSG - VulkanSceneGraph - currently being worked on. Wouldn't it be nice if the API was almost exactly compatible with OSG and all you had to do was run a script to replace the string 'osg' with 'vsg'?