OSG port roadmap and design plans

Everything about development and the OpenMW source code.
Post Reply
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

OSG port roadmap and design plans

Post by scrawl » 27 Feb 2015, 01:01

Time for another brain dump.

For most of our code we want to do a 1:1 port and thus not much explanation is required. However there are two systems that will highly benefit from a redesign:

NIF loader
(all done)
  • With the OSG stateset system, properties can now be applied to the node hierarchy, instead of having to collect parent properties for each renderable.
  • Setting visibility flags (node mask) and render queue group (render bin) can be done on the node hierarchy instead of on the renderables only. This means we no longer need the "mEntities" "mParticleSystems" vectors in ObjectScene. Same applies to the environment map effect for enchanted objects, we can do that on the hierarchy too.
  • Skinned body parts should be attached directly to the character skeleton instead of a manually updated child skeleton (performance improvement).
  • No static geometry batching. The inherent problems (lighting issues, enable/disable issues, and increased loading time) will be gone with it. The new renderer has a much lower batch cost, so we will not need any form of batching, at least not for the 1.0 release.
  • Get rid of mesh names and skeleton names (loading time improvement).
  • Node filtering should be done while traversing the hierarchy, i.e. not create filtered meshes at all, instead of just hiding the created meshes (loading time improvement).
  • Node transform should not be baked into the meshes. This will allow for more accurate transparency-sorting so we can finally enable alpha blending again.
  • Creating multiple instances of the same NIF file should be overhauled. Currently we have to parse the file again for every instance that is created. I expect this change to greatly reduce loading time.
    The idea is to use the OSG defined copy constructor for a scene graph, which allows detailed control over deep vs. shallow copy for each component. A shallow copy should be used, except for anything that has a controller attached to it.
    As a bonus, we do not need to implement resource sharing (for geometry, etc), because the shallow copy behaviour takes care of that automatically.
  • Controllers should be part of the scene graph (as UpdateCallbacks), thus updated automatically, and alleviating copy-constructor problems with the various target pointers we used to have in the controller classes. Storing these pointers is not needed anymore, because the NodeVisitor will give them to us.
Resource system

The main bonus is that we can finally have anonymous (unnamed) resources, e.g. for the savegame screenshot, loading screen FBO, character preview textures, font textures, video textures, local map and world map textures. Anonymous resources would bypass the resource manager entirely, so we don't have to worry about name collisions.

All data directories and BSAs should be merged into a single file tree for improved performance. Currently we are using a separate Archive for each and have to search them in order, which is slow.

The resource manager should track when a resource was last used so it can be clever when deciding which resources to unload.

In fact some of the Ogre resource managers we have currently implemented (skeleton manager, collision shape manager) don't have unloading implemented at all, so porting them to the new manager we would benefit from lower memory usage automatically.


So far about the design goals, here is the roadmap.

1. Standalone components
  • [done] Port ConstrainedFileDataStream to use an std::istream so that OSG's image loaders can interface with it
  • [done] Port BSAFile
  • [done] Port NIF reader
  • [done] Port ESM reader
  • [done] Create the new NIF loader
    • [done] Geometry
    • [done] StateSets
    • [done] Controllers
    • [done] Skinning (RigGeometry)
    • [done] Vertex morphing (MorphGeometry)
    • [done] Particles
    • [done] Billboard nodes
  • [done] Scene utilities
    • [done] Load filtered parts onto an existing skeleton
    • [done] Investigate StateSet caching
  • [in progress] Create the new resource system
    • [done] Virtual file system
    • [done] Scene object (NIF) template manager
    • [in progress] Texture manager
    • [in progress] Collision shape manager
  • [done - thanks to Chris for help] MyGUI backend
  • [done - thanks to Chris for help] SDL2 GraphicsWindow
  • [done] Video player
2. Begin integration in OpenMW
  • [done] stub Ogre-related code so we can gradually port it over
  • [done] sky renderer
  • [done] terrain renderer (just the basic 3x3 grid for now, distant terrain should be added later)
  • [done] lighting (directional lights can be created as an osg::Light, for point lights we need to collect the closest lights for a particular sub-graph)
  • [done] Animation, character controller
  • [done] Camera, player controls
  • [done] GUI
  • [done] Inventory preview
  • [done] Physics
  • [done] Local map, fog of war, world map
  • [done] Basic water and ripple particles
At this point the game will be playable again, and we should upload the branch to the official repo for interested players to try out.

3. Further porting
  • [not started] shader pipeline
  • Note OSG supports "shader composition", so we should not need an extra library like Shiny again: http://forum.openscenegraph.org/viewtopic.php?t=14680
  • [not started] shadows
  • [not started] water reflections
  • unrelated to OSG, but I also want to try replicating the original MW water shader, i.e. cubemapping, a normal map for rain drops, and re-add the ripple normal map effect, which will look much better with a cubemap.
  • [done] OpenCS
  • [done] Launcher
At this point the port will have all features that are present in the Ogre branch, while at the same time beating it hands-down in framerate, loading performance, memory usage, and additional features. Merge it to the master branch ASAP.

4. Bonus improvements
  • NPC width

    Thanks to the new skeleton system we can now properly scale NPCs on their X and Y axes for the authentic Morrowind look. I already tried this with the NIF loader test app and it's working.
  • Try out hardware skinning and hardware vertex morphing

    To begin with we will be using software skinning/morphing like the Ogre branch did. Getting hardware skinning to work would be a bonus performance improvement. I never got this to work with Ogre, because it hit the limit on the number of bones it was able to pass to the shader at a time. AFAIK the problem was that even bones not used for skinning contributed towards the limit. osgAnimation does not appear to have this problem, so we should give HW skinning a try.
  • Render the loading screen in a separate thread

    OpenMW has both the loading screen and the actual loading procedure happening in the same thread. This is of course problematic because the rendering blocks the loading procedure. Doing the loading and loading screen rendering from independent threads should reduce loading times, and also allow us to increase the loading screen "fps", i.e. have it look smoother, with no performance penalty.

    I *think* that is already done, or rather, not an issue in the first place. With the default threading mode, osgViewer uses a separate graphics thread for OpenGL calls, which goes on drawing (and swapping) while the main thread is already running the next frame's update (or in the case of the loading screen, loading stuff).
  • Background cell loading

    The new NIF loader should make it much easier to implement background loading of cells. Creating an OSG scene graph seems to be inherently thread-safe, so long as that scene graph is not connected to the root node yet. The OpenGL objects (textures, geometry) for the disconnected scene graph can be uploaded gradually over several frames using osg's IncrementalCompileOperation.
  • Raycasting

    OSG seems to have built-in raycasting support, so we would be able to get rid of the Bullet raycasting shapes. If I understand correctly, raycasting against a skinned mesh also works, so we could get rid of the selection buffer used for the inventory preview and use raycasting instead. Should be much faster, because we do not have to copy the selection buffer from the GPU to the CPU.
  • Camera relative rendering

    Experiment with camera-relative rendering to fix the large world precision issues.
  • Get rid of boost

    With Ogre gone we will no longer need the boost libraries. We are still using filesystem and shared_ptr in a lot of places though. Switching to C++11 would be a migration path.
Last edited by scrawl on 30 Jun 2015, 23:23, edited 12 times in total.

User avatar
psi29a
Posts: 4612
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OSG port roadmap and design plans

Post by psi29a » 27 Feb 2015, 09:27

Nice, and professional, progress report!

Great job!

Have you had a chance to visit this thread?
http://forum.openscenegraph.org/viewtopic.php?t=14705

I know that right now you are in the honeymoon phase with OSG, is there anything that you've come across that you would say is difficult in OSG to do coming from an Ogre3D background?

Would you say that the first OSG based OpenMW release will no longer have Bullet as a dependency? Where do we use Bullet and for what things specifically?

Are you using anything version specific in OSG? What version are you using so that I can get our deps in order.

If I wanted to do some pre-packaging work, do you have a branch that I could look at?

matthewfarmery
Posts: 51
Joined: 03 Feb 2015, 13:27

Re: OSG port roadmap and design plans

Post by matthewfarmery » 27 Feb 2015, 09:49

this is certainly exciting, and looks like your making good progress, would be interesting to test the build as soon as its playable. the loading time improvements sounds really cool, and also performance should certainly be good, (hoping that GL rendering will be on par or better to what directx is at the moment or better ) (fingers crossed) but it sounds from the progress report this will be the case.

can't wait to see OSG openmw in action!

User avatar
raevol
Posts: 2998
Joined: 07 Aug 2011, 01:12
Location: Caldera

Re: OSG port roadmap and design plans

Post by raevol » 27 Feb 2015, 10:46

So so so excites.

Chris
Posts: 1558
Joined: 04 Sep 2011, 08:33

Re: OSG port roadmap and design plans

Post by Chris » 27 Feb 2015, 11:41

scrawl wrote:The resource manager should track when a resource was last used so it can be clever when deciding which resources to unload.
This'll be a bit difficult with 'passive' reference-counted resources, like what Ogre and OSG use (where you get a handle to the resource and then just let it go out of scope when you're done). We can only tell if there's any remaining external references, not when it was last unreferenced.
[not started] lighting (directional lights can be created as an osg::Light, for point lights we need to collect the closest lights for each drawable)
I don't think we want to do it for every drawable. One of the supposed benefits of OSG is that it can better sort drawables by GL state to avoid unnecessary state changes. But if each drawable has a unique set of lights, thus unique state that's changing on every draw call, that would defeat the purpose.
[*]Render the loading screen in a separate thread

OpenMW has both the loading screen and the actual loading procedure happening in the same thread. This is of course problematic because the rendering blocks the loading procedure. Doing the loading and loading screen rendering from independent threads should reduce loading times, and also allow us to increase the loading screen "fps", i.e. have it look smoother, with no performance penalty.
I think it would be more appropriate to have rendering in general happen on another thread. Or alternatively, don't block the main render loop when loading, just take a different code path which loads piecemeal. e.g.

Code: Select all

while(!viewer.done())
{
    if(gamestate == loading)
        keep_loading();
    else if(gamestate == playing)
        update_logic();

    viewer.frame();
}
Rendering should just respond to game state, and 'loading' is as much of a state as 'playing'.
Use ProgramBinary to speed up shader loading
What is ProgramBinary? If it's that newish extension for intermediate bytecode shaders, I don't think it'll actually help. According to nVidia, the majority of the shader compile time is spent in the optimizer, and the time spent parsing the text is relatively negligible. Plus, since the bytecode itself is not native code to the hardware (even at the opcode level), it still has to be interpreted and converted/recompiled to what the GPU wants.

The main "benefit" of binary shaders is that it keeps wandering eyes from being able to copy them without putting forth at least some effort to do so. It doesn't really help load times, from what I've heard.

swick
Posts: 96
Joined: 06 Aug 2011, 13:00

Re: OSG port roadmap and design plans

Post by swick » 27 Feb 2015, 13:53

Chris wrote:
Use ProgramBinary to speed up shader loading
What is ProgramBinary? If it's that newish extension for intermediate bytecode shaders, I don't think it'll actually help. According to nVidia, the majority of the shader compile time is spent in the optimizer, and the time spent parsing the text is relatively negligible. Plus, since the bytecode itself is not native code to the hardware (even at the opcode level), it still has to be interpreted and converted/recompiled to what the GPU wants.

The main "benefit" of binary shaders is that it keeps wandering eyes from being able to copy them without putting forth at least some effort to do so. It doesn't really help load times, from what I've heard.
It's a wrapper for GL_ARB_get_program_binary.

User avatar
dEnigma
Posts: 248
Joined: 24 Nov 2013, 21:24
Location: Hla Oad

Re: OSG port roadmap and design plans

Post by dEnigma » 27 Feb 2015, 14:58

Very nice, seems like you have it all planned out already ;)

Looking forward to the performance improvements, since I'm still stuck with my stupid laptop. :mrgreen:
I'll make sure to start testing as soon as it's ready.
Some pirates achieved immortality by great deeds of cruelty or derring-do. Some achieved immortality by amassing great wealth. But the captain had long ago decided that he would, on the whole, prefer to achieve immortality by not dying

matthewfarmery
Posts: 51
Joined: 03 Feb 2015, 13:27

Re: OSG port roadmap and design plans

Post by matthewfarmery » 27 Feb 2015, 16:41

dEnigma wrote:Very nice, seems like you have it all planned out already ;)

Looking forward to the performance improvements, since I'm still stuck with my stupid laptop. :mrgreen:
I'll make sure to start testing as soon as it's ready.
I will be curious about the performance, and also the loading times, and it does seem that the move will cut down that, so will be interesting how quick we move around vivec and how quick they load and the FPS while there. Its one of the areas that does drop FPS even on my system, (using Directx)

I7 4770k
geforce 780 OC edition
16gig of ram.
windows 64bit

I'm a bit concerned still on GL performance, but if the renderer is better optimised in OSG, then hopefully we should see some good improvements for all OS's. still, I think vivec will be the best test, as it does run fairly slow at the mo when going there. so maybe we should have a FPS thread / comparison when, OSG openmw is fully functional. going to be interesting on the results.

User avatar
psi29a
Posts: 4612
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OSG port roadmap and design plans

Post by psi29a » 11 Mar 2015, 23:37

I know you're busy Scrawl, but just wondering about the OSG progress.

The more I read and work with OSG in my spare time, the more I like it.

User avatar
raevol
Posts: 2998
Joined: 07 Aug 2011, 01:12
Location: Caldera

Re: OSG port roadmap and design plans

Post by raevol » 16 Mar 2015, 08:12

psi29a wrote:I know you're busy Scrawl, but just wondering about the OSG progress.
Bump, in case he didn't see this. :)

Post Reply