Generic nif engine importer for openmw

Everything about development and the OpenMW source code.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Generic nif engine importer for openmw

Post by Chris »

Zini wrote: 19 Sep 2017, 10:29 I am still convinced that a game engine is generally better off with only supporting a single file format for things like meshes (ideally one that is closest to the internal data structures and requires at little effort as possible for reading. Providing resources in the right format (converting if necessary) is best done with external tools.
Technically in that case then, we should be focusing on supporting OSG's model format(s). Morrowind's meshes require a fair amount of effort to load into OSG and use as they're intended, even needing custom handlers because of how NIF works compared to OSG (w.r.t. things like animations). Granted it's far better now that it was when using Ogre3D, but I'd blame that on us trying to use Ogre3D in ways it apparently wasn't meant to.

Personally, I'm of the opinion that for an open source engine that welcomes moddability, focusing on support for open and widely supported formats that modders can easily use and distribute should be a big priority. Obviously being that we're replicating the Morrowind engine it's more important that we support Morrowind's NIFs, but that's largely done aside from an occasional bug or mishandling that pops up. I would actually prefer to not really focus on other NIF formats because of spotty support (and lack of proper documentation), but instead turn the attention to more readily usable formats like OpenGEX and glTF 2.0.

Especially after 1.0, we won't be able to stick with supporting only Morrowind's NIF format if we're to add new model-related features, but we obviously will always need to support Morrowind's NIF format. Extending Morrowind's NIF format is just going to be a waste of resources (not only would it duplicate the work of adding support for things other formats already have, but support for various modeling software packages will not be good either). So really the only good option will be to have support for Morrowind's NIF format, and add support for more modern formats that offer features we want and are supported well enough for modders to use.

Ideally we should leverage a general model loading API to support multiple formats. Like how we use FFmpeg to support not only the wav, mp3, and bik media files, but also gain support for ogg, flac, and webm with no extra work on our part. OSG has a general model loading API, and while I understand there are issues with handling specific features of Morrowind's NIFs through it, if we could work that out it would be a big boon as we'd not have to worry much about model formats further. We can assume OSG has or will have import plugins for formats modders would like to use, and anyone that wants to add support for certain other formats would work with OSG rather than us (though we could still be available for advice), so it eases the overall amount of work we have to do for future format support.
User avatar
psi29a
Posts: 5356
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Generic nif engine importer for openmw

Post by psi29a »

I would love (and prefer) to see OpenGEX, glTF 2.0 and collada support be included.

That doesn't help OpenMW being used to play Oblivion or Skyrim however. If people want to spend their time implementing/extending our NIF format, I say go ahead. It's a neat technical challenge.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Generic nif engine importer for openmw

Post by scrawl »

Agreed on all but the last paragraph of Chris' post.
Ideally we should leverage a general model loading API to support multiple formats.
The 'API' for a loader is really simple: file goes in, osg::Node comes out. That's how the NIF loader was designed as well.

That said, sometimes our engine still needs to look 'inside' the structure of the loaded graph, e.g. when we want to start certain animations based on in-game events / time sources. This is where a 'standard' API becomes a bigger issue (what is an animation and how is it controlled?).
We can assume OSG has or will have import plugins for formats modders would like to use, and anyone that wants to add support for certain other formats would work with OSG rather than us (though we could still be available for advice), so it eases the overall amount of work we have to do for future format support.
OSG has a bazillion such plugins, yes. Last I looked though, some had the issue of not allowing our virtual filesystem to be used (e.g. the .obj plugin would look for a .mtl of the same name, opening it through the regular filesystem).

IMO, encouraging all these plugins to be used is going to open a can of worms that we may not want to deal with. At a bare minimum we should decide on a sub-set of plugins that we officially support and should be included in our release builds (remember, plugins are enabled at build time). But I'm still not sure this is a good idea at all, with some of the maintenance burden inevitably falling on us.

It should be noted that even with no support in OpenMW, one can still use osgconv to convert insert-obscure-format-here into .osg and load that into OpenMW.

Animations are another issue. The closest thing to an animation 'standard' in OSG is the osgAnimation node kit, and this is also what the model importers turn animations into. But OpenMW uses a custom animation component, mostly for optimization reasons and because osgAnimation doesn't support a peculiarity in Morrowind's way of skinning. So the osg importers wouldn't work for animations anyway, unless we did some sort of conversion that loads osgAnimations into our custom system, or change the animation playback in the engine to look for both Morrowind-style controllers and 'osgAnimations' (sounds like a maintenance nightmare).
Last edited by scrawl on 19 Sep 2017, 16:26, edited 1 time in total.
raven
Posts: 66
Joined: 26 May 2016, 09:54

Re: Generic nif engine importer for openmw

Post by raven »

OpenGEX and collada are exchange formats. Afaik they are text based and very generic and thus slow to parse. gltf is a bit better, as it looks more like a text binary hybrid.

For performance you'll probably want something binary that does not require much, if any, parsing.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Generic nif engine importer for openmw

Post by Chris »

scrawl wrote: 19 Sep 2017, 16:21 At a bare minimum we should decide on a sub-set of plugins that we officially support and should be included in our release builds (remember, plugins are enabled at build time).
Yes, this is what I'd expect. We declare what formats we officially support and provide the plugins for them in our official releases (we already do for textures, don't we?). We don't need to be too stringent on what to include though, as long as we can be assured the plugin won't be going away.
Animations are another issue. The closest thing to an animation 'standard' in OSG is the osgAnimation node kit, and this is also what the model importers turn animations into. But OpenMW uses a custom animation component, mostly for optimization reasons and because osgAnimation doesn't support a peculiarity in Morrowind's way of skinning. So the osg importers wouldn't work for animations anyway, unless we did some sort of conversion that loads osgAnimations into our custom system, or change the animation playback in the engine to look for both Morrowind-style controllers and 'osgAnimations' (sounds like a maintenance nightmare).
What exactly is the issue with Morrowind's way of skinning? I'm sure it's been mentioned before, but I don't know where. Would it be possible to improve osgAnimation to support what we need? Would it be possible to convert Morrowind's way of skinning to something osgAnimation can work with?
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Generic nif engine importer for openmw

Post by scrawl »

What exactly is the issue with Morrowind's way of skinning? I'm sure it's been mentioned before, but I don't know where. Would it be possible to improve osgAnimation to support what we need? Would it be possible to convert Morrowind's way of skinning to something osgAnimation can work with?
It's a conceptual difference that can't be converted (well, you can convert the standard way to NIF's way, but not always the other way around). Most skinning systems will assign a bind matrix to each bone. NIFs assign a set of 'bone, bind matrix' pairs to each skinned mesh. This just complicates things, for the minimal 'benefit' of being able to use different bone bind matrices for different meshes on the same skeleton, something that's not really required if you design all of your assets with the same conventions. I don't think it makes sense to implement this in osgAnimation.

As for the 'optimization reasons' thing, our component is mainly faster due to:
* Software skinning is done in the Cull phase, not the Update phase, to skip unneeded work for objects not on screen.
* Not using a synchronized draw traversal (instead, the internal geometry is double-buffered)
When I briefly brought up these ideas on the osg forum, there didn't seem to be much interest. The first one breaks when using camera-multithreading (which OpenMW doesn't use), and I haven't gotten to revising the solution to work with that. The second one is controversial because it complicates the implementation, uses more memory, and not everyone will see the same speed-up OpenMW did (mainly helps cpu-limited applications).

Anyway, I get the impression that osgAnimation isn't, or wasn't intended to be, much of a standard or a mature component. The way callback objects are used for updates makes serialization awkward (i.e. implementation details slip into the format). And recently, there's been a PR that happily broke backwards compatibility for a basic feature (defining bone weights), until I advised them not to.

I think we really only have two good options:
* Write a osgAnimation -> OpenMW converter and run this at load-time. This way we can, in theory, make use of osg's importer plugins for all sorts of formats and use the osg blender exporter as-is.
* Ditch the idea of using pre-made importers and write our own ones that will use OpenMW's animation system to begin with. We'll also need to write our own blender exporter for our own .osg format.

I'd probably prefer option 1, but I'm not sure how well osgAnimation's keyframe system maps to OpenMW's (never used the former).
Last edited by scrawl on 19 Sep 2017, 19:45, edited 1 time in total.
User avatar
Deltaxus
Posts: 39
Joined: 12 Aug 2017, 17:43

Re: Generic nif engine importer for openmw

Post by Deltaxus »

So in an effort to consolidate how we can accept new assets from various sources, I have begun writing a parser generator from the nif.xml. It will make a performant parser tailored to each game and it's corresponding nif version / versions. It will also share the same glue code between common nif objects to the engine. I'm writing the generator in perl so it's easy to change the scripting for it.
This would definitely come handy eventually, especially when attempting to finally load up Oblivion. However, as it was mentioned before, I don't think it is necessary (or rather not even advisable) to write a parser generator able to generate a parser for all the games which are using the .nif format, mainly when taking the maintenance cost into consideration. We should limit ourselves to Elder Scrolls III-V and maybe Fallout, but that's just my personal preference.
crassell
Posts: 49
Joined: 26 Aug 2017, 21:10

Re: Generic nif engine importer for openmw

Post by crassell »

A lot of discussion happened while I slept it seems.
crassell wrote: 19 Sep 2017, 06:42 So development is convoluted, with submodules and the rest. The idea that the nif format is defined in xml is not that bad (read once), once loaded into memory is nifxml not longer needed.
Good to hear that nifxml is a solid choice at least from a format spec. I'm going 1 step further in this generator by not loading the nifxml at all for the engine parsing code i'm creating but instead generating the c++ code that matches the format directly.
psi29a wrote: 19 Sep 2017, 10:09 There was some talk about using python bindings on niflib, which would be interesting in terms of speed when loading large NIFs (40+MiB files).
This was my fear with both the blender importers and even niflib after reviewing the version check code that's required. The speed is a huge concern which is why I'm going down generating parser code to target an exact subset of nif versions optimized for our uses.
scrawl wrote: 19 Sep 2017, 11:21 My first concern is how fast niflib would be. When I profiled our code a while ago, a lot of time spent loading was in parsing NIF files. So it's a bottleneck that we don't want to make any worse. As I understand niflib was mainly written in the context of NifSkope which uses QT's model/view for the Nif scene graph, and the niflib parsing is built around this model (everything is indexed by strings, etc). Which brings me to the concern that it might not be as fast as our custom written parser.
I'm definitely concerned about this as well which is why i'm not using niflib but instead going down a hopefully more performance tuned path. I already see some slight optimizations to be made on the existing nif format reads in openmw assuming we are not limited by file I/O already. I'll be doing benchmarking throughout this work so in the worst case we're no worse off and in the best case we get a perf improvement.
scrawl wrote: 19 Sep 2017, 11:21 Second, parsing the file is only half the puzzle. Non-Morrowind versions include non-Morrowind features that need to be handled by the game (e.g. rigid body physics), or converted into OSG equivalents, and there's no point in reading something that can't be used. Some games also use the extra data fields to control game-specific features, like the shader tags in skyrim's nifs. 'simple' files could potentially work just fine if we ignore the new fields, while others will require support for the new features before the files can be used.
This was already going to be a mixed bag as we have to unravel how the engine would even support some of the later nif versions and all of the asset modifications they introduce. Hopefully this will lay the groundwork for supporting other elder scrolls games even if there is much work to be done to support everything that is being read. I'm also going to use this parser to write converters for other formats which may lead into the OSG native support we want.
scrawl wrote: 19 Sep 2017, 11:21 So if we go for additional support we'll want to have a reasonable and useful goal, say supporting Oblivion or Skyrim files
This is more of the scope of work I would like to keep with the other recent Bethesda RPG's as an option (Fallout 3 / Fallout New Vegas / Fallout 4).
scrawl wrote: 19 Sep 2017, 11:21 I agree, partly because I don't want to deal with reports like "horses from empire earth 2 don't load into openmw". Don't want, don't care. We can reject them as unsupported, but that somewhat defeats the point of adding the code in the first place.
This is not my goal either and I should have better stated that in my initial post. The parser generation is merely a means to an end. I could for instance use a tweaked version of the nif.xml dtd to define a custom OSG format for animation. It could then share the import conversion logic between both the nif format path and the osg format path. I realize i'm speaking in ideals but it has potential.
Chris wrote: 19 Sep 2017, 14:04 I would actually prefer to not really focus on other NIF formats because of spotty support (and lack of proper documentation), but instead turn the attention to more readily usable formats like OpenGEX and glTF 2.0.
My thought here was to extend nif support for future endeavors with other bethesda titles as well getting new functionality with what the later asset versions offer but I do agree that reverse engineering is not for the faint of heart if nif still has a lot of mystery object data.

I'm starting to understand there's a lot of differing views on the right path forward going by what we discussed previously in this post:
viewtopic.php?f=22&t=4591&start=10
Chris wrote: 19 Sep 2017, 14:04 Ideally we should leverage a general model loading API to support multiple formats. Like how we use FFmpeg to support not only the wav, mp3, and bik media files, but also gain support for ogg, flac, and webm with no extra work on our part. OSG has a general model loading API, and while I understand there are issues with handling specific features of Morrowind's NIFs through it, if we could work that out it would be a big boon as we'd not have to worry much about model formats further. We can assume OSG has or will have import plugins for formats modders would like to use, and anyone that wants to add support for certain other formats would work with OSG rather than us (though we could still be available for advice), so it eases the overall amount of work we have to do for future format support.
Except for the problems with animations, couldn't this be done today if it's entirely in the realm of OSG supported plugins? I guess someone has to look through all the plugin options and separate the useful from the redundant.
scrawl wrote: 19 Sep 2017, 16:21 It should be noted that even with no support in OpenMW, one can still use osgconv to convert insert-obscure-format-here into .osg and load that into OpenMW.
I could conceivably make the generic nif parser i'm creating spit out OSG format. That would alleviate the maintenance issue on openmw directly (you wouldn't officially support oblivion/skyrim/fallout 4/etc. formats), but you could point to this tool as a means to convert bethesda archives of static assets to OSG. From there the problem is all the custom things for physics data and such that have nothing to do with the graphics engine. Not sure how that fits in.
scrawl wrote: 19 Sep 2017, 16:21 Animations are another issue. The closest thing to an animation 'standard' in OSG is the osgAnimation node kit, and this is also what the model importers turn animations into. But OpenMW uses a custom animation component, mostly for optimization reasons and because osgAnimation doesn't support a peculiarity in Morrowind's way of skinning. So the osg importers wouldn't work for animations anyway, unless we did some sort of conversion that loads osgAnimations into our custom system, or change the animation playback in the engine to look for both Morrowind-style controllers and 'osgAnimations' (sounds like a maintenance nightmare).
This was discussed in my previous developer application post. It seems like regardless of format, we will have to tailor some conversion to fit into openmw's animation format. This means OSG / OpenGEX / glTF 2.0 are all on the table. I still haven't given up on this.
raven wrote: 19 Sep 2017, 16:25 For performance you'll probably want something binary that does not require much, if any, parsing.
For any consideration of a text based object format without a binary option, I wouldn't even consider it without rolling a binary format to complement the text version.
scrawl wrote: 19 Sep 2017, 19:29 * Write a osgAnimation -> OpenMW converter and run this at load-time. This way we can, in theory, make use of osg's importer plugins for all sorts of formats and use the osg blender exporter as-is.
This is most likely the best path of the 2 options. Rolling our own exporters for blender would create redundant work.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Generic nif engine importer for openmw

Post by Chris »

scrawl wrote: 19 Sep 2017, 19:29 It's a conceptual difference that can't be converted (well, you can convert the standard way to NIF's way, but not always the other way around). Most skinning systems will assign a bind matrix to each bone. NIFs assign a set of 'bone, bind matrix' pairs to each skinned mesh. This just complicates things, for the minimal 'benefit' of being able to use different bone bind matrices for different meshes on the same skeleton, something that's not really required if you design all of your assets with the same conventions. I don't think it makes sense to implement this in osgAnimation.
If there's no models that rely on being able to bind different matrices to different meshes on the same skeleton, I would imagine it's possible to take the 'bone, bind matrix' pairs of the skinned meshes and pre-map them to bone-specific bind matrices when loading (IIRC the PC/NPC skeleton nif contains a dummy mesh, perhaps giving a default set of bind matrices to map skinned meshes to). That's probably just a bit of hopeful optimism, though. It's been a while since I dealt with the horrors that is the NIF format.
I'd probably prefer option 1, but I'm not sure how well osgAnimation's keyframe system maps to OpenMW's (never used the former).
Option 1 does seem to be the better option to me too. I imagine the biggest issue may be that most animation systems prefer to split each animation group into a separate track, rather than having one large track with text keys indicating where each group is. Back before when we were using Ogre3D I always wanted to split the groups into individual animations (if for no other reason than to make selecting a group more efficient, since we need to do a linear search through almost the whole set of text keys to find where a group starts and ends, but also it would make it easier to replace individual groups), but didn't manage to resolve the issues. I probably could take another stab at it, but I'd need to get acquainted with how NIFs and OSG are being glued together nowadays first.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Generic nif engine importer for openmw

Post by scrawl »

If there's no models that rely on being able to bind different matrices to different meshes on the same skeleton, I would imagine it's possible to take the 'bone, bind matrix' pairs of the skinned meshes and pre-map them to bone-specific bind matrices when loading (IIRC the PC/NPC skeleton nif contains a dummy mesh, perhaps giving a default set of bind matrices to map skinned meshes to). That's probably just a bit of hopeful optimism, though.
I'm afraid so. The bind positions are actually different (for some files like werewolves and better bodies, anyway), which was IIRC the reason why our initial animation attempts with Ogre only worked for certain files and not others. The 'solution' to that was to duplicate the skeleton for each skinned mesh, which allowed us to use the different bind matrices but also created a redundant scene graph, negatively affecting the rendering performance.
Post Reply