osgb/osgt animation format support

Everything about development and the OpenMW source code.
crassell
Posts: 49
Joined: 26 Aug 2017, 21:10

osgb/osgt animation format support

Post by crassell »

I'm not liking the idea of continuing the topic of adding support for this underneath other loosely related posts, so I'm creating this thread separately to organize our thoughts on this. Re-pasting the conclusion of scrawl from this thread:
viewtopic.php?f=6&t=4648&start=10
scrawl wrote: 19 Sep 2017, 19:29
What exactly is the issue with Morrowind's way of skinning? I'm sure it's been mentioned before, but I don't know where. Would it be possible to improve osgAnimation to support what we need? Would it be possible to convert Morrowind's way of skinning to something osgAnimation can work with?
It's a conceptual difference that can't be converted (well, you can convert the standard way to NIF's way, but not always the other way around). Most skinning systems will assign a bind matrix to each bone. NIFs assign a set of 'bone, bind matrix' pairs to each skinned mesh. This just complicates things, for the minimal 'benefit' of being able to use different bone bind matrices for different meshes on the same skeleton, something that's not really required if you design all of your assets with the same conventions. I don't think it makes sense to implement this in osgAnimation.

As for the 'optimization reasons' thing, our component is mainly faster due to:
* Software skinning is done in the Cull phase, not the Update phase, to skip unneeded work for objects not on screen.
* Not using a synchronized draw traversal (instead, the internal geometry is double-buffered)
When I briefly brought up these ideas on the osg forum, there didn't seem to be much interest. The first one breaks when using camera-multithreading (which OpenMW doesn't use), and I haven't gotten to revising the solution to work with that. The second one is controversial because it complicates the implementation, uses more memory, and not everyone will see the same speed-up OpenMW did (mainly helps cpu-limited applications).

Anyway, I get the impression that osgAnimation isn't, or wasn't intended to be, much of a standard or a mature component. The way callback objects are used for updates makes serialization awkward (i.e. implementation details slip into the format). And recently, there's been a PR that happily broke backwards compatibility for a basic feature (defining bone weights), until I advised them not to.

I think we really only have two good options:
* Write a osgAnimation -> OpenMW converter and run this at load-time. This way we can, in theory, make use of osg's importer plugins for all sorts of formats and use the osg blender exporter as-is.
* Ditch the idea of using pre-made importers and write our own ones that will use OpenMW's animation system to begin with. We'll also need to write our own blender exporter for our own .osg format.
And Chris replied:
Chris wrote: 20 Sep 2017, 08:22
scrawl wrote: ↑19 Sep 2017, 13:29
It's a conceptual difference that can't be converted (well, you can convert the standard way to NIF's way, but not always the other way around). Most skinning systems will assign a bind matrix to each bone. NIFs assign a set of 'bone, bind matrix' pairs to each skinned mesh. This just complicates things, for the minimal 'benefit' of being able to use different bone bind matrices for different meshes on the same skeleton, something that's not really required if you design all of your assets with the same conventions. I don't think it makes sense to implement this in osgAnimation.
If there's no models that rely on being able to bind different matrices to different meshes on the same skeleton, I would imagine it's possible to take the 'bone, bind matrix' pairs of the skinned meshes and pre-map them to bone-specific bind matrices when loading (IIRC the PC/NPC skeleton nif contains a dummy mesh, perhaps giving a default set of bind matrices to map skinned meshes to). That's probably just a bit of hopeful optimism, though. It's been a while since I dealt with the horrors that is the NIF format.
I'd probably prefer option 1, but I'm not sure how well osgAnimation's keyframe system maps to OpenMW's (never used the former).
Option 1 does seem to be the better option to me too. I imagine the biggest issue may be that most animation systems prefer to split each animation group into a separate track, rather than having one large track with text keys indicating where each group is. Back before when we were using Ogre3D I always wanted to split the groups into individual animations (if for no other reason than to make selecting a group more efficient, since we need to do a linear search through almost the whole set of text keys to find where a group starts and ends, but also it would make it easier to replace individual groups), but didn't manage to resolve the issues. I probably could take another stab at it, but I'd need to get acquainted with how NIFs and OSG are being glued together nowadays first.
Chris I have started to work on this so I hope that's ok.

The link below is my first stab at an object level comparison of the nif and osg formats using the objects I found in base_anim.nif and xbase_anim.kf:
https://docs.google.com/spreadsheets/d/ ... sp=sharing

I've put N/A in the osg object column for the objects in these files which I feel should already serialize for import to osg just fine (haven't confirmed in all cases). I also need to still fill out the current openmw object being used for each nif object. Let me know if something looks out of place.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: osgb/osgt animation format support

Post by Chris »

crassell wrote: 25 Sep 2017, 06:36 The link below is my first stab at an object level comparison of the nif and osg formats using the objects I found in base_anim.nif and xbase_anim.kf:
https://docs.google.com/spreadsheets/d/ ... sp=sharing

I've put N/A in the osg object column for the objects in these files which I feel should already serialize for import to osg just fine (haven't confirmed in all cases). I also need to still fill out the current openmw object being used for each nif object. Let me know if something looks out of place.
Maybe I'm reading it wrong, but this comparison seems backwards. The idea we settled on was to allow OSG to load a model with its osgAnimation component as normal, and then we convert the osgAnimation into the objects OpenMW needs to animate meshes. osgAnimation won't have everything a NIF animation will and not everything may map 1:1, but as long as it has the necessary information, we can construct OpenMW's animation objects from osgAnimation's objects.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: osgb/osgt animation format support

Post by scrawl »

I've put N/A in the osg object column for the objects in these files which I feel should already serialize for import to osg just fine
Yes, they should.
NiSkinInstance/NiSkinData -> osgAnimation Object: RigTransformHardware
This should be 'RigGeometry' as well (except it's under the osgAnimation namespace, in contrast to OpenMW's RigGeometry). The RigTransformHardware is just the implementation path (software or hardware skinning). Another reason to not like the osgAnimation format - the implementation path really has no business being in the file, it should be chosen by the application at runtime.

I wouldn't worry about KF files for now. This task is large enough as it is. Try using a creature model like nix hound.nif that doesn't require extra KF files to be used.

KF files just contain more keyframe animations (but without the skeleton) and are merged at runtime with the skeleton from the base_anim.nif. This feature is used by the game to conditionally add animations (e.g. for females) while still using the 'base' animation by default for animations that don't need to be specialized. Arguably, this is a little messy. There's also "xmodel.nif" and "xmodel.kf" files for every "model.nif" that has animations, which are basically equivalent and we have no idea why these duplicates are there, but the game will always use 'xmodel.nif+xmodel.kf' if it exists instead of 'model.nif'.
Chris was suggesting a while back we could do things differently for new files, but I'm not quite sure how.

If we stick to the separate KF-file way, I'm not sure how exporting 'only' keyframes from Blender would look like. Can we just export as normal (including the skeleton), but when loading in OpenMW only look at the animations and ignore the skeleton?
crassell
Posts: 49
Joined: 26 Aug 2017, 21:10

Re: osgb/osgt animation format support

Post by crassell »

Chris wrote: 25 Sep 2017, 12:56 Maybe I'm reading it wrong, but this comparison seems backwards. The idea we settled on was to allow OSG to load a model with its osgAnimation component as normal, and then we convert the osgAnimation into the objects OpenMW needs to animate meshes. osgAnimation won't have everything a NIF animation will and not everything may map 1:1, but as long as it has the necessary information, we can construct OpenMW's animation objects from osgAnimation's objects.
I promise if you repeat your intent multiple times in multiple different ways it eventually gets through my skull :lol: .

Here was my original approach in my mind:
animation.osgb/osgt -> custom serializer on openmw animation classes to match osg animation format

Here is what you are arguing for:

Code: Select all

void convertAnimationToOpenMW(){
osg::ref_ptr<osg::Node> osgAnimationNode = osgDB::readNodeFile("animation.osgb/osgt");
/*
code to read elements / children of osgAnimationNode pointer and de-compose it into the openmw objects
*/
/* 
let osg::ref_ptr to osgAnimationNode go out of scope and delete the default osg animation data
*/
}
That being said, my excel is still going to be useful in a high level attempt to show what OSG animation objects will contain the data for the counterpart openmw / nif objects. For the objects we already 1-1 map with OSG it helps show me what I can just copy as is from the normal OSG animation format.
scrawl wrote: 25 Sep 2017, 13:27 I wouldn't worry about KF files for now. This task is large enough as it is. Try using a creature model like nix hound.nif that doesn't require extra KF files to be used.
Sounds good. I will ignore KF for now although it seems critical to how humanoids are animated so I will definitely come back to it.
scrawl wrote: 25 Sep 2017, 13:27 KF files just contain more keyframe animations (but without the skeleton) and are merged at runtime with the skeleton from the base_anim.nif. This feature is used by the game to conditionally add animations (e.g. for females) while still using the 'base' animation by default for animations that don't need to be specialized. Arguably, this is a little messy. There's also "xmodel.nif" and "xmodel.kf" files for every "model.nif" that has animations, which are basically equivalent and we have no idea why these duplicates are there, but the game will always use 'xmodel.nif+xmodel.kf' if it exists instead of 'model.nif'.
Chris was suggesting a while back we could do things differently for new files, but I'm not quite sure how.

If we stick to the separate KF-file way, I'm not sure how exporting 'only' keyframes from Blender would look like. Can we just export as normal (including the skeleton), but when loading in OpenMW only look at the animations and ignore the skeleton?
Yuck. I think our only option is to export both together to not have to alter the current blender exporter. Player / NPC modding overhauls will be more complicated because of this. I'm guessing we should provide a base_anim.osgb/osgt of the original nif and kf files for modders to start with. Is there a way just to allow support for a unified file in the current framework and ignore applying an external skeleton everytime?
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: osgb/osgt animation format support

Post by Chris »

scrawl wrote: 25 Sep 2017, 13:27 Chris was suggesting a while back we could do things differently for new files, but I'm not quite sure how.
I'm not quite sure I had a specific idea for anything. But one thing we should do in the future is support multiple animation sources (rather than the current single-patch override an esm/p defines per-NPC). Rather than a single KF file that has one large animation track with various groups tagged on it at various time points, allow one KF/track per group. We could split the large track into multiple tracks given where each group starts (for instance of a group name in the text keys) and stops (last instance of it), copying the relevant text keys and adjusting the time points. That way we could define and replace individual groups separately.
crassell wrote: 25 Sep 2017, 15:45 Here was my original approach in my mind:
animation.osgb/osgt -> custom serializer on openmw animation classes to match osg animation format
My idea is to take a model.osgb/t and get a root Node for it from OSG. Then we search the returned Node tree for an osgAnimation node (or whatever it uses), read that node and any child nodes for information about how it animates, then replace it with what OpenMW needs.
scrawl wrote: 25 Sep 2017, 13:27 If we stick to the separate KF-file way, I'm not sure how exporting 'only' keyframes from Blender would look like. Can we just export as normal (including the skeleton), but when loading in OpenMW only look at the animations and ignore the skeleton?
That's what I'd suggest. Get the skeleton and animation from the osgb/t, remap the animation node targets to the "real" skeleton, then drop the bones from the animation file.
crassell
Posts: 49
Joined: 26 Aug 2017, 21:10

Re: osgb/osgt animation format support

Post by crassell »

Chris wrote: 25 Sep 2017, 17:25 My idea is to take a model.osgb/t and get a root Node for it from OSG. Then we search the returned Node tree for an osgAnimation node (or whatever it uses), read that node and any child nodes for information about how it animates, then replace it with what OpenMW needs.
Then we're on the same page.

So I have some issues with the current key data handling. nifkey.hpp is doing double duty as the read data piece for NikeyframeData and the storage for key information in the KeyframeController class. This inevitably leads me into having to import knowledge about nif data types as in the below example:

Code: Select all

 typedef ValueInterpolator<Nif::QuaternionKeyMap, QuaternionSlerpFunc> QuaternionInterpolator;
    typedef ValueInterpolator<Nif::FloatKeyMap, LerpFunc> FloatInterpolator;
    typedef ValueInterpolator<Nif::Vector3KeyMap, LerpFunc> Vec3Interpolator;
I could just get ignore this for now and continue or I could try to pull out the nif dependencies and have a common keymap and interpolator data structure (template in this case) between the 2 components. Thoughts?
crassell
Posts: 49
Joined: 26 Aug 2017, 21:10

Re: osgb/osgt animation format support

Post by crassell »

I took the liberty of making a keytype.hpp file that both nif and sceneutil can share for now.

Some more clarity I need is again on this statement:
- Move KeyframeController out of nifosg into a component (e.g. sceneutil) and make it not depend on NifOsg::NodeUserData
After taking a look at NifOsg::NodeUserData there doesn't seem to anything tied to the nif in that derived object.

Couldn't we just do this same call in the osgAnimation -> openmw animation code?

Code: Select all

            // UserData used for a variety of features:
            // - finding the correct emitter node for a particle system
            // - establishing connections to the animated collision shapes, which are handled in a separate loader
            // - finding a random child NiNode in NiBspArrayController
            // - storing the previous 3x3 rotation and scale values for when a KeyframeController wants to
            //   change only certain elements of the 4x4 transform
            node->getOrCreateUserDataContainer()->addUserObject(
                new NodeUserData(nifNode->recIndex, nifNode->trafo.scale, nifNode->trafo.rotation));
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: osgb/osgt animation format support

Post by scrawl »

After taking a look at NifOsg::NodeUserData there doesn't seem to anything tied to the nif in that derived object.

Couldn't we just do this same call in the osgAnimation -> openmw animation code?
Hmm.. in theory, yes. I'd prefer getting rid of it in one way or another eventually, but we can worry about that later I suppose.
I could just get ignore this for now and continue or I could try to pull out the nif dependencies and have a common keymap and interpolator data structure (template in this case) between the 2 components. Thoughts?
Dependencies can probably be ignored for the initial 'proof of concept', we'll just want to fix them at some point.
crassell
Posts: 49
Joined: 26 Aug 2017, 21:10

Re: osgb/osgt animation format support

Post by crassell »

How should we determine in the scenemanager.cpp load function that an osgb/osgt file is intended for animation? Should we call the file something like nixhound.anim.osgt for example? The only other alternative would be to search the return osgDB node tree for animation nodes but that seems unnecessary and expensive.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: osgb/osgt animation format support

Post by scrawl »

You can use the root node's 'getNumChildrenRequiringUpdateTraversal'. If it's 0, the file shouldn't have anything animated. If it's > 0, we still can't be sure if there are actually animations or just some something else like a material controller or particles, but at least we won't have to scan the majority of files redundantly.
Post Reply