Alpha channel with osgt objects

Feedback on past, current, and future development.
Post Reply
unelsson
Posts: 227
Joined: 17 Mar 2018, 14:57

Alpha channel with osgt objects

Post by unelsson »

I didn't find this on bug tracker - osgt objects with transparent textures don't have transparency enabled in current OpenMW.

This is fixed by adding this to renderingmanager.cpp

Code: Select all

sceneRoot->getOrCreateStateSet()->setMode(GL_BLEND, osg:: StateAttribute:: ON);
sceneRoot->getOrCreateStateSet()->setRenderingHint (osg:: StateSet:: TRANSPARENT_BIN);
but doing that seems to destroy vanilla Morrowind's Red Mountain area. Also, I've experienced similar issues as in this topic, even when doing the trick above.
viewtopic.php?f=2&t=4550&p=49033&hilit=alpha#p49031
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Alpha channel with osgt objects

Post by scrawl »

There's either a setting you need to toggle on in Blender or a missing feature in the blender-osg exporter. The osg format supports this for sure.
Setting the entire world to use alpha blending does of course break things.
unelsson
Posts: 227
Joined: 17 Mar 2018, 14:57

Re: Alpha channel with osgt objects

Post by unelsson »

scrawl wrote: 29 Apr 2018, 17:16 There's either a setting you need to toggle on in Blender or a missing feature in the blender-osg exporter. The osg format supports this for sure.
Setting the entire world to use alpha blending does of course break things.
Heh, yes. I'll see if there is something with blender, but the thing is that I got the alpha working with my blender-exported objects by turning the entire world for alpha blending. Therefore my exports do have alpha in .osgt format, textures in png format, however OpenMW engine for some reason doesnt use it.
User avatar
AnyOldName3
Posts: 2666
Joined: 26 Nov 2015, 03:25

Re: Alpha channel with osgt objects

Post by AnyOldName3 »

If you could share an OSGT file with us that doesn't work (preferably a small cube with the simplest material exhibiting the issue so it's easy to read) someone could take a look at it to see if the OSGT file actually contains any alpha information.
unelsson
Posts: 227
Joined: 17 Mar 2018, 14:57

Re: Alpha channel with osgt objects

Post by unelsson »

I'm using this https://opengameart.org/content/oak-tree converted to osgt, it uses texture quercus_a_1024.png that most certainly has alpha. I've attached my converted files. Morrowind's bushes etc. render alpha like they should. There are screenshots of non-working alpha in OpenCS preview, and how it should render in Blender view.
Attachments
tree_blenderview.png
tree_preview.png
tree_preview.png (76.41 KiB) Viewed 6484 times
quercus_A_packed_v1.osgt.zip
(19.37 KiB) Downloaded 249 times
Quercus_A_1024.png
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Alpha channel with osgt objects

Post by scrawl »

No, that's looking at the wrong thing. Let me reiterate. OpenMW most certainly loads the osgt file correctly because it's not even OpenMW loading it but OSG does. The osg file is missing the alpha blending flag, which needs to be present. OpenMW won't make a guess that you intend to use alpha blending just because the texture is transparent. That's not how model formats work. So you're looking at either an issue in Blender or the exporter. If you look at the osgexport code, it does attempt to add the alpha blending flag if the images depth is > 24, so that in principle should work. However, if your image is an external reference that Blender can't load (e.g. because it's only in OpenMW's data path but not present relative to the Blender file) then that can't work. Check if your texture displays in blender. If it does, I'm afraid you'll have to debug the exporter code to see what goes wrong. If you want us to do so, you'll need to show us the blend file before you export it - the broken osgt file is useless.
unelsson
Posts: 227
Joined: 17 Mar 2018, 14:57

Re: Alpha channel with osgt objects

Post by unelsson »

Right.. Thanks for a really informative answer! I'll look up to it in a few days, and when it's resolved, I'll post a solution here and/or to wiki.

edit: As of day later, I don't yet have a proper solution, but the problems with model above were fixed by forcing exporter to use flags GL_BLEND ON, GL_CULL_FACE OVERRIDE. So far I've just forced these on in exporter's osgdata.py. I don't yet have a good understanding why blending flag doesn't work with exporter, culling flag doesn't seem to have any implementation in code.

edit2: It seems there are other issues with blender exporting to osg. I tested some osg examples, and there seems to be no problem in OpenMW, just need to get flags and statesets ok. I currently get sky-colored edges around the tree object when terrain is behind, and it seems stateset isn't properly set to TRANSPARENT_BIN. Alpha blending works nicely with other objects and sky though. OSG example files that are properly set to TRANSPARENT_BIN have no rendering errors.
Last edited by unelsson on 02 May 2018, 10:21, edited 2 times in total.
unelsson
Posts: 227
Joined: 17 Mar 2018, 14:57

Re: Alpha channel with osgt objects

Post by unelsson »

Investigating some osgt-files created with osgconv, I've been able to manually fix the exported osgt-file. Now I only wish I could fix this to blender's osg exporter to automate things.

The crucial findings were:

Code: Select all

ModeList 3 {
    GL_BLEND ON
    GL_CULL_FACE OVERRIDE
}
GL_BLEND gives branches proper transparency, and at least with this particular model, cull-face setting "override" is required to fix culling-related errors with transparent branches.

Code: Select all

RenderingHint 2
RenderBinMode USE_RENDERBIN_DETAILS 
BinNumber 10 
BinName "DepthSortedBin"
Without this alpha-channeled branches would get sky-colored artifacts, as the model wasn't placed in correct rendering bin. Using bin 10 (TRANSPARENT_BIN) makes alpha blending work properly.
unelsson
Posts: 227
Joined: 17 Mar 2018, 14:57

Re: Alpha channel with osgt objects

Post by unelsson »

I had trouble getting Blender 2.79 to osgt exporting working fluently. Good people at OpenMW-discord instructed me what's this transparency all about, and so I incorporated alpha blending (blender's "Z Transparency") and alpha testing (blender's "Mask"), and backface-culling (Blender Game rendering the "backface-culling" checkbox) to osgexport. I'm still very much a beginner in graphics and 3d-modeling, so I'm not sure if it's perfect, but I'm getting results that satisfy. Note: This is nothing "official" OpenMW-stuff, just a way I found that works.

https://github.com/unelsson/osgexport

https://imgur.com/a/429pbc7

So, how to make a osgt-model with Blender to OpenMW?
1) Create a model in blender.
2) Add material or multiple materials to your model (there might be performance advantages in using the same texture file)
3) Add texture to your material, tune all settings of texture and material in blender until you are satisfied with the results. At least png works, probably some other file formats too.
4) Set Blender rendering to "Blender Game". If you don't want backface-culling for some material, uncheck material->game settings->backface culling.
5) Save. After saving, scale bigger to 1:128 (edit: or 1:70, hinted elsewhere). This is a workaround, as osgt-exporter doesn't yet understand Blender unit scale.
6) Don't save the scaled version, but export that to osgt.
7) Copy osgt-file to data/meshes, copy textures to data/textures. Check that your osgt-file has correct path for texture (should be "textures/filename.png"
8) Open OpenMW-CS and create a new plugin (or game), Objects -> Add Record, e.g. type Static (other if you want something else) -> At Model/Animation column, write your osgt-filename (it should suggest and auto-complete).
9) Open a Cell, drag&drop the Object to the Scene, that will create an Instance of your Object.
10) Save your game/plugin, open it through openmw-launcher, play and enjoy.

As a sidenote, some other learning experiences for me were:
a) Join all objects in blender to one before exporting, to reduce drawables, therefore getting insanely much higher performance.
b) Alpha testing should be faster than alpha blending, but textures might need tuning, you likely need a big enough resolution to get texture alpha-edges look nice. There's probably a ton more to learn. :)
c) I've had occasional rendering problems of alpha edges, at least with alpha blending, when ctrl-D duplicating branches and placing them over each other. These problems are visible in blender, so according to my experience, if you can make a good blender model, it works in OpenMW too.
d) Placing planes that cross each other sometimes resulted in rendering problems, I have no idea why, and I haven't tested this with the latest exporter settings. Because of this, I prefer to avoid planes that cross each other.
Post Reply