Shadows

[AnyOldName3]

I got the laptop back and tested both with and without your PR.

• Shadows work about as well as on the desktop when they work.
• The debug HUD never worked, with or without your shader tweak.
• Without your tweak, I just see the simple water.
• With your tweak, I see the water shader, but it sometimes looks weird, and sometimes the shadows fade away or disappears (I think that's two separate issues, one with the shader, and one with multi-texturing getting muddled.

[imec]

Compiling is about as far as I go with computers, but I've noticed that equipping any weapon with a magic/enchanted effect around it and then going third person will cause all shadows to disappear. You can also put the weapon on the ground, and as long as it's in your view, shadows will also disappear. I noticed shadows kept vanishing near plantations and I think it might actually have been the slave bracelets causing it.

I've also noticed that any small change in the camera results in the shadow resolution going crazy. I've had the problem since the shadow build has been available and I thought everyone had it, but your screenshot looks much more crisp than anything that I've been able to accomplish. Scrawl sounded like he had the same problem, except it was happening with his water shader. Very strange.

That's about all I have to add as far as weird bugs. Thank you for all of your hard work. This was a really big improvement and I have had fun testing things out.

[AnyOldName3]

I've got two methods of making the sky not receive shadows working.

Method 1 involves using nodemasks like osgShadow seems to want us to, but it's a little bit messy. I reckon that that issue could be resolved if we made our hierarchical nodemasks work a bit differently. Currently, parent nodemasks don't include the nodemasks of their potential children, so a traversal looking for that specific child won't find it unless we also search for the parent nodemask. That's a bit of a problem because sometimes a nodemask is used for both parent nodes and child nodes, so erroneous extra children would be found. To solve that, we either need to propagate child nodemasks to their parents or make parent nodemasks include any potential child nodemasks. Also, the moons were using the nodemask 2, which is the debug nodemask, so I changed them to use the sun nodemask. This is available at https://github.com/AnyOldName3/openmw/t ... method-one

Method 2 is closer to what Scrawl suggested, where I've disabled a texture unit for the sky. This also has mess, as there's not a nice way to let the sky know which texture unit to disable, so it's put in as a magic number. This feels worse to me as there's no way to disable shadows for things with shaders if we ever want to do that. This is available at https://github.com/AnyOldName3/openmw/t ... method-two

[psi29a]

Are there any performance differences between the two approaches?

Should we try to keep to OSG's shadow implementation?

[scrawl]

In theory, there should be no performance impact for either approach. Number 1 could have an impact if there were non-shadow receivers somewhere deep in the scene graph then that would make the 2nd cull pass more expensive, but right now it's just the sky and nothing else.

This feels worse to me as there's no way to disable shadows for things with shaders if we ever want to do that.

But the first approach has the same problem, no? It does the same thing, render the object without the shadow texture bound. If there's a shader used that expects shadowing, we'll still need to adjust that, which either approach.

I'd prefer method 2, because it's less code and because I don't like the conflating node masks thingy.

[AnyOldName3]

If we go for method one, ensuring the texture isn't bound is done for us. I'd presumed that before any of this got merged I'd have to fence off the shadow stuff in the shaders behind #ifdef @shadows@ anyway, so they could be turned off by user settings, meaning that method one doesn't really require any work we wouldn't need anyway.

The open issue I have with method two is how we'd get the sky to know about which texture units have shadow maps bound. The two possible solutions I have are giving the sky access to the shadow settings object or just iterating through every possible texture unit (which involves checking how many there might be, which I'm still not totally sure how to get for fixed-function mode).

[AnyOldName3]

I've just merged Scrawl's tweaks into my test branches, including the two don't-shadow-the-sky ones.

I'm still wondering why the water shader disappears on my desktop and does weird things for other people. Unfortunately, I won't have access to the laptop for a few days, so if I'm going to work out what weird things are happening, I need to work out why my desktop doesn't like them. I'm hoping it isn't a driver bug and is actually just that two things are interfering, and other people's computers are getting it right by blind luck, but don't know where to look.

[AnyOldName3]

Actually, I just tried it under an Ubuntu VM on my desktop, and the water shader appeared there, and the weirdness basically seemed to be that the shadow map was being used as one of the other textures by the water shader. Once I figure out why the shadow map breaks if bound to anywhere other than texture unit one I can just put it in another slot and the problem will resolve itself.

Weirdly, in the VM, the camera insisted on pointing at the ground which made navigating my way to some water a bit of a nuisance.

[scrawl]

I'm still wondering why the water shader disappears on my desktop and does weird things for other people. Unfortunately, I won't have access to the laptop for a few days, so if I'm going to work out what weird things are happening, I need to work out why my desktop doesn't like them. I'm hoping it isn't a driver bug and is actually just that two things are interfering, and other people's computers are getting it right by blind luck, but don't know where to look.

If I had to make a guess, it might be the reflection/refraction renders are attempting to retrieve a shadow map that's not actually bound in this case; or accessing a non-shadow texture (default 0 if no uniform is set) as the shadow texture. Does the error go away if the shadowing part of the object shader is removed? Does the error go away if you leave the water shader as is, but remove its render textures?

To debug OpenGL errors, I've always found GDebugger helpful. The program is discontinued now, but it still works if you can find a download. Alternatively, you might try ApiTrace which, I think, shows errors when you replay a trace file (but I might be misremembering)

[AnyOldName3]

I've done a bit of investigating, and it seems that osgShadow::VDSM doesn't work in a minimal test case when the shadow texture unit is set to anything other than one. This gives me something much simpler to debug than OpenMW, so that's where I'll be focusing. I'm pretty sure that once I've fixed that, most of the rest of the issues will go away magically.