Shadows

Everything about development and the OpenMW source code.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Shadows

Post by AnyOldName3 »

Can you show me a before and after screenshot of that issue (and ideally tell me somewhere where it's always happening and is immediately visible that wouldn't involve spoilers for someone who's not actually played Morrowind, as I'm waiting for 1.0 before I actually play it, hence doing my utmost to hurry 1.0 along).
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Shadows

Post by scrawl »

Wow, it really works and there are practically no artifacts anywhere. Amazing! FWIW, I had to change the gl_TexCoord[0] to gl_TexCoord[0].xy to get the debug overlay to work on my end.

Link to branch for the lazy: https://github.com/anyoldname3/openmw/t ... -test-vdsm - if you want to try it, I think 'force shaders' should be enabled in your settings.

So what actually changed, compared to mine attempt where everything glitched out? The commit 'Update the shadow frustrum bounds properly' I had already applied to my osg build, or so I think. Other than that, I can't see any other changes to ViewDependentShadowMap, or were there already changes in your subclass when you initially added it to git?
Potentially change how non-translucent shadow casters are rendered so unnecessary alpha testing is skipped.
Binding the texture is the main thing that's unnecessary and performance intensive, the alpha testing as such shouldn't be a big deal.
I imagine we could run a "post-process" on the state graph collected by the cull visitor that will remove any statesets with textures that aren't eventually used for alpha testing.
Stop the sky from receiving shadows.
Unlike everything else, the sky is rendered without shaders. So all that needs doing is disabling the shadow texture. Try to set the shadow texture unit to OFF|PROTECTED in the sky's stateset.
Find an efficient way to have VDSM support objects that don't receive shadows so we don't have to rearrange the scene graph and don't have to put up with my current workaround of rendering non-receivers twice, once with shadows, and then a second time without, which draws over the first.
Which objects exactly do we need to not receive shadows? The easiest way is to change the material parameters to use emissive lighting rather than diffuse, so the shadow practically has no effect. I can't think of an easy way that would allow the shadow technique to control what receives shadows. It's really up to that object.
For whatever reason, the terrain isn't making it into the shadow map, so hills and the like don't cast shadows.
Add Mask_Terrain to the caster mask. After doing that I see the terrain shadows just fine.
For performance, we'll want to make sure the terrain is only rendered once in the shadow map (as opposed to once per each texture layer). relevant code
I've also noticed that this branch seems to break terrain texture blending, making the entire world a massive patchwork of seams everywhere. That should probably be resolved down the line before this is merged.
I suspect this is due to the shadow texture unit being set to a fixed slot (e.g. 1) that then overrides whatever multitexturing was originally being done in that slot and breaks the terrain blending. This will also be an issue for normal objects that use more than one texture (normal maps, detail maps, etc.)
I just added shadow support to the water shader, too, but I'm not exactly sure if this is what shadows striking water looks like because it feels weird. It's quite possible that the convenient 'shadow' variable in the water shader doesn't actually mean what I think it means, so changing its value from a constant 1.0 to what I read from the shadow map may be incorrect.

OpenMW-Shadow-Water.png
Eh, that's not the water shader is it? Looks like the simple 'animated texture' water.

For me, the water shader looks a little strange now. There are shadows (or reflections?) that seem to move about as I turn the camera.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Shadows

Post by AnyOldName3 »

I'm adding numbers so that it's easier to keep track of which point is which going forward.
  1. The main changes compared to your work are implementing the shader changes to make shadows work at all, creating a subclass so I could add a debug HUD, and copying the Drawable handling bit into the subclass. Your drawable handling got merged into the master branch of OSG but wasn't backported to the 3.4 branch and isn't in the master branch of OSG on Steroids. There are some other tweaks I made that seem to have gone missing, but they don't seem necessary right now.
  2. I'll look into meddling with the sky's stateset.
  3. I'm not sure what objects are supposed to not receive shadows other than the sky, but my mention of it was because the way the VDSM technique is set up, it never actually renders things that don't have a nodemask saying they should receive shadows. I convinced myself that the workaround I found might result in some things being drawn twice, once with unwanted shadows, and then a second time without, but it turns out that when I actually check this, that wasn't the case and it was perfectly fine as it was. Part of the issue was apparently that I'd not put this workaround in my subclass, only my experimental modifications directly to the VDSM class, and things weren't working as I thought they should. I'll have a look at this.
  4. At my end, adding the terrain mask to the caster mask doesn't work. Can you post a screenshot of it working, including the debug hud? It's possible that it's something to do with AMD's Vega drivers being immature because OSG's spamming a lot of error reports to the console on my desktop, but not on my laptop which has a GTX 850M.
  5. If we're hitting issues with multi-texturing, we might have a problem. Not including the shadow map, the object shader might use up to eight textures. If we tell the shadow map to use slot 8, leaving 0-7 free for the other textures, the shadow map doesn't get rendered because of (what I assume is) some fixed function stuff not liking to use slots above seven. I found that OpenGL 3 mandates that each shader stage have at least sixteen slots available, but I'm not sure about older versions.
  6. I'm no longer convinced that that is the water shader, as when I manually ruin the output of the water shader by setting the red channel to 1.0, nothing changes. However, I have the water shader turned on, so it's supposed to be working. If I change the shadow texture unit to anything sufficiently high (I think it was three or five) the water shader came back to life, but...
  7. all shadows disappear if the shadow texture unit isn't one, despite the shadow map still being rendered.
  8. There are a couple of extra issues I've spotted. Sometimes, the shadows just disappear, despite the shadow map still being rendered.
  9. I can't remember the other issue.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Shadows

Post by scrawl »

I'm no longer convinced that that is the water shader, as when I manually ruin the output of the water shader by setting the red channel to 1.0, nothing changes. However, I have the water shader turned on, so it's supposed to be working. If I change the shadow texture unit to anything sufficiently high (I think it was three or five) the water shader came back to life, but...
I recall an odd change in your fork that changed the main camera's cull mask. The mask should include either Mask_Water or Mask_SimpleWater, but not both.
If we're hitting issues with multi-texturing, we might have a problem. Not including the shadow map, the object shader might use up to eight textures. If we tell the shadow map to use slot 8, leaving 0-7 free for the other textures, the shadow map doesn't get rendered because of (what I assume is) some fixed function stuff not liking to use slots above seven. I found that OpenGL 3 mandates that each shader stage have at least sixteen slots available, but I'm not sure about older versions.
That's assuming all features are used simultaneously, which is probably never going to happen. For example, 'detail map' and 'dark map' are essentially the same thing just with a different factor, so it doesn't really make sense to use both at the same time. I think we can safely assume that 8 texture slots should be enough, even with split shadows.

In fixed function mode, some drivers (like NVIDIA's) actually only support 4 texture slots. This shouldn't matter though, because we're not going to use shadows with fixed function materials.

Setting the shadow texture to the 8th slot is probably the easiest solution, though I wonder if it's the optimal one. IIRC, OSG uses a vector internally to keep track of textures, so if we use the 8th slot without using 2-7, we get a bunch of empty slots that are not being used but still generate overhead during the cull phase or even while rendering.
I'm not sure what objects are supposed to not receive shadows other than the sky, but my mention of it was because the way the VDSM technique is set up, it never actually renders things that don't have a nodemask saying they should receive shadows.
Ah, okay. I think it's fine to just say 'everything receives shadows' then so nothing gets excluded, but we can still use diffuse/emissive components to remove the actual effect of shadows if needed.
all shadows disappear if the shadow texture unit isn't one, despite the shadow map still being rendered.
Did you set the uniform for the shadow map texture to the correct index?
At my end, adding the terrain mask to the caster mask doesn't work. Can you post a screenshot of it working, including the debug hud? It's possible that it's something to do with AMD's Vega drivers being immature because OSG's spamming a lot of error reports to the console on my desktop, but not on my laptop which has a GTX 850M.
You'll need either very steep terrain or a low sun altitude to see it, but I think it did work. I'll try to make a screenshot later.

Actually, let me deal with the terrain. I'm going to fix the drawable to not draw its shadows more than once and if that doesn't fix things I'll debug what's going on, then make a PR.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Shadows

Post by AnyOldName3 »

  1. The weird mask change was in your fork, so I didn't touch it in case it was important. The water shader can be induced to work despite this change, but when I try putting it back to the same state as master, the water just disappears altogether. When it's Mask_Water, it has the same behaviour as what's currently on the online version.
  2. The uniform appears to be being set correctly by the parent class, so it should mean the right texture unit is being used.
  3. I think I'm getting a bug that you aren't with the terrain shadows. In my case, not only am I not seeing terrain shadows (no matter how hard I hunt for them) there are big teal regions on the debug hud, which I'm pretty sure is the maximum distance. That means that the shadow camera isn't seeing anything there, but that shouldn't be possible as that's where the terrain is.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Shadows

Post by AnyOldName3 »

One more thing, the shadows make the minimap really dark.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Shadows

Post by scrawl »

Made a small pull request, explanation follows. I'll leave you to the rest.

Yes, you need to go hunting for a suitable location but the terrain shadows definitely work for me, here is me standing under a hill, and then a hill shadowing itself: (I even disabled all the rocks to make sure the shadow is really the terrain)
terrain_1.png
terrain_2.png
there are big teal regions on the debug hud, which I'm pretty sure is the maximum distance. That means that the shadow camera isn't seeing anything there, but that shouldn't be possible as that's where the terrain is.
I see the same. That's because during shadow casting, we cull the front faces, so the flat terrain you are walking on is not going to be visible. Only back-faces (facing away from the light) are going to appear. If we don't cull front faces, everything would be shadowing itself resulting in a flickering mess.
The weird mask change was in your fork, so I didn't touch it in case it was important
Oh :lol: Looks like I left quite a mess. Don't remember why I did this one, probably a part of some frantic attempt to reduce everything to the minimum to see if shadows would work then. I've just restored the mask now and don't see anything broken by this.
One more thing, the shadows make the minimap really dark.
Right, we'll want to disable shadow receiving while rendering the minimap. Binding a white dummy texture that will make the shadow test always fail, should work.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Shadows

Post by AnyOldName3 »

Yeah, I forgot that we were front-face culling instead of back-face culling. I guess I got muddled with the LispSM version, because that was culling things it shouldn't, so I turned it off altogether for that. I'll believe you that I've just underestimated the steepness required to become shadowed.

Changing the mask back, however, definitely does bad things at my end:
I'd say there was a drought, but it's raining, and there are hovering ripples
I'd say there was a drought, but it's raining, and there are hovering ripples
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Shadows

Post by scrawl »

I didn't have that. Is your water shader on or off? Are you using the PC that was spamming errors or the NVIDIA one?
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Shadows

Post by AnyOldName3 »

Water shaders are on, and they work under most circumstances, just not any of the ones where shadows also work. This is on the error-spamming desktop as someone else is using the laptop.
Post Reply