Improving distant terrain distance

Everything about development and the OpenMW source code.
Post Reply
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Improving distant terrain distance

Post by Chris »

This topic is to discuss/plan relatively simple ways distant terrain can be improved, both from a user perspective, and a design perspective. If some consensus can be reached, I could attempt to do something (or if not, we'll have something down for someone else). Note also that this is relatively simple things -- not distant statics, LODs, or such things -- what I'm discussing should be doable now with (hopefully) little hassle.

To start with, an issue that arises is that simply enabling distant terrain does nothing from the user's perspective. This is because the far clipping plane remains tied to the viewing distance, which is set for being in the default 3x3 cell configuration. The user has to also increase the viewing distance to see the distant terrain, but this causes a few problems. First, most users will have no idea what appropriate values would be, or what effect it will actually have on certain aspects of rendering (e.g. depth buffer resolution). Secondly, increasing the viewing distance has an effect on the atmospheric fog (all but removes it, because the viewing distance is both the far clip plane and the fog end point, with various weathers altering fog relative to that). Third, because the viewing distance is vastly increased, it makes the in-game viewing distance option a liability; touching it will undo the extended view distance and there's no way to set it back other than exiting and re-editing the config. Fourth, the viewing distance that gets set (for the far clipping plane) is ultimately arbitrary because there's no ideal (or even "good enough") value. Given that the purpose of distant terrain is to include everything the eye can see, the far clip plane should be set as far as realistically possible and not need any adjustment.

The first issue can be easily fixed by automatically using a more distant far clip plane when distant terrain is enabled. The viewing distance setting can still control the fog distance, but having the fog end separate from the far clip plane when distant terrain is enabled would then solve the second and third issues (fixing the fog itself to look good with an increased view can then be done with shaders or something). The remaining points (the effects of the far clip distance on depth resolution, and an arbitrary far clip distance) are a little trickier to deal with, but I think are more than doable.

For the far clip distance, it can default to something that works okay for a typical Morrowind install (with Tamriel Rebuilt in mind, if possible). Doesn't have to necessarily be able to show everything if it would cause issues with Z-fighting, but a good chunk would be nice. Eventually, we can try utilizing a reversed depth buffer to be able to set a far clip distance so large, no mod would realistically hit it (practically infinite, even if technically not). Note that while that post mentions that trick only works in Direct3D, and OpenGL needs to use a custom projection matrix and the GL_NV_depth_buffer_float extension to make it work, there is GL_ARB_clip_control which allows it to work like in Direct3D (without the custom projection matrix or NV extension).


To sum up, when distant terrain is enabled:
Separate the far clip plane from the fog end distance, leaving the fog end distance controlled by the viewing distance setting.
Automatically use a farther far clip plane not tied to the viewing distance setting.
Set the far clip plane to something that's okay for most games, possibly with a new setting to adjust it (separate from the already-existing viewing distance setting). And consider the option to use a reversed depth buffer for extending the far clip plane to no man's land, where no further adjustments are needed.

I could work on separating the fog distance and far clip plane and setting an alternative far clip plane distance for distant terrain. However, I'm not quite as well-versed in OSG as I'd like, and it's been a while since I poked around in the rendering code, so I may stumble when it comes to that.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Improving distant terrain distance

Post by scrawl »

Separate the far clip plane from the fog end distance, leaving the fog end distance controlled by the viewing distance setting.
Makes sense to me (assuming it's opt-in, so the default game without distant terrain behaves as-is, and there's still a way to manually set the far clip plane in case I only want to see a few more cells and not everything)
Automatically use a farther far clip plane not tied to the viewing distance setting.
Set the far clip plane to something that's okay for most games, possibly with a new setting to adjust it (separate from the already-existing viewing distance setting). And consider the option to use a reversed depth buffer for extending the far clip plane to no man's land, where no further adjustments are needed.
Might also be worth trying OSG's support for auto-calculated near and/or far clip planes (See Camera::setComputeNearFarMode). This basically calculates the clipping planes every frame according to the objects in the camera frustum, so it's always set perfectly for however far you could possibly see, and the near plane can be extended if there isn't anything in the way (e.g. in first-person mode with no weapons out) in order to increase precision.
Last edited by scrawl on 17 Sep 2017, 14:44, edited 1 time in total.
User avatar
raevol
Posts: 3093
Joined: 07 Aug 2011, 01:12
Location: Caldera

Re: Improving distant terrain distance

Post by raevol »

I apologize for not being technical enough to completely understand this post, but I do agree that making distant terrain more user-friendly would be very beneficial for us. You didn't mention adding gui options for your proposed feature changes, would that be something that you would like to see? If I remember right one hurdle with the gui is localization.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Improving distant terrain distance

Post by Chris »

scrawl wrote: 17 Sep 2017, 14:42 Makes sense to me (assuming it's opt-in, so the default game without distant terrain behaves as-is, and there's still a way to manually set the far clip plane in case I only want to see a few more cells and not everything)
The default game without distant terrain would continue to behave as-is. There probably could be another setting to modify the far clip plane even without distant terrain on, something like 'far clip plane offset' which specifies an offset from the viewing distance (positive only, to push it out farther than the fog end distance). Either that or automatically increase the far clip plane when the cell radius is increased.
Might also be worth trying OSG's support for auto-calculated near and/or far clip planes (See Camera::setComputeNearFarMode). This basically calculates the clipping planes every frame according to the objects in the camera frustum, so it's always set perfectly for however far you could possibly see, and the near plane can be extended if there isn't anything in the way (e.g. in first-person mode with no weapons out) in order to increase precision.
I remember trying that in a personal project that had with a large terrain size, and wasn't very impressed. The near clip plane went crazy, sometimes being clearly visible and other times not, and the far clip distance wasn't very impressive. I got better results (stable clipping and a farther view distance) using set values. It could be possible I just wasn't using it right, but I wouldn't expect much.
raevol wrote: 17 Sep 2017, 14:44 You didn't mention adding gui options for your proposed feature changes, would that be something that you would like to see? If I remember right one hurdle with the gui is localization.
In-game options likely need to wait for localization options. Although there are non-vanilla video options already there, so I don't know. Perhaps there are GMST strings that could be used to make suitable labels. At the very least, there's been discussions about adding more or all of the settings from settings.cfg (where this would be) to the launcher, so it would be available there for the less config-file-savvy users.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Improving distant terrain distance

Post by scrawl »

The default game without distant terrain would continue to behave as-is. There probably could be another setting to modify the far clip plane even without distant terrain on, something like 'far clip plane offset' which specifies an offset from the viewing distance (positive only, to push it out farther than the fog end distance). Either that or automatically increase the far clip plane when the cell radius is increased.
An 'auto' mode sounds good to me. This mode would use the formula we recommend here, except if distant terrain is enabled it'll be based on the size of the world. Or have 3 different modes to choose from (manual, cell grid size, world size).
I remember trying that in a personal project that had with a large terrain size, and wasn't very impressed. The near clip plane went crazy, sometimes being clearly visible and other times not, and the far clip distance wasn't very impressive. I got better results (stable clipping and a farther view distance) using set values. It could be possible I just wasn't using it right, but I wouldn't expect much.
I think I saw some weird results with the near plane too. Possibly an osg bug needs fixing. Anyway, you can also set it to calculate only the far plane and not the near plane, try if that works better. We actually use the autocalculated near/far for the character preview render and there aren't any issues there, so that's strange.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Improving distant terrain distance

Post by Chris »

scrawl wrote: 17 Sep 2017, 16:11 An 'auto' mode sounds good to me. This mode would use the formula we recommend here, except if distant terrain is enabled it'll be based on the size of the world. Or have 3 different modes to choose from (manual, cell grid size, world size).
With distant terrain off, I was thinking of leaving the viewing distance setting as-is, but add an extra 8192 to the far clip plane distance for every cell radius added after the first (e.g. viewing distance + 8192*(exterior cell load distance - 1)). That way the viewing distance option in the game settings still works and can be used to adjust how much of the farthest cell can be seen, without ever fully cutting out loaded cells. I guess the other option is to ignore the viewing distance setting if exterior cell load distance is greater than 1, forcing the use of (8192*exterior cell load distance - 1024)*0.93.
I think I saw some weird results with the near plane too. Possibly an osg bug needs fixing. Anyway, you can also set it to calculate only the far plane and not the near plane, try if that works better. We actually use the autocalculated near/far for the character preview render and there aren't any issues there, so that's strange.
I think a general method to calculate an appropriate near- and far-clip planes is just a very difficult thing to do. It depends on testing everything that's in view to find what doesn't clip with the left, right, top, or bottom planes, and it depends on the depth resolution you need to avoid Z-fighting with your meshes (which isn't something you can easily detect for a given distance). And you have to allow for the fact that clipping will unavoidably happen on occasion, but there's no 'priority' system for OSG to know what things are more okay to clip and what should stay unclipped as much as otherwise possible (and to say what's okay to clip with the far plane and mustn't clip with the near plane).

I get the feeling that the auto-calculated near and far planes are probably most appropriate for rendering things that are entirely in-frame, like the character preview, where the near and far planes can be set to the minimum and maximum distances of the bounding shapes being rendered. Once things start going behind the camera or exceed the available depth resolution, where clipping must happen to avoid Z-fighting, all bets are off.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Improving distant terrain distance

Post by Chris »

I've managed to get the far plane separate from the fog distance, and move the far plane farther out when distant terrain is enabled. However, the results are not good. With as heavy as the fog is, and with how it affects the sky, you can't even make out the outline of things exposed by the extended far plane. A few places you can kind of make things out, but ultimately you still need to push out the fog as well to see any worthwhile difference.

I remember asking at one point how MGE XE handled fog with its distant terrain, and I believe the answer was that it still worked like vanilla. I find that hard to believe though, given how useless it is to simply change the far plane and leave the fog alone. Either there's something more going on, or OpenMW doesn't calculate the fog settings quite like vanilla.
magamo
Posts: 109
Joined: 28 May 2016, 17:28

Re: Improving distant terrain distance

Post by magamo »

I'm fairly certain that MGE XE generates fog very differently than Vanilla when Distant Land is used. It has several different fog modes (pixel/vertex, range/depth, exponential/linear I believe)
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Improving distant terrain distance

Post by Chris »

magamo wrote: 19 Sep 2017, 18:26 I'm fairly certain that MGE XE generates fog very differently than Vanilla when Distant Land is used. It has several different fog modes (pixel/vertex, range/depth, exponential/linear I believe)
Those fog modes just dictate how fog is rendered. Exponential/linear is the only thing there that would affect the actual apparent depth, but as far as I know, you can't just use an exponential fog mode with what Morrowind's game data provides.

Looking into a bit more, it does seem that MGE XE overrides the default fog calculations. This seems to be how MGE sets up the fog prior to rendering the distant terrain (which then persists for the rest of the scene):
https://github.com/Hrnchamd/MGE-XE/blob ... d.cpp#L412
And the various Configuration.DL.* values (their defaults) can be seen here:
https://github.com/Hrnchamd/MGE-XE/blob ... ata.h#L156
The *FogStart and *FogEnd values are in cells.

I should be able to replicate the fog calculations MGE XE does, and it would effectively disable the viewing distance setting when distant terrain is enabled. However, I'm unsure what to do about those default MGE XE settings. I could add them to settings.cfg, but I don't think they're appropriate there. I'd imagine we'd want to expose them properly to omwaddon files when de-hardcoding the weathers, so I'm not sure what to do in the meantime.

It's interesting to note that Morrowind seems to have a max view distance of 7168 units, while we max at 6666 units. Also, MGE XE's "exponential" fog doesn't seem to actually be exponential, but uses linear and adjusts the fog start/end using exponential curves.
Post Reply