Distance Rendering

General discussion regarding the OpenMW project.
For technical support, please use the Support subforum.
User avatar
psi29a
Posts: 4620
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Distance Rendering

Post by psi29a » 25 Nov 2018, 20:25

No... as talked about elsewhere, the real issue is that you're loading in all the cells which NO sane game engine does and for a very good reason. OpenMW doesn't have LoD because Morrowind doesn't have LoD and we're trying to figure out how best to approach this. In addition to this, distant statics would be introduced as well following the work on LoD. At a certain point, the player will not see statics nor actors, so these can be disabled (no rendering, no physics simulations and etc.)

So by loading all the cells... you're turning on EVERYTHING. Simulating the whole game, all the actors doing their thing, all the physics doing their thing.

There is good news however, in 0.45, there has been work done on the actors side of things where we disable physics simulations for actors outside a player's radius which helps quite a bit. That means you could go back to cranking up your number of loaded cells even though we advise you not to.

Xenuria
Posts: 39
Joined: 26 Feb 2017, 22:35

Re: Distance Rendering

Post by Xenuria » 26 Nov 2018, 00:26

psi29a wrote:
25 Nov 2018, 20:25
No... as talked about elsewhere, the real issue is that you're loading in all the cells which NO sane game engine does and for a very good reason. OpenMW doesn't have LoD because Morrowind doesn't have LoD and we're trying to figure out how best to approach this. In addition to this, distant statics would be introduced as well following the work on LoD. At a certain point, the player will not see statics nor actors, so these can be disabled (no rendering, no physics simulations and etc.)

So by loading all the cells... you're turning on EVERYTHING. Simulating the whole game, all the actors doing their thing, all the physics doing their thing.

There is good news however, in 0.45, there has been work done on the actors side of things where we disable physics simulations for actors outside a player's radius which helps quite a bit. That means you could go back to cranking up your number of loaded cells even though we advise you not to.
Please don't do this, it confuses newer members.
Nobody here is saying that the engine should be able to handle all grids loaded, and you already knew that. :roll:

Getting back on topic...
The best use case would be one wherein the engine can use resources efficiently and either have LOD or 3-6 grids loaded. I don't expect this to happen any time soon because as discussed in numerous other threads the engine isn't fully multi-core yet. For distance rendering you are always going to have a bottleneck at the CPU level because of the engine.

User avatar
AnyOldName3
Posts: 1498
Joined: 26 Nov 2015, 03:25

Re: Distance Rendering

Post by AnyOldName3 » 26 Nov 2018, 02:45

The ability to use lots of threads isn't especially tightly coupled with the ability to cope well with distant things. The limitation is that there's no simple way to generate good LoD data and Morrowind's data doesn't provide any. No matter how well-threaded OpenMW is, it's not a simple problem to resolve, even though better threading might make an eventual solution perform better.
AnyOldName3, Master of Shadows

User avatar
psi29a
Posts: 4620
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Distance Rendering

Post by psi29a » 26 Nov 2018, 09:18

It wasn't a flame, it was a legitimate and technical response to your comment and you should know the difference. Had you searched/checked the forums in the first place you would already know that we've been working on this from another direction with something that should help in the next release: 0.45

The fact that we time after time have to tell people not to do something and yet people keep doing it and claiming it as a problem is absurd. It's not just me, ask another developer: "Do not do it."

Have a read here:
viewtopic.php?f=8&t=5496&p=59261&hilit= ... ics#p59233

Back to the topic at hand:

Throwing more threads at the problem isn't going to resolve the problem, it'll help but it isn't _the_ solution.

That doesn't mean that we have been resting on our laurels, have a look at this:
https://github.com/bulletphysics/bullet3/issues/2004

Xenuria
Posts: 39
Joined: 26 Feb 2017, 22:35

Re: Distance Rendering

Post by Xenuria » 26 Nov 2018, 13:22

psi29a wrote:
26 Nov 2018, 09:18
The fact that we time after time have to tell people not to do something and yet people keep doing it and claiming it as a problem is absurd. It's not just me, ask another developer: "Do not do it."

Have a read here:
viewtopic.php?f=8&t=5496&p=59261&hilit= ... ics#p59233

Back to the topic at hand:

Throwing more threads at the problem isn't going to resolve the problem, it'll help but it isn't _the_ solution.

That doesn't mean that we have been resting on our laurels, have a look at this:
https://github.com/bulletphysics/bullet3/issues/2004
The physics sub system is a totally different mess. Even if replaced with a perfect and flawless physics system the bottleneck would still rest heavily on the core limitations of the graphics engine.
As demonstrated in these 2 videos https://youtu.be/bzszGVBLvrU https://youtu.be/_7SqiArkyR0
and discussed ad nausium in this thread viewtopic.php?f=2&t=4505&sid=a6db922a16 ... c4221b6d17

The issue isn't going to go away once the physics engine is fixed. Pushing the narrative that the end user isn't supposed too or shouldn't use more than 1 distant grid is preposterous. Even if this nutty ideology held any real merit outside of being used as a tool to troll, it still wouldn't explain away the issues with the CPU use at 1 grid loaded and default view distance. The notion that making OpenMW fully 64-bit and multi-core wont lead to a notable performance increase is also a myth. VMware experimentation has totally debunked that and shown that the performance increase in frames per second is proportional to the amount of fps while using a single core times the amount of additional cores. Is there a DR effect and some static in the math? Sure there is, it's a proof of concept but telling new people that multi-core is a waste of time is at best ignorant and at worst outright trolling.

User avatar
lysol
Posts: 1307
Joined: 26 Mar 2013, 01:48
Location: Sweden

Re: Distance Rendering

Post by lysol » 26 Nov 2018, 14:00

I'm... not sure you guys understand what the other one is trying to say.
Normal mapped texture replacers, exclusive for OpenMW:
My Nexus page

User avatar
AnyOldName3
Posts: 1498
Joined: 26 Nov 2015, 03:25

Re: Distance Rendering

Post by AnyOldName3 » 26 Nov 2018, 14:15

Do you even know what you're saying when you say 'make OpenMW fully 64-bit'? I don't think we do. AMD64 brought three main benefits over IA32. The first is that it can address much more memory, and there's no part of OpenMW that isn't using 64-bit pointers, so there's no way we're hitting an issue where we can't address all the memory. The second is that there were more instructions which worked directly on 64-bit data types. Unless you're using inline assembly, it's the compiler's job to actually use 64-bit instructions for 64-bit data types, and that's what it's doing. This doesn't make using 64-bit data types any faster than 32-bit ones (64-bit ones are always going to be the same speed or slower) so it's not like switching every use of float to double and int to long will make anything any faster. The final main benefit is that 64-bit has more registers so register thrashing is less of a concern. Again, unless you're using inline assembly, this is the compiler's job, and it's doing it.

The reason most people are focussing on the physics being slow is that for most people who are CPU-bound, the physics system is the main bottleneck. There are parts of the renderer that also take a long time, but there's not much low-hanging fruit there - apart from animation, most of the time is spent in OSG code, and as that's a widely-used library, lots of effort has already gone into making it as fast as possible. Lots of the work it's doing wouldn't get faster if it was split across multiple threads, either - adding all the extra synchronisation needed to make accessing shared data structures possible would be very likely to counteract any speedup.
AnyOldName3, Master of Shadows

i30817
Posts: 58
Joined: 07 Nov 2018, 05:56

Re: Distance Rendering

Post by i30817 » 26 Nov 2018, 16:19

You can get a good performance increase on the lower end GPUs by using the 'Project Atlas' mod. The texture atlas reduce the number of drawcalls somewhat drastically (if you don't ruin things by adding 10 other mesh and texture mods that use the original morrowind scheme later).

I'm expecting shadows to ruin things again later, but that is one setting that is permanent disabled in games and morrowind especially.

User avatar
AnyOldName3
Posts: 1498
Joined: 26 Nov 2015, 03:25

Re: Distance Rendering

Post by AnyOldName3 » 26 Nov 2018, 17:22

Is that mod far enough along that there's any actual performance data yet? I thought work had only just started.
AnyOldName3, Master of Shadows

Xenuria
Posts: 39
Joined: 26 Feb 2017, 22:35

Re: Distance Rendering

Post by Xenuria » 26 Nov 2018, 17:32

AnyOldName3 wrote:
26 Nov 2018, 14:15
Do you even know what you're saying when you say 'make OpenMW fully 64-bit'? I don't think we do. AMD64 brought three main benefits over IA32. The first is that it can address much more memory, and there's no part of OpenMW that isn't using 64-bit pointers, so there's no way we're hitting an issue where we can't address all the memory. The second is that there were more instructions which worked directly on 64-bit data types. Unless you're using inline assembly, it's the compiler's job to actually use 64-bit instructions for 64-bit data types, and that's what it's doing. This doesn't make using 64-bit data types any faster than 32-bit ones (64-bit ones are always going to be the same speed or slower) so it's not like switching every use of float to double and int to long will make anything any faster. The final main benefit is that 64-bit has more registers so register thrashing is less of a concern. Again, unless you're using inline assembly, this is the compiler's job, and it's doing it.

The reason most people are focussing on the physics being slow is that for most people who are CPU-bound, the physics system is the main bottleneck. There are parts of the renderer that also take a long time, but there's not much low-hanging fruit there - apart from animation, most of the time is spent in OSG code, and as that's a widely-used library, lots of effort has already gone into making it as fast as possible. Lots of the work it's doing wouldn't get faster if it was split across multiple threads, either - adding all the extra synchronisation needed to make accessing shared data structures possible would be very likely to counteract any speedup.
I understand what you are saying. I simply disagree with the assertion that OpenMw is such a niche and specific edge case that it won't benefit from using more than 1 core. I will try to explain better.
Consider the following...

You have 3 VMware boxes all 3 of them have the same hardware and all 3 of them are using a vanilla, clean version of openMW. We use Vmware to test things in a vacuum to find where the problems are without static or other variables influencing.

Vmware box 1 is using a CPU that is under-clocked so it's running at 2.5 GHz
Vmware box 2 is using the same CPU at base clock 3.5 GHz
Vmware box 3 is using the same CPU but at an overclock of 3.9 GHz
In all 3 of these boxes OpenMw experiences a bottleneck in 2 places. The Physics sub system and the graphics system; specifically the GPU isn't getting instructions fast enough because the game isn't properly utilizing all cores and threads. Frame rates are HIGHER when the base clock of the CORE being used by OpenMW is higher. Frame Rates are NOT changed when the base clock of CORES NOT being used by OpenMW are increased or decreased. You can individually underclock cores and then run OpenMw in this environment observing so change in performance so long as the physical core being used by the game isn't the one you underclocked.

If everything you said is 100% correct than this scenario I describe should be impossible. The fact that the scenario I describe is not only possible but reproducible with ease tells me that you either don't know what you are talking about or have misunderstood what is meant by Multi-core and multi-thread. This problem isn't going away by arguing semantics or vocabulary. We shouldn't be arguing with each other we should be working the problem.

Edit: If making more videos and demoing this is what it takes to finally put the issue to bed I will invest the time to do it. I will note however that I really shouldn't have too. The people up to their knees in code should be able to nip this in the but and clarify but unfortunately they seem to prefer using vague examples and smear best use cases. Instead of saying "yup our game is acting like second life" they say "welll yaa know... in the vanilla morrowind there wasn't multi-core support anyway." or "Well but in a true morrowind experience there would be no distance land or more than 6666 view distance. "

Post Reply