Unless the underlying data structures are amenable to it, in general, making something use more cores doesn't make it faster, it actually makes it slower. Lots of things require the ability to access other bits of data, and if there might be another thread making changes to that data at the same time, you need some kind of lock or mutex so that each thread can read the data either before it gets changed or after it gets changed, but not while it was halfway through being changed and doesn't make any sense. Doing so requires messages to be passed between cores by some mechanism, and this is really slow in comparison with most other tasks a CPU does. If you split some work across four cores, you might expect it to take a quarter as long to finish, but when the performance hit of inter-thread communication is taken into account, it might not be any more.
This is analogous to how if you have a group of twenty people and ask them to write one sentence of a letter each, only communicating to each other by writing messages onto paper aeroplanes and throwing them to each other's desks, it's going to take a lot longer to have a sensible letter than if you'd just asked one of them to write the whole thing.
In the case of the draw thread, splitting it onto multiple threads would be more like asking them to also type their sentence out on a typewriter, but you only give them the one typewriter and don't let anyone leave their desk, so the typewriter would need to be thrown from desk to desk, too. OpenGL and DirectX 11 and earlier are like the single typewriter - they can only be accessed from one thread at once.
Distance Rendering
Re: Distance Rendering
As AnyOldName3 alluded to, adding more cores doesn't necessarily improve CPU bottlenecks. You need to find work that can be parallelized to split across multiple cores, otherwise the work is serial and splitting that work across cores will add serialization overhead. It's the difference between:Xenuria wrote: ↑26 Nov 2018, 17:32 I understand what you are saying. I simply disagree with the assertion that OpenMw is such a niche and specific edge case that it won't benefit from using more than 1 core.
[...]
In all 3 of these boxes OpenMw experiences a bottleneck in 2 places. The Physics sub system and the graphics system; specifically the GPU isn't getting instructions fast enough because the game isn't properly utilizing all cores and threads.
Code: Select all
core1: S-work-work-work-work-work-work-E S-work-work-work-work-work-work-E
// vs
core1: S-work-work-work-v ... waiting ... S-work-work-work-v ... waiting
| |
core2: >-work-work-work-E ... waiting ... >-work-work-work-E
Code: Select all
subsys1: S-work1-work1-work1--v S-work1-work1-work1-v S-work1-work1-work1-v
| | |
subsys2: >-work2-work2-work2-E >-work2-work2-work2-E >-work2-work2-work2-E
- psi29a
- Posts: 5361
- Joined: 29 Sep 2011, 10:13
- Location: Belgium
- Gitlab profile: https://gitlab.com/psi29a/
- Contact:
Re: Distance Rendering
I personally don't think we need to waste our time here. From a PM:
https://www.digitalcitizen.life/3-ways- ... -or-32-bit
You pick the right one for your arch because a 64-bit compiled executable will not run on a 32-bit machine.
Please don't confuse this with PAE (Physical Address Extension) which allows 32-bit machines, that have this flag and have support in software for it, to address up to 4GiB and sometimes beyond. Windows XP had issues with this and they ended up disabling support for beyond 4GiB. Stated here:
https://en.wikipedia.org/wiki/Physical_ ... _Extension
Your system has something called VT-x (vmx), it allows for a hypervisor to run in either 32 or 64-bit mode regardless of underlying OS. You can't run a 64-bit VM session on a 32-bit processor. However, you can run a 64-bit VM session if you have a 64-bit processor but have installed a 32-bit host OS and your processor supports the right extensions. You can validate that yourself here:
https://my.vmware.com/web/vmware/detail ... hkYmRAZQ==
Yes, we ship two versions of OpenMW: 64-bit and 32-bit, yes MSVC/GCC/Clang know how to target these architectures.
Yes, OpenMW is multi-threaded, meaning that sub-systems are spun out into a thread and sync/join to main thread. OSG, our rendering library uses OpenThreads. We make use of OpenThreads as well for the music and now with navmesh generation, that is also in an apart thread. Cell-preloading is in a thread as well to allow seamless cell transitions.
OpenMW is open-source, you can read it yourself. There are no smoke and mirrors here.
And with your last comment, you've just earned your first troll token. Thanks for wasting everyone's time while we could be doing something productive. You're are wrong on so many levels and when you're the only one that holds an intenible position then the only thing anyone can assume is that you are a troll. We've tried to be nice, we've tried to help, but you persist. Now you start with the personal attacks.
If you think we're wrong, great, prove it. The code is available, have at it.
For windows, we release two binaries. One compiled for 32-bit and the other 64-bit. You can check these yourself.Xenuria wrote: Sent: 26 Nov 2018, 18:49
by Xenuria
I hold no malice either. I am simply frustrated with the dogma perpetuated by many people who should know better.
Telling new people that OpenMw is 64-bit and Multi-core isn't true. It's just demonstrably incorrect and counter productive.
Just because I can kernel patch a 32-bit copy of windows XP, causing it to use more than 4GB of RAM dosn't mean that I am using a 64-bit copy of windows XP.
Just because I can run a 64-bit Vmware Box on a 32-bit copy of windows 7 and use 30GB of RAM dosn't mean that my copy of Windows 7 is 64-bit.
Saying that OpenMw is 63-bit and Multi-core is reductive and one of the most counterproductive things to come out of these forums in over a year. You guys are a team of dedicated developers not a bunch of used care salesmen.
https://www.digitalcitizen.life/3-ways- ... -or-32-bit
You pick the right one for your arch because a 64-bit compiled executable will not run on a 32-bit machine.
Please don't confuse this with PAE (Physical Address Extension) which allows 32-bit machines, that have this flag and have support in software for it, to address up to 4GiB and sometimes beyond. Windows XP had issues with this and they ended up disabling support for beyond 4GiB. Stated here:
https://en.wikipedia.org/wiki/Physical_ ... _Extension
Even that being the case, OpenMW does not officially support XP. I doubt our releases run on XP, feel free to test.The original releases of Windows XP and Windows XP SP1 used PAE mode to allow RAM to extend beyond the 4 GB address limit. However, it led to compatibility problems with 3rd party drivers which led Microsoft to remove this capability in Windows XP Service Pack 2.
Your system has something called VT-x (vmx), it allows for a hypervisor to run in either 32 or 64-bit mode regardless of underlying OS. You can't run a 64-bit VM session on a 32-bit processor. However, you can run a 64-bit VM session if you have a 64-bit processor but have installed a 32-bit host OS and your processor supports the right extensions. You can validate that yourself here:
https://my.vmware.com/web/vmware/detail ... hkYmRAZQ==
Yes, we ship two versions of OpenMW: 64-bit and 32-bit, yes MSVC/GCC/Clang know how to target these architectures.
Yes, OpenMW is multi-threaded, meaning that sub-systems are spun out into a thread and sync/join to main thread. OSG, our rendering library uses OpenThreads. We make use of OpenThreads as well for the music and now with navmesh generation, that is also in an apart thread. Cell-preloading is in a thread as well to allow seamless cell transitions.
OpenMW is open-source, you can read it yourself. There are no smoke and mirrors here.
And with your last comment, you've just earned your first troll token. Thanks for wasting everyone's time while we could be doing something productive. You're are wrong on so many levels and when you're the only one that holds an intenible position then the only thing anyone can assume is that you are a troll. We've tried to be nice, we've tried to help, but you persist. Now you start with the personal attacks.
If you think we're wrong, great, prove it. The code is available, have at it.
Re: Distance Rendering
Being overtly hostile and openly flaming users for asking questions or offering feedback is not the way to success. We are on the same team. The past few days reading this forum have caused me no shortage of upset and distress. I have a vested interest in the success of this project but the community here has become so pointlessly toxic. If you want to insult me and disparage me further I won't try to stop you. I have been accused of trolling by core members of this community for trying to put out fires and steer threads back on track. Just because you put in hours, weeks or years of work doesn't give you the right to demean people especially when your own development team contradicts the claims you make.
I stand by my statement.
OpenMW is NOT Multi-core.
OpenMW is 64 bit in the same way second life is 64-bit and that isn't a good thing.
The Physics subsystem is NOT multi-threaded.
I will continue to make demo and benchmark videos demonstrating the performance of various builds but I will be taking a break from posting in this community.
Goodbye
I stand by my statement.
OpenMW is NOT Multi-core.
OpenMW is 64 bit in the same way second life is 64-bit and that isn't a good thing.
The Physics subsystem is NOT multi-threaded.
I will continue to make demo and benchmark videos demonstrating the performance of various builds but I will be taking a break from posting in this community.
Goodbye
- AnyOldName3
- Posts: 2676
- Joined: 26 Nov 2015, 03:25
Re: Distance Rendering
Everyone agrees about the physics problem, so I'm not totally sure why that was brought up, but if you're not intentionally being a troll, you really need to go and look up what the other two things mean. The fact of the matter is, either accidentally or deliberately, you're saying wrong things.OpenMW is NOT Multi-core.
OpenMW is 64 bit in the same way second life is 64-bit and that isn't a good thing.
The Physics subsystem is NOT multi-threaded.
From what I can tell, the only controversy surrounding the way in which Second Life is 64-bit is that the 64-bit builds still use a 32-bit build of Havok (presumably because the 32-bit Havok SDK is free of charge and the 64-bit one costs quite a bit of money). As there's a 64-bit binary interoperating with a 32-bit one, there's extra work required to pass data between them. That's definitely not a problem with OpenMW - all our dependencies are compiled however is needed, so 32-bit builds are entirely 32-bit and 64-bit builds are entirely 64-bit.
OpenMW spawns significantly more threads that work in parallel than Skyrim does, and on the box of my copy of Skyrim it says "Enhanced for: multi-core", so you need to lawyer up and start talking to Bethesda about fraud before you say OpenMW isn't multi-core. In fact, you can test the impact of using fewer threads than normal yourself - if you set the environment variable OSG_THREADING to SingleThreaded before running OpenMW, you'll see a big performance hit. There might be more scope to split more things into separate threads, but you've said you're limited by the draw thread, and no OpenGL program can split that into different threads.
Re: Distance Rendering
It seems that at some point, we'll have to add "Why isn't openmw using all my CPU?", "What about using X instead of Y, because Z game uses it and the game is nice-looking", and "OPENMW ETA WEN?!" to the Faq.
- psi29a
- Posts: 5361
- Joined: 29 Sep 2011, 10:13
- Location: Belgium
- Gitlab profile: https://gitlab.com/psi29a/
- Contact:
Re: Distance Rendering
And you have absolutely no idea what you're talking about and that is very sad and I pity you.Xenuria wrote: ↑26 Nov 2018, 22:34 Being overtly hostile and openly flaming users for asking questions or offering feedback is not the way to success. We are on the same team. The past few days reading this forum have caused me no shortage of upset and distress. I have a vested interest in the success of this project but the community here has become so pointlessly toxic. If you want to insult me and disparage me further I won't try to stop you. I have been accused of trolling by core members of this community for trying to put out fires and steer threads back on track. Just because you put in hours, weeks or years of work doesn't give you the right to demean people especially when your own development team contradicts the claims you make.
I stand by my statement.
OpenMW is NOT Multi-core.
OpenMW is 64 bit in the same way second life is 64-bit and that isn't a good thing.
The Physics subsystem is NOT multi-threaded.
Actually, we have every right to ban you for your aggressive and abrasive behaviour. We haven't flamed you, but you appear to take it that way because everyone who works on OpenMW is trying to educate you while you blindly disagree. We've tried our best to be reasonable but we are also tired of your shit. Please leave.
Re: Distance Rendering
The last is certainly planned and the other two have been added to my list. Also for future reference to all, if you have ideas for FAQ items, feel free to post in either the Documentation subforum or on the updating the FAQ topic. I don’t read through every thread in every subforum so they are likely to get lost!
For what it’s worth, I also have an entry on Distant Land vs Distant Statics vs LOD in the works as well.