Collision detection system discussion [Original title: Very low FPS]

Feedback on past, current, and future development.
User avatar
Husaco
Posts: 25
Joined: 07 Aug 2011, 11:54
Location: Australia

Re: Very low FPS

Post by Husaco » 29 Apr 2019, 14:28

Stomy wrote:
28 Apr 2019, 11:06
I tend to get distracted by other projects but I am still working on the dedicated collision/physics system, and I fully intend on giving the same treatment to the rendering system eventually. I'm very disappointed with OpenMW's performance and want to improve it as best I can but it will take a whole lot of time to unravel.
Neat. Good to hear you're still working on it. A custom rendering engine is a far out prospect, but not an unwelcome one.

User avatar
lysol
Posts: 1325
Joined: 26 Mar 2013, 01:48
Location: Sweden

Re: Very low FPS

Post by lysol » 29 Apr 2019, 15:58

Personally, if chaning rendering engine is something you want to do later Stomy, maybe vulkan scene graph would be the go-to engine? Ogre3D wasn't exactly a performance horse back in the <0.37.0 era, and outside of that, do we really have much other choice in GPL-friendly engines?
Normal mapped texture replacers, exclusive for OpenMW:
My Nexus page

User avatar
AnyOldName3
Posts: 1559
Joined: 26 Nov 2015, 03:25

Re: Very low FPS

Post by AnyOldName3 » 29 Apr 2019, 16:38

I think he's suggesting a from-scratch rendering engine rather than an existing library, which I'm not certain of the wisdom of. We'll definitely end up with something mostly like OSG in the end, but with a much bigger maintenance burden.
AnyOldName3, Master of Shadows

User avatar
Stomy
Posts: 47
Joined: 11 Dec 2018, 02:55
Location: Fhloston Paradise!
Contact:

Re: Very low FPS

Post by Stomy » 29 Apr 2019, 17:20

I do mean from scratch. I wouldn't be suggesting it if I didn't think OSG was a serious bottleneck itself. Despite its versatility OSG is not designed for speed and it shows in cull and draw times. It's possible to optimize within OSG's architecture just as its possible to improve Bullet's performance for collision detection. But there comes a point where you're fighting with the library to do what the project needs rather than simply working naturally.

At this stage our use of OSG and Bullet have laid out some clear patterns of what OpenMW needs and how it uses which features, so its the best time to start slicing out the considerable amount of glue code and unused features. As for maintenance burden, consider that our dependencies' problems are our problems too, and in adding any library you're requiring that new contributers learn the middleware in detail in addition to the engine code, leading to more burdens rather than less. The only reason another library would mean less complexity would be if it was highly likely that new contributors were already familiar with it, which is not really the case with OSG, what with this being the most significant project known to use it for a video game.
Never attribute to bad code, that which can be adequately explained with clever hacks

Chris
Posts: 1559
Joined: 04 Sep 2011, 08:33

Re: Very low FPS

Post by Chris » 29 Apr 2019, 19:25

Stomy wrote:
29 Apr 2019, 17:20
I do mean from scratch. I wouldn't be suggesting it if I didn't think OSG was a serious bottleneck itself. Despite its versatility OSG is not designed for speed and it shows in cull and draw times. It's possible to optimize within OSG's architecture just as its possible to improve Bullet's performance for collision detection. But there comes a point where you're fighting with the library to do what the project needs rather than simply working naturally.
Though it's also important to ask if we're fighting with the library because it isn't designed to do what we need, or if because we don't understand how to most effectively use it. The former case is the issue we had with OGRE. OGRE wanted us to work at a higher level than we were able since we were trying to replicate behavior it didn't provide. OSG is a lot less stringent there, and actually maps better to what we're trying to do.

Bullet's problem is that its documentation is horrible and there's no real help with learning how to use it correctly. Plus the fact that it was the first time I've ever done any serious physics-based work, and had to learn everything as I went along in what's not my field of expertise (and much of the work done since then has been built on top of it). I wouldn't at all doubt I made tons of mistakes with it and the problem with Bullet's performance is a fundamental misunderstanding of how to properly use it.
The only reason another library would mean less complexity would be if it was highly likely that new contributors were already familiar with it, which is not really the case with OSG, what with this being the most significant project known to use it for a video game.
Another reason is if there's a lot of cookie-cutter/boilerplate work that the library takes care of. As it is, OSG is handling things like culling and depth sorting, state tracking and grouping (ordering draws to avoid as much state change as reasonably possible, which is important for performance-oriented work), and giving us a threaded renderer (which can run in parallel to the main engine update, also important for performance). Certainly OpenMW could do it itself, but would it really be less complex for users to have to learn a completely custom renderer and how its used by the engine, vs learning how OpenMW uses an existing renderer?

OSG isn't perfect, but neither would a custom renderer be. It becomes a question of if we could make a renderer that's so much better than OSG that it warrants the additional maintenance.

User avatar
psi29a
Posts: 4633
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Very low FPS

Post by psi29a » 29 Apr 2019, 20:26

In addition to this, keep in mind the bus-factor. A rock-star comes in, says the current library is shit and they can do something better. It happens, greatest thing since sliced bread, then rock-star goes away. No one is left to maintain it. At least with OSG, it is a maintained 3rd-party library with many eyes on it. If we have to maintain it ourselves, then we're it...

It's more or less the same argument between writing our own custom animation format on top of osgt _or_ supporting DAE/Collada animation. With the former, we have to not only support our own animation format, we have to provide importer and exporters. I honestly think that is outside the scope of the project and we benefit from having DAE/Collada support provide for us by OSG and we map it over top our animation system.

I'm not saying a flat NO, but I'm not convinced a custom renderer (with all the things OSG does for us reimplemented) is the best way to go unless you want to do it for fun and surprise us all. :)

User avatar
AnyOldName3
Posts: 1559
Joined: 26 Nov 2015, 03:25

Re: Very low FPS

Post by AnyOldName3 » 29 Apr 2019, 23:44

I'm of the opinion that lots of the things that make OSG awkward are simply because it's been around a long time. VSG isn't going to have everything done in a way that mimics OSG as, even though it'll make porting existing apps harder, OSG has loads of stuff that would be gone if backwards compatibility wasn't a concern, and other stuff that is a reasonably good idea, but not necessarily the best idea, yet is always around even if it's not being used.
AnyOldName3, Master of Shadows

User avatar
Stomy
Posts: 47
Joined: 11 Dec 2018, 02:55
Location: Fhloston Paradise!
Contact:

Re: Very low FPS

Post by Stomy » 30 Apr 2019, 04:15

I genuinely believe that in-house code being considered more complicated and error-prone is a misconception from programmers teaching each other not to trust their own abilities or that of their fellows.

We already have 360k lines of code in the codebase, Bullet is another 200k, and then OSG is 480k. In order to reasonably contribute to either physics or rendering a contributor has to understand the underlying concepts and navigate through about 500k lines of code combined, Nearly a million if they want to do both, and yet we have mulitple developers capable of doing so just fine. I posit that hand-written solutions will bring the total line-count drastically down and make contribution from others easier, not harder. The only reason why I would avoid hand-writing would be in a commercial setting where you have a deadline, but this is a hobby project and has no hard time constraint, so we can afford to spend as much time simplifying and refactoring as we want.

Regardless this is still a long way off, the dedicated collision system itself isn't anywhere near complete. I would consider that a showcase not of my abilities but of what hand-written code can do in the first place, as I don't consider anything I write to be particularly novel or complex. There's not a lot more I can say to defend this because what I'm proposing is typically considered insane by modern programming wisdom so from here I can only let the code speak for itself.
Never attribute to bad code, that which can be adequately explained with clever hacks

User avatar
raevol
Posts: 3015
Joined: 07 Aug 2011, 01:12
Location: Caldera

Re: Very low FPS

Post by raevol » 30 Apr 2019, 05:49

psi29a wrote:
29 Apr 2019, 20:26
In addition to this, keep in mind the bus-factor.
I'm not throwing my hat on either side of this debate (I'm not qualified) but I would like to point out that elegantly designed code with well-written documentation should eliminate the bus factor. And that goes for hand-written systems or using libraries.

Chris
Posts: 1559
Joined: 04 Sep 2011, 08:33

Re: Very low FPS

Post by Chris » 30 Apr 2019, 08:05

Stomy wrote:
30 Apr 2019, 04:15
I genuinely believe that in-house code being considered more complicated and error-prone is a misconception from programmers teaching each other not to trust their own abilities or that of their fellows.
It really depends on a person's knowledge and skillset. If a developer is knowledgeable about OpenGL and regularly uses it, then yeah they'll probably be able to produce a good OpenGL renderer because they already have. If someone has a lot of time on their hands and a knack for learning, then sure they may be able to produce a decent OpenGL renderer eventually. But those are some big ifs. Someone who doesn't know OpenGL well, though, is going to make unoptimal and error-prone OpenGL code because they're unfamiliar with how to use it correctly. See the number of games that were originally written using Direct3D, then had an OpenGL renderer tacked on when a Mac or Linux port was considered, where the OpenGL renderer ends up lacking features, being slow, and/or prone to crashing. For many of these kinds of games, it's not unusual to see the Windows version work better under Wine (an external solution), vs the native version with in-house code. It's to the point where even bigger porting companies, like Feral, use Wine to make their ports.
I posit that hand-written solutions will bring the total line-count drastically down and make contribution from others easier, not harder.
It's not just total line count, but also compartmentalization and maintainability. Compartmentalizing code can easily increase the line count, but significantly improve maintenance cost and ease of contributing by having less code to consider when looking at a particular thing. You just need to learn the API, not what's behind the API. The point is more that we can treat that code largely as a black box, significantly reducing the amount of stuff to worry about when tracking down a bug or adding a new feature. Of course we can still look in that box and tinker with it if we want (and we do, the benefits of FOSS), but it's not a necessity for newcomers to worry about.
The only reason why I would avoid hand-writing would be in a commercial setting where you have a deadline, but this is a hobby project and has no hard time constraint, so we can afford to spend as much time simplifying and refactoring as we want.
I would think it goes the other way around. If you're on a deadline, you shouldn't rely on other people's code who may or may not be able to get back to you in a timely manner regarding questions (unless you pay them to prioritize your needs, effectively making them contract workers), and instead just rely on your own skills. In contrast, with us being a hobby project we're in no rush if getting a response from OSG or Bullet takes time. But also being a hobby project, we can't assume anyone willing to contribute has all the time in the world to learn a bunch of custom code before starting. The less they have to learn before starting, the better.
There's not a lot more I can say to defend this because what I'm proposing is typically considered insane by modern programming wisdom so from here I can only let the code speak for itself.
It's not the code we need to hear/see. I don't really know you, but I have no reason to doubt you can write a good solution for handling physics. But after we have the code, what then? Can you guarantee you'll always be available to deal with issues as they arise, or add new features as they become desirable? If someone wants to contribute, will you be there to help them learn what they need to know about your code? If you leave, is someone else going to be able to pick up where you left off?

Post Reply