Hey let's switch away from bullet to anything else (discussion)

Everything about development and the OpenMW source code.
User avatar
akortunov
Posts: 471
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation
Github profile: https://github.com/akortunov

Re: Hey let's switch away from bullet to anything else (discussion)

Post by akortunov » 05 Dec 2018, 11:58

Just for reference: most of time is spent here. Even if we will use something else instead of Bullet, we will need split workload somehow to different threads, or find another ways to optimize actor's processing.

User avatar
psi29a
Posts: 4366
Joined: 29 Sep 2011, 10:13
Github profile: https://github.com/psi29a/
Contact:

Re: Hey let's switch away from bullet to anything else (discussion)

Post by psi29a » 05 Dec 2018, 12:11

Have we gotten any further on this?
https://github.com/bulletphysics/bullet3/issues/2004

Anyone test to see if setting -DBULLET2_USE_THREAD_LOCKS=ON when building bullet will help here?

You want me to make test packages and host on the PPA?

User avatar
akortunov
Posts: 471
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation
Github profile: https://github.com/akortunov

Re: Hey let's switch away from bullet to anything else (discussion)

Post by akortunov » 05 Dec 2018, 12:24

psi29a wrote:
05 Dec 2018, 12:11
Anyone test to see if setting -DBULLET2_USE_THREAD_LOCKS=ON when building bullet will help here?
Since I am on Gentoo, I can build the Bullet by myself.
Unfortunately, even if additional locks will prevent crashes, it will not improve performance automatically - someone will need to rewrite the PhysicsSystem::applyQueuedMovement() and related stuff to use additional threads.

User avatar
psi29a
Posts: 4366
Joined: 29 Sep 2011, 10:13
Github profile: https://github.com/psi29a/
Contact:

Re: Hey let's switch away from bullet to anything else (discussion)

Post by psi29a » 05 Dec 2018, 12:25

akortunov wrote:
05 Dec 2018, 12:24
psi29a wrote:
05 Dec 2018, 12:11
Anyone test to see if setting -DBULLET2_USE_THREAD_LOCKS=ON when building bullet will help here?
Since I am on Gentoo, I can build the Bullet by myself.
Unfortunately, even if additional locks will prevent crashes, it will not improve performance automatically - someone will need to rewrite the PhysicsSystem::applyQueuedMovement() and related stuff to use additional threads.
Hmmm... sounds like fun. This also means that maintainers have more work to do to make sure -DBULLET2_USE_THREAD_LOCKS=ON is used.

I'm game if you're game. :)

User avatar
akortunov
Posts: 471
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation
Github profile: https://github.com/akortunov

Re: Hey let's switch away from bullet to anything else (discussion)

Post by akortunov » 05 Dec 2018, 12:31

psi29a wrote:
05 Dec 2018, 12:25
Hmmm... sounds like fun. This also means that maintainers have more work to do to make sure -DBULLET2_USE_THREAD_LOCKS=ON is used.
Yes, but only in case if btCollisionWorld really supports parallel sweep tests. I did not find much info about it.

User avatar
akortunov
Posts: 471
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation
Github profile: https://github.com/akortunov

Re: Hey let's switch away from bullet to anything else (discussion)

Post by akortunov » 09 Dec 2018, 12:22

Here are results of my experiments: https://github.com/OpenMW/openmw/pull/2070
It seems convexSweepTests are not fully threadsafe yet.

User avatar
wareya
Posts: 252
Joined: 09 May 2015, 13:07

Re: Hey let's switch away from bullet to anything else (discussion)

Post by wareya » 10 Dec 2018, 23:23

What does that branch do, broadly speaking? Does it make actors move independently of one another on separate threads?

That will probably have bad side effects, or at least behave differently than it currently does, even if you find a way to make it threadsafe (e.g. having each thread use an immutable version of the world as it existed when the physics frame started), because actors can move and then collide with one another. You'll probably get actors penetrating one another slightly, or projectiles passing through actors (if projectiles are threaded too), etc.

However, it would work without any issues if you checked for actors that could reasonably collide with one another and ensured that they were dealt with on the same thread, assuming you find a way to fix the threadsafety problem.
paying attention to #1751 #2473 #3609 #3862/#3929 #3807 #4297 #4623

User avatar
Stomy
Posts: 5
Joined: 11 Dec 2018, 02:55
Location: Fhloston Paradise!
Contact:

Re: Hey let's switch away from bullet to anything else (discussion)

Post by Stomy » 11 Dec 2018, 03:42

In my (Admittedly limited) experience, switching to any other physics engine will just add frustration from having to adapt to their new API, the only worthwhile improvements I've seen are either optimizing towards bullet's preferred conditions or scrapping it and making a custom solution.

For now there should still be plenty that can be done. For starters does every static have its own btScaledBvhTriangleMeshShape or are all statics in a cell combined into a single mesh? Bullet prefers a smaller number of large BVHs over a large number of small ones (as each is a miniature broadphase in its own right). Secondly, and this one might be a little harder to manage as Bullet encourages OOP organization, but is there any memory management being done with collision bodies? Because just letting the OS take care of new is a recipe for disaster, and I've seen writing for cache-awareness tends to give much more benefit than multi-threading.

A hand-written solution is a bigger investment of effort but will go a lot further than any physics middleware that could be chosen. The absolute best library in the world would struggle to compete with a relatively naive custom solution specifically written for OpenMW's conditions.

I'd be willing to investigate some of this myself in a separate branch, walking around in Old Ebonheart and seeing the framerate dip to ~20 fps makes me cry and collision is about a quarter of that. The rest I'd blame on OSG but that's a beast to be tackled another day.
Spoiler: Show
Never attribute to bad code, that which can be adequately explained with clever hacks

User avatar
akortunov
Posts: 471
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation
Github profile: https://github.com/akortunov

Re: Hey let's switch away from bullet to anything else (discussion)

Post by akortunov » 11 Dec 2018, 07:30

IIRC every static has its owns collision shape. Currently we have at least two reasons for such implementation:
1. We have a collision tracker for scripting system - we need to know if actor collides with given object.
2. It allows to quite simple add new objects to scene or remove them from it - we can just add or remove shapes, without need to rebuild the combined shape.

If we can use a some kind of batching to add/remove objects and keep info about original objects in the merged shape, we indeed can use combined shapes.
Last edited by akortunov on 11 Dec 2018, 09:22, edited 1 time in total.

User avatar
Stomy
Posts: 5
Joined: 11 Dec 2018, 02:55
Location: Fhloston Paradise!
Contact:

Re: Hey let's switch away from bullet to anything else (discussion)

Post by Stomy » 11 Dec 2018, 08:00

I remember finding btMultimaterialTriangleMeshShape a while back which stores data on a per-triangle basis. It would probably be wasteful of memory with all of the extra physics properties OpenMW won't be using, but it is a start for annotating which triangles are supposed to be part of each mesh. That would make Bullet a bit happier while keeping the required flexibility. There's also the option of extending one of the TriangleShape classes as well, though maintainers would then need to understand how both Bullet and OpenMW work to make changes to it.

In all seriousness I'd be eager to try my hand at a branch with a hand-written physics implementation so it could be specifically tailored for OpenMW's needs rather than trying to hack up some other team's middleware to work as intended. There are no complex physics interactions in the engine anyway and I've done collision detection, handling, and some broadphase before (BSP trees, which probably take too long to generate to be useful here. Though they are blazing fast). However I come from an entirely different philosophy to how OpenMW is structured so far, so this collision system would be much more C-like than C++ as OOP is not conducive to performance on modern processors.

In the future a hand-written broadphase solution could also be re-used for rendering too, and it would be easier to make extensions to it in-house rather than hoping for devs upstream to take notice.
Never attribute to bad code, that which can be adequately explained with clever hacks

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests