Lua scripting in OpenMW

Everything about development and the OpenMW source code.
Post Reply
ptmikheev
Posts: 45
Joined: 01 Jun 2020, 21:05
Gitlab profile: https://gitlab.com/ptmikheev

Re: Lua scripting in OpenMW

Post by ptmikheev » 08 Nov 2020, 19:49

AnyOldName3 wrote:
08 Nov 2020, 01:05
Could we have something like a bunch of scripts that need to be run at some point during the frame, and the main thread sticks them in a queue of waiting scripts that get picked up by a pool of worker threads, then once it needs the results of things, it can either deal with what the worker threads have already processed, or pick a script from the front of the queue to run itself if no results are ready yet. This architecture would still leave the main thread in control, if OpenMW was run on a machine with as many cores as scripts it could scale, and we can reuse the worker threads for other async tasks like preloading for the part of the frame where we've got no pending scripts yet.

I sort of anticipate that we're going to have a bunch of work that absolutely needs doing before any scripts can be run and a bunch that needs doing after all scripts have finished and then a bunch of work that all arrives at once and needs to be finished as soon as possible, so we'll want a system that copes well with that.
Not sure I understand. The main problem with multithreaded scripting is concurrent access to the world state, since every script needs the access.

Let me illustrate the approaches I see with code samples:

1. Current state (no separate thread)

It is how OpenMW function now.

Code: Select all

// Main thread
while (true) {
  executeScripts();
  updateMechanics();
  updatePhysics();
  updateWorld();
  updateGUI();
  
  mViewer->eventTraversal();
  mViewer->updateTraversal();
  mViewer->renderingTraversal();
}

2. Separate thread for scripting

Here we have one worker thread that evaluates scripts while the main thread does rendering.
We will need to verify that scripting and rendering never work with the same data at the same time.

Having several workers is more complicated due to the same concurrency problem. I.e. we can crash if one script reads an inventory at the same time as another script removes an item from it. Locking objects with critical sections may have too big overhead.
However even with a single worker separating scripting and rendering will help a lot.

Code: Select all

// Main thread
startScriptingThread();
while (true) {

  // Update world and apply changes to OSG scene tree
  updateMechanics();
  updatePhysics();
  updateWorld();
  
  updateGUI();
  
  mExecuteScriptsFlag = true;  // start scripting
  
  // Render OSG scene tree. Shouldn't use MWWorld.
  mViewer->eventTraversal();
  mViewer->updateTraversal();
  mViewer->renderingTraversal();
  
  while (mExecuteScriptsFlag) wait();  // wait scripting if it is not finished yet
}

// Scripting thread
initializeLuaAPI();
loadScripts();
while (true) {
  while (!mExecuteScriptsFlag) wait();
  
  executeScripts();  // Shouldn't use OSG scene tree.
  mExecuteScriptsFlag = false;
}

3. Separate thread for scripting; copy data before use

Here we copy world data to a separate object (ScriptingWorld) before running scripts.

Benefits from ScriptingWorld:
  • It is easy to verify that scripting and rendering work with different data.
  • Possibility to have several worker threads for scripting (just create several instances of ScriptingWorld).
  • Scripting rate can differ from frame rate (i.e. no need to drop frame rate if scripts are slow).
Disadvantage: additional overhead of copying data to and from ScriptingWorld.

Code: Select all

// Main thread
startScriptingThread();
while (true) {

  if (!mExecuteScriptsFlag) {  // start scripting if the previous scripting iteration is already finished
    // Apply results of previous scripting frame from ScriptingWorld to World
    applyChangesToWorld(mScriptingWorld);
    
    // Update data in ScriptingWorld
    updateScriptingWorld(mScriptingWorld);
    mExecuteScriptsFlag = true;
  }

  updateMechanics();
  updatePhysics();
  updateWorld();  
  updateGUI();
  
  mViewer->eventTraversal();
  mViewer->updateTraversal();
  mViewer->renderingTraversal();
}

// Scripting thread
initializeLuaAPI();
loadScripts();
while (true) {
  while (!mExecuteScriptsFlag) wait();
  
  executeScripts(mScriptingWorld);  // Only ScriptingWorld is used
  mExecuteScriptsFlag = false;
}

User avatar
urm
Posts: 61
Joined: 02 Jun 2017, 16:05
Contact:

Re: Lua scripting in OpenMW

Post by urm » 08 Nov 2020, 20:33

Lua doesn't support threads. Async logic can be implemented with coroutines, but it's not real threads.
We don't need to manage threads from Lua itself
If all the communication between scripts happens through eventsc we could split the scripts into an arbitrary amount of separate Lua machines on different threads without any real issues

User avatar
urm
Posts: 61
Joined: 02 Jun 2017, 16:05
Contact:

Re: Lua scripting in OpenMW

Post by urm » 08 Nov 2020, 20:41

ptmikheev wrote:
08 Nov 2020, 19:49

Could we have something like a bunch of scripts that need to be run at some point...
Personally, I think a combination of 1 and 3 is the way to go.
We could have local scripts run on the main thread, and delegate to them features which are necessary to run every frame (e. g. smooth UI animations, or manipulating actor positions in a very specific way). And if they need something particularly number crunchy, we can always provide a simple "Worker thread" API.
Also keep in mind that some particularly heavy parts of the Lua API (like I/O) should be done through coroutines, and nothing is stopping us from implementing them in a way which offloads most of the work to a different thread. This would allow us to apply the third approach only when it's effective to do so - in slow and latency insensitive cases. It would also simplify thread communication - it's always nicer when threads only need to send immutable data to each other.

User avatar
urm
Posts: 61
Joined: 02 Jun 2017, 16:05
Contact:

Re: Lua scripting in OpenMW

Post by urm » 08 Nov 2020, 22:52

Actually, reading it again, option 2 is also quite promising. One thing that's not perfectly clear to me, is whether we still need access to world data during rendering. If we do, we would still end up with locking behavior, since scripts will be writing data at well.
Last edited by urm on 09 Nov 2020, 09:00, edited 1 time in total.

ptmikheev
Posts: 45
Joined: 01 Jun 2020, 21:05
Gitlab profile: https://gitlab.com/ptmikheev

Re: Lua scripting in OpenMW

Post by ptmikheev » 08 Nov 2020, 23:07

urm wrote:
08 Nov 2020, 20:41
Personally, I think a combination of 1 and 3 is the way to go.
We could have local scripts run on the main thread, and delegate to them features which are necessary to run every frame (e. g. smooth UI animations, or manipulating actor positions in a very specific way).
Since there was some misunderstanding, I want to clarify:

My examples are about data synchronization between MWWorld and local Lua scripts. "1" and "3" are principally different and both are about local scripts. Communication between local and global scripts is a separate question that is out of scope of my previous post.

In the variant "2" the scripting thread works always synchronously with the main thread and accesses world state only during rendering, when it is not used by the main thread. Just a performance optimization of the current implementation.
urm wrote:
08 Nov 2020, 22:52
Actually, reading it again, option 2 is also quite promising. One thing that's not perfectly clear to me, is whether we still need access to world data during rendering. If we do, we would still end up with lacking behavior, since scripts will be writing data at well.
As I understand mwphysics updates RefData for every object and then updates the corresponding OSG node (linked with RefData::getBaseNode).
Then rendering uses only the scene graph, and during rendering the RefData is safe to be used from scripting. Please correct me if I am wrong.

User avatar
urm
Posts: 61
Joined: 02 Jun 2017, 16:05
Contact:

Re: Lua scripting in OpenMW

Post by urm » 09 Nov 2020, 12:44

ptmikheev wrote:
08 Nov 2020, 23:07
Since there was some misunderstanding, I want to clarify:

My examples are about data synchronization between MWWorld and local Lua scripts. "1" and "3" are principally different and both are about local scripts. Communication between local and global scripts is a separate question that is out of scope of my previous post.

In the variant "2" the scripting thread works always synchronously with the main thread and accesses world state only during rendering, when it is not used by the main thread. Just a performance optimization of the current implementation.
My main point was that regardless of local scripts running in the main thread, we should still run many tasks on different thread(s) - certainly openmw API's I/O, but possibly also provide a Lua API for mods to do the same.

User avatar
AnyOldName3
Posts: 2048
Joined: 26 Nov 2015, 03:25

Re: Lua scripting in OpenMW

Post by AnyOldName3 » 09 Nov 2020, 15:04

The thing I was proposing would have all the stuff OpenMW currently does before running scripts happen before any scripts get run, and all the stuff OpenMW currently does after scripts happen after all scripts have been run. That would mean the only worries about concurrent access to world state would be if two scripts tried meddling with the same thing. As Urm pointed out, in the cases where two scripts genuinely had a dependency on each other, that could be solved by making them register for events, so the script that needed to run later wouldn't even be added to the pending scripts queue until the script that needed to run first had put it there.
AnyOldName3, Master of Shadows

User avatar
urm
Posts: 61
Joined: 02 Jun 2017, 16:05
Contact:

Re: Lua scripting in OpenMW

Post by urm » 09 Nov 2020, 17:32

AnyOldName3 wrote:
09 Nov 2020, 15:04
The thing I was proposing would have all the stuff OpenMW currently does before running scripts happen before any scripts get run, and all the stuff OpenMW currently does after scripts happen after all scripts have been run. That would mean the only worries about concurrent access to world state would be if two scripts tried meddling with the same thing. As Urm pointed out, in the cases where two scripts genuinely had a dependency on each other, that could be solved by making them register for events, so the script that needed to run later wouldn't even be added to the pending scripts queue until the script that needed to run first had put it there.
Isn't this identical to running all scripts on the main thread?

User avatar
AnyOldName3
Posts: 2048
Joined: 26 Nov 2015, 03:25

Re: Lua scripting in OpenMW

Post by AnyOldName3 » 10 Nov 2020, 01:53

No, as you'd have a thread pool running multiple scripts at the same time when they didn't depend on each other. Provided we/modders correctly determined which scripts depended on each other, it'd programmatically be very similar to the approach where the main thread did scripts (and potentially if we detected we were running on a machine with very few cores so didn't populate the thread pool it'd end up literally doing that) but should go faster if scripts are doing a lot of heavy lifting. We have other bits of work that can be run during other parts of the frame, but not during the whole frame (e.g. physics), so the thread pool could be used for those, too, and always/usually be kept busy, but we'd have the architectural niceness of a frame consisting of distinct phases that didn't overlap and could be trusted to run in a particular order.
AnyOldName3, Master of Shadows

User avatar
urm
Posts: 61
Joined: 02 Jun 2017, 16:05
Contact:

Re: Lua scripting in OpenMW

Post by urm » 10 Nov 2020, 08:22

AnyOldName3 wrote:
10 Nov 2020, 01:53
No, as you'd have a thread pool running multiple scripts at the same time when they didn't depend on each other. Provided we/modders correctly determined which scripts depended on each other, it'd programmatically be very similar to the approach where the main thread did scripts (and potentially if we detected we were running on a machine with very few cores so didn't populate the thread pool it'd end up literally doing that) but should go faster if scripts are doing a lot of heavy lifting. We have other bits of work that can be run during other parts of the frame, but not during the whole frame (e.g. physics), so the thread pool could be used for those, too, and always/usually be kept busy, but we'd have the architectural niceness of a frame consisting of distinct phases that didn't overlap and could be trusted to run in a particular order.
We would still likely have thread locks, since all of those scripts could be writing data. Also, the dependencies between scripts could potentially end up being circular, there is no guarantee they would resolve into a tree

Post Reply