OpenMW Lua roadmap
Initial Lua MR was merged some time ago and basic (still very limited)
OpenMW Lua is available in latest unreleased builds (can be found
here).
Here are plans for further development.
Dehardcoding
Dehardcoding means reimplementing game mechanics (which are currently implemented in C++) as built-in Lua mods.
Now it is one of the main priorities for development.
Why is it so important?
Without dehardcoding Lua scripts are not fully functional and can not alter game mechanics. For example without dehardcoding we can not write a script that prevents player death, because when health goes to zero, C++ code immediately starts death animation before scripts can do anything. OpenMW Lua doesn't have an analogue of TES3MP validators that allow to cancel something that is done by C++ code.
Instead we plan to use script
interfaces and
event system. But this approach works only if game mechanics are already implemented in Lua.
Public API and temporary internal API
Suppose that there is an important C++ function
actorDoSomething(ptr) and we want this functionality to be available from Lua as soon as possible. But in future we plan to dehardcode this function and need a way to do it without breaking mods.
Then the steps are:
- Design an interface that will still be relevant after dehardcoding.
In our example we may decide that after dehardcoding "actorDoSomething" will be implemented in a builtin script 'basic_ai.lua' and will be available in local scripts via a script interface as interfaces.BasicAI.doSomething().
- Add an internal function self:_doSomething() to the Lua package "openmw.self". This function doesn't need documentation and shouldn't be used by modders.
- Create 'basic_ai.lua' and implement interfaces.BasicAI.doSomething() as a simple wrapper around self:_doSomething(). It is public interface that can be used in mods.
- Later during dehardcoding we can remove self:_doSomething() and replace the wrapper with a real implementation without changing the public interface.
All functions and engine handlers with prefix "_" are internal API that shouldn't be used outside of built-in scripts. But note that it is a temporary thing. At the end there will be no internal API at all and built-in scripts will not have any "special powers" comparing to normal mods.
Principles of extending Lua API
- It is easy to add a new command, but it is hard to remove if it was designed badly because somebody may already depend on it. So we should be careful. At the current stage we should focus on major stuff (like designing an approach how to work with records from Lua). There is no sense in adding a lot of minor commands (unless they are needed for major ones), because too lot of things are yet unclear and something can change.
- New commands should be conceptually compatible with multiplayer:
A local script can see only some area of the game world (cells that are active on a specific client). Any data from inactive cells can’t be used, as they are not synchronized and could be already changed on another client.
A local script can only modify the object it is attached to. Other objects can theoretically be processed by another client. To prevent synchronization problems the access to them is read-only.
Global scripts can access and modify the whole game world including unloaded areas, but the global scripts API is different from the local scripts API and in some aspects are limited, because it is not always possible to have all game assets in memory at the same time.
Though the scripting system doesn’t actually work with multiplayer yet, the API assumes that there can be several players. That's why any command related to UI, camera, and everything else that is player-specific can be used only by player scripts.
- We need to remember that scripts and rendering work in separate threads. It is possible to run some callbacks from the main thread if it is really needed, but let’s try to minimize the number of such cases.
Adding new commands to Lua API
- Create an issue on gitlab with a proposal on what should be added to Lua API.
- Discuss the proposal and check that it meets the principles above. We will add a special gitlab tag "Approved feature request" (or something like this) for issues that have passed this step and are ready to work on.
- Create an MR with the change. It should also update documentation and increment core.API_REVISION. Once a framework for integration tests is ready, there will be a requirement that every new Lua command should be used in at least one integration test.
What must be done before 0.48 release
For now we are free to change Lua API in any way without thinking about compatibility. But after 0.48 (the first stable release with Lua) we will not be able to remove commands. So the main thing that must be done before the release is to change in advance everything that we may want to change in the future.
In particular:
- Design a good interface for AI packages that will not be changed after dehardcoding (i.e the built-in script "basic_ai.lua" as described above). Remove from public API (i.e. add prefix "_") all functions that work with AI directly:
Code: Select all
self:enableAI(v), self:getCombatTarget(), self:setDirectControl(control),
self:startCombat(target), self:stopCombat()
They should be available via script interfaces instead.
- Review the whole API one more time.
- Decide whether we continue to use lua-scripts=something.omwscripts in openmw.cfg, or somehow store it inside omwaddons.
Of course before the release we also want to add to OpenMW Lua as much new functionality as possible, but it is less critical than the points above.
Work in progress
- Package “openmw.input” – access user input and input bindings (merge request 1073)
- New testing framework (merge request 1005)
- Camera dehardcoding (merge request 1030)
- Lua UI (Uramer works on it)
- Safe way to pass Lua callbacks to C++ code, needed at least for UI and async raycasting (I am going to work on it, but not yet started)
How to contribute
It is recommended to contact me first
There are some tasks in queue:
- Mod settings and a persistent storage for settings.
- Design an interface for AI packages.
- Create/remove/pick/drop objects in Lua (see some ideas here).
- Issues on gitlab with tag "Lua".
And there is a lot of
major not yet designed stuff.
Each item in this list requires a forum thread, discussion, and probably a design doc. One of the questions for each item is "how is it going to work with multiplayer". Another question is how it will work after dehardcoding.
- Work with records from Lua. Currently Lua scripts can get recordId (string) from any object, but can not get any info from the record itself. Records should also support queries.
- Call MWScript from Lua. Maybe access variables from MWScript. Also need to think how to get rid of (i.e. replace with a converter) MWScript in future.
- How to work with a player’s journal? Need to take into account both multiplayer (each player has their own journal, but some notes and quests can be common) and later games like Skyrim. Requires significant generalization of current journal implementation.
- ...