Evidently, not every part of VR can be moved to a Lua script. The engine must implement a few things.
- Stereo Rendering
A code layer implementing all we need for stereo rendering. This is already merged: https://gitlab.com/OpenMW/openmw/-/merge_requests/1757
- VR runtime
A code layer implementing all we need from OpenXR. Such as managing frame synchronization, swapchains, and tracking data. This currently is part of the draft VR MR: https://gitlab.com/OpenMW/openmw/-/merge_requests/948
A summary of what the Lua API needs to achieve.
- Tracker to actor bone mapping.
- Tracker to camera mapping.
- Modifying visibility of actor parts.
Motivation: Many people will find disembodied limbs to be a more comfortable VR experience.
- Arbitrary control of actor bones.
Motivation: Implementing visual gestures such as finger pointing.
- Read world space or model space position of any part/bone
Motivation: Implementing realistic combat and archery
- Input subactions
Motivation: VR inputs differ from other inputs in that they may differentiate between what limb triggered the input.
- Lua API for GUI - For modifying existing UI items and providing new UI items, rather than having to modify existing UI layout files.
A summary of intended APIs and their motivation
- Actor Part Manipulation API
Needed to map tracking data to actor bones. This API shall allow arbitrary manipulation of the position, orientation, and visibility of any and all actor parts.
- Input Actions and sub-actions
Needed to extend the input API with VR specific controls. Sub-actions are needed to differentiate control source, such as left vs right hand controllers, if this is relevant for an action.
- VR API
An API providing access to VR specific information and methods.
- 3D GUI API
Needed to render and interact with GUI as 3D geometry placed within the scene.