Lua API for GUI

[urm]

With the Lua MR taking form, it's probably time to consider how to implement various parts of the Lua API. This thread is meant to be a discussion on how to best approach it for the in-game GUI.

Current state of UI in OpenMW:

OpenMW uses MyGUI to render in-game UI. The implementation consits of XML layouts, with some parts generated with C++ code, which also defines any logic or event handling. We have a few custom Widgets, such as elements which automatically resize to fit their content.

Overall most of the UI code is quite old, as it changed little after reaching Morrowind.exe parity, and has some quirks which are not acceptable for a modding API (e. g. some widgets are hard coded to ignore a part of their texture to work around vanilla assets).

Existing UI APIs for Morrowind:
1. MWScript - simple popups and status messages, with no real customization, outside of button text.
2. MWSE Lua - defined directly and entirely through Lua code. AFAIK, has limited customization, but it's easy to make the UI fit the vanilla style. https://mwse.readthedocs.io/en/latest/l ... ght=tes3ui
3. tes3mp Custom UI PR - not a part of any official tes3mp release (yet?), implemented by me. It is essentially an extension of every Widget both from MyGUI and OpenMW, which allows for some simple UI logic to be described right in the XML layout, and to bind some properties and events to network messages. Due to tes3mp only having server-side Lua API, all of the significant logic is handled there, which causes some limitations. https://github.com/uramer/openmw/tree/0.7.1-custom-ui
4. Editing MyGUI layout and resources files included with OpenMW - technically, we already allow some interface customization, which could be expanded by reducing the portion of the layouts that's entirely generated by C++ code. The look of the UI can also be altered by providing replacement textures, which a few mods already do.

The simple approach to the new API:

Expose relevant MyGUI classes through Lua bindings, and allow mods to provide their own resource and layout MyGUI files (probably registered in the script, rather than in openmw.cfg).

There are some issues with this one. Firstly, MyGUI has some questionable or buggy behaviour. Secondly, we would be bound to the current MyGUI API. Both make it clunky to expose to modders, as updating MyGUI would be difficult due to backwards compatibility. Also, MyGUI layout syntax is relatively complex, and is overkill for most mods.

An API abstracted from MyGUI:

A much more flexible approach would be to design an API from scratch, and "implement" it by using MyGUI. It would solve the issues related to MyGUI, and would keep the possiblity of replacing MyGUI with a different UI library (which we might have to do if we realize we can't use MyGUI for later Bethesda games, for instance).

The problems are rather obvious: it's a lot more work, and we could introduce our own bugs.

Specifics to discuss:
1. Should we provide a layout declaration, or construct UI entirely in code? If we do have a layout, which features should it have? (e.g. only widget relations and style, like in MyGUI, or maybe registering events and property binds, like in my PR)
2. Most mods would prefer to edit as little of the UI as possible (e.g. add new info to item popups), how can we offer that option?
3. Multiple mods editing the same UI should be as compatible as possible. An entirely different inventory UI (e. g. Skyrim-like lists with 3d item previews) should not necessarily appear any differently to other scripts interacting with it (e.g. ones that handle right click interactions on items).
4. We should probably provide simple functins similar to the MWScript ones, as many modders would probably still have a use for them.
5. Currently styling MyGUI layouts in vanilla style requires a decent amount of work and boilerplate. No matter which approach we pick, we should provide tools which make it easy.

Both approaches imply we would have to "dehardcode" existing UI into the new API (probably easier with the first option), to allow mods to override parts of them.

Please comment if you have any thoughts, ideas or objections.

[wazabear]

2. Most mods would prefer to edit as little of the UI as possible (e.g. add new into to item popups), how can we offer that option?

Note, what I'm about to say bleeds into (3). This is something really important to get right! I think a good solution to this is to give every widget its own mini API and not differentiate between core components and a modded component. Every component must then be registered with some UID such that other mods can utilize them. You'd also want every component to have some shared dynamics, ideally controlled via events. For example, onHide(UID, handler).

A possible API for the tooltip could be something like this.

Code: Select all

Tooltip (id=UID)
    setFilter(pred(focus_object)) -> what makes this popup show? 
    setWidth()
    setHeight()
    setFadeDelay()
    hide()
    show()
    blockCount() -> number of blocks including the default ones
    getBlock(index) -> returns component, can be anything!
    insertBlock(index)
    removeBlock(index)
    popBlock()

Ideally, builtin components shouldn't be handled any differently then anything else. The problem is that it is extremely dangerous, OpenMW can crash if you mess around with the tiniest UI element from my experience with MorroUI. Things will need to change on the mwgui side, it's mostly just OpenMW unconditionally expecting widgets to be there. You hid that textbox, too bad you're done for.

Another advantage to this is that any mod can override any other mod by simply handling the appropriate event. If events can cascade (events overrides can pass on execution as normal) this becomes even more powerful.

[urm]

We already have a concept of script interfaces in the Lua API. Essentially, it allows a script to assign a table to a string, which is then available to all other scripts attached to the same object. In that case, we could have every UI element be a script attached to the player, and smaller UI elements (e. g. item tooltips) are used by larger UI elements (e. g. inventory). Other scripts can override the interface through assigning an interface with the same name while being later in the load order. Since the later script can access the older interface (unlike the others), it can pass any unchanged calls up to the standard UI (or a mod above it).

And yeah, I think the idea of splitting the UI elements into pieces that are as small as possible, and have each of them provide an API is good. It might be a decent compromise between the two suggestions I've given: if every UI script is expected to generate a MyGUI widget (potentially with some children), we can combine them as if in a component system.

We will need to rewrite/refactor the existing UI in any case. Making it robust to changes will not be trivial though, I agree.

I'm not sure that exact API is good for tooltips, but some kind of way to attach blocks of information makes a lot sense.

[psi29a]

There is already a refactor MR to get things usable for VR... not sure if that will cause conflicts.

[urm]

There is already a refactor MR to get things usable for VR... not sure if that will cause conflicts.

From what I know about that one, it doesn't really touch the same things I'm talking about. The main change there is the ability to render parts of the UI into separate buffers. It would, of course, be nice to merge that first, before we do another big refactor of UI code.