On Lua Scripting Limitations

[wazabear]

It appears all but set in stone that the Lua framework will be sandboxed, and I think this brings to table a lot of concerns regarding the future of OpenMW modding that I have yet to see addressed. I would like to take this time to bring those concerns to light and hopefully get some feedback. Keep in mine, I'm writing this post with the perspective of a modder.

• No direct access to filesystem. This is a huge concern for a few reasons. For one, it means that every Lua mod needs an associated omwaddon if custom assets are used. This may not seem so important at first but it has some pretty serious drawbacks.
  It makes some advanced configuration impossible. If you want to add some music files so your fancy Lua mod can play some specific mp3 in the grazelands, it is impossible. If you setup some shareable config system, importing them from in-game is impossible. I could go on with this one for a long time, but I just want to make clear the filesystem is a big deal when creating cool mods and its loss will be felt.
• If saves cannot be created, deleted, and reloaded this makes any attempt at a different save interface impossible. A mod that wanted to do autosaves every 20 minutes and only keep the 10 most recent ones per player would be impossible. A mod that allowed you to take screenshots with a Dwemer camera and put them in a film for you to inspect and load from would be impossible. The same general issue arises when we can only exit the game with one specific button due to an Exit command simply not being available. It means certain aspects of the UI will be impossible to change, this doesn't sit right with me and certainly puts a huge damper on any creative ideas for otherwise ordinary systems.
• No network access is also a major issue. It straight up makes a chunk of mods/mod functionality impossible. In the grand scheme of things, most mods won't be using the network, but there are still a fair few who will need it. Discord RPC, leaderboard systems, loading remote configuration files, auto-update with a click, automatic mod downloads, saving configurations and saves to cloud, syncing weather & time, utilizing any web API. Again, I could go on, but I don't want to give too many of my ideas away

I should also note that MWSE would be able to do everything I mentioned above, and I would also urge everyone reading this to consider the history of script extenders, which have been around for over a decade across various games (not just BGS titles).
These features, while potentially incredibly dangerous and destructive to your system, also have the potential to do many "cool" and useful things (like in the examples I mentioned above) with little effort and that would otherwise be impossible.

If scripts could read from anywhere and write to their mod folder this would be enough. If REST APIs could be used, that would cover the majority of use cases as well. There are clear boundaries that can be set instead of just declaring unrestricted (or completely restricted) access to everything.

[psi29a]

This is a bit of an alarmist post considering that this was discussed years ago already. Having a sandboxed scripting language was the only way of getting anything other than mwscript into OpenMW. That is just a fact that the modding world will have to live with. So this position of direct access to underlying system resources is a non-starter. No, full stop.

That being said, we can provide a safe access to system resources via the OpenMW-Lua API. This means read-only access to any file inside the VFS. So having your mod be able to play a specific mp3 will be doable so long as the mp3 is seen in the VFS.

Having a way to store information (state for example) is also possible, there have been discussions about this already about how to do this in a safe way. By safe, we mean in a thread-safe (avoiding race conditions for writing to persistent storage) and safe by not allowing malicious content from being run either in OpenMW or outside of OpenMW.

Extending the API for additional ways to parts of OpenMW that handles things like savegames, screenshots, cameras are not out of the question either however it has to be done with a clearly defined API that abstracts away the internals of OpenMW.

Unfettered network access is a non-starter. If there is a way to mitigate security concerns about having network access, then we can provide a mechanism for a specific protocol to be used, such as REST or some other Pub/Sub mechanism.

It has already be discussed, years ago as well, that OpenMW-Lua will not be MWSE-Lua; in that OpenMW has a responsibility to not allow people to shoot themselves in the foot. Script extenders are only necessary because of the limitations of the game engines they are extending which are closed source. OpenMW is on the other end of the spectrum where such things are not needed as OpenMW can provide mechanisms that can be made use of in OpenMW-Lua. So a balance has be struct between usability and security.

With that said, having a OpenMW-LUA API that is versioned that can change over time will be OpenMW's greatest asset moving forward. Being able to adapt means we can try new things and also deprecate/remove others over time. Treating modders better than how Bethesda does by providing developer level documentation in an open way with feedback to its future development.

[akortunov]

If you want to add some music files so your fancy Lua mod can play some specific mp3 in the grazelands, it is impossible.

Nothing stops modmakers from using VFS instead of direct access to filesystem.

[wazabear]

Nothing stops modmakers from using VFS instead of direct access to filesystem.

Modmakers yes, but not for people using the mods. As a modder, the last thing you want is for users to go around messing with your folders. In this particular instance you want people to choose specific files to add to the VFS. It's not uncommon for you to choose files from some in-game file browser. In Minecraft, it's used for choosing shader packs. In general having access to a file browser (even if that's the *only* way to read files outside the mods VFS) would be extremely beneficial.

[AnyOldName3]

It would be a good idea for you to go back to the last time someone made each of your points and work out what we said to address it. This is all stuff that's been covered in the past as far as I can tell, so it'll save everyone some time if you can get up to speed. The arguments are compelling enough that I dropped my initial position of thinking there should be a system for native-code engine plugins all the way back to the other end of the spectrum and wanting fully sandboxed Lua.

It's not impossible that something critical got missed in past debates, but there's no point in someone new coming along every six months and saying things we've gone over repeatedly in the past.

[wazabear]

This is all stuff that's been covered in the past as far as I can tell, so it'll save everyone some time if you can get up to speed.

I've gone through the relevant posts, and I believe I have the gist of things. Sandboxing is happening, and there is no debate left to be had. If someone wants a feature down the road then they can submit a pull request or issue request such that it can be properly and safely implemented, ensuring stability and compatibility. I did not come across any concrete answers, everything was all rather abstract to me. Here, I've given explicit examples of mods that I see as being impossible.

So correct me if I'm wrong here, consider the scenario where my mod absolutely relies on selecting arbitrary files from anywhere in the filesystem and copying them over to the mods directory (all my POC mods rely on such functionality, and yes there are other ways to go about it but they all are objectively inferior from a users point of view). I would implement this myself, or if I didn't have the knowledge would open up an issue for such functionality. It would be discussed, then rejected or accepted after some arguments over whether this is an actual good idea or not.

The problem is that from what I have read such a functionality would be rejected. Another actual example in which I would expect a resounding no is being able to do some processing of NIFs in a python script.

The argument seems to be you'd never need any of these functionalities since the API can always be extended and should be powerful enough. I just don't see that happening in any realistic time-frame, and I would love to see OpenMW-Lua take off.

[akortunov]

Another actual example in which I would expect a resounding no is being able to do some processing of NIFs in a python script.

There are existing tools to do that (PyFFI, for example). There is no need to reinvent a wheel.

In Minecraft, it's used for choosing shader packs.

It is irrelevant to OpenMW since you more likely will need to setup shaders prior to game launch rather that during runtime anyway.
The same thing for content files.

So correct me if I'm wrong here, consider the scenario where my mod absolutely relies on selecting arbitrary files from anywhere in the filesystem and copying them over to the mods directory

Again, there is no need to do it during runtime via OpenMW scripting system - you can just use any 3d-party tools for that.
Otherwise your design is a quite weird since you mix different responsibilities in one application. In theory, content creation should be handled by content creation tools, not by the game itself.

[wazabear]

There are existing tools to do that (PyFFI, for example). There is no need to reinvent a wheel.

This is exactly my point, you can't use them from within Lua. So in the current implementation, you certainly would need to reinvent the wheel.

It is irrelevant for OpenMW since you more likely will need to setup shaders prior to game launch rather that during runtime anyway.
The same thing for content files.

For one, when post processing comes, I highly doubt a restart will be necessary (it's not even necessary right now, it's just not implemented), and even then this is not the point I was making with this. Being able to choose a file from your system is incredibly useful for certain mod mechanics. It is not just shaders but configuration files, potentially other Lua scripts, images, previewing nifs, and that's just the 3 off the top of my head. These have real use cases, telling users to put all the resources they want into a single mod folder is a fantasy. I should stress again you do not want users meddling with the mod folder itself, that should be the duty of the Lua script.

Again, there is no need to do it during runtime via OpenMW scripting system - you can just use any 3d-party tools for that.
Otherwise your design is a quite weird since you mix different responsibilities in one application. In theory, content creation should be handled by content creation tools, not by the game itself.

Offloading these things to 3rd party tools is already what I do, and it's horrible. Publishing mods which require 3rd party tools is hard to maintain and plain annoying for users. I've shipped binaries with mods before, users run them just as they would run a regular mod. This doesn't make much sense to me at all if you're enforcing sandboxing but encouraging such tools to be used as an alternative. A reminder that mods can be utility.

Mods are not just mechanics, quests, and UI changes, they make possible what others thought to be impossible, and they do things which developers cannot always foresee.

[akortunov]

It is not just shaders but configuration files, potentially other Lua scripts, images, previewing nifs, and that's just the 3 off the top of my head.

Configs can be handled by OpenMW itself, mods do not even need to know which data format configs will have and where they will be stored.
Scripts which can modify and copy another scripts look like a bad idea for me.
Game itself does not work with NIF's directly, so NIF preview makes no sense. OpenMW works with scene graph, so NIF file (as well as file of any other supported mesh format) needs to be taken from VFS and deserialized to OSG scene node, result should be cached and only then mesh can be used in scene graph to be rendered.

[wazabear]

Game itself does not work with NIF's directly, so NIF preview makes no sense.

I think my point is not getting across at all. It doesn't matter that the engine doesn't deal with NIFs directly, modders only care that a NIFs are what they used to show things in game. Keep in mind any other format aren't realistically going to be used for any semi-complex model in foreseeable future. There is no exporter which can handle advanced features such as geomorphs, multiple animated UV layers, material controllers, or even specific flags/nodes (such as ZBuffer or Billboard). Previewing NIFs was a very specific example, and likely would be most useful for modders anyways.

As for the config example, consider this specific use case which the simplest I could think of.

You have a mod which changes weather configurations which is advanced and works independently of openmw.cfg values. You can export these configs to share, or import an existing ones from your Downloads folder (which you conveniently downloaded from the Nexus). Offline mod configs are not very useful here. For one, it's been made clear in other discussions OpenMW is not in the business of becoming a mod manager, and that certainly is in the domain of a mod manager. Secondly, changes like these are best viewed in real-time.