The new MWSE-Lua interface

Everything about development and the OpenMW source code.
Post Reply
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: The new MWSE-Lua interface

Post by AnyOldName3 »

That may be true, but few people have the skill to do so. Making an MWSE plugin requires much less skill, and thus greatly increases the potential number of people who are able to contribute changes to the game.
If an MWSE plugin requires you to write native code (the main thing we're blocking with sandboxing), you're writing C++ or C which interacts with OpenMW. At that point, you're just jumping through extra hoops as you could just write C++ as part of OpenMW. The other things sandboxing blocks are network and file access, which can be done directly within Lua. With file access, if you're using it for non-nefarious reasons, there's not much that can't still be achieved through a well-designed sandbox (for example, we could just give each mod a directory which it is free to write and read, which would be fine until you need to access specific files in other locations, which could potentially include a user's bank details, which we can't allow). With network access, I can't think of any way to have a partial sandbox which couldn't be used to exfiltrate sensitive information unless there was no way of letting Lua access the sensitive information in the first place, which requires a good sandbox.
User avatar
SeaFox
Posts: 34
Joined: 29 Feb 2016, 17:30

Re: The new MWSE-Lua interface

Post by SeaFox »

AnyOldName3 wrote: 03 Oct 2018, 21:30 If an MWSE plugin requires you to write native code (the main thing we're blocking with sandboxing), you're writing C++ or C which interacts with OpenMW. At that point, you're just jumping through extra hoops as you could just write C++ as part of OpenMW.
But making changes to the OpenMW code base requires more knowlege and skill than is required of making a plugin.
AnyOldName3 wrote: 03 Oct 2018, 21:30 The other things sandboxing blocks are network and file access, which can be done directly within Lua. With file access, if you're using it for non-nefarious reasons, there's not much that can't still be achieved through a well-designed sandbox (for example, we could just give each mod a directory which it is free to write and read, which would be fine until you need to access specific files in other locations, which could potentially include a user's bank details, which we can't allow). With network access, I can't think of any way to have a partial sandbox which couldn't be used to exfiltrate sensitive information unless there was no way of letting Lua access the sensitive information in the first place, which requires a good sandbox.
I agree sandboxing is the proper route for the main branch of the OpenMW project. But I also think it may be worth considering a sub-project fork of OpenMW which allows plugins without the strict requirements of sandboxing. This would make many people happy, and would potentially allow greater contributions from mod creators who lack the skill to contribute directly to the OpenMW code base.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: The new MWSE-Lua interface

Post by AnyOldName3 »

But making changes to the OpenMW code base requires more knowlege and skill than is required of making a plugin.
If the plugin is something which absolutely requires the ability to execute native code (i.e. isn't just calling some other library which only exists as non-Lua because it wasn't made specifically for OpenMW), it's always going to be easier to add to the engine directly. If your argument is based around this premise, then it doesn't hold any weight.
User avatar
lysol
Posts: 1513
Joined: 26 Mar 2013, 01:48
Location: Sweden

Re: The new MWSE-Lua interface

Post by lysol »

SeaFox wrote: 03 Oct 2018, 22:32
AnyOldName3 wrote: 03 Oct 2018, 21:30 If an MWSE plugin requires you to write native code (the main thing we're blocking with sandboxing), you're writing C++ or C which interacts with OpenMW. At that point, you're just jumping through extra hoops as you could just write C++ as part of OpenMW.
But making changes to the OpenMW code base requires more knowlege and skill than is required of making a plugin.
To further elaborate on what AnyOldName3 said: If you're just talking of a simple plugin that don't require writing C++, then you probably don't need non-sandboxed lua to begin with. OpenMW WILL have Lua support. It will however be sandboxed. And this is probably not an issue in the case you're describing.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: The new MWSE-Lua interface

Post by Chris »

SeaFox wrote: 03 Oct 2018, 20:57 That may be one reason why not everyone uses OpenMW instead of MWSE. But other factors include the fact that MWSE makes it easy for low skill mod creators to make changes. Changes that OpenMW programmers may or may not have time to make in the immediate future if ever.
OpenMW doesn't develop on its own, it gets developed when people work on it. If people don't develop OpenMW itself because it's "easier" to make an un-sandboxed plugin, who's going to help get those features into OpenMW?
SeaFox wrote: 03 Oct 2018, 20:57 But old mods often don't work well with newer mods, or with the improved systems of OpenMW. Therefore, people who wish to use newer mods will eventually be forced to give up the older mods for the improved versions.
It was an analogy. The point is, if an un-sandboxed mod works just fine for 90% of players, fewer people will be motivated to implement the feature properly in the engine, let alone fix the mod for the remaining 10%.
User avatar
SeaFox
Posts: 34
Joined: 29 Feb 2016, 17:30

Re: The new MWSE-Lua interface

Post by SeaFox »

AnyOldName3 wrote: 03 Oct 2018, 22:42
But making changes to the OpenMW code base requires more knowlege and skill than is required of making a plugin.
If the plugin is something which absolutely requires the ability to execute native code (i.e. isn't just calling some other library which only exists as non-Lua because it wasn't made specifically for OpenMW), it's always going to be easier to add to the engine directly.
If you have the knowlege and skill to add functions directly to the OpenMW code base, then it probably would be easy to do. But if you lack such knowlege, then it may not be easy, or even possible. Even with such knowlege and skill, some people may prefer to add native functions without having to recompile the entire OpenMW project. Jast saying.
User avatar
SeaFox
Posts: 34
Joined: 29 Feb 2016, 17:30

Re: The new MWSE-Lua interface

Post by SeaFox »

Chris wrote: 03 Oct 2018, 23:30 OpenMW doesn't develop on its own, it gets developed when people work on it. If people don't develop OpenMW itself because it's "easier" to make an un-sandboxed plugin, who's going to help get those features into OpenMW?

It was an analogy. The point is, if an un-sandboxed mod works just fine for 90% of players, fewer people will be motivated to implement the feature properly in the engine, let alone fix the mod for the remaining 10%.
I understand that point of view, but disagree. Obviously we all want OpenMW to succeed, but using "vendor lock-in" seems like a bad strategy for promoting OpenMW and gaining new developers. You are essentially saying, that mod creators should only be allowed to make mods which extend the core functionality of the scripting engine if they have the knowledge, skill, and willingness to directly modify the OpenMW code base. Even then they may not be allowed to do it until OpenMW developers reach a consensus as to proper methods of doing what the mod creator wants to do. That stifles creativity. This will not bring new people to the OpenMW project. It instead provides an incentive for them to use MWSE and other alternatives.

In my opinion, more people will come to OpenMW by giving them more options for making mods, not less. Then, when competent OpenMW developers discover good mods with capabilities which merit inclusion in the core OpenMW code base, they can do so with their much greater knowledge of how the OpenMW code base works and the best ways to improve it.
CMAugust
Posts: 285
Joined: 10 Jan 2016, 00:13

Re: The new MWSE-Lua interface

Post by CMAugust »

When it comes to vendor lock-in, it's important to remember this:
akortunov wrote: 03 Oct 2018, 05:54 I am pretty sure that at least 90% of MWSE mods, which will use custom binaries, will be Windows-only anyway, even if we will not sandbox Lua.
Loading a custom executable or library on a random Linux distribution can be a hell because of software dependencies.
A Windows-only mod certainly sounds like vendor lock-in to me. Does it make sense for OpenMW to support this?
User avatar
SeaFox
Posts: 34
Joined: 29 Feb 2016, 17:30

Re: The new MWSE-Lua interface

Post by SeaFox »

CMAugust wrote: 04 Oct 2018, 02:29 When it comes to vendor lock-in, it's important to remember this:
akortunov wrote: 03 Oct 2018, 05:54 I am pretty sure that at least 90% of MWSE mods, which will use custom binaries, will be Windows-only anyway, even if we will not sandbox Lua.
Loading a custom executable or library on a random Linux distribution can be a hell because of software dependencies.
A Windows-only mod certainly sounds like vendor lock-in to me. Does it make sense for OpenMW to support this?
The dominance of Windows does that. People who use MWSE are going to make windows only mods anyway because MWSE is windows only. Having an official fork of OpenMW which supports the plugin archetecture of MWSE does not incentivise Windows-only mods. Instead, it would incentivise them to make mods which work with OpenMW, and to even provide source code so that others can compile for other platforms.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: The new MWSE-Lua interface

Post by Chris »

SeaFox wrote: 04 Oct 2018, 01:09 I understand that point of view, but disagree. Obviously we all want OpenMW to succeed, but using "vendor lock-in" seems like a bad strategy for promoting OpenMW and gaining new developers.
It's not vendor lock-in considering OpenMW is open source. Anyone can take OpenMW's source, fork it, and become a new "vendor". Anyone can duplicate OpenMW's modding extensions in their own engine. Similarly, this isn't "use OpenMW or else", it's "if you use OpenMW, we're doing things this way".

Currently OpenMW seeks to be compatible with vanilla Morrowind, and after 1.0 will add new modding capabilities. Some compatibility with MWSE mods is also something for after 1.0 (both preexisting mods that use the extended vanilla scripts, to a degree, and potentially MWSE-Lua scripts). The only thing is, OpenMW is going to sandbox its use of Lua for stability, compatibility, and security reasons.
In my opinion, more people will come to OpenMW by giving them more options for making mods, not less.
Giving people more slack to do anything they want sounds all fine and dandy, until they get enough slack that they end up hanging themselves.

This isn't an issue unique to OpenMW/Morrowind modding. When the Doom engine got open sourced, similar issues cropped up between certain "source ports" where some allowed native code plugins, and others didn't. One of the biggest engines to support native code plugins was Doomsday, and in its day was quite popular with modders. One of the bigger engines that refused to support native-code plugins was ZDoom.

Over the years, a number of big mods were made for Doomsday. But the engine was always plagued with stability issues as a result of native code plugins. Many mods were Windows-specific, because the modders were unwilling or unable to make Linux or Mac plugins. OS updates would occasionally break plugins too. There was nothing the engine developers could do about these issues because it didn't involve their code. Unfortunately this also means some mods no longer work and can never be made to work again, because something broke and the modder has moved on.

These days, Doomsday is rarely used. ZDoom* is what many modders target, because it offers a wealth of modding extensions and continues gaining new features that modders want. It developed compatibility with certain features of other source ports where it could (Doomsday's hi-res texture pack and model support, Doom Legacy's custom FraggleScript, etc). The engine is very stable, the vast majority of mods made for it continue working, new features are still being added, and it all works for everybody regardless of their preferred system. The engine's even been used in some recent commercial games.

* Technically, GZDoom is what many use. It was created as a fork of ZDoom that added an OpenGL renderer that the ZDoom dev wasn't interested in adding. The two developed side-by-side and often in collaboration (the GZDoom dev even eventually got write privs for ZDoom's repo). However the main ZDoom dev has since moved on to other things, and rather than take over someone else's project, GZDoom has effectively become its successor as it continues development in the same vein.
Post Reply