Post 1.0 Planning Update

Anything related to PR, release planning and any other non-technical idea how to move the project forward should be discussed here.
NullCascade
Posts: 120
Joined: 16 Jan 2012, 07:58

Re: Post 1.0 Planning Update

Post by NullCascade » 04 Apr 2018, 19:40

Zini wrote:
04 Apr 2018, 10:54
Regarding MWSE: As discussed before that isn't an option, but this design document should come fairly close to cover all features of MWSE. We can hope that at least some of the MWSE mods will be updated for OpenMW eventually (trivial in some cases, a lot of work in others).
If there's anything I can do to help there, let me know. MWSE 2.1's upcoming release brings Lua scripting support (LuaJIT w/full FFI support), would be kind of amazing if you were heading in a similar direction and we could make our APIs compatible.
MWSE 2.x lead.

User avatar
AnyOldName3
Posts: 1477
Joined: 26 Nov 2015, 03:25

Re: Post 1.0 Planning Update

Post by AnyOldName3 » 04 Apr 2018, 21:20

Zini is very anti-'calling arbitrary C functions from within OpenMW scripts' even though there are some of us who see it as vital if OpenMW is to become the dominant Morrowind engine. I doubt that the APIs will be compatible unless either he can be persuaded that users who'd download mods that pose a security risk would find easier ways to install malware or someone forks OpenMW, which will be a horrible mess and should be avoided.
AnyOldName3, Master of Shadows

User avatar
lysol
Posts: 1289
Joined: 26 Mar 2013, 01:48
Location: Sweden

Re: Post 1.0 Planning Update

Post by lysol » 04 Apr 2018, 21:32

AnyOldName3 wrote:
04 Apr 2018, 21:20
Zini is very anti-'calling arbitrary C functions from within OpenMW scripts' even though there are some of us who see it as vital if OpenMW is to become the dominant Morrowind engine. I doubt that the APIs will be compatible unless either he can be persuaded that users who'd download mods that pose a security risk would find easier ways to install malware or someone forks OpenMW, which will be a horrible mess and should be avoided.
Would it be possible to implement some kind of security-ish functions, something like:

1. Users have to "activate" advanced-but-potentially-harmful scripts in settings.cfg
2. When running the game with a mod that uses the above mentioned scripts, you'll end up with a pop-up saying "Warning: sneakytrojanmod.omwaddon wants to run advanced scripts that can potentially be harmful to your computer. Only allow this if you trust the author of the mod. Allow? Y/N".

Kind of like UAC on windows and the root access password pop-up in linux distros.

Or is this stupid/hard/both?
Normal mapped texture replacers, exclusive for OpenMW:
My Nexus page

NullCascade
Posts: 120
Joined: 16 Jan 2012, 07:58

Re: Post 1.0 Planning Update

Post by NullCascade » 04 Apr 2018, 22:23

lysol wrote:
04 Apr 2018, 21:32
AnyOldName3 wrote:
04 Apr 2018, 21:20
Zini is very anti-'calling arbitrary C functions from within OpenMW scripts' even though there are some of us who see it as vital if OpenMW is to become the dominant Morrowind engine. I doubt that the APIs will be compatible unless either he can be persuaded that users who'd download mods that pose a security risk would find easier ways to install malware or someone forks OpenMW, which will be a horrible mess and should be avoided.
Would it be possible to implement some kind of security-ish functions, something like:

1. Users have to "activate" advanced-but-potentially-harmful scripts in settings.cfg
2. When running the game with a mod that uses the above mentioned scripts, you'll end up with a pop-up saying "Warning: sneakytrojanmod.omwaddon wants to run advanced scripts that can potentially be harmful to your computer. Only allow this if you trust the author of the mod. Allow? Y/N".

Kind of like UAC on windows and the root access password pop-up in linux distros.

Or is this stupid/hard/both?
It's not stupid/harsh, it's just that it's a lot of engineering cycles for mostly a "feel good" moment. If you've convinced someone to download your malicious thing from the internet, you've probably already won. An Android or UAC-style security measure is only going to massively complicate things while not really making you much more secure. Like, telling Facebook on your phone that it can't have access to your contacts isn't going to significantly improve your security -- you still are choosing to download and use the app, and it's probably doing plenty you don't necessarily want.

As for calling arbitrary C functions from within engines, well... that's kind of how Oblivion and Skyrim keep coming out with awesome mods that you can't make in Morrowind (or couldn't, until recently). You can do some sandboxing if it makes you feel better, but never in the history of TES modding has someone delivered a malicious OBSE/SKSE plugin, mod repository sites (like the Nexus) would catch those things quickly if they did, and I don't feel like putting those restrictions on modders is going to be in line with the screenshot in the first post. If you want to incentivize and empower modders, you need to give them some power and trust. If you keep them super locked down in your engine, their creations are limited by your imagination and your API, not their own imagination and whatever they can make work.

Just my $0.02 though. I'd just love it if modders could make a fancy mod and have it run on both engines. With tes3mp and MWSE both allowing for Lua scripting, I'd happily work on OpenMW's scripting efforts if it meant some level of synergy for modders that work between the three projects.
MWSE 2.x lead.

User avatar
Zini
Posts: 5536
Joined: 06 Aug 2011, 15:16

Re: Post 1.0 Planning Update

Post by Zini » 05 Apr 2018, 12:45

Adding the ability to execute arbitrary code from a script is not going to happen as long as I have anything to say. Sorry, to be harsh here, but security over functionality any time.

That being said the situation with OpenMW is completely different from Morrowind/Oblivion/Skyrim. We have access to the source, meaning that if there is a demand for a feature that can't be done from the scripting system, (at least in general) we can enhance the engine in a way that the feature can be done from the scripting system.

User avatar
heilkitty
Posts: 129
Joined: 11 Aug 2011, 07:57
Location: Vivec City, MW
Contact:

Re: Post 1.0 Planning Update

Post by heilkitty » 05 Apr 2018, 13:40

Zini wrote:
05 Apr 2018, 12:45
security over functionality any time
To be honest, I don't think there's so much functionality in executing native code, aside from rather superficial things like that discord integration and not superficial things like prototyping stuff. Lua support, on the other hand, would be nice, IMHO.
In Soviet MoЯЯowind, Almalexia kills YOU!!

NullCascade
Posts: 120
Joined: 16 Jan 2012, 07:58

Re: Post 1.0 Planning Update

Post by NullCascade » 05 Apr 2018, 16:39

Zini wrote:
05 Apr 2018, 12:45
Adding the ability to execute arbitrary code from a script is not going to happen as long as I have anything to say. Sorry, to be harsh here, but security over functionality any time.

That being said the situation with OpenMW is completely different from Morrowind/Oblivion/Skyrim. We have access to the source, meaning that if there is a demand for a feature that can't be done from the scripting system, (at least in general) we can enhance the engine in a way that the feature can be done from the scripting system.
This is extremely untenable. OpenMW wasn't written with dehardcoding in mind from the start, meaning each bit of dehardcoding is going to be an uphill battle, refactoring potentially major parts of the engine if you want to expose things to modders. This is compounded harshly if you still plan on updating mwscript to try to cram new features into it (which very likely will cause regressions with old mods that relied on Bethesda's terrible CS compiler if you continue to ignore the bytecode for legacy mods).

And... recompiling from source, merging mods like that is just really, really unappealing. It'd be like playing Skyrim, and needing to merge sources and recompile (which with the Linux-focused nature of the dev environment, is a pain on Windows/macOS, normal modders aren't going to want to touch it with a 10' pole) if I want to install Frostfall and SkyrimSouls at the same time. Almost no one is going to want to do that.

Is this just straight-up not up for discussion? I'd really like to help here, but your response is very discouraging. I can't think of any major engine out there that has such a limited scripting environment as to prevent the game from running code. You say you want to make OpenMW the go-to engine to play Morrowind with. And, you're right, the way to do that is to let you do things in OpenMW that you could never do in the original engine. I'd like to see this too, and think OpenMW is definitely the future of Morrowind modding. But how are you going to compete if it has fewer modding opportunities than the original engine + MWSE? This is a major limitation of the engine all in the name of the illusion of security.

At the very least, can it just be an option that mod-users have to enable? LuaJIT's FFI can be disabled, and Lua is very friendly to sandboxing. I worked as an engineer at a cross-platform game engine company that exposed Lua to our users. Again, I'd really love to help with this stuff, and would hate for OpenMW to stick with the horror that is mwscript.
Last edited by NullCascade on 05 Apr 2018, 17:22, edited 1 time in total.
MWSE 2.x lead.

User avatar
Jemolk
Posts: 180
Joined: 20 Mar 2017, 02:12
Location: Sadrith Mora

Re: Post 1.0 Planning Update

Post by Jemolk » 05 Apr 2018, 16:57

Zini wrote:
05 Apr 2018, 12:45
Adding the ability to execute arbitrary code from a script is not going to happen as long as I have anything to say. Sorry, to be harsh here, but security over functionality any time.

That being said the situation with OpenMW is completely different from Morrowind/Oblivion/Skyrim. We have access to the source, meaning that if there is a demand for a feature that can't be done from the scripting system, (at least in general) we can enhance the engine in a way that the feature can be done from the scripting system.
Just to reiterate something from NullCascade -- If the malware author has already convinced someone to do something truly stupid and destructive, we simply cannot place enough barriers in the fool's way to stop them without crippling the sane, non-idiot users. Programmers are never going to be able to make something truly foolproof -- the universe just keeps developing bigger fools. This merely ensures that OpenMW never fully supplants the vanilla engine -- though a fork might, if said fork adds the functionality that you insist against.

While we're on the topic, MWSE script compatibility should be simple enough. We will simply have to define functions that do the same things as MWSE functions. They don't have to work in the exact same way, they just need to produce the same result with the input given to the MWSE versions. Then all that's left is changing the names in scripts from the MWSE names to the OpenMW versions. Defining certain functions in advance shouldn't produce any real problems with more open-ended functionality, and as I said, they'd mainly be there so that MWSE mods would be easy to convert. This would have a far greater effect on OpenMW becoming the engine of choice than you seem to realize. Even just a function that can get the name of a player's current interior cell without having to be fed it first would be absolutely massive.

User avatar
AnyOldName3
Posts: 1477
Joined: 26 Nov 2015, 03:25

Re: Post 1.0 Planning Update

Post by AnyOldName3 » 05 Apr 2018, 17:38

Many MWSE functions can't be directly supported as they work by exposing internal engine behaviour for the original engine, but that behaviour is different (and often better) in OpenMW.

As a hypothetical example that might actually exist but I don't know if it does actually exist, let's say that MWSE has a function that gets the load order index of the plugin that a specific object came from. It would be perfectly reasonable for MWSE to return an unsigned char or a pointer to an unsigned char, as an unsigned char is eight bits so can count from 0 to 255 and the original engine supports up to 255 plugins being loaded at once. (I'm pretty sure that) OpenMW supports a much larger number of plugins being loaded at once, so suddenly the value returned by such a function won't fit in an unsigned char and so it's no longer the same function.

One possible solution that may somewhat placate Zini's concerns would be to support two types of plugins - game plugins and engine plugins. The former would be safe and sandboxed and would be able to change only the things exposed by OpenMW (or installed engine plugins). Engine plugins, however, would have the freedom to call native code, be written in precompiled languages, and override big chunks of OpenMW. For example, the animation system could be replaced so that it supports hair and cloth physics for characters (although this isn't the best example as I'm pretty sure that if someone made a PR, that would get merged).

I suppose that a big problem here is that some of us are making the assumption that there'll be engine modifications that players will want to install, but Zini won't want to merge. I'm struggling to think of any mods I know of that require their game's script extender that would still require a script extender if the original engine was more flexible but wouldn't be considered mergeable even if people wanted to use it. Without such an example, the assertion that running arbitrary code is a necessary feature doesn't hold much weight.
AnyOldName3, Master of Shadows

fraang
Posts: 12
Joined: 26 Apr 2013, 13:48

Re: Post 1.0 Planning Update

Post by fraang » 05 Apr 2018, 17:45

NullCascade wrote:
05 Apr 2018, 16:39
This is extremely untenable. OpenMW wasn't written with dehardcoding in mind from the start, meaning each bit of dehardcoding is going to be an uphill battle, refactoring potentially major parts of the engine if you want to expose things to modders.
What is the problem with properly dehardcoding stuff instead of ugly hacks? Yes this is time consuming but the only sane way to do it.
NullCascade wrote:
05 Apr 2018, 16:39
I can't think of any major engine out there that has such a limited scripting environment as to prevent the game from running code.
In what use case do you need to access stuff outside the engine from mwscripts?

Post Reply