The new MWSE-Lua interface

Everything about development and the OpenMW source code.
Chris
Posts: 1626
Joined: 04 Sep 2011, 08:33

Re: The new MWSE-Lua interface

Post by Chris »

SeaFox wrote: 05 Oct 2018, 23:46 Yes, but as I said before:
And as I said, that's entirely self-defeating.
User avatar
SeaFox
Posts: 34
Joined: 29 Feb 2016, 17:30

Re: The new MWSE-Lua interface

Post by SeaFox »

Chris wrote: 05 Oct 2018, 23:54
SeaFox wrote: 05 Oct 2018, 23:46 Yes, but as I said before:
And as I said, that's entirely self-defeating.
Not really. Nexus scans for viruses in mods, so the risk is minimal. Also, since sandboxing remains the default behavior, anyone who disables it would be fully responsible for any risk that they are taking.

You could even flash a big warning message on the screen:

WARNING: You may be endangering your data, your computer, or even your life by disabling sand-boxing. Are you absolutely certain you wish to continue?
davidcernat
Posts: 256
Joined: 19 Jul 2016, 01:02

Re: The new MWSE-Lua interface

Post by davidcernat »

NullCascade wrote: 04 Oct 2018, 11:41
Chris wrote: 04 Oct 2018, 05:03Giving people more slack to do anything they want sounds all fine and dandy, until they get enough slack that they end up hanging themselves.
You're probably right. I went and exposed the Morrowind scene graph to people, and they went and did the worst shit, like releasing weapon sheathing and state-based models for meshes that change depending on time of day. All this capability to innovate will be the death of modding. Think of what people could truly accomplish if they had a well-regulated interface, or had to fork the entire engine instead.
Those can be done just fine in sandboxed Lua with a few script functions that let you attach or replace meshes, i.e. you added some mesh-related C++ code to MWSE that could be called from your Lua scripts and now OpenMW can add some somewhat similar (but much more elegant) C++ code that can be called from its Lua scripts to achieve the same functionality.

Similarly, the mod this thread was initially about (Tamrielic Lore Tooltips) was made possible by your addition of GUI-related C++ code to MWSE that allowed for GUI manipulation in your Lua scripts, which OpenMW can have its own equivalent C++ code for that can be called from its own Lua scripts.

Can you come up with some actual examples that really require a non-sandboxed environment? The examples I remember from you seemed so lightweight so as to make your whole obsession with the topic seem a little silly.
Cammera
Posts: 19
Joined: 08 Oct 2017, 22:35

Re: The new MWSE-Lua interface

Post by Cammera »

Nothing sekrit about OpenMW, it's not developed in a vacuum. I believe there are many knee-jerk reactions here _or_ that reading comprehension skills need sharpening.
No, nothing sekrit, you just have to:
A)Learn everything about a complex engine's internals and then hope the klub likes it and accepts the puil req
or
B)Ask nicely that the klub does it and hopes they like the idea.
or
C) Compile your own fork and then ask that people who wants your mod uses it, which makes it much harder (Actively impossible without C knowledge) to use other forks.
It's not a sekrit klub, granted, I guess there's windows in the klubhouse so you can see in and take pictures.
akortunov wrote: 05 Oct 2018, 18:04 A trade-off approach:
1. Implement a Windows32-only compile-time flag which determines if Lua should be sandboxed (since modders will not bother to support other OS-es anyway).
FTFY:
"People will only do this thing, so better only let them do this specific thing because else they might not and I might have to eat my words, which is the worst possible thing." Also only 32 bits, seriously? Why limit that aspect arbitrarily too?
I'll assume this one was a joke, but to be honest I am not wholly convinced.
OpenMW has never rejected your offer to work with OpenMW
To my knowledge Zini never replied to Null's offer to help him with the unified API. That sounds like a pretty rotund (Weakspinned, but still rotund) "No" if you ask me.
Instead of helping build OpenMW, you would rather bring it down. If that makes you happy, cool, OpenMW wishes that you keep making MWSE and Morrowind awesome anyway.
Who' bringing down what? Who said they'd bring down anything?
M8, if bringing such a humongous and well organized project as OpenMW is as easy as offering a rational, modder-friendly approach to modding that took... seven-ish months of development? Then the problem here isn't NullCascade's actions.

(EDIT:)
Besides, giving the possibility of unsandboxed mods (I support the choice of having the sandbox be an opt-out choice, for the record, it does seem like a good middle ground) is simply the optimal way to go. It'd give modders the possibility of loading apis on and off quickly, and not having to recompile the entire engine every time they change something would provide a massively better workflow. Not to mention that for every extra step you add to anything more people will give up and go elsewhere.

I do support the base engine adding apis over time. I don't support that being the only choice, or walled-garden approaches in general. They're not what makes open source great.
User avatar
AnyOldName3
Posts: 2676
Joined: 26 Nov 2015, 03:25

Re: The new MWSE-Lua interface

Post by AnyOldName3 »

As the most prominent Windows OpenMW dev, I can say that it's easiest to build OpenMW on Windows (when it works).

To build OpenMW on Windows:
  1. Check out the repo with Git.
  2. Using the Bash shell which comes with Windows builds of Git (including the Microsoft-recommended package bundled with Visual Studio), run CI/before_script.msvc.sh (probably with some command line options specifying the Visual Studio version).
  3. Open the Visual Studio solution and build it just like any other piece of software on Windows.
To build OpenMW on Ubuntu:
  1. Check out the repo with Git.
  2. Add the OpenMW PPA to Apt's sources.
  3. Install the OpenMW deps (copying and pasting the command from the Wiki as it's far too long to remember or guess).
  4. Run CMake.
  5. Run make, remembering that it only uses a single core by default, so you need to give it a command line switch to enable a multicore build.
To build OpenMW on a more obscure Linux distro:
  1. Check out the repo with Git.
  2. Hope that your package repo has a dev package for each of OpenMW's dependencies, and for those it does, install them.
  3. For those that it doesn't, search their website for a precompiled version. In the rare cases it exists, install it.
  4. For the rest, download the source and their dependencies (hoping that you don't have to compile those, too). Figure out how to build them and then install them.
  5. Run CMake, hoping it autodetects where the dependencies have ended up, and for those it doesn't, manually specifying where they are with command line options or the CMake GUI.
  6. Run make, remembering that it only uses a single core by default, so you need to give it a command line switch to enable a multicore build.
To build OpenMW on a Mac:
  1. Follow the instructions on the wiki.
  2. When they inevitably fail, ask on Discord and get told to ask Thunderforge because no one else has a clue.
  3. Maybe get things to work eventually.
The idea that we're hostile to Windows developers and cater only to Linux ones is patent bullshit (unless you count Mac users as people - they do have it harder than the average Linux user).

However, there are a couple of caveats:
  • Unless they've fixed it, which they might have, CMake cocks up when setting the debugger working directory for VS 2017 as Microsoft changed which file you have to specify it in since VS 2015. This can be worked around with one right click and four left clicks, though, so isn't the end of the world, and is far less effort than learning GDB on Linux.
  • There seems to be an issue with CMake's OpenSceneGraph detection script meaning it fails on NullCascade's computer sometimes. No one else has reported it, I can't replicate it on four different computers, and technically, as NullCascade hasn't made an actual issue report but just complained on Discord, I have nothing to forward to the CMake people so they can fix it.
NullCascade may have got a bad impression of how things are supposed to work, but there wasn't much we could have done to help him at the time as he went about things the wrong way. The main problem was that for months he didn't even mention the bugs anywhere any OpenMW developers would look, so we didn't know he was having issues. The specific things he was complaining about have now either been fixed ordetermined to be CMake's fault and not something we can fix.

I've worked on Mod Organizer, even trying to get it to build back in the days when Tannin had just abandoned it to start working for the Nexus and it literally could only be built on his machine. I've seen appauling build systems. The only way building OpenMW could be easier would be to nag all of our dependency maintainers until they host Nuget packages, and then include a solution file for each of the tens or hundreds of combinations of Visual Studio version and major dependency versions which we're supposed to maintain compatibility with. This would mean you'd have to keep track of which of these solutions you're using and it would kill the potential for people who prefer other Windows build systems, such as MinGW, Qt Creator or NMake, all of which our current system theoretically supports (although probably not in practice without some tweaking).

Either NullCascade is having this debate in bad faith, with no intention of determining which approach to anything is actually better as long as MWSE looks better to someone who only knows the gist of what's going on, or he has a major issue with reading comprehension and keeping track of OpenMW's (roughly consistent) stance on things, as he keeps bringing up imaginary problems or things which aren't the official stance of OpenMW, showing that they're bad, and basing his whole argument around that.

If an MWSE mod can do something without accessing arbitrary files on disk, sending arbitrary data across a network, or executing C/C++ code provided by the mod, an equivalent (maybe even compatible) option will be made available in OpenMW.

If building OpenMW on Windows is not completely painless for you (excluding Visual Studio 2017 needing to be reinstalled six times because the installer is buggy) we will help you get it working, and if necessary, make changes so that no one ever encounters the same issue again.
Chris
Posts: 1626
Joined: 04 Sep 2011, 08:33

Re: The new MWSE-Lua interface

Post by Chris »

SeaFox wrote: 06 Oct 2018, 00:02 You could even flash a big warning message on the screen:
Windows proves this doesn't do anything. If someone has already decided to toggle the option, they'll click through any warning or message to get the mod to work (and may even complain that there's a warning or message in the way). And modders that want to do something more quickly with a plugin rather than improving the engine (or utilizing a different preexisting method that may be a little less elegant or a little more work to get the same effect), they'll be encouraging it.
Cammera wrote: 06 Oct 2018, 01:19 To my knowledge Zini never replied to Null's offer to help him with the unified API. That sounds like a pretty rotund (Weakspinned, but still rotund) "No" if you ask me.
Currently OpenMW is focused on getting 1.0 features done, which is where many devs are focused. That doesn't mean there can't be discussions or working out a design, but you can't expect the devs to constantly jump into discussions about post-1.0 things.

As it is, OpenMW has already bent on supporting MWSE (mostly, somewhat*), partly at the behest of NullCascade and others, where previously the idea was sketchy because Zini worried it could interfere with improvements. OpenMW has also already bent on using Lua, again at the behest of NullCascade and others, rather than improving on the vanilla scripting which Zini was pushing strongly for.

But because sandboxing is the one non-negotiable thing for mod compatibility, stability, and security reasons, OpenMW is not willing to work with others and is trying to force everyone to do things its way?

* Full support possibly won't happen since certain MWSE extensions could have dependencies on the vanilla engine implementation details that may not have an analogue in OpenMW. But we'll still do what we can.
davidcernat
Posts: 256
Joined: 19 Jul 2016, 01:02

Re: The new MWSE-Lua interface

Post by davidcernat »

Cammera wrote: 06 Oct 2018, 01:19
Nothing sekrit about OpenMW, it's not developed in a vacuum. I believe there are many knee-jerk reactions here _or_ that reading comprehension skills need sharpening.
No, nothing sekrit, you just have to:
A)Learn everything about a complex engine's internals and then hope the klub likes it and accepts the puil req
or
B)Ask nicely that the klub does it and hopes they like the idea.
or
C) Compile your own fork and then ask that people who wants your mod uses it, which makes it much harder (Actively impossible without C knowledge) to use other forks.
That's no different from having to add APIs to MWSE before you can use them, because you can't just magically use some functionality you've thought up out of thin air unless Null has actually made it possible in his C++ code first, with the MWSE C++ code being much harder to contribute to than the OpenMW C++ code is.

The only exceptions are out-of-engine libraries doing out-of-engine stuff. How many of those do you really use and need? Give me examples.
Cammera wrote: 06 Oct 2018, 01:19
OpenMW has never rejected your offer to work with OpenMW
To my knowledge Zini never replied to Null's offer to help him with the unified API. That sounds like a pretty rotund (Weakspinned, but still rotund) "No" if you ask me.
I'm sure Null would have had much greater success if he had some basic people skills, instead of – you know – constantly trying to antagonize every developer around. At this point the "oooh, I'm so annoying, why won't you let me help you?" shtick is getting tiresome.
Cammera wrote: 06 Oct 2018, 01:19
Instead of helping build OpenMW, you would rather bring it down. If that makes you happy, cool, OpenMW wishes that you keep making MWSE and Morrowind awesome anyway.
Who' bringing down what? Who said they'd bring down anything?
M8, if bringing such a humongous and well organized project as OpenMW is as easy as offering a rational, modder-friendly approach to modding that took... seven-ish months of development? Then the problem here isn't NullCascade's actions.
psi29a meant bringing it down emotionally. It's one of the possible meanings of "bring down" that you'll find in a dictionary. Considering how very rude Null has been elsewhere, it's certainly an observation that psi29a is entitled to make.

As for the seven-ish months of development... Well, it's a brief victory for quick 'n' dirty hacks instead of serious software. Enjoy it while it lasts.
NullCascade
Posts: 121
Joined: 16 Jan 2012, 07:58

Re: The new MWSE-Lua interface

Post by NullCascade »

davidcernat wrote: 06 Oct 2018, 00:54Can you come up with some actual examples that really require a non-sandboxed environment? The examples I remember from you seemed so lightweight so as to make your whole obsession with the topic seem a little silly.
There are countless libraries out there on a variety of platforms. OpenMW wants to not just be a Morrowind engine replacement, but a fully fledged game engine in its own right. One of the best things we did when I worked on a professional game engine was open our platform up to 3rd party plugins. My so-called obsession comes from this experience, seeing how it affected our product, and working with all the 3rd party developers. It's an absolutely critical feature of any engine. If OpenMW is to succeed, it needs native plugins for both its engine and its editor. Engine bloat isn't very desirable, and not every game, or modded install, needs the same libraries. So how do you figure out a situation where you aren't filling your engine with every library under the sun? How do you alleviate developer bottlenecks so that Zini isn't manually putting stamps of approval on every library to be considered? You use a plugin infrastructure. Every serious game engine does this, and I'm frankly shocked at how OpenMW wants to both be taken seriously as a game engine and hard-core reject such a feature.

What do Oblivion, Skyrim, Minecraft, Terraria, and other modders do? They assemble common code, which doesn't necessarily have anything to do with OpenMW itself, in common libraries that they can share. Every modding scene I've been in as either a modder or a player (with the exclusion of NWN2) has had common library support in native or equivalently powerful (and not sandboxed) code. I'm not going to enumerate the countless possible examples that native extensions can bring to modders. Look at any open source library or software service that could be integrated, or any library that may make creating a particularly difficult system possible. It's not hard.
NullCascade
Posts: 121
Joined: 16 Jan 2012, 07:58

Re: The new MWSE-Lua interface

Post by NullCascade »

davidcernat wrote: 06 Oct 2018, 02:25psi29a meant bringing it down emotionally. It's one of the possible meanings of "bring down" that you'll find in a dictionary. Considering how very rude Null has been elsewhere, it's certainly an observation that psi29a is entitled to make.

As for the seven-ish months of development... Well, it's a brief victory for quick 'n' dirty hacks instead of serious software. Enjoy it while it lasts.
psi29a has been an ass to me here, in PMs, and in Discord. I'm not going to pretend he and I get along, but thanks for bringing that to the forums.
davidcernat
Posts: 256
Joined: 19 Jul 2016, 01:02

Re: The new MWSE-Lua interface

Post by davidcernat »

NullCascade wrote: 06 Oct 2018, 02:30 psi29a has been an ass to me here, in PMs, and in Discord. I'm not going to pretend he and I get along, but thanks for bringing that to the forums.
Is there anyone you get along with these days? I used to have some degree of respectful neutrality towards you and I don't think it's there anymore.
Post Reply