The Future of OpenMW Scripting

Everything about development and the OpenMW source code.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: The Future of OpenMW Scripting

Post by psi29a »

Just had an interesting conversation and thought I'd share a bit here about MWSE compatibility.

So instead of extending oldscript to support mwse, in effect creating oldscript+ which we don't want to do, we just freeze oldscript. No more development on that front.

We build out newscript.

Then we see about a translator that could convert common mwse extensions into newscript. Just to get this "on paper", Zini replied with:
Zini wrote:That would require a pretty advanced translator, but if people want to put the time and effort into it I have no problem with that.
So this is a possible way forward for MWSE support in OpenMW if anyone would like to tackle that.
User avatar
faerietree
Posts: 14
Joined: 14 Aug 2018, 19:33
Contact:

Re: The Future of OpenMW Scripting

Post by faerietree »

Interesting read.
I think what goes hand in hand with the scripting decision is the modularity.
Of course not of the engine code. Rather of the content.
Multiple data folders are already one of my favorite changes compared to vanilla MW engine. ;)

No matter which scripting language, and of course limiting OS,filesystem libs makes sense for security reasons, yet it should have read only access to files.
Are JSON or XML files or file references considered?
Currently content looks bundled in ESM|omwgame, ESP|omwaddon files. Scripts can be embedded but need not necessarily, at least I remember references to external script files on the wiki somewhere. But content like which model uses which texture and which variation is not mapped. As are base skills and different predefined characters or am I wrong?

Defining content separately (in XML, JSON) sounds a bit more friendly to me for the following reasons:
  • readability
  • re-usability and
  • version control
While we at solution C and a translator anyway, could we not generate the omw{game,addon} files via a "combinator" like tool instead of hard-coding it in a binary format. Okay, I know blender also uses binary and it has advantages but for game source files I feel like a more modular approach is more desirable.

Is there any chance on a 1vs.1 comparison that makes it easier to carve in stone that Lua is really more desirable than oldscript+? When people talk about Lua then LuaJIT for sure as without just in time compiling Lua generally falls behind highly optimized scripting languages like v8.JavaScript (due to google) or SpiderMonkey.JavaScript (due to mozilla) or PyPy.

Apart from 0 vs 1 based arrays what I like about JavaScript is
* Both the V8 and SpiderMonkey implementation are free and highly maintained.
* Javascript is everywhere there is a browser. People could test and debug their mod's logic in the browser.
* source files may be smaller in size due to {} vs begin/end

Generally I try to be neutral to Java and not join the hatred to it. I heard something good about Clojure and especially Go, but of course these are full blown languages.

Python, especially PyPy, also looks very nice and is very short syntax and well established numeric libraries numpy, grumpy, ..., but if it can not be sandboxed and this is important to OpenMW then it will - as so often and even more since Py2 and 3 mess - have less chance to be adapted than AngelScript or maybe even BASIC or hey GUI enthusiasts out there, what about Lava ;-).
I like Ada as much as C and Go, Python and JavaScript (yes even if object orientation is strange and debugging asynchronous code is not easy but still it gets useful tasks done, consider browser addons).
My preferred route to create using OpenMW is to code C++. I plan on content mods only and provide functionality only once the scripting side and planned API is really stable which I'm not sure when to expect, maybe post 2.0, i.e. 2.1?


Btw. anyone can explain why wrapper code and an API is strictly necessary considering that LuaJIT can hook directly into function calls if the binary is compiled as a shared library. I guess it is safety, but still, someone could provide a mod with LuaJIT calling any C++ functions of libs instead of only those in the (potentially superfluous?) wrapper code anyway (have a look at ffi).
Isn't the C++ code to be kept safe anyway? This then meant we could save developer time. No wrappers to maintain. Full freedom for modders. Or is this overly idealistic?
User avatar
AnyOldName3
Posts: 2666
Joined: 26 Nov 2015, 03:25

Re: The Future of OpenMW Scripting

Post by AnyOldName3 »

We're not going to include JavaScript. If we start seriously considering it, I'm going to hold shadows hostage until we stop. It's an evil, evil language.
User avatar
raevol
Posts: 3093
Joined: 07 Aug 2011, 01:12
Location: Caldera

Re: The Future of OpenMW Scripting

Post by raevol »

AnyOldName3 wrote: 14 Aug 2018, 20:42 We're not going to include JavaScript. If we start seriously considering it, I'm going to hold shadows hostage until we stop. It's an evil, evil language.
I, also, will probably choose to leave the project if we choose javascript.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: The Future of OpenMW Scripting

Post by psi29a »

Had this run it's course yet? I mean... no one has offered up any convincing technical and social arguments for another scripting language to take the place of Lua.
User avatar
silentthief
Posts: 456
Joined: 18 Apr 2013, 01:20
Location: Currently traversing the Ascadian Isles

Re: The Future of OpenMW Scripting

Post by silentthief »

psi29a wrote: 15 Aug 2018, 09:41 Had this run it's course yet? I mean... no one has offered up any convincing technical and social arguments for another scripting language to take the place of Lua.
Not a dev - just some guy who throws my 2 cents in now and then (or occasionally help with scripting - from vanilla standpoint, or perhaps a funny post).

But - I agree with this completely. I don't think there is any merit for any scripting language unless it is for something like MWSE compatibility (*which is why I thought that LUA was decided on in the first place - and assuming that that is still a desired outcome). If not - then simply oldscript+ would suffice (and I would assume be easier to manage).

If someone wants an alternate language - then I assume the option to do a pull request and start coding a branch still exists...

ST
User avatar
AnyOldName3
Posts: 2666
Joined: 26 Nov 2015, 03:25

Re: The Future of OpenMW Scripting

Post by AnyOldName3 »

My main complaints about OldScript+ were:
  • Its weird syntax - that's just going to catch people out and annoy experienced programmers.
  • Its total lack of pre-existing documentation, tutorials, StackOverflow filled with beginner questions and example code for common situations, and other people completely unrelated to OpenMW willing to provide support for it.
  • The maintenance burden of doing everything ourselves from scratch.
As the average modder is not as good at programming as the average OpenMW developer, and the average person who wants to make a mod, but will give up if it turns out to be hard has even less skill. These are the people we need to cater for, as most people can't just look at a couple of simple examples and work out how to do everything complex.
User avatar
Thunderforge
Posts: 503
Joined: 06 Jun 2017, 05:57

Re: The Future of OpenMW Scripting

Post by Thunderforge »

I agree, the discussion has run its course. Things we have come to a consensus on:
  • Oldscript+ is not desirable
  • Newscript is the way to go
  • Regarding which language to implement Newscript in:
    • Key developers are vetoing JavaScript (and it is suboptimal for a number of reasons)
    • AngelScript has what we are looking for, but seems too niche to use
    • Python lacks sandboxing
    • Lua is popular, has the functionality we need, is easy to embed, includes sandboxing, and is already used by TES3MP and MWSE. Therefore, it is going to be the one that Newscript will be written in (unless something changes in the future).
I don't think that any further discussion will change these points. Therefore, I'm going to lock this topic.

If you think you have something new to add, create a new topic. If it is just a rehash of what we've been saying for the last 13 pages, then that will be locked as well.
Locked