Page 3 of 13

Re: The Future of OpenMW Scripting

Posted: 15 Jun 2018, 10:05
by werdanith
I'm aware that I haven't contributed to the project in a while, but in the spirit of discussion I'd like to make a case specifically against the proliferation of additional competing standards.

The fact is that while many people are looking forward to OpenMW 1.0+ for all its benefits, fewer ones are anticipating oldscript+ primarily because nobody knows its planned capabilities and what it will look like. For now and the immediate future, if a modder wants to write a mod that utilizes advanced scripting features, they're faced with either the old MWSE or the new Lua/MWSE, both of which are documented and community-supported, to an extent.

Even if oldscript+ is engineered to be a technically sound, featureful and the Right Tool For The Job™, it's still a new tool for people to learn and documentation to be written for. A new potential wedge to separate the target audience. Quite a deterrent, that could lead to a chicken and egg problem with regards to adoption.

Regardless of method of implementation, I think Lua is the way to go because it is currently the way forward for both MWSE and TES3MP, as well as being an established game scripting language in both code and documentation.

Re: The Future of OpenMW Scripting

Posted: 15 Jun 2018, 10:09
by psi29a
werdanith wrote: 15 Jun 2018, 10:05Even if oldscript+ is engineered to be a technically sound, featureful and the Right Tool For The Job™, it's still a new tool for people to learn and documentation to be written for. A new potential wedge to separate the target audience. Quite a deterrent, that could lead to a chicken and egg problem with regards to adoption.

Regardless of method of implementation, I think Lua is the way to go because it is currently the way forward for both MWSE and TES3MP, as well as being an established game scripting language in both code and documentation.
This is pretty much my thought as well. I don't think it is OpenMW's best interest to be the _only_ game engine that supports oldscript+.

For me, supporting oldscript via newscript (however that happens) is the way to go. Not just for Morrowind but for other Beth games as well.

Re: The Future of OpenMW Scripting

Posted: 15 Jun 2018, 10:30
by Shnatsel
As someone who has a teeny tiny bit of experience writing translators, I can tell you this is not really a fun prospect. There is a reason there is still no functional translators between languages out there, despite the presence of numerous compilers. The only actually functional translator out there is emscripten and it targets an assembly-like subset of a programming language anyway. Targeting a human-readable programming language is a pain in the tail because sometimes there is just no corresponding construct in the target language that would preserve all the semantics of the source language, so the translation is imperfect; you end up having to choose between less imperfect but not human readable/editable version and a more imperfect but human-readable/editable version, and the "I just want to get my script to run!" use case is poorly served by either of these options.

It would be simpler to target an existing VM instruction set than to target a programming language; you could get oldscript to run in the same VM as newscript. That would reduce oldscript compiler to just the parser, leaving the optimization and code-generation passes up to an existing VM (but I guess you already do that, right?). You can also get oldscript and newscript able to call into each other this way, but I doubt that would be actually useful. Sadly, going down this route would limit newscript choice because there aren't all that many general-purpose VMs out there - I can name .net/mono, java VM, LLVM, and that new shiny polyglot VM from Oracle, and all of them sound like a terrible choice. And hooking into a VM not designed for such shenanigans is probably more trouble than it's worth.

At this point I am genuinely curious how MWSE/Lua managed to sidestep the interoperability issues. If Lua that works well enough when bolted on the side of the original engine, it surely can be made to work well in OpenMW in parallel to oldscript compiler/interpreter.

Also, If you're serious about error reporting, debugging, and other tooling for oldscript+, I've got bad news for you: you're never going to even come close to an established language in terms of tooling quality (error reporting, debugging, execution tracing, profiling, static analysis, etc) - not by virtue of being bad at it, but simply by virtue of being the only project with that language. Some of the big-name programming languages are actually good because half the frigging world has invested in them, and because they had 10+ years in development and then several more years of actual use in practice to mature. Python was started in 1989 yet barely anyone heard of it until the 2000s and not until it got a major overhaul; Rust was started in 2006 yet it is only becoming usable now; etc. And after all that some of those languages still don't have tooling that's worth a dime (still no tracing for Python, are you bucking kidding me?!). And there's also the issue of documentation; established languages have extensive tutorials, reference guides, howtos, and stackoverflow; oldscript+ got nothing. This is one more area that you'd have to sink a lot of manpower on, and then keep sinking because there is never enough documentation. These issues are going to be exacerbated if an event system ever surfaces, because writing and debugging asynchronous code is almost as hard as parallel code, and writing and debugging parallel code is a lot harder than it seems.

When you could get most of the documentation and vastly superior tooling for free, it's really a shame to pass it up. Hell, I bet execution tracing or profiling were not even on the roadmap for oldscript+, and developers are going to need them for debugging event-driven scripts, if those ever happen.

Based on the above, I see option B as the only feasible one. Although turning oldscript interpreter into a frontend for newscript VM might be worth looking into.

Re: The Future of OpenMW Scripting

Posted: 15 Jun 2018, 12:19
by AnyOldName3
It would be simpler to target an existing VM instruction set than to target a programming language; you could get oldscript to run in the same VM as newscript. That would reduce oldscript compiler to just the parser, leaving the optimization and code-generation passes up to an existing VM (but I guess you already do that, right?). You can also get oldscript and newscript able to call into each other this way, but I doubt that would be actually useful. Sadly, going down this route would limit newscript choice because there aren't all that many general-purpose VMs out there - I can name .net/mono, java VM, LLVM, and that new shiny polyglot VM from Oracle, and all of them sound like a terrible choice. And hooking into a VM not designed for such shenanigans is probably more trouble than it's worth.
Lua runs on a VM, so it's not like we'd have to translate OldScript to NewScript as we could go straight to NewByteCode

Re: The Future of OpenMW Scripting

Posted: 15 Jun 2018, 15:50
by Chris
raevol wrote: 15 Jun 2018, 04:06 Option C will still let us play Morrowind on OpenMW right? The translator will be able to take the scripts from the original content and convert them into newscript?
Yes, I don't imagine any option that would break vanilla compatibility is ever going to be considered. It's just a question of how the original content will be handled, and how to evolve the engine's scripting capabilities going into the future.

Re: The Future of OpenMW Scripting

Posted: 16 Jun 2018, 08:48
by raven
If there is so much interest in Lua, one of the Lua proponents could implement Lua bindings(there are numerous binding libs to help with this) and rewrite current script compiler generate Lua bytecode.

It shouldn't be too hard given how primitive mw script is. And you'd have something tangible to argue about. ;)

Re: The Future of OpenMW Scripting

Posted: 16 Jun 2018, 08:50
by NullCascade
raven wrote: 16 Jun 2018, 08:48If there is so much interest in Lua, one of the Lua proponents could implement Lua bindings(there are numerous binding libs to help with this) and rewrite current script compiler generate Lua bytecode.

It shouldn't be too hard given how primitive mw script is. And you'd have something tangible to argue about. ;)
People have added Python and Lua support before. It was never merged. The problem isn't getting people to do the work, it's getting the project to accept it.

Re: The Future of OpenMW Scripting

Posted: 16 Jun 2018, 09:05
by psi29a
NullCascade wrote: 16 Jun 2018, 08:50People have added Python and Lua support before. It was never merged. The problem isn't getting people to do the work, it's getting the project to accept it.
Python, yes, to see if it could be done. The developer, a friend of mine who is a Debian Developer, did it to see if it was possible, he wasn't serious about it. You _can't_ practically sandbox python so that's a no go for OpenMW but not for OpenMW-CS where it can be used.

Can you provide the PR or MR of the Lua support because of the all the years I've been with the project, I've never see it.

Re: The Future of OpenMW Scripting

Posted: 16 Jun 2018, 10:00
by NullCascade
psi29a wrote: 16 Jun 2018, 09:05
NullCascade wrote: 16 Jun 2018, 08:50People have added Python and Lua support before. It was never merged. The problem isn't getting people to do the work, it's getting the project to accept it.
Python, yes, to see if it could be done. The developer, a friend of mine who is a Debian Developer, did it to see if it was possible, he wasn't serious about it. You _can't_ practically sandbox python so that's a no go for OpenMW but not for OpenMW-CS where it can be used.

Can you provide the PR or MR of the Lua support because of the all the years I've been with the project, I've never see it.
I haven't monitored PRs, but this is on the wiki: OpenMW: Scripting Extension.

Re: The Future of OpenMW Scripting

Posted: 16 Jun 2018, 10:52
by psi29a
https://wiki.openmw.org/index.php?title ... on=history

Yeah, that is by Maqifrnswa aka Scott Howard our friendly Debian Developer. There was no Lua work done, just his observations in working with python and how it could work with adding Lua to OpenMW.

Here is when he added those: 10 February 2015
https://wiki.openmw.org/index.php?title ... oldid=2072

Lua work has never been done so it couldn't have been rejected either.