Deno/Javascript scripting in OpenMW

Everything about development and the OpenMW source code.
Post Reply
nif
Posts: 14
Joined: 14 Apr 2015, 03:11

Deno/Javascript scripting in OpenMW

Post by nif »

sorry to fuel more flames, but I wanted to point out there is a secure, standalone 1 file JavaScript runtime environment called Deno

https://github.com/denoland/deno

to quote their readme:

* Secure by default. No file, network, or environment access, unless explicitly enabled.
* Supports TypeScript out of the box.
* Ships only a single executable file.

As a longterm Morrowind fan/OpenMW player and occasional tinkererer, I've played with Lua myself on a few other gamedev hobby projects and can see the merits of its simple syntax and approachability. However, when I tried to get more serious I became frustrated by the issue of versioning specifically; it seems fragmented and the developer ecosystem seems quite a mess actually. My impression was not very good compared to JavaScript/NodeJS where the syntax is equally intuitive and overall just easier to get things done. The JS ecosystem has near unlimited reference/documentation not to mention a million modules if you don't feel like re-inventing the wheel, the landscape is just not as evolved in Lua world and correct me if I'm wrong it doesn't seem that that is changing very fast; if ever. I am about as excited to code more Lua as I am to code Ruby, Perl or PHP (nothing against those just wanted to generalize together some other 'slightly stagnant' ecosystems). Python is more attractive choice than Lua in my mind, but again the same issue of ecosystem comes to forefront; its a bit more reasonable comparison but I still tend to side with the JS ecosystem not because it is of course much bigger but again there is just a lot more you can do with JS; as such its easier to get things done. And given Morrowind is built on C++, JS seems like a more natural fit; the two languages share a lot in common and not to mention there is a technique you can use for NodeJS to load C++ as a module or to go the other way by transpiling JavaScript to C++ via WebAssembly.

Anyway, just wanted to put in a vote for Deno (JavaScript and TypeScript) ! I don't mind Lua being apart of OpenMW but personally even if an amateur I will try to stick with C++ and automate/script where I can using JavaScript; ie- maybe not for mods officially but I can at least use it to help automate my developer environment or to generate code; or to find other workarounds such as WebAssembly to get JS running in the game even if indirectly. With an OpenMW API with official scripting support via Deno of course that would be ideal!

if it can be coded in JavaScript it will be coded in JavaScript
User avatar
urm
Posts: 83
Joined: 02 Jun 2017, 16:05
Gitlab profile: https://gitlab.com/uramer

Re: Lua scripting in OpenMW

Post by urm »

nif wrote: 06 Mar 2021, 20:44 sorry to fuel more flames, but I wanted to point out there is a secure, standalone 1 file JavaScript runtime environment called Deno

https://github.com/denoland/deno

to quote their readme:

* Secure by default. No file, network, or environment access, unless explicitly enabled.
* Supports TypeScript out of the box.
* Ships only a single executable file.

As a longterm Morrowind fan/OpenMW player and occasional tinkererer, I've played with Lua myself on a few other gamedev hobby projects and can see the merits of its simple syntax and approachability. However, when I tried to get more serious I became frustrated by the issue of versioning specifically; it seems fragmented and the developer ecosystem seems quite a mess actually. My impression was not very good compared to JavaScript/NodeJS where the syntax is equally intuitive and overall just easier to get things done. The JS ecosystem has near unlimited reference/documentation not to mention a million modules if you don't feel like re-inventing the wheel, the landscape is just not as evolved in Lua world and correct me if I'm wrong it doesn't seem that that is changing very fast; if ever. I am about as excited to code more Lua as I am to code Ruby, Perl or PHP (nothing against those just wanted to generalize together some other 'slightly stagnant' ecosystems). Python is more attractive choice than Lua in my mind, but again the same issue of ecosystem comes to forefront; its a bit more reasonable comparison but I still tend to side with the JS ecosystem not because it is of course much bigger but again there is just a lot more you can do with JS; as such its easier to get things done. And given Morrowind is built on C++, JS seems like a more natural fit; the two languages share a lot in common and not to mention there is a technique you can use for NodeJS to load C++ as a module or to go the other way by transpiling JavaScript to C++ via WebAssembly.

Anyway, just wanted to put in a vote for Deno (JavaScript and TypeScript) ! I don't mind Lua being apart of OpenMW but personally even if an amateur I will try to stick with C++ and automate/script where I can using JavaScript; ie- maybe not for mods officially but I can at least use it to help automate my developer environment or to generate code; or to find other workarounds such as WebAssembly to get JS running in the game even if indirectly. With an OpenMW API with official scripting support via Deno of course that would be ideal!

if it can be coded in JavaScript it will be coded in JavaScript
First to address Deno: we don't need any of it in our API. Deno is quite similar to node.JS in the way that it's essentially just a bunch of C/C++ libraries with JavaScript bindings. So what you are really saying, is that we should have JavaScript instead of Lua.

While you are right that JavaScript is much more popular, I can't say I agree with your other points against Lua. I'm not sure what you mean by versioning issue. The ecosystem is indeed fragmented, but we don't really need any libraries, as we want to keep mods securely sandboxed, with relatively minimal possibilities outside of gameplay related ones. Syntax being intuitive is purely being used to it, JavaScript has plenty of weird quirks like implicit type conversion in comparison operators.

JavaScript also has many drawbacks compared to Lua:
1. A significantly bigger dependency. Something like V8 (comparable performance to LuaJIT) is massive and tricky to build.
2. If you don't use an enormous JS engine, performance will be significantly worse than LuaJIt or even normal Lua.
3. Trickier to provide bindings for, and less focused on sand boxing.
4. Many more language features, which can create issues when running as a part of a C++ application (e. g. you have to be careful about coroutines in Lua. JS has two features which do the same thing: generators and async/await, so now you have to think about both).
5. JS has many conventions, which we don't necessarily want to follow. Lua is a language where every small ecosystem is expected to have its own conventions, which is what we want.
6. MWSE and tes3mp already use Lua, so a significant part of the Morrowind modding community is familiar with it, which is more important than JS being more popular overall - a random WEB developer isn't likely to start making Morrowind mods just because they are in JavaScript.

TypeScript is nice, and Lua has a similar static typing option: Teal. I'm planning to look into supporting it with OpenMW out of the box, not sure how much work it would be yet.

P.S.: also, your point about C++ and JavaScript sharing a lot in common is... laughable, to be honest.
User avatar
AnyOldName3
Posts: 2666
Joined: 26 Nov 2015, 03:25

Re: Lua scripting in OpenMW

Post by AnyOldName3 »

A lot of the discussion regarding which scripting language we should have happened before I got a job requiring me to write a lot of Javascript. It didn't change my opinion that I should veto JavaScript being the scripting language, but it did give me the confidence to say I've got some experience to back up my opinion rather than just a general feeling that I'm right. The good subset of JavaScript The Language is basically stuff that any decent modern language has, and the good subset of JavaScript The Ecosystem that isn't shared with alternatives like Lua isn't relevant to OpenMW.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Deno/Javascript scripting in OpenMW

Post by psi29a »

I split this topic off since it doesn't have anything to do with Lua scripting in OpenMW
nif wrote: 06 Mar 2021, 20:44 sorry to fuel more flames, but I wanted to point out there is a secure, standalone 1 file JavaScript runtime environment called Deno
There were no flames to begin with and prefixing your opinion with "sorry" doesn't absolve you of anything and just makes anything you have to say after the fact insincere which sets the tone of the rest of your message. Don't do that.

You're rather late to the party when it comes to language preference, that boat has sailed.

Lua will be a first class citizen in both OpenMW and OpenMW-CS.

I would like to see Python go into the OpenMW-CS purely for the benefit of the huge ecosystem when it comes to data sciences and manipulation of large sets of data, not to mention GDAL which would help create worlds with GIS data which can expand the scope of OpenMW from just a game engine but a walking simulator for real-world environments for things like risk analysis. Take for example the Dutch coastline. Raise the sea-level by a half a meter, have a dike break open... see how much of the coast-line and Poulder area is flooded. You can visualise this in OpenMW.

Sorry, Javascript just isn't the go-to language with ready made libraries for that kind of thing either. There is literally nothing in Javascript that Lua and Python don't already do, but better.
nif
Posts: 14
Joined: 14 Apr 2015, 03:11

Re: Deno/Javascript scripting in OpenMW

Post by nif »

urm wrote: 06 Mar 2021, 22:09 P.S.: also, your point about C++ and JavaScript sharing a lot in common is... laughable, to be honest.
Fair enough, I'm not a C++ programmer but in my early experimentation with it - of course the syntax has various differences but general control flow seems quite similar. And with modern JS, you can mimic C++ class driven architecture. Libraries/modularity is also similar. Anyway, I'm excited to learn more particularly alongisde the OpenMW codebase which has a ton of great reference. Thanks for your feedback.
urm wrote: 06 Mar 2021, 22:09 I'm not sure what you mean by versioning issue.
If I recall, the pain was getting different Lua codebases or frameworks to play nicely with the different versions of Lua. The problem may just be that there aren't a lot of frameworks or librariesout there for so the maintainers don't care to update them as the language evolves and things quickly break or seem broken in such a small ecosystem.

I guess frustration in coding in Lua also comes from being spoiled as a JS developer, where backwards compatibility is never a problem since "do not break the web" is a rule in the language's development; as such a feature you used a decade ago still works to this day and likely another decade from now - this just isn't the case in other languages like Lua and Python.

For these reasons I quickly took hard pass on using Lua for serious development; but in context of a silo'ed scripting environment like for OpenMW where as you suggest that outside dependencies would not be welcome - then ya that makes a little more sense I am more open to playing with it in that regard. And forgive me if I'm being oblivious to Lua's more modern developments.
urm wrote: 06 Mar 2021, 22:09 A significantly bigger dependency. Something like V8 (comparable performance to LuaJIT) is massive and tricky to build.
I understand I am beating a dead horse here, but there are embeddable JavaScript alternatives to LuaJIT:

https://duktape.org
https://bellard.org/quickjs

but does seem LuaJIT has a good reputation for speed; so not necessarily debating the choice obviously OpenMW should choose the most performant option.


psi29a wrote: 08 Mar 2021, 09:45 I split this topic off since it doesn't have anything to do with Lua scripting in OpenMW
Good call, thx
psi29a wrote: 08 Mar 2021, 09:45 There is literally nothing in Javascript that Lua and Python don't already do, but better.
Okay well if my statement earleir was laughable... than this one is too! When Lua and Python drive most of the internet maybe you can say this :p
ptmikheev
Posts: 69
Joined: 01 Jun 2020, 21:05
Gitlab profile: https://gitlab.com/ptmikheev

Re: Deno/Javascript scripting in OpenMW

Post by ptmikheev »

See Javascript vs Lua comparison.

While JS is obviously better for web design and is more popular in general, Lua definitely wins in the category "scripting languages for game development".
nif wrote:
urm wrote: I'm not sure what you mean by versioning issue.
If I recall, the pain was getting different Lua codebases or frameworks to play nicely with the different versions of Lua. The problem may just be that there aren't a lot of frameworks or librariesout there for so the maintainers don't care to update them as the language evolves and things quickly break or seem broken in such a small ecosystem.
I hope that for in-game scripting we don't need so many frameworks as in web design :D .
Some mods can be kind of frameworks and provide functionality for other mods, but I can not imagine a situation where we would like to use an external non openmw-specific framework.

I am going to stick to one specific version - Lua 5.1. LuaJit simply doesn't support later versions.
Last edited by ptmikheev on 10 Mar 2021, 17:10, edited 1 time in total.
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Deno/Javascript scripting in OpenMW

Post by psi29a »

nif wrote: 10 Mar 2021, 04:59 Okay well if my statement earleir was laughable... than this one is too! When Lua and Python drive most of the internet maybe you can say this :p
In this particular context, OpenMW, then yes my statement is as serious as they come. You're presenting a strawman argument by using "the Internet" which isn't any way comparable. We're not talking about websites and visual gimmicks, we're talking about a game engine scripting language and plugin system for use in in the CS.

We're just focusing on Javascript's utility (or lack there-of) in OpenMW/CS and game-engines in general.
nif
Posts: 14
Joined: 14 Apr 2015, 03:11

Re: Deno/Javascript scripting in OpenMW

Post by nif »

ptmikheev wrote: 10 Mar 2021, 09:19 I hope that for in-game scripting we don't need so many frameworks as in web design .
Some mods can be kind of frameworks and provide functionality for other mods, but I can not imagine a situation where we would like to use an external non openmw-specific framework.
Indeed; and thats a good point and it would be neat to see a sort of internal cross-dependency network (albeit a light amount of it) amongst OpenMW mods. Good suggestion with regards to sticking to Lua 5.1.
psi29a wrote: 10 Mar 2021, 17:04 In this particular context, OpenMW, then yes my statement is as serious as they come. You're presenting a strawman argument by using "the Internet" which isn't any way comparable. We're not talking about websites and visual gimmicks, we're talking about a game engine scripting language and plugin system for use in in the CS.

We're just focusing on Javascript's utility (or lack there-of) in OpenMW/CS and game-engines in general.
Point taken; thankyou for clarifying context. Though with the general lack of confidence in JS here in this community it does inspire me to double down on making games with it - because I do think it is the future and already leaps and bounds ahead in some key areas. The web is simply one target; really the language is now its own beast with NodeJS and its not just about web stuff for example there's a good chance you are using VSCode as your daily driver for programming which is built on Electron + JS/TypeScript; as are a growing number of desktop games too using that same stack or similar tek.

Furthermore, it excels in areas condusive to game scripting. For example, there is a robust set of libraries available to address event driven or complex asynchronous control flow & state management needs and while yes the techniques can involve a number of libraries this simply is just another strong point of the ecosystem ie- micro-modularity/insta-install of modules; one command to install any one of the million modules available via npm (which include libs for handling large data sets, GIS, and GDAL btw).
psi29a wrote: 10 Mar 2021, 17:04 There is literally nothing in Javascript that Lua and Python don't already do, but better.
Well, if I was making strawmen argument fine but then it was in defense; a response to your unfair blanket statement :p
User avatar
psi29a
Posts: 5355
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Deno/Javascript scripting in OpenMW

Post by psi29a »

nif wrote: 12 Mar 2021, 05:54
psi29a wrote: 10 Mar 2021, 17:04 There is literally nothing in Javascript that Lua and Python don't already do, but better.
Well, if I was making strawmen argument fine but then it was in defense; a response to your unfair blanket statement :p
Apologies, I was operating from a OpenMW context. I still believe that both Lua and Python have a leg up on you in the amount available plugins/modules already available, especially in the data-science world (which is where I come from professionally). Things are ready made, including AI (tensor could be used to AI assisted texture up-scaling) and GIS data (topography/heightmaps) which are already highly optimised with C (ctypes/cython/bindings) anyway. So in terms of speed and efficiency, Python is just "Ready". I would suspect that Javascript doesn't have this type of useful ecosystem available and if you wanted it, it would be a non-trivial amount of time to implement. So the question then becomes, why bother, when it already exists in a well supported and understood language? Why re-invent the wheel?

If you want to go at it yourself, great, good learning experience. I look at it from a more managerial point of view of getting OpenMW from point A to point B, removing roadblocks and getting us further along so we make OpenMW more accessible and useful to people.
Post Reply