SWIG Scripting Extensions project

Everything about development and the OpenMW source code.
mike29
Posts: 24
Joined: 18 Dec 2012, 19:54

Re: SWIG Scripting Extensions project

Post by mike29 »

Is Lua faster? It's supposed to be but is it? If modder has to split script to 5 to make it perform well or rewrite it to Lua it might be worth to support new language.
maqifrnswa
Posts: 180
Joined: 14 Jan 2013, 03:57

Re: SWIG Scripting Extensions project

Post by maqifrnswa »

mike29 wrote:Is Lua faster? It's supposed to be but is it? If modder has to split script to 5 to make it perform well or rewrite it to Lua it might be worth to support new language.
Using lua as the scripting language and interpreter was considered at the very begining
viewtopic.php?f=13&t=23

Namely, tranlate all mwscript to lua and use the lua interpreter/libraries. After the discussion, the following design decisions were made:
1) The one unified scripting language will be mwscript "Who is the audience (morrowind scripting modders or other game modders that already use another langauge)?" The answer to that pointed to keeping mwscript as the one language.
2) A custom interpreter/compiler should be writen instead of using lua's "How do we support the large body of existing mwscript scripts/mods?" Use knowledge of mwscript's esotericism to make an optimized compiler/interpreter and not make a transcoder, especially since zini was doing all the work and knew how he wanted to do the mwscript implementation better than what the lua transcoding would look like.

OpenMW has a great interpreter and compiler now (probably better than vanilla) that is designed around mwscript's quirts, so it won't be faster if we just switch to lua. The only thing you gain is the abillity to use all the "extra" stuff that lua can do that the openmw interpreter can't Post 1.0, zini mentioned revisiting the interpreter to improve it, so those might go away as well.

This SWIG project was to see if generic extensions (which actually just compile the code in openmw interpreter byte code and send it to the openmw interpreter, so it is inherently slower) could be possible. And it is, in case someone wants to use it in the future.

Honestly, maybe I'm going about this backwards - what about writing opcodes that enable Lua/python/etc functionallity in mwscript rather than trying to enable mwscript functionallity in Lua/python/etc? just a thought.

One of the relevant quotes:
Zini wrote:The primary reason for having a custom format is, that we can exploit our the knowledge about the specific structure of Morrowind scripts and the underlying world model to gain a higher efficiency.

The secondary reason is, that translating to Lua sounds like more work than a custom interpreter (for which I might have reusable bits and pieces of code lying around from another project too).
The original Morrowind scripting language has so much bizarreness, that a straight forward translation would be pretty much impossible. Also, I am not really sure if a move to Lua as primary scripting language is a good idea. Yes, the Morrowind scripting language has its odd sides, but it is what everybody is using when it comes to Morrowind scripting and it actually isn't that much worse than other scripting languages (from an end-user's point of view), if you ignore how buggy it is. I would rather extend it, than offering a different language.

And the tertiary reason finally is, that I don't know much about Lua and I am only weakly motivated to change that. While I am aware that Lua is rather popular, so far I have failed to see the attractiveness of this language.
melrief
Posts: 9
Joined: 04 Dec 2015, 20:51

Re: SWIG Scripting Extensions project

Post by melrief »

I feel forced to add a few comments:
  • Python can be sandboxed using PyPy or similar tools. For a videogame scripting language, you want PyPy or something like RPython anyway.
  • Lua is better than MWScript, it is a quite mature and polished language for scripting. For the record, having AND and OR operators in a language is not that complicated. MWScript is full of limits and that's why we have projects like MSE that desperately try to overcome MWScript problems...
  • For people who don't want to add the support to another language because "everybody use MWScript": you don't need to discard MWScript, you could just add the support for Python or Lua or other proper languages. Mods and modders that want to use MWScript regardless could use it without problems. The rest of us could use a more mature and advanced language. Everybody wins, what's the problem?
User avatar
sjek
Posts: 442
Joined: 22 Nov 2014, 10:51

Re: SWIG Scripting Extensions project

Post by sjek »

viewtopic.php?f=4&t=3225#p35923 in posting time in there.

i think it's pr mostly and community.

most of the limitation is already deleted so to speak and remaining can be handled with script file/function flavours or extending existing functions

morrowind scripting for dummies could become only 10 pages long

edit: but for other parts than overcoming limitations..... maybe, don't know, definitely doable and also will be done in sometime by somebody or as external addon like MCP via assembly code just for the heck of it : P
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: SWIG Scripting Extensions project

Post by Chris »

melrief wrote:For people who don't want to add the support to another language because "everybody use MWScript": you don't need to discard MWScript, you could just add the support for Python or Lua or other proper languages. Mods and modders that want to use MWScript regardless could use it without problems. The rest of us could use a more mature and advanced language. Everybody wins, what's the problem?
The biggest isue is maintenance, I think. We have to have mwscript, because the original game and mods all use it. Another scripting language would be more code that has to be kept working. It becomes a bigger issue if you want to add new scripting features, and allow mwscript to grow some as well.

Personally, I think it would be best to effectively consider mwscript as "for compatibility only", and have something else as the "real" scripting language/engine for OpenMW. No new features for mwscript beyond vanilla (except maybe stupidly simple ones, like some extra functions MWSE has), and really just treat it as the old outdated format it is. Between all the nastiness mwscript allows for, and its really poor syntax, it'd be really nice to break from it and only keep it around for compatibility purposes. Have OpenMW use a different scripting language that rigidly adheres to a specific syntax and doesn't give an inch to poorly written scripts, with mwscript just being a converter that takes in vanilla scripts and writes out OpenMW scripts.

That's how I'd go about it, though some others disagree and want to keep mwscript as the main language, extending it as needed for new features.
User avatar
Okulo
Posts: 672
Joined: 05 Feb 2012, 16:11

Re: SWIG Scripting Extensions project

Post by Okulo »

Imagine all these modders who have been waiting for OpenMW to do what they wanted to do. Suddenly they have a well-documented scripting language available, with a large, active community behind it for any help they need. A language that can be learned from Codecademy, in the case of Python!

It would attract so many new people to a community that can use any boost. Hell, I'm probably one of these people. I'm going to need to learn Lua and/or Python anyway, if not for school then for any other kind of development. This would just kill another bird with the same stone. I'd love to make mods for OpenMW, but I doubt I'd bother with MWscript... and I've heard similar sentiments around the community back when this was debated a bit more intensely.
User avatar
DestinedToDie
Posts: 1181
Joined: 29 Jun 2015, 09:08

Re: SWIG Scripting Extensions project

Post by DestinedToDie »

I've learned Python for 2 semesters, mostly because it was a beginner language that gave a basic idea of syntax. I'm not very good at it, but I do love the idea of doing my scripting with it. It's fairly easy and user-friendly, even an amateur can pull it off.
User avatar
sjek
Posts: 442
Joined: 22 Nov 2014, 10:51

Re: SWIG Scripting Extensions project

Post by sjek »

that's for sure needed but how would go doing the reference structure that's is essential for scripting. in skyrim what have looked it's kinda barebones while, stack, list and such commands with some essential handless to game mechanics.

for that i think it would be basically syntax upgrade and the state machining what is in mwscript would still be there.

like this one what wrote and got working this morning

Code: Select all

Begin travelingtrigger

float timer
short ander
short x
short once

if once==0
 set x to 4
 eldafire->aitravel -11380 -68559 236
 set once to 1
endif

set timer to timer+getsecondspassed

if ander==0
 eldafire->aitravel -11380 -68559 236
  player->rotate z 20
 if timer>x
  if eldafire->getaipackagedone==1
   player->modspeed 50
   fargoth->disable
   set timer to 0
   set ander to 1  
  endif
 endif
endif


if ander==1
 eldafire->aitravel -11390 -69994 241
  player->rotate z -20
 if timer>x
  if eldafire->getaipackagedone==1
   player->modspeed -50
   fargoth->enable  
   set timer to 0
   set ander to 0
  endif
 endif
endif

End travelingtrigger

it's basically moves eldafire from one end of the bridge to other and in one direction makes player spiin clockwise? which works independetly of decreasing player speed by 50 and enabling fargoth. to other direction fargoth is disabled and speed returned while spinning is reversed. the if once is to get aitravel to roll and to set time x between direction change (doesn't override while traveling)

this could be used to move boulders with trigger limits or making drunk companion with animation which takes you to dance. as the position and presumably future animation functions can take variables

the trick was that first for variable, trigger was needed to get eldafire staying in place while getaipackagedone would return 1, as the NPC aiwander setting would otherwise overwrite it to 0.
this could be post 1 handled with extra variable making the issued aipackage semipermanent.

and variable also for separating turning around and commads as the timer>1 is reventing the aipackage to be checked in same frame which, althought the other aitravel is issued on next if block, would jump it with checking aipackage right to first block with set ander to 0
this could be handled by separate function timers

for this would think that incrementing number would allow to skip between kinda neatly or using some kind of different jumping but what the gamestate handling would look like as mwscript is basically state / number handling

in scripting the UI, handling the animations and such, advanced jumping and mouse handling would be handy.

basically is there workarounds game state machining with scripting syntax and helper functions which on the other hand would be hard or unpractical to fit into mwscript ?
melrief
Posts: 9
Joined: 04 Dec 2015, 20:51

Re: SWIG Scripting Extensions project

Post by melrief »

Actually, for Python and/or Lua maintenance is minimal after you have created the bindings unless you change something core which I don't think you'll do easily. You do have to maintain MWScript, that's true, but I think MWScript would still be required to be compatible with most of the mods.
MWScript would require years of work to just be as good as Lua and Python is way ahead of Lua. Let me quote one of the most embarrassing parts of Morrowind Scripting for Dummies 9.0:
A warning: There are different opinions on the forums on the use of several operators in one
line. Some people report a lot of problems with this, I myself have successfully used at least
four operators and variables in a single line. There appears to be an issue with very long
additions (e.g. adding up more than 20 variables in one line of code) that causes a mod to
crash the game upon loading. If this happens, split the calculations to several lines.
Nobody has a clue on how MWScript really works. It's about opinions, not facts! There is no clear specification and yes, that's bad. For the record, "I myself have successfully used at least four operators and variables in a single line" is not the sign of a good programming language :lol:.

You could fix the problems in OpenMW but then you would not have MWScript anymore but another language with all the compatibility problems that a better language would have. This is a nightmare to maintain. Ask Python 3 people about easy are to maintain Python 2 scripts when you use Python 3 :lol:.
User avatar
sjek
Posts: 442
Joined: 22 Nov 2014, 10:51

Re: SWIG Scripting Extensions project

Post by sjek »

set ln to ( ln + ( k10 * math_ln10 ) + ( k2 * math_ln2 ) )
that should be no problem even with 100 xP
depending where's c++ limit on one frame calculation if it's not threaded which takes it to processor limit

and the point is that there's already done 5-3 years on it + 7 from modding vanilla + MCP research like recently that in random) the ) is somewhat corrupted in one case leading to portion of script not working

if you don't count CS and morrowind compiler errors in hex level it fairly fell in package but individual test / unittest are needed to solve details

what from wikipedia: https://en.wikipedia.org/wiki/Lua_%28pr ... anguage%29

metatables could be implemented as archive scripts

https://en.wikipedia.org/wiki/First-class_function seems like would be same as "function" running strings as commands althought more fine revined would maybe need extra parameters?

https://en.wikipedia.org/wiki/Information_hiding that would need it's own block naturally at least on mwscript

https://en.wikipedia.org/wiki/Associative_array that looks like list but more general approach. extra array functions and such would need to be script based with only those

for loops:
"while condition do" would be handy

"repeat" both are basically same as while or xwhile which would have feature stopping at call before running whole script and determining how many times even if condition doesn't fullfil. ( details... )

"for i = first,last,delta do ;delta may be negative, allowing the for loop to count down or up"
that could be implemented as is

record accessing via point.x is basically in as script accessing. list and such need possibly different syntax to handle list inside another script or namespace script bundle if prefer

parsing whole page in time maybe but is there limitation inherit on c++ which would prevent cherrypicking across multiple languages for most fitting for game coding

part of the MWSE functions would be good as their purpose is kinda right access the game logic but how do you define those properties if not inside engine or would it hit performance if one script is accesses for thousands objects in scene to read properties and then determining values while many can be under different script and such.
(maybe defining in files in the end)
i think lua's fastness would be needed for that but how good it is at runrime multithreading and queue

edit: otherwise with extension it's easier to test how it handles the game itself 8-)
Post Reply