[post-1.0] Scripting enhancemnts

Everything about development and the OpenMW source code.
Post Reply
User avatar
Zini
Posts: 5536
Joined: 06 Aug 2011, 15:16

[post-1.0] Scripting enhancemnts

Post by Zini » 23 Mar 2012, 18:42

As announced in another thread here are my thoughts on scripting improvements after OpenMW 1.0. Please note, that I am still not interested in discussing post-1.0 in more details, because it is too important that we focus on 1.0 right now. I am making an exception for scripting, because I have planned out most of it anyway and there are so many vague and largely fruitless discussions over scripting improvements going on in this forum.

Code: Select all

This is a proposal for the first batch of post-1.0 scripting improvements. It should keep us (me) busy during 1.1, 1.2 and probably 1.3. It is not exhaustive and the scripting system can certainly be improved further in later versions.
For this first batch we should stay focussed though. Most of these improvements either fill glaring holes or are motivated by my own work with the MW scripting language on a total conversion. The later improvements are aimed mostly at large projects with scripts of high complexity that add completely new features to the MW engine. I believe that these will be of use to more ordinary scripters too.



I am not listing the following types of scripting enhancements in this document:

- enhancements that are specific to the post-1.0 de-hardcoding effort (moving away from hardcoded features/settings and let ESXs decide about them instead)
- enhancements that are required for new features
- enhancements that are already part of 1.0 (improvements that were possible without additional effort by choosing a different implementation path).


Contents
1. Compatibility
2. Missing Basic Functionality
3. Script types
4. Throttling
5. Variable scopes
6. Variable types
7. Additional control structures


1. Compatibility

Each script can have an optional compatibility line. Proposed keyword and syntax:

flavour x

where x is a sequence of one or more compatibility settings.

The line must be placed at the very beginning of the script. The scripting flavour of the console should be made globally configurable.

If no flavour statement is given, OpenMW will default to

flavour 0

which is the same as MW.

This will allow scripts of different flavours to be used at the same time, e.g. a plugin can add a "flavour 5 sane" script, without the "flavour 0" scripts of the master needing to be rewritten.

Settings:

1.1. Compatibility number

This is an integer number that is increased each time we release a new version that introduces new keywords or syntactic enhancements and thus potentially could cause an incompatibility with existing scripts.

1.2 Sane mode

Keyword: sane

This removes some oddities from the MW scripting language, namely:

1.2.1 Disallow the use of keywords as variable names

1.2.2 Change the operator precedence to something more reasonable (details still need to be worked out)

Note that the sane mode might become mandatory from a certain compatibility number on, in which case it would be added implicitly.


2. Missing Basic Functionality

2.1 Logical operators

Keywords: and, or, not, xor

2.2 Additional loop control instructions

Keywords: break, continue

2.3 Object deletion

In vanilla MW objects in a cell can be deleted with the setdelete instructions, while objects in a container can be deleted with the removeitem instruction.

Removeitem is mostly sensible, which is why we should keep it (with some addition for new variable types, see section 6).

The same can not be said about setdelete. It has an awkward syntax and is very limited. The former is reason enough to declare it deprecated.

As a replacement a new delete function shall be added. Examples:

(i) Delete            - This will delete the object the script is attached to (can only be used in local scripts).
(ii) Laire -> Delete  - This will delete the first reference to the ID "Lair" OpenMW can find.

(i) is an example of an implicit reference and (ii) is an example of an explicit reference. This terminology is currently used in the OpenMW codebase and will also be used in several places in this document.

Note that (i) will also work when the object is in a container. In this case the whole stack is deleted.

2.4 Variable initialisation at declaration

A variable of local, in or out scope (see section 5) can be given a value at declaration. Example:

local long x = 10

(for the local keyword see section 5)

Variables without a explicit value are default initialised.

Note that if a variable of scope in is explicitly initialised with a value all following variables of scope in need to be initialised too.

2.5 Increment and decrement

New keywords: inc, dec, by

For convenience the scripting language needs equivalents to the C++ operators += and -=. Since some hard to fix syntactic oddities make it impossible at this time to move from the "Set x to y" syntax to a "x = y" syntax, these need a different syntax too:

inc x by y

dec x by y

The "by y" part is optional and defaults to 1 if not present.


3. Script types

Currently we have 4 script types: local, global, dialogue and targeted (5 if we count console scripts, which we won't do here).

3.1 New script types

(i) Region scripts: Executed while the player is in the respective region

(ii) Cell scripts: Executed while the player is in the respective cell

(iii) Late global scripts: same as global scripts

(iv) Function scripts: Executed when called from another script


3.2 Script type execution order

Each frame scripts shall be executed in the following order

(i) global scripts
(ii) region scripts
(iii) cell scripts
(iv) local scripts
(v) late global scripts

This extended script system will allow scripts of different types to work together and provide features that can only be implemented with ugly workarounds in MW.

The region and cell scripts can also serve as repositories for region- and cell-specific state (in their member variables), which would go into global variables in vanilla MW.

One area where this feature could be utilised is the much discussed music player feature. The general consent on the forum was that special slots should be added that determine what music track is played (region, cell, ...). But this would never be flexible enough.
If we change the streammusic instruction so that it turns into a no-op, if the requested track is already playing and also only changes the music at the end of the frame we are getting this feature for free with the new script types.
A script of each type could specify a desired track, which can be overridden by a more specific script type (if needed), with late global handling tracks of the highest priority (like combat music).

There are too many areas where this new script structure can be utilised to list them all and modders will certainly come up with even more over time.


3.3 Script type ID

Each script can optionally specify which type it is by giving a type name after the script name, e.g.:

Begin SomeScript local

Keywords: global, region, cell, local, lateglobal, function

Dialogue scripts do not need a separate keyword, because they don't have a begin statement and also they are stored and handled separately from other scripts.

Targeted scripts also don't have a separate keyword. This script type is a bit of an oddity and is made largely redundant by the enhanced scripting system. While we can't drop support for it without breaking compatibility, we should discourage its use by declaring it obsolete and not adding any improvements to it.

The purpose of the script type ID is mostly to make it more clear how the script is used. But the editor could also offer filter functions for script types when listing scripts. It should also complain when a script is used in a way not compatible with the specified script type (e.g. attaching a global script to a MW-object).

Later we might utilise the script type ID to perform special script optimisations.


4. Throttling

MW performs a crude throttling on global scripts. This "feature" has to go. But throttling is required, especially since post 1.0 more hardcoded functionality will be replaced by scripts.

Therefore each global-, region-, cell-, local- and late global-script will receive a priority (specified by an integer number in the range of 0-9; higher values mean higher priority).

Each script can set its priority by the following line:

priority x

where x is a number between 0 and 9.

Scripts that do not provide a priority setting shall get one assigned based on a script type specific GMST.

The following guarantees are provided:

(i) If one script of priority i is executed during a frame, all scripts of priority i are executed during this frame.
(ii) Scripts of priority 9 are executed every frame.
(iii) Given a high enough number of frames, a script of priority i will be executed at least as often as a script of priority i-1.
(iv) If scripts of priority i have been executed and scripts of priority i-1 have been executed in a later frame, scripts of priority i-1 will not be executed again, until scripts of priority i have been executed.


5. Variable scopes

MW provide two variable scopes: global and local. The term local is misleading though, since these variables behave more like member variables. Therefore we shall call this scope "member" in this document.

The following scopes need to be added:

5.1 Local scope

This is a true local scope, i.e. the variables exist only for the duration of the script execution and lose their value afterwards. This will allow variables like loop counters to be kept out of the world state (and therefore also out of the saved game files).

Keyword: local

Example:

Local Short x

5.2. In/Out scope

These scopes are related to the function calling mechanism and are explained in section 7.

Keywords: in, out

Example:

In short x

5.3 Static scope

This is the equivalent to the static keyword in C++ used on member variables. A variable declared with static scope will be shared by all references of the same ID. The static scope will help to reduce the flood of global variables in larger projects.

Keyword: static

Example:

Static short x


6. Variable types

Currently we have 3 variable types: short, long, float.

6.1 Global variables

In MW all global variables are internally stored as float. This causes problems especially for longs. We can't fix this without extending the ESX format, so this needs to be addressed as a post-1.0 feature instead.

6.2 New variable types

6.2.1 Cell

New keywords: cell, isinterior, isexterior, interior, exterior, nocell, cellof

Example:

Cell x

This variable type stores a reference to a cell (the term reference here should not be confused with a regular MW-reference, a.k.a. object). Cells require a special treatment, because a cell record is the only addressable record type, that can not be specified by a string or an integer (interiors can be specified by a unique name, exteriors by unique coordinates).

Newly declared local/member/static cell variables are implicitly set to nocell.

The following functions needs to be added (c being a variable of type cell):

(i) isinterior c     - returns 1, if c is an interior cell, and 0 if c is not an interior cell
(ii) isexterior c    - returns 1, if c is an exterior cell, and 0 if c is not an exterior cell
(iii) exterior x, y  - returns the exterior cell specified by the given coordinates
(iv) interior name   - returns the interior cell specified by the given name
(v) cellof           - returns the cell a MW-reference is in (both implicit and explicit references are supported)

Note that nocell is neither interior nor exterior.

6.2.2 List

New keywords: list

New operators: #, []

Examples:

List long x

List y

This variable stores a list of values. List of lists are not allowed.

The first example creates a single type list. The second example creates a mixed type list.

Newly declared local/member/static list variables are empty.

A list literal can be specified by listing list items in square brackets. Example:

Set x to [ 1, 2, 3, 4 ]

The type of the list literal shall be automatically chosen based on the listed value. An empty list literal is a mixed list and can be implicitly casted to any other list type.

The size of a list can be read by the # operator. Example:

#x

The n-th element of a list (starting at index 0), can be accessed by the [] operator. Example:

x[1]

This only works for single type lists. Since this scripting language is statically types, for mixed type list a separate syntax is required. Example:

x[1, long]

If the list item is not of the given type an implicit cast is attempted.

Note:

(i) This is a very basic list support. Additional features can be added later, but that is beyond the scope of this specification. Advanced list support might be worth a design document of its own.
(ii) List of lists are not supported because they would be hard to implement. We may reconsider this at some point in the future.

6.2.3 String

New keywords: string

Example:

String x

This variable stores a piece of text.

Newly declared local/member/static string variables are set to an empty string.

String literals should be placed into pairs of quotation marks. If the string does not contain any spaces or other special characters, the quotation marks are optional.

Strings can by concatenated by the + operator. Example:

"abc" + "def"

Any value of type short, long, float, cell, list or ref can be turned into a string by use of the string function. Example:

string (1.3)

When using the string function on a ref value, the name of the ID and not the ID itself will be returned. Using the string on nocell or noref will return an empty string.

The length of a string can be determined by the # operator (see list type).

Access to a single character in a string is possible through the [] operator (see list type). The returned value is of type string again.

String variables can be used in any place where an ID can be used. In this case the resulting ID will be the value of the string variable.

Note: As with lists the initial implementation of strings will be somewhat basic. Additional features may be added later. Amongst others we might borrow the % operator from Python.

6.3.4 Ref

New keywords: ref, noref. self, content, cellcontent, getid

Example:

Ref x

This variable type stores a reference to a MW-reference (a.k.a. an object), which can either be located in a cell or a container.

Newly declared local/member/static ref variables are set to noref.

The following functions need to be added (r being a variable of type ref, c being a variable of type cell):

(i) self             - returns a ref to the object the script is attached to (can only be used in local scripts)
(ii) content         - returns a list of the references in the inventory of an object. Can be used either with an implicit or explicit reference. The object must be a container, creature or NPC.
(iii) cellcontent c  - returns a list of the references in c.
(iv) getid r         - returns a string containing the ID of r. The ID of a noref is an empty string.

Ref variables can be used instead of IDs when using explicit references. Example:

x -> enable

Ref variables can also be used instead of IDs as function/instruction arguments. In cases where an individual ref doesn't make sense, the ref will automatically degrade to its ID.

Note: Ref variables must be of local, in or out scope. Any other scope would be a nightmare to implement with OpenMW's current world model and also very error prone when put in the hand of an average modder.


7. Additional control structures

7.1 for loop

New keywords: for, forend, in

Iterates over the elements in a list.

Example:

local ref r
local long count

for r in content
    inc count by GetItemCount r
endfor

This little script will count the total number of items contained in the object this script is attached to (e.g. a NPC).

Note: In case of the content/cellcontent function, since the for loop is operating on a list of refs and not on the data structure of the object used as argument for content, removing the current ref from the object/cell is perfectly valid. In other words cutting off the branch you are sitting on is not a problem.

Example:

local ref r

for r in cellcontent player -> cellof
    if getid r == "iron dagger"
        r -> delete
    endif
endfor

This would delete any iron dagger in the cell the player is currently in.

7.2 switch/case

New keywords: switch, endswitch, case, default

Example:

switch player -> getItemCount "Gold_001":
    case 0
        MessageBox "no coins at all"
    case 1
        MessageBox "a single coin"
    default
        MessageBox "filthy rich"
endswitch

The switch variable can have the type short, long, float, cell or string.

7.3 function calls

New keywords: runscript

This will execute a script as a function.

Example:

local long x
local long y

runscript double in x out y

In this case the script double could look like this:

Flavour 1 sane

begin double function
    in long a
    out long b
    Set b to 2*a
end

or shorter:

Flavour 1 sane

begin double function
    in long a
    out long b = 2 * a
end

In and out variables must match in type and number the arguments of the runscript instruction. If no in or out variables are required, the respective in/out clause can be omitted. The caller is free to provide less out variables than defined in the function script, in which case the remaining return values are discarded.

Yacoby
Posts: 119
Joined: 30 Sep 2011, 09:42

Re: [post-1.0] Scripting enhancemnts

Post by Yacoby » 23 Mar 2012, 19:46

To play devils advocate, given that we are ditching backwards compatibility, is there any reason why we don't implement something small and fast like Lua/Python? Just run it along side the default scripting language. Some of these changes seem flawed. Like trying to make an octopus by sticking more legs on a dog in a flawed way.

For example, with a decent language you could probably allow the modders more control over how a script is run etc rather than having to have script types. You also don't have the pain of backwards compatibility. (And lets face it, C++ was bad enough and C was a fairly well designed language)

Pros:
* Lua is fast. Like really fast: http://shootout.alioth.debian.org/u32/b ... l&lang=all
* Its code we don't have to write
* Less compatibility issues
* The design can actually be good, if we improve the Morrowind scripting system things will (and do) looked hacked in to work with a system not designed for it. To be honest,

Cons:
* More complex maintaining new System
* Requires people to learn a new language
* Probably more complex w.r.t. some of the code design

Anyway, 10 minutes of thoughts on the biggest issues I can see. I will try and look at it in more detail soon.
Zini wrote: so many vague and largely fruitless discussions over scripting improvements going on in this forum.
I am glad we now have a proper thread for them then ;)

First off, no mention of namespaces or some sort of package mechanism? This is one of the biggest issues that Morrowind scripting has. Look at any script by a big author and the script will start with something like YAC_ or FG_ to try and avoid name conflicts.
Each script can have an optional compatibility line.
Why the need for the number? flavour sane extended?
1.2.2 Change the operator precedence to something more reasonable (details still need to be worked out)
Out of curiosity, what is it? MWSFD doesn't mention.
local long x = 10
This syntax makes no sense. = is equality not assignment. Why would you want to use equality for assignment? It would also lead to confusing things like

Code: Select all

short bar;
; ...
bool foo = bar = 10
Which is just horrible.
Currently we have 4 script types: local, global, dialogue and targeted (5 if we count console scripts, which we won't do here).
targeted is the same as global, it just has a default reference.
3.1 New script types
I am in two minds about this, possibly a good idea. IDK. But you need a targeted script type. You can't change the handling for globals to something sane.
(iv) Function scripts: Executed when called from another script
So this makes all functions global scope? Sounds like a terrible idea. If you need me to list the issues I can.

(i) global scripts
(ii) region scripts
(iii) cell scripts
(iv) local scripts
(v) late global scripts
what is the reasoning behind late globals? How would you specify what cell something can be run in. How can a script be run in both a cell and a region. How can you change it at run time?
Each script can set its priority by the following line:

priority x
scriptname->priority x? Right. Because I can't see any reason why priority should be anything other than a function.
(i) isinterior c - returns 1, if c is an interior cell, and 0 if c is not an interior cell
(ii) isexterior c - returns 1, if c is an exterior cell, and 0 if c is not an exterior cell
Why duplicate the functionality of already existing function (getInterior)? Also, if there is isInterior there is no need for isExterior.
This variable stores a list of values. List of lists are not allowed.
Why? If a list can contain shorts, floats, longs then why can't it contain lists?

User avatar
Zini
Posts: 5536
Joined: 06 Aug 2011, 15:16

Re: [post-1.0] Scripting enhancemnts

Post by Zini » 23 Mar 2012, 20:24

To play devils advocate, given that we are ditching backwards compatibility,
We are not ditching backward compatibility. Old scripts can run unmodified. If you want to extend a new script with features from newer compatibility levels, most likely they will work too without modifications (after you added the required flavour line). The compatibility setting is a safety measure to absolutely make sure that newer OpenMW versions don't break older ESX files. Remember we have a huge and diverse body of work here.
First off, no mention of namespaces or some sort of package mechanism? This is one of the biggest issues that Morrowind scripting has. Look at any script by a big author and the script will start with something like YAC_ or FG_ to try and avoid name conflicts.
Namespaces are not a script specific issue. If we introduce namespaces at some point we need to address the whole ID/record system. I am not sure how to do that yet and it is definitely out of scope for scripting improvements.
Why the need for the number? flavour sane extended?
With OpenMW 1.1 we may introduce a few new keywords. So we need to increment the number by one, so that older scripts that use one of these keywords as a variable name don't break. With OpenMW 1.2 we may introduce even more new keywords. Again we need to increment the number by one, so 1.1 scripts which happen to use one of these keywords as a user defined name don't break.
Out of curiosity, what is it? MWSFD doesn't mention.
I don't have the specifics at hand right now, but I think it was something with the comparison operators. Would have to look it up.
This syntax makes no sense. = is equality not assignment
Nope. == is equality. = is unused in MW so we can as well borrow it for initialisation.

Code: Select all

short bar;
; ...
bool foo = bar = 10
Huh? That makes no sense. Certainly not legal MW script (neither vanilla nor OpenMW post-1.0). You must have misunderstood something.
targeted is the same as global, it just has a default reference.
That is a pretty big difference.
But you need a targeted script type. You can't change the handling for globals to something sane.
I have no idea what you mean by this.
So this makes all functions global scope? Sounds like a terrible idea. If you need me to list the issues I can.
Except for namespace issues I don't see any and that is a separate topic.
what is the reasoning behind late globals?
See the example I gave. Combat music.

In the Redemption TC we work with a lot of features distributed across several scripts. One script does a bit of work then communicates with another script that does another bit of the work. The execution order is very important there. I don't have any specific use cases at hand, but I am pretty sure I would regret it if this script type would not be available (I vaguely remember that I regretted it when working on Redemption; that was a few years back though).
How would you specify what cell something can be run in.
I don't understand the question.
How can a script be run in both a cell and a region.
The same script can't be a cell script and a region script, if that is the question.
How can you change it at run time?
Change what? I don't follow you.
scriptname->priority x? Right. Because I can't see any reason why priority should be anything other than a function.
Makes sense. Instruction instead of function actually, but okay.
Why duplicate the functionality of already existing function (getInterior)
getInterior checks if the player is in an interior. These function check if a cell is an interior. These are two completely different things.
Also, if there is isInterior there is no need for isExterior.
nocell is neither.
Why? If a list can contain shorts, floats, longs then why can't it contain lists?
Hard to implement. As I wrote, we might consider adding it later, but for now it is just too much work.

BenBat
Posts: 1
Joined: 24 Mar 2012, 01:04

Re: [post-1.0] Scripting enhancemnts

Post by BenBat » 24 Mar 2012, 02:06

I have been lurking this project forum for quite a long time now and I would like to intervene in this discussion because scripting is actually the part I'm interested in the most.

I would like to strongly back Yacobi. Reimplementing and extending an interpreter seems to be mostly irrelevant when there are so many good VMs out there that this project can use. Extending scripts this way would result in free optimisation (modern VMs do quite a lot of really clever code analysis) and allows to avoid hackish solutions like flavour, sane syntax and scripts limited to be called by scripts.

Let's face it, Morrowind scripting language is an incredibly bad object-oriented language. The syntax and priority are convoluted and verbose. The type system is really poor with no support for really basic thing like strings, arrays, lists and booleans. There are only primitives and no proper way to define functions. There is no way to import code properly. Loops are limited to while. The list of flaws is still long.

Implementing a proper VM like Lua inside the project seems to me like the right solution. Lua is the de facto standart in the video game industry nowadays. The language is quite simple but can be really powerful thanks to its gestion of array and the VM has been designed to be integrated in this kind of project and has a fairly good level of optimisation.

Furthermore, given the state of OpenMW interpreter, integrating Lua at this point seems quite straight-forward. If I understand the code correctly, there is already a tokenizer so a parser outputting code targeting the Lua VM (probably directly in Lua) needs to be written. Script functions have already been properly implemented inside the engine in C++ so a Lua API on top of it is enough for them to become accessible.

On the other hand, trying to extend the actual interpreter boils down to designing your own language which means a lot of nasty issues will have to be addressed (things like scoping, variables passing, tail-call optimisation).

EmbraceUnity
Posts: 28
Joined: 24 Oct 2011, 04:41

Re: [post-1.0] Scripting enhancemnts

Post by EmbraceUnity » 24 Mar 2012, 04:55

I see where Zini is coming from and support that approach since everyone in the MW modding community is used to the Bethesda scripting language. Extending that really does seem like the right way to go, and Zini's suggestions show how it could be done while maintaining backward compatibility.

Of course, this is an open source project so anyone is free to add Lua themselves, or fork if necessary.

The ideas aren't mutually exclusive. Zini's idea of having something like a meta tag means that you could differentiate between different script styles, and that approach could be extended indefinitely, to include Lua or anything else.

Chris
Posts: 1558
Joined: 04 Sep 2011, 08:33

Re: [post-1.0] Scripting enhancemnts

Post by Chris » 24 Mar 2012, 06:08

I'm with Yacoby and BenBat. MW's scripting language is pretty poorly designed and implemented compared to what's possible nowadays, and its scope is quite limited. Trying to extend MW's scripting language is like trying to put band-aids on a gash. Sure it can help (and things like MWSE have no choice but to work that way), but there's more effective ways to do it when the engine is completely under our control. Especially if OpenMW's going to introduce extensions that (when enabled) break backwards compatibility anyway, I don't see the point in trying to keep anything new tied to MW's inflexible 10-year-old scripting design.

User avatar
Zini
Posts: 5536
Joined: 06 Aug 2011, 15:16

Re: [post-1.0] Scripting enhancemnts

Post by Zini » 24 Mar 2012, 10:30

Only commenting on a few points for now.
Morrowind scripting language is an incredibly bad object-oriented language.
I do not agree with this statement. it isn't an object oriented language at all. And in my opinion it doesn't have to be. OOP is a very powerful paradigm, no doubt about that. But that doesn't make it the answer to everything. For a highly domain specific language like the scripting language of an RPG engine IMO staying mostly clear of OOP is a good idea.
This script language is used to implement quests, certain parts of story telling, customisations and new features. It's not meant to write an operating system with it. And from my own experience the MW scripting language is very good at this; or rather it would be good if it weren't implemented so lousy.
Furthermore, given the state of OpenMW interpreter, integrating Lua at this point seems quite straight-forward.
Not really. There are syntactic oddities in the MW scripting language that would be hard to handle, e.g. keywords can be used as user-defined names and identifiers can have spaces.
MW's scripting language is pretty poorly designed and implemented
No doubt about the implementation. But I do not agree with the part about the design; yes, it has a few annoying oddities, yes, it has a few unfortunate omissions, yes it is pretty simple, but overall it does its job well. I am not wearing my OpenMW developer hat, but my MW modder hat, when making this statement.
Especially if OpenMW's going to introduce extensions that (when enabled) break backwards compatibility anyway
We are not doing that. Even when enabled these extensions will break compatibility only in a few places. The majority of old script should work even with higher compatibility numbers. The whole flavour system is intended for the few cases were it does not work.
I see where Zini is coming from and support that approach since everyone in the MW modding community is used to the Bethesda scripting language.
Exactly that is my point. Tossing out the old script language would not be helpful to the existing modding community at large. The situation would be different, if the existing scripting language would be unfixable. But I really don't see that this is the case.
Trying to extend MW's scripting language is like trying to put band-aids on a gash.
I think that is the core point where we are disagreeing here.

Yacoby
Posts: 119
Joined: 30 Sep 2011, 09:42

Re: [post-1.0] Scripting enhancemnts

Post by Yacoby » 24 Mar 2012, 13:42

Zini wrote:
To play devils advocate, given that we are ditching backwards compatibility,
We are not ditching backward compatibility. Old scripts can run unmodified. If you want to extend a new script with features from newer compatibility levels, most likely they will work too without modifications (after you added the required flavour line). The compatibility setting is a safety measure to absolutely make sure that newer OpenMW versions don't break older ESX files. Remember we have a huge and diverse body of work here.
We are not doing that. Even when enabled these extensions will break compatibility only in a few places. The majority of old script should work even with higher compatibility numbers. The whole flavour system is intended for the few cases were it does not work.
What I meant was that new scripts won't be compatible with the old. New functions, types, control structures.
First off, no mention of namespaces or some sort of package mechanism? This is one of the biggest issues that Morrowind scripting has. Look at any script by a big author and the script will start with something like YAC_ or FG_ to try and avoid name conflicts.
Namespaces are not a script specific issue. If we introduce namespaces at some point we need to address the whole ID/record system. I am not sure how to do that yet and it is definitely out of scope for scripting improvements.
True. There is no reason why we can't have namespaces in the scripting system now anyway though. We can work it out with objects at a later date/if at all.
Why the need for the number? flavour sane extended?
With OpenMW 1.1 we may introduce a few new keywords. So we need to increment the number by one, so that older scripts that use one of these keywords as a variable name don't break. With OpenMW 1.2 we may introduce even more new keywords. Again we need to increment the number by one, so 1.1 scripts which happen to use one of these keywords as a user defined name don't break.
Sort of makes sense. We should be able to put together a list of reserved keywords beforehand though?
This syntax makes no sense. = is equality not assignment
Nope. == is equality. = is unused in MW so we can as well borrow it for initialisation.
Both = and == are equality. I know of several mods that use = as equality.
Huh? That makes no sense. Certainly not legal MW script (neither vanilla nor OpenMW post-1.0). You must have misunderstood something.
So I won't be able to initialise a type from the result of a comparison? That sounds like something PHP would do.
targeted is the same as global, it just has a default reference.
That is a pretty big difference.
Not really. You still have all the problems such that the script can only have one instance of itself running at a time.
But you need a targeted script type. You can't change the handling for globals to something sane.
I have no idea what you mean by this.
If you want to have target scripts, you may as well make it so they are not such a pain to deal with. Look at things like Vampire Embrace. There are about 20 - 40 scripts that are just duplicates of other scripts as an ugly work around for that issue. (And from experience, making changes to them is just painful)
So this makes all functions global scope? Sounds like a terrible idea. If you need me to list the issues I can.
Except for namespace issues I don't see any and that is a separate topic.
The main thing is lack of namespace. Having to name a function YAC_Max would make code readability suffer. Also, why would I want to expose a simple two line function to the world.

Just make function a type and then you get all the nice scoping rules for functions as well. And then I can have higher order functions.
In the Redemption TC we work with a lot of features distributed across several scripts. One script does a bit of work then communicates with another script that does another bit of the work. The execution order is very important there. I don't have any specific use cases at hand, but I am pretty sure I would regret it if this script type would not be available (I vaguely remember that I regretted it when working on Redemption; that was a few years back though).
It is just that it sounds a bad way to solve the problem. Chaining functions would work better surly?
How can you change it at run time?
Change what? I don't follow you.
Where the script should be run. There is no way I know all the cells at compile time. Lets say I want to run a script with all cells with water in (See: Swimming Realism), there is no way I can list these cells as mods add more. I would need to be able to add that cell to so that the script is run in it.

Although maybe better we have a WaterCell script type? ;) The issue is that it isn't that extensible and there will be some modders that want to do more things that just what you think. It is probably a good idea to give them the power to if they want to.

Maybe it would be better to pass a function that indicates when the script should be run?
scriptname->priority x? Right. Because I can't see any reason why priority should be anything other than a function.
Makes sense. Instruction instead of function actually, but okay.
Instruction as in inbuilt function?
Why duplicate the functionality of already existing function (getInterior)
getInterior checks if the player is in an interior. These function check if a cell is an interior. These are two completely different things.
Ah. My Bad
Also, if there is isInterior there is no need for isExterior.
nocell is neither.
isNoCell? Just don't allow isInterior on non cell objects?
Why? If a list can contain shorts, floats, longs then why can't it contain lists?
Hard to implement. As I wrote, we might consider adding it later, but for now it is just too much work.
It just strikes me that now allowing recursive types is bad design and indicates a problem somewhere. Lists of lists seems like a fairly critical feature to me.
This script language is used to implement quests, certain parts of story telling, customisations and new features. It's not meant to write an operating system with it. And from my own experience the MW scripting language is very good at this; or rather it would be good if it weren't implemented so lousy.
No doubt about the implementation. But I do not agree with the part about the design; yes, it has a few annoying oddities, yes, it has a few unfortunate omissions, yes it is pretty simple, but overall it does its job well. I am not wearing my OpenMW developer hat, but my MW modder hat, when making this statement.
*Puts on Modder Hat*

I disagree. No functions. No libraries. No imports. No unit testing. Script extenders that have worked with the language but suffered for it (Iteration over NPCs is so close to C using MWSE it is painful). Lack of ability to do things (such as run multiple instances of a script on different objects).

Compared to doing something in a sane language with some good tools, development is slow. Debugging 5000 lines of interconnected code with code running on the same actor split into multiple scripts for performance reasons with excessive script intercommunication and started/stopped at specific points depending on the world state is horrible. But it is the only way to get decent performance (To give you an idea, older versions of Pursuit Enhanced could cut my FPS in half)

I wrote Pursuit Enhanced and did a lot of work on a mod called Werewolf Infection (Similar to Vampire Embrace, but working with Werewolves which was hard work as it involved lots of targeted scripts and swapping actors with werewolf models. I gave up on it, although someone has taken it on I think).

As Chris said, complicated configuration systems involving message boxes were a pain.

In other words, doing simple things like enabling or disabling something is fine. Doing complicated things such as implementing new features is horrible. Horrible to the point I am never scripting with the old system again.



With regard to OO, I agree that Morrowind isn't OO, OO isn't always a solution and in this case ramming it down peoples throats isn't a good solution. But the non OO languages that I like (Haskell for example) have very good and well designed type systems. (Far better type systems than "OO" languages like Java)
Last edited by Yacoby on 24 Mar 2012, 14:19, edited 6 times in total.

Chris
Posts: 1558
Joined: 04 Sep 2011, 08:33

Re: [post-1.0] Scripting enhancemnts

Post by Chris » 24 Mar 2012, 13:50

Zini wrote:No doubt about the implementation. But I do not agree with the part about the design; yes, it has a few annoying oddities, yes, it has a few unfortunate omissions, yes it is pretty simple, but overall it does its job well. I am not wearing my OpenMW developer hat, but my MW modder hat, when making this statement.
The problem I see is that the scripts are basically just lumps of code that the engine fully executes on each frame. For each frame, it runs each active script from top to bottom regardless of what the script needs to do at the time. This doesn't play nice with complex algorithms, nor does it parallelize well.

There's also the horrible syntax. If I want to assign something, I have to do

Code: Select all

set foo to bar
instead of

Code: Select all

foo = bar;
like most common languages. You can't break up large lines because line breaks end the command instead of a semicolon. Additionally, to access and manipulate object properties you have to do verbose stuff like

Code: Select all

set var to ref.GetActorValue val
set var to var+1
ref.SetActorValue val var
instead of something easier like

Code: Select all

ref.actorvalue("val") += 1;
There's no inheritance so you can't use an object reference to manipulate base object properties. You can't provide specific object types, like "Actors" or "Statics", and they're instead manipulated through functions that work on generic IDs (where they'll then silently fail at run-time if they're given an improper object type).

This is me talking as an ex-Oblivion scripter. I really despised the flakiness induced by the simplified syntax and non-type-safe methods, and it was a pain in the rear to make scripts do anything non-simple without turning them into their own mini state-machines (I still have nightmares about trying to add multi-choice message boxes to existing scripts, or nested message boxes). Pausing and blocking functions were out of the question.. a script ran when it ran, it must make its way to the end of the block, and ain't nothing stopping it.

I'd strongly urge taking a look at Papyrus for a more modern scripting design. Even Bethesda abandoned the old scripting language. It's very error prone, it's unintuitive when trying to do anything complex, and the design just doesn't take good advantage of resources available on today's systems. Not to say Papyrus is perfect, but it does seem better from what I've seen.

User avatar
Zini
Posts: 5536
Joined: 06 Aug 2011, 15:16

Re: [post-1.0] Scripting enhancemnts

Post by Zini » 24 Mar 2012, 17:28

What I meant was that new scripts won't be compatible with the old. New functions, types, control structures.
Yeah, that is obvious. If we extent the scripting language, vanilla MW won't run newer scripts
Sort of makes sense. We should be able to put together a list of reserved keywords beforehand though?
We can that with the next couple of versions, but certainly not for 2.0 and 3.0 or whatever. At some point we will have to introduce new keywords again, that were not in the first batch.
Both = and == are equality. I know of several mods that use = as equality.
Wasn't aware of that. IIRC our current parser does not handle = (would have to look that up). Well, we can flag = used as equality a deprecated feature.
o I won't be able to initialise a type from the result of a comparison?
It can. With your explanation about = your example makes sense, though I would definitely use == instead of = here.
It is just that it sounds a bad way to solve the problem. Chaining functions would work better surly?
I do not agree with this statement. It is very powerful and also elegant. Its definitely the way I would want to continue scripting in MW/OpenMW.
Where the script should be run. There is no way I know all the cells at compile time. Lets say I want to run a script with all cells with water in (See: Swimming Realism), there is no way I can list these cells as mods add more. I would need to be able to add that cell to so that the script is run in it.
I still don't follow you. Cell and region scripts are attached to a single cell/region each the same way a local script is attached to a MW-reference.
Instruction as in inbuilt function?
Instructions don't have return values.
Just don't allow isInterior on non cell objects?
All objects are non-cells. A cell is not an object. nocell is like NULL or nullptr.

@Chirs: Yes, the syntax is a bit suboptimal. But I wouldn't call it horrible.

Post Reply