Elder-scrolls IV Oblivion

Feedback on past, current, and future development.
User avatar
AnyOldName3
Posts: 1936
Joined: 26 Nov 2015, 03:25

Re: Elder-scrolls IV Oblivion

Post by AnyOldName3 » 23 Mar 2020, 23:54

We suspect that the original MWScript compiler did dumb stuff like read uninitialised memory, which means it's technically not even guaranteed to produce the same output if run twice on the same code. That's not even something that can be expressed by a formal grammar (although I guess you could argue that it's just an ambiguous language and multiple parse trees are valid for the same input without specifying which you'll get when).

Oblivion is probably better than MWScript. By the time they got to Skyrim, they used Bison, and Bethesda using an external tool or library is usually a sign that something's gone horribly wrong.
AnyOldName3, Master of Shadows

1Zero
Posts: 38
Joined: 09 Feb 2020, 18:44

Re: Elder-scrolls IV Oblivion

Post by 1Zero » 24 Mar 2020, 15:02

AnyOldName3 wrote:
23 Mar 2020, 23:54
We suspect that the original MWScript compiler did dumb stuff like read uninitialised memory, which means it's technically not even guaranteed to produce the same output if run twice on the same code. That's not even something that can be expressed by a formal grammar (although I guess you could argue that it's just an ambiguous language and multiple parse trees are valid for the same input without specifying which you'll get when).
The size of the set of of possible strings in memory is 2^(Memory Size), an astronomic value for modern memory sizes. The set it recursively numerable (You can write an algorithm listing all possible finite strings), however not necessarily a recursive set.
Well, based on your description, MWScript really sounds like a hack job on BGS part.
Oblivion is probably better than MWScript. By the time they got to Skyrim, they used Bison, and Bethesda using an external tool or library is usually a sign that something's gone horribly wrong.
They used Bison? Nice, Bethesda ascending.
I hope Oblivion scripting engine does not access random memory addresses, we want out compilers to be nice and deterministic. I expect it to be implementable with a common parser generator, which wont be necessary if the OpenMW parser can be sufficiently easily adapted to parse Oblivion.
I have a plan.

User avatar
psi29a
Posts: 4915
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Elder-scrolls IV Oblivion

Post by psi29a » 15 Apr 2020, 20:18

I'll just drop this bit of OpenMW w/ Fallout here...

https://youtu.be/xKt0bbaJs9E

1Zero
Posts: 38
Joined: 09 Feb 2020, 18:44

Re: Elder-scrolls IV Oblivion

Post by 1Zero » 09 May 2020, 03:56

@cc9cii

Did you ever figure out Speedtree?
Apparently based on a L- System which is just some Formal Grammar in additional to some Hilbert Calculus like Axiom rule. No Man's Sky actually also utilizes it as part of their Fauna Procedural Generation algorithms. But it seems to be fairly obnoxious and tedious to translate the Production sequence into 3D meshes. I suppose using placeholder trees would be "good enough" for now unless you already did some work on it.

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Elder-scrolls IV Oblivion

Post by cc9cii » 09 May 2020, 07:17

I've not spent any time looking at speedtree at all - at the moment I'm kinda leaning towards using the tree NIFs from Skyrim which comes with LOD meshes and textures.

1Zero
Posts: 38
Joined: 09 Feb 2020, 18:44

Re: Elder-scrolls IV Oblivion

Post by 1Zero » 09 May 2020, 16:03

cc9cii wrote:
09 May 2020, 07:17
I've not spent any time looking at speedtree at all - at the moment I'm kinda leaning towards using the tree NIFs from Skyrim which comes with LOD meshes and textures.
Makes sense for development convenience.
I started playing around with L Systems in Python for a bit
Depending on the coefficients, they can create varying kinds of fractals, here
a Barnsley Fern, a special kind of L System.

Standard one:
2020-05-09 16_35_42-Photos.png
Barnsley fern 2
2020-05-09 16_35_42-Photos.png (172.7 KiB) Viewed 1264 times
Changed Coefficients:
2020-05-09 16_38_04-Photos.png
Barnsley fern 1
2020-05-09 16_38_04-Photos.png (151.13 KiB) Viewed 1264 times
(https://en.wikipedia.org/wiki/Barnsley_fern)
So that can indeed be used for Fauna variety. Ofc. the plant isn't continuous here and only monochromatic.

You can generate tree like structures from this system, which is actually what Speedtree apparently uses.
Gif From the L Systems article on wiki:
Fractal_Farn.gif
Fractal Tree
Fractal_Farn.gif (443.73 KiB) Viewed 1264 times
You can generalize this system to take arbitrary production rules
for the generation of fractals, in the end you can get a (hopefully Kolmogorov complexity) algorithmic mechanism to give a blueprint for another algorithm to generate Fauna.

The hard part is to translate this into OpenMW's 3D engine, and I just started with 3D programming so I likely going to wait out a bit. Integrating Scripting has precedence either way once you release your fork.

User avatar
AnyOldName3
Posts: 1936
Joined: 26 Nov 2015, 03:25

Re: Elder-scrolls IV Oblivion

Post by AnyOldName3 » 10 May 2020, 03:01

We only need to replicate a subset of SpeedTree. It's a big library that does a lot of different things, ranging from just deciding to place pre-generated models to providing materials to movie-quality destruction simulation, and obviously most applications don't need everything. That means that if something says SpeedTree uses a certain technique, it doesn't imply Oblivion uses it, and it might only be in big-budget feature films with a renderfarm at their disposal.
AnyOldName3, Master of Shadows

1Zero
Posts: 38
Joined: 09 Feb 2020, 18:44

Re: Elder-scrolls IV Oblivion

Post by 1Zero » 10 May 2020, 07:09

AnyOldName3 wrote:
10 May 2020, 03:01
We only need to replicate a subset of SpeedTree. It's a big library that does a lot of different things, ranging from just deciding to place pre-generated models to providing materials to movie-quality destruction simulation, and obviously most applications don't need everything. That means that if something says SpeedTree uses a certain technique, it doesn't imply Oblivion uses it, and it might only be in big-budget feature films with a renderfarm at their disposal.
Yes sure, there is no need to reimplement all of it, just the features needed for Oblivion (and general flora generation in the long term, because it's always LSystem based). Luckily, the theoretical framework already exists
http://algorithmicbotany.org/papers/Tre ... .large.pdf
http://algorithmicbotany.org/papers/lpy.fps2012.pdf

This is actually super impressive
https://blenderartists.org/t/treesketch ... pad/577784

It's not a priority to implement right now for me, but the theoretical background is already there.

ponyrider0
Posts: 14
Joined: 11 Jun 2019, 23:53

Re: Elder-scrolls IV Oblivion

Post by ponyrider0 » 10 May 2020, 07:22

@cc9cii

I just got around to watching your youtube videos for this year. Looking very nice! Did you figure out the parameters for the FGTS? I have not had time to confirm it yet, but I believe there are fifty [4 byte] values (32bit float?) corresponding to the 50 images contained within an EGT file. According to the SDK, scale factor (32bit float from FGTS?) * RGB component == RGB mode

Also, if you have not figured out the pathgrid priority node encoding yet: it is stored in the odd/even value of the z position: even value == (red/orange) general/auto node, odd value == (blue) priority node.

FYI, here is how I implemented the TES3 to TES4 pathgrid translation algorithm: https://github.com/ponyrider0/openmw/bl ... #L302-L325

For reference:
- https://github.com/TES5Edit/TES5Edit/bl ... .pas#L3921
- http://tesalliance.org/forums/index.php ... utorial-4/

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Elder-scrolls IV Oblivion

Post by cc9cii » 10 May 2020, 11:37

ponyrider0 wrote:
10 May 2020, 07:22
I just got around to watching your youtube videos for this year. Looking very nice! Did you figure out the parameters for the FGTS? I have not had time to confirm it yet, but I believe there are fifty [4 byte] values (32bit float?) corresponding to the 50 images contained within an EGT file. According to the SDK, scale factor (32bit float from FGTS?) * RGB component == RGB mode
FaceGen textures for TES4 work ok - not too bad given that I'm not using shaders and using workarounds. However they don't work the same way in FO3/FONV and I've not worked out the differences.
Also, if you have not figured out the pathgrid priority node encoding yet: it is stored in the odd/even value of the z position: even value == (red/orange) general/auto node, odd value == (blue) priority node.
Yes, I worked it out a day or two later. I've pushed that to my github repo around the end of March (the FaceGen stuff is also in the repo). I believe the pathgrids work the same way (both Morrowind and Oblivion), at least within the cell. How they work over long distance via roads, I don't know yet.

Post Reply