.ess Importer

Everything about development and the OpenMW source code.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51
Contact:

.ess Importer

Post by scrawl » 17 Jan 2015, 22:42

I have started an implementation for a .ess (Morrowind save game) importer.
Branch

What it currently does:
Filling in the metadata is completed, i.e. all the information you see in OpenMW when selecting the save.
Global map and Fog of War are loaded and written to .tga files, but more work is needed to get them into a format usable by openmw.
Some player state is converted, everything you see in the stats window, and the skill books read by the player.
Dynamic records (such as created enchantments, custom spells, custom potions etc) are converted. Custom player classes work. Global variables work.

Since 99% of the format is still unknown, I figured I'd post the method I am using in case others want to help.
Basically what I'm doing is load a save game in Morrowind, change one specific thing I'm looking for, save to a new file, then compare the two files. That is why I added a "--compare" switch to essimporter that compares two ess files.
This should be sufficient for figuring out the important stuff. Hrnchamd has offered to help in case there's something we can't make any sense of that way.

Before someone mentions the existing "documentation", yes I have seen it and it is vastly incomplete. But it might still be useful in some way, so here's a link.

I do not recommend using the current importer on a savegame created with the standard Morrowind.esm (still too many unknown subrecords to be able to load it). Instead use a mostly blank .esm with no start scripts or anything:
blank_esm.zip
(2.13 MiB) Downloaded 220 times
So, here's a rough TODO list:

- Dialogue & Journal
- Cell references
- Container/inventory state
- Global/local script state (variables), targeted scripts
- AI
- Magic effects

Currently I am looking at the cell references myself. Help in the other areas would be most welcome. Either just figuring out the format and noting it down somewhere or adding the code directly. The code has been structured to separate the original format from the conversion process, so there isn't actually any knowledge on OpenMW required.

User avatar
AlphaMode2
Posts: 28
Joined: 24 Aug 2014, 20:50

Re: .ess Importer

Post by AlphaMode2 » 18 Jan 2015, 02:43

YES! YES! YES!

This is the feature i've been waiting for.

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51
Contact:

Re: .ess Importer

Post by scrawl » 18 Jan 2015, 23:03

The player is placed in the correct cell now. Progress on CellRefs: placing NPCs and Creatures seems to work more or less. For other object types I need to fix the OpenMW save format first, so as to not require the object type in the file (which we do not know when importing).

I also added the names and order for various subrecords in CellRef and others. This makes all the save files on my computer load. However these are all testing save files and none of them have a significant amount of playtime.

So this is where I need your help: I need more .ess files to test with! The more, the better (I have a batch script for regression testing). Preferably a large save file with lots of playtime and lots of mods. It does not matter if the file is using mods that I don't have locally (the importer doesn't need the esm files to work).

User avatar
sjek
Posts: 425
Joined: 22 Nov 2014, 10:51

Re: .ess Importer

Post by sjek » 19 Jan 2015, 00:45

can the safefiles be sent directly to you via PM or posted here .?

is there a possibility to publish standalone executables later on as getting the build up and running in the first time can take considerable amount of time :)
"life is crazy"
"craziness has beauty which only crazies understand" some movie clip in the head.
https://wiki.openmw.org/index.php?title=Testing

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51
Contact:

Re: .ess Importer

Post by scrawl » 19 Jan 2015, 01:12

can the safefiles be sent directly to you via PM or posted here .?
Posting them in a .zip as forum attachment in this topic would be fine.
is there a possibility to publish standalone executables later on as getting the build up and running in the first time can take considerable amount of time :)
I just opened a pull request to get the (WIP) essimporter in the main branch, then it should start appearing in the nightly builds once Ace starts building it (which may need some changes to his build scripts). If they never show up, just ask him about it.

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51
Contact:

Re: .ess Importer

Post by scrawl » 21 Jan 2015, 21:53

More progress...
essimport1.png
essimport2.png
essimport3.png
The first two shots are of a savegame I downloaded from the Nexus. The last one is one of my testing characters.

It took a fair amount of head-scratching, but I believe I've got the numbering system for references all figured out now. As you can see, objects in the cell are now correctly loaded. Also equipped items work, which was a minor source of trouble.

When I started writing the importer, the main goals were:
- The importer shouldn't require loading the content files (esm/esps) in order to do its job.
- Changes to the OpenMW save format should be backwards compatible.
This is proving extremely difficult and I may have to compromise on either one of these goals later. There are tiny differences in the format all over the place that can be difficult or impossible to resolve without having the full context (i.e. esm's and esp's loaded).
For example, the OpenMW save format would say "this object is equipped in the left hand", whereas the ess format would say "this object is equipped in the first slot it can be equipped in". Obviously, since the importer doesn't know what kind of object it is, it can't know where the object can be equipped. What I settled for, was changing the OpenMW format to match, which I was able to do in a (mostly) backwards compatible manner.

The strategy further is to continue decoding the rest of the format, and worry about the conversion later, just so I have a good picture on whether or not I have to break compatibility in the OpenMW save format.
I'm going to have a look at the dialogue and journal now.

And FYI: the "I need more files" part is still accurate. I may have asked this in the wrong forum, since it is OpenMW after all and probably not a lot of people are playing Morrowind here. Maybe we should reach out on the Bethesda forums? I don't have an account over there though.

User avatar
AlphaMode2
Posts: 28
Joined: 24 Aug 2014, 20:50

Re: .ess Importer

Post by AlphaMode2 » 21 Jan 2015, 22:08

scrawl wrote:More progress...
Damn, this is really good. When it works, is there a chance you could send me one that works for a UNIX command line?

I have a tip on how to get saves: You could download saves from nexus, there are quite a few of them there. Also try reddit.

User avatar
sjek
Posts: 425
Joined: 22 Nov 2014, 10:51

Re: .ess Importer

Post by sjek » 21 Jan 2015, 22:12

And as of bethsoft can post it anytime : )

This is the current thread: http://forums.bethsoft.com/topic/149832 ... ad8/page-5

Edit: i have kinda habit to jump on forward and such.... but gonna craft tomorrow taking pick from upper text. Would put simply done and.... oh well..
Aaand done :mrgreen:
Last edited by sjek on 22 Jan 2015, 09:03, edited 2 times in total.
"life is crazy"
"craziness has beauty which only crazies understand" some movie clip in the head.
https://wiki.openmw.org/index.php?title=Testing

User avatar
AlphaMode2
Posts: 28
Joined: 24 Aug 2014, 20:50

Re: .ess Importer

Post by AlphaMode2 » 22 Jan 2015, 00:46

Could you send me a compiled version? For some reason it wound find my ogre installation :(.

User avatar
Lazaroth
Posts: 220
Joined: 30 May 2012, 05:04

Re: .ess Importer

Post by Lazaroth » 22 Jan 2015, 02:43

I searched the depths of my harddrive for some old ess backups and found one! It's big, so I've probably played for a while.
Attachments
quiksave.rar
(1.08 MiB) Downloaded 111 times

Post Reply

Who is online

Users browsing this forum: No registered users and 11 guests