Load order: reuse of mlox generated list?

[scrawl]

Continuing our PM discussion.

2. If you run openmw-iniimport from command line, is content=Blank_ESM_2.0.esm written to OpenMW.cfg? (Obviously, remove all content= lines from OpenMW.cfg before doing the import.) This will tell me if probem is in openmw-inimport or openmw-launcher.

Note: I used commit 834f4d016d068656 from your (now closed) pull request.

I just tested mwiniimport from the command line, like so:

Code: Select all

mwiniimport -g -c <openmw.cfg> -i <Morrowind.ini>

and the correct game files are in openmw.cfg (no Blank_ESM_2.0.esm in there), though they're in the wrong order:

Code: Select all

content=Tribunal.esm
content=Bloodmoon.esm
content=Morrowind.esm

When I then start the launcher, it reverts to "Blank_ESM_2.0.esm" as the selected game file. I assume that's because Tribunal.esm, the first file listed in openmw.cfg, is not a valid game file. When I change the openmw.cfg moving Morrowind.esm to the top, then start the launcher, everything works as expected.

One cause of trouble may be using symlinks. For instance, my Bloodmoon.esm is a symlink to "Bloodmoon (en).esm". Using stat on the files, it recognizes the correct modification date for the original file, but not for the symlink. Perhaps the importer should resolve symlinks before getting the modification date?

Code: Select all

$ file Bloodmoon.esm
Bloodmoon.esm: symbolic link to `Bloodmoon (en).esm'
$ stat Bloodmoon.esm | grep Modify
Modify: 2014-12-23 19:40:30.750540465 +0100
$ stat Bloodmoon\ \(en\).esm | grep Modify
Modify: 2003-07-07 16:27:56.000000000 +0200

In my Steam install, which isn't using any symlinks, the modification dates are not set correctly at all, so we can not rely on them, I guess.

Code: Select all

$ stat SteamApps/common/Morrowind/Data\ Files/Morrowind.esm | grep Modify
Modify: 2014-12-26 22:46:25.294051091 +0100
$ stat SteamApps/common/Morrowind/Data\ Files/Tribunal.esm | grep Modify
Modify: 2014-12-26 23:16:32.284746767 +0100
$ stat SteamApps/common/Morrowind/Data\ Files/Bloodmoon.esm | grep Modify
Modify: 2014-12-26 20:03:10.542421946 +0100

[dteviot]

@scrawl,

Thank you so much! Finally starting to make some progress.

When I then start the launcher, it reverts to "Blank_ESM_2.0.esm" as the selected game file. I assume that's because Tribunal.esm, the first file listed in openmw.cfg, is not a valid game file. When I change the openmw.cfg moving Morrowind.esm to the top, then start the launcher, everything works as expected.

OK, so that's probably a new bug, not directly related to the iniimporter.
I'll look into it.

One cause of trouble may be using symlinks. For instance, my Bloodmoon.esm is a symlink to "Bloodmoon (en).esm". Using stat on the files, it recognizes the correct modification date for the original file, but not for the symlink. Perhaps the importer should resolve symlinks before getting the modification date?

I've updated my private branch of the iniimporter to resolve symlinks.
https://github.com/dteviot/openmw/tree/Bug2346
If you care to try running it from the command line and see what the output looks like.

In my Steam install, which isn't using any symlinks, the modification dates are not set correctly at all, so we can not rely on them, I guess.

That's hard to explain. If they're not correct, how does Vanilla manage to load them in the correct order?
That said, if we do have these in the wrong order we should get the warning icons on the Data Files page in the loader.
With the exception of Tribunal and Bloodmoon getting mixed up. That said, I could probably arrange for a warning there also.
Do some hardcoding. When read the files, when Bloodmoon.esm is seen, add Tribunal as a dependancy.

[dteviot]

@scrawl,

When I then start the launcher, it reverts to "Blank_ESM_2.0.esm" as the selected game file. I assume that's because Tribunal.esm, the first file listed in openmw.cfg, is not a valid game file. When I change the openmw.cfg moving Morrowind.esm to the top, then start the launcher, everything works as expected.

OK, so that's probably a new bug, not directly related to the iniimporter.
I'll look into it.

Yup. It's a new bug. https://bugs.openmw.org/issues/2394Short form: launcher does NOT handle "root" content files well.

[scrawl]

I've updated my private branch of the iniimporter to resolve symlinks.
https://github.com/dteviot/openmw/tree/Bug2346
If you care to try running it from the command line and see what the output looks like.

Seems to be getting the actual timestamp now at least. But I just noticed that my Morrowind.esm doesn't have a correct timestamp at all. I can't remember if I tampered with it manually or if it was like that to begin with.
In fact, the Morrowind launcher shows the files in the wrong order too, with Morrowind.esm after Tribunal.esm. But when running the game it appears to "fix" the order according to dependencies.
So I think the least we need to do in the importer, is reorder files when necessary for loading a dependency.
And we do indeed need a warning for loading Bloodmoon.esm/Tribunal.esm in the incorrect order, maybe emit a load order warning in the launcher. The hardcoding is unfortunate, but I do not see another option.

[scrawl]

I found how to reproduce the bug where a deleted profile can show up again (commit f6128a85b2531806e from dteviot repo).

1. Make some modification to the content= lines in openmw.cfg.
2. Start the launcher, so that it creates a new timestamped profile.
3. Repeat steps 1 and 2 at least once, so you have at least 2 timestamped profiles.
4. Delete the timestamped profiles through the launcher UI.
5. Go to the Settings tab, and import Morrowind.ini gamefiles into a new profile.
6. Observe the profiles you deleted are back.

[mym]

However, the fact that the files are written to openmw.cfg in alphabetical order very strongly suggests problem is with mwiniimport. Can you add the logging to it, run it, and send me a copy of your morrowind.ini file and the logging mwiniimport writes to the console?.

I'm not sure of what log you need, so I produced 2 of them.
One by running directly the importer, and one by running the launcher and redirecting the output (and error channel) to a file:

Code: Select all

mehdi@mehdi:~/src/openmw/build$ ./openmw-iniimporter -v ~/Apps/Morrowind/Morrowind.ini ~/.config/openmw/openmw.cfg 
load ini file: /home/mehdi/Apps/Morrowind/Morrowind.ini
Warning: ignored empty value for key 'General:Beta Comment File'.
Warning: ignored empty value for key 'General:Editor Starting Cell'.
load cfg file: /home/mehdi/.config/openmw/openmw.cfg
write to: /home/mehdi/.config/openmw/openmw.cfg

Code: Select all

Loading config file: launcher.cfg 
Loading config file: /home/mehdi/.config/openmw/launcher.cfg 
Loading config file: /home/mehdi/.config/openmw/openmw.cfg 
Loading config file: /home/mehdi/.config/openmw/openmw.cfg 
Loading config file: openmw.cfg 
Loading config file: /usr/local/etc/openmw/openmw.cfg 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/Default/content", "Profiles/currentprofile", "Settings/language") 
Loading config file: /usr/local/etc/openmw/settings-default.cfg 
Loading config file: settings-default.cfg 
Loading config file: /home/mehdi/.config/openmw/settings.cfg 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/Default/content", "Profiles/currentprofile", "Settings/language") 
current profile is:  "Default" 
arguments  ("--game-files", "--encoding", "win1252", "--ini", "/home/mehdi/Apps/Morrowind/Morrowind.ini", "--cfg", "/home/mehdi/.config/openmw/openmw.cfg") 
Loading config file: launcher.cfg 
Loading config file: /home/mehdi/.config/openmw/launcher.cfg 
Loading config file: /home/mehdi/.config/openmw/openmw.cfg 
Loading config file: /home/mehdi/.config/openmw/openmw.cfg 
Loading config file: openmw.cfg 
Loading config file: /usr/local/etc/openmw/openmw.cfg 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/Default/content", "Profiles/currentprofile", "Settings/language") 
Loading config file: /usr/local/etc/openmw/settings-default.cfg 
Loading config file: settings-default.cfg 
Loading config file: /home/mehdi/.config/openmw/settings.cfg 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/2015-02-22T18:29:06/content", "Profiles/Default/content", "Profiles/currentprofile", "Settings/language") 
current profile is:  "2015-02-22T18:29:06" 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/2015-02-22T18:29:06/content", "Profiles/Default/content", "Profiles/currentprofile", "Settings/language") 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/2015-02-22T18:29:06/content", "Profiles/Default/content", "Profiles/currentprofile", "Settings/language") 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/2015-02-22T18:29:06/content", "Profiles/Default/content", "Profiles/currentprofile", "Settings/language") 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/2015-02-22T18:29:06/content", "Profiles/Default/content", "Profiles/currentprofile", "Settings/language") 
Loading config file: launcher.cfg 
Loading config file: /home/mehdi/.config/openmw/launcher.cfg 
Loading config file: /home/mehdi/.config/openmw/openmw.cfg 
Loading config file: /home/mehdi/.config/openmw/openmw.cfg 
Loading config file: openmw.cfg 
Loading config file: /usr/local/etc/openmw/openmw.cfg 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/Default/content", "Profiles/currentprofile", "Profiles/test/content", "Settings/language") 
Loading config file: /usr/local/etc/openmw/settings-default.cfg 
Loading config file: settings-default.cfg 
Loading config file: /home/mehdi/.config/openmw/settings.cfg 
("General/MainWindow/height", "General/MainWindow/posx", "General/MainWindow/posy", "General/MainWindow/width", "General/firstrun", "Profiles/Default/content", "Profiles/currentprofile", "Profiles/test/content", "Settings/language") 
current profile is:  "test" 

[dteviot]

@mym

I'm not sure of what log you need, so I produced 2 of them.
One by running directly the importer, and one by running the launcher and redirecting the output (and error channel) to a file:

Code: Select all

mehdi@mehdi:~/src/openmw/build$ ./openmw-iniimporter -v ~/Apps/Morrowind/Morrowind.ini ~/.config/openmw/openmw.cfg 
load ini file: /home/mehdi/Apps/Morrowind/Morrowind.ini
Warning: ignored empty value for key 'General:Beta Comment File'.
Warning: ignored empty value for key 'General:Editor Starting Cell'.
load cfg file: /home/mehdi/.config/openmw/openmw.cfg
write to: /home/mehdi/.config/openmw/openmw.cfg

Sorry for not being clear. What I wanted was the output to the console when openmw-iniimport was run from the command line. i.e. The first "log" you sent me. (and shown in above quote)

Unfortunately, looks like I also wasn't clear about the command line to use either. It seems to be missing the "--game-files" parameter that tells the openmw-iniimport to copy the content files as well. The command line needs to be:

Code: Select all

mehdi@mehdi:~/src/openmw/build$ ./openmw-iniimporter ~/Apps/Morrowind/Morrowind.ini --cfg  ~/.config/openmw/openmw.cfg --game-files

at least, I believe the --game-files parameter is missing, as I don't see it on the command line, and there's no output lines that I would expect to see if it was importing the content files.
Could you please re-run?

@scrawl

I found how to reproduce the bug where a deleted profile can show up again (commit f6128a85b2531806e from dteviot repo).

Thanks for that. I see what's gone wrong.

• The launcher doesn't write any changes to launcher.cfg or openmw.cfg until you hit "run game" or "close" buttons. So when you delete a profile, it's just deleting it from the in-memory structures.
• After the openmw-iniimporter runs, owlauncher re-reads openmw.cfg and launcher.cfg to get the changes.
• So fix should be, after openmw-iniimporter runs, owlauncher only reads openmw.cfg.

I've updated my private branch of the iniimporter to resolve symlinks.
https://github.com/dteviot/openmw/tree/Bug2346
If you care to try running it from the command line and see what the output looks like.

Seems to be getting the actual timestamp now at least. But I just noticed that my Morrowind.esm doesn't have a correct timestamp at all. I can't remember if I tampered with it manually or if it was like that to begin with. In fact, the Morrowind launcher shows the files in the wrong order too, with Morrowind.esm after Tribunal.esm.

That says to me that the the changes to the ini-importer to get the time modified are OK.

In fact, the Morrowind launcher shows the files in the wrong order too, with Morrowind.esm after Tribunal.esm. But when running the game it appears to "fix" the order according to dependencies.
So I think the least we need to do in the importer, is reorder files when necessary for loading a dependency.
And we do indeed need a warning for loading Bloodmoon.esm/Tribunal.esm in the incorrect order, maybe emit a load order warning in the launcher. The hardcoding is unfortunate, but I do not see another option.

I've already suggested to Zini that the launcher should give a warning when there is a load order problem.
He was, let's say, less than enthusiastic about the idea. Probably because I suggested using a message box to give the warning. Zini REALY hates message boxes.
If I just disable the "run game" button on the launcher and indicate some how that it's because there's a load order problem (maybe a tooltip?) he might be agreeable.

If I do that, then I don't need to Re-ordering the files to comply with load order as given by the game file entry in each file, as doing that conflicts with requirement to show the load order as it IS in openmw.cfg.

And we do indeed need a warning for loading Bloodmoon.esm/Tribunal.esm in the incorrect order,

Now, THAT I can do easily.

[Zini]

He was, let's say, less than enthusiastic about the idea. Probably because I suggested using a message box to give the warning. Zini REALY hates message boxes.

To clarify this: I don't mind message boxes so much for actual errors. But just for informational messages I don't think they are a good option. Repeatedly throwing message boxes at the user that he might be able to ignore will just result in him ignoring them all together (combined with becoming really annoyed). We should save messages boxes for situations where they are direly needed, so we can avoid the user numbing himself to them.

If I just disable the "run game" button on the launcher and indicate some how that it's because there's a load order problem (maybe a tooltip?) he might be agreeable.

Don't do that. There may be situations where even in case of a load order issue the user will want to continue. A tooltip would be fine though. And maybe change the text of the run button to include a warning sign or something?

And we do indeed need a warning for loading Bloodmoon.esm/Tribunal.esm in the incorrect order,

Agreed. I don't like hardcoding things, but this is a special case. This needs to be flagged in the content list with the usual warning symbol and via the mechanism discussed above.

[mym]

Unfortunately, looks like I also wasn't clear about the command line to use either. It seems to be missing the "--game-files" parameter that tells the openmw-iniimport to copy the content files as well. The command line needs to be:

Code: Select all

mehdi@mehdi:~/src/openmw/build$ ./openmw-iniimporter ~/Apps/Morrowind/Morrowind.ini --cfg  ~/.config/openmw/openmw.cfg --game-files

Unfortunately it doesn't change anything:

Code: Select all

mehdi@mehdi:~/src/openmw/build$ ./openmw-iniimporter -i ~/Apps/Morrowind/Morrowind.ini -c ~/.config/openmw/openmw.cfg -g -v
load ini file: /home/mehdi/Apps/Morrowind/Morrowind.ini
Warning: ignored empty value for key 'General:Beta Comment File'.
Warning: ignored empty value for key 'General:Editor Starting Cell'.
load cfg file: /home/mehdi/.config/openmw/openmw.cfg
write to: /home/mehdi/.config/openmw/openmw.cfg

[dteviot]

Unfortunately it doesn't change anything:

Code: Select all

mehdi@mehdi:~/src/openmw/build$ ./openmw-iniimporter -i ~/Apps/Morrowind/Morrowind.ini -c ~/.config/openmw/openmw.cfg -g -v
load ini file: /home/mehdi/Apps/Morrowind/Morrowind.ini
Warning: ignored empty value for key 'General:Beta Comment File'.
Warning: ignored empty value for key 'General:Editor Starting Cell'.
load cfg file: /home/mehdi/.config/openmw/openmw.cfg
write to: /home/mehdi/.config/openmw/openmw.cfg

@mym
That first line of logging does not match what I get. I'm not sure where those -i, -g and -v params come from.
I wonder if maybe, and please don't be offended, but somehow you're not running the patched importer.

That said, I think Scrawl said it was now working for him. So it's possible I no longer need your assistance.