Testing performance in linux (and workaround for load order)

[xander2077]

Ok, so now i have openmw installed and working, it is not complaining about missing textures any more or taking forever to load because of a plugin error. but what is happening is this

1: the load order seems to be haphazard, and different each time on the same save game. or if i start a new one on what is supposedly the same load order it gives weird results, like one load when i went into the census office and the old man is there, he was a young man with no beard, the next time he was an old man with a brown wig on. this is definitely a load order problem and openmw is not being consistent. trying to force the load order only screws up the config file and it resets it to default or rewrites/erases everything based off of how it interprets the morrowind.ini differently each time. nothing relevant shows in the log when it does this. i also cannot manually change the load order in the config file either, or it will wipe my entries and i have to import morrowind.ini all over again. i also cannot add plugin data locations into the config file or they will be scrubbed by openmw as if they were never there.

2: when entering interiors, the speed is great. i have not played a working version of morrowind so long that i forgot what it felt like to be able to move freely without being sluggish, however, when i go through a door to an exterior location, that is when it gets sluggish. and i don't have any idea how i can change settings in the config to get it to load faster, or keep from having to load cells every 100 feet or so. whenever it loads exterior cells it loads slow enough, but then at the end it freezes for almost 3 minutes before it finally lets me move in the direction i was going. and i think this also has to do with the way openmw handles the load order (or at least part of the reason), and something got screwed up because from what i was reading it is supposed to be something that was already fixed long ago.

any way to fix this? could the engine be tweaked to handle exterior cells in a similar way to the interior ones? why is the load order implementation so random and finicky? why is there no decent tutorial on how to tweak these things on the user end for those using linux? i looked everywhere for a decent explanation on how to add plugins or different data directories in openmw on linux by editing the config file, or how to force a load order a certain way in the config file, but the explanation is vague, it assumes that whoever is running it already knows how to do it, but for those of us new to openmw, we have no idea how to do that properly step by step and don't keep up with build changes like insiders and devs have access to. but it could also be that the config file doesn't allow changes so it reverts back to the default settings and everything that was edited is lost? is there any way to fix these things? it is not something related to my permissions in linux, because if that were true, then i would not be able to edit and save the config file at all. it would be read only, so it has to do with something in openmw, and the log (once again) doesn't show anything relevant to this problem, so i am assuming the engine doesn't recognize it as a bug or error. i realize this engine is a WIP, but it can be frustrating for those of us who dont know what was changed or why, or how to do user side fixes.

UPDATE: i have done a bit of reading, and apparently the reason why there was a lag in load time is because the value for loaded cells was 9 instead of 1 (default) so the load speed is much better now, almost instantaneous, as well as while you walk, which of course is a minor annoyance, but pretty smooth even if it does have the loading bar for a split second.

but i am still not sure how to solve the load order issue. supposedly there was a patch developed that could fix the issue, but i am not sure how to use it or if i have to make my own build of openmw launcher for the patch to work using the patch in the build. so i wont mess with it. it was supposed to fix the load order problem but i guess that got forked or has to be redone after the migration to OSG, so i dont want to bork my game with experimentation.

what would be cool is if there was a way to implement the ability in the menu tab for data files (where you check or uncheck morrowind archives) to be able to move them up or down in the list which would then be written in the same order in the config file. the best way to do this (imho) is to adjust the order by simply highlighting the mod you want in the list to move up or down, and then have up and down arrows to the side that will only appear if the whole line of the mod is selected (not the check box) and then click on one or the other (up or down) to move them, similar to the menu for the panel in lubuntu that can move panel items back and forth. and im sure there are other examples of this in mac or windows. it would be an easy way to move the load order around. not sure if coding it would be easy, but it would eliminate the need to try and base the load order off of a variable like alpha numeric order, or the date a mod was created. reason being is some mods should be loaded later than others. the main BSA files should always be first before mods, and then mods come later depending on the user preference usually, and in some cases compatibility between mods comes into play.

the lat thing i am still curious about is how in the heck do users in linux add extra mod directories to their config file? i have tried every combination i can think of and nothing works. i guess i will just have to open the morrowind directory and paste the mod in i want after i back the current folders up, because there are too many directories to clean if it screws things up.

UPDATE: I figured out a way to fix the load order in openmw (probably not the first to think of this) so the launcher doesn't load things out of order. this may be useful to someone since the launcher doesn't give the option to deviate from the date of the mod file creation. it may not seem like an easy fix to some, but i simply used wyremash to change the file creation date on each mod i wanted to load in a specific order.

sadly most of the features in wyremash don't work correctly in wine, but the upside to it is it will allow you to point to a directory outside of wine so you can edit the morrowind.ini file that is in the directory you use with openmw. the dates can be changed to anything you desire. it doesn't care if you change the date to something that is in the future, it still works. the danger to this is>>>> ***WARNING, DO NOT change the dates on any of the main archives like morrowind tribunal or bloodmoon BSA/esm files***. that will cause a failed attempt to load correctly and the launcher will not allow any of those to be loaded out of order, in fact it will have a caution symbol next to the mods that either depend on another BSA/esm, or if say bloodmoon had a date set before morrowind, then it would not allow clicking on tribunal and it can really screw up mods when they depend on one of the main BSA files, so just don't change that. if you do then try to put the earliest dates on those main three files in order of how they should load so they load before anything else at all. or you could just date anything that has an earlier date than those three to something a year or two later than bloodmoon and tribunal. that seems to fix a lot of things when some official mods/archives that were updated have a creation date that is later than user created mods, and you want everything to load after the main three.

anyway, so now after i did that, i simply went back to the settings tab, and either run the wizard again, or import settings again, and it loads the edited morrowind ini, and it also accepts the new dates for mod file creation and logs it correctly in the config file in the openmw directory. this should be done prior to selecting any mods, in fact i just deleted all the profiles already there so i would not choose the wrong one later. once the new settings are imported, then you can go back to the data files selection tab and check all the mods you want to run and they will appear out of order in the launcher list, but will load according to creation date (as edited in wyremash) and everyone is happy. perhaps this last post (not the whole thread) should be pinned and added to the wiki for anyone who wants to use this method to order their mods if all else fails. i know it will probably be just about the only way for people using linux to get it to work, unless something changes at a later date.

for anyone wondering how to change the dates for each mod file, its is pretty simple. i wont go through how to install wyremash, but winedb has a lot of help there, i use q4wine to install it and load it after using winetricks to make sure all the dependencies are installed. then i use q4wine to run the program (seems to be the best way if you don't have a shortcut on the desktop, and it is less messy than playonlinux, but ymmv) once it starts it will ask you to point to your morrowind directory (it is up to you to point it to the correct folder) then wyremash loads the mods, and in the mod tab it has them listed similarly to things like mlox or even the openmw launcher. in linux under wine this is the only tab that really does anything, but i am glad it does something. simply click on the mod entry (not the check box but the text of the name in the list) you want to edit, (date, name, etc), and to the right there are details in the sidebar. each field can be edited, even the name can be changed or the description (especially if there is none and you want to remember what a mod is and the mod name itself doesn't jog your memory) but there in the date section you can edit the date to whatever you want, even some time in the future, and then at the bottom of the window you click the save button and its done. do this to any mod you want besides the main files. you can change the dates to reflect the order you want your mods to load in, so if you want one to load before another, you simply change the date to an earlier date, or change the one you want to load last to a date after everything else, and so on. loading last means it overrides anything that was loaded before it. so, for example, if you want one facepack to override another, so you can allow the first one to fix faces that another one missed, then load the better or smaller of the two last. or if you want a certain scripted mod to run even though another mod prevents it, then load that mod after the mod that would normally prevent it from working.

[xander2077]

UPDATE: i have done a bit of reading, and apparently the reason why there was a lag in load time is because the value for loaded cells was 9 instead of 1 (default) so the load speed is much better now, almost instantaneous, as well as while you walk, which of course is a minor annoyance, but pretty smooth even if it does have the loading bar for a split second.

but i am still not sure how to solve the load order issue. supposedly there was a patch developed that could fix the issue, but i am not sure how to use it or if i have to make my own build of openmw launcher for the patch to work using the patch in the build. so i wont mess with it. it was supposed to fix the load order problem but i guess that got forked or has to be redone after the migration to OSG, so i dont want to bork my game with experimentation.

what would be cool is if there was a way to implement the ability in the menu tab for data files (where you check or uncheck morrowind archives) to be able to move them up or down in the list which would then be written in the same order in the config file. the best way to do this (imho) is to adjust the order by simply highlighting the mod you want in the list to move up or down, and then have up and down arrows to the side that will only appear if the whole line of the mod is selected (not the check box) and then click on one or the other (up or down) to move them, similar to the menu for the panel in lubuntu that can move panel items back and forth. and im sure there are other examples of this in mac or windows. it would be an easy way to move the load order around. not sure if coding it would be easy, but it would eliminate the need to try and base the load order off of a variable like alpha numeric order, or the date a mod was created. reason being is some mods should be loaded later than others. the main BSA files should always be first before mods, and then mods come later depending on the user preference usually, and in some cases compatibility between mods comes into play.

the lat thing i am still curious about is how in the heck do users in linux add extra mod directories to their config file? i have tried every combination i can think of and nothing works. i guess i will just have to open the morrowind directory and paste the mod in i want after i back the current folders up, because there are too many directories to clean if it screws things up.

[Cramal]

for the load order, if you want to force 1 mod to load before another you should put these mods in 2 different directories.

Then you can control the load order by the order in which you write the 2 directories in your config file

[xander2077]

for the load order, if you want to force 1 mod to load before another you should put these mods in 2 different directories.

Then you can control the load order by the order in which you write the 2 directories in your config file

well, if i had known how to do that, then it probably would not be an issue, but it is kinda too late for it now. i asked how to do that in the OP. in fact i don't even know what to do step by step, so telling me what i should do, without instructions on how to do it, kind of defeats the advice. the wiki doesnt explain it, and neither do the forum topics.

the few forum topics i could find explain how to use extra directories in windows and mac *after* people come here and ask because the wiki is so vague, but so far nobody has explained the correct directories to use for linux, you think that information like this or other helpful topics would be pinned so people don't have to go fighting through hundreds of hours of searching or ask the same questions over and over.

and as i explained in detail above, i tried to create another directory that i could point openmw launcher and the config file to, and nothing i typed in the file stayed, it would always erase and revert back to default settings. so adding it there doesn't help at all no matter what i type in the config file related to load order. i dont know if it is becasue i have added the extra directories to the morrowind folder, or if i need to instead add the directories somewhere else that openmw expects. i also dont know if because the directory is not found for some reason by openmw, why it erases the entry i typed so i have to try again. there is no warning or anything, it just erases what it considers to be invalid information.

i really think that if the openmw launcher is going to rewrite entries when it opens the config file, that it should be the tool that controls the load order, not the user. a simple way to do that is to add functionality to the data files section of the launcher, so instead of a simple list with check boxes, each list entry is select-able to highlight, and buttons are there so the entry can be moved up or down in the load order. that would go a long way to solving the problem for linux users who cant edit the config file because openmw launcher keeps rewriting or erasing whatever they wrote in the config file. as it stands right now, the coding for the load order is set to order the list according to alphanumeric order first, then by whatever other parameters are there. but i think that is a mistake, its not thinking outside the box. instead of trying to make the launcher set the order based off of parameters, it should just copy the list from the order in morrowind.ini, adding parameters doesnt seem to work and just makes it more complicated. and if it just loads the list without those parameters on how to order it, then the next step would be easier, adding buttons to allow the user to move things up or down in the list themselves so they can see the visual effects or mods they want in the priority they want. that way if one mod cancels parts of another out, they can pick the best of the two.

i also think detailed instructions for all three major OS's on how to add extra directories that openmw will recognize, should be pinned. and that perhaps they should also be added to the wiki when people go looking there. they can be edited when and if the methods change.

my goal here is to try and help figure out how to run this on linux since it is the only option for many of us who cannot gain the benefits of MGE any more, or morowind runs so bad under wine it is unplayable. so whatever i find out here is bound to help others looking for the same morrowind satisfaction.

just reading the thread on load order [ viewtopic.php?f=8&t=2751&start=10 ] shows me that it is being approached in the wrong way. the coding in it is already overcomplicating things that don't need to be complicated, so any further attempts to fix the issue in linux are defeated by the strict coding when it loads the list from morrowind.ini

if it just copied the load order from morrowind.ini, without strict parameters based off of alphanumeric order, or capitalization, then functionality to change the load order would be easier to add to the launcher itself. but the stumbling block is already written into the code. it can be simplified and streamlined in a much more logical fashion.

[xander2077]

UPDATE: I figured out a way to fix the load order in openmw (probably not the first to think of this) so the launcher doesn't load things out of order. this may be useful to someone since the launcher doesn't give the option to deviate from the date of the mod file creation. it may not seem like an easy fix to some, but i simply used wyremash to change the file creation date on each mod i wanted to load in a specific order.

sadly most of the features in wyremash don't work correctly in wine, but the upside to it is it will allow you to point to a directory outside of wine so you can edit the morrowind.ini file that is in the directory you use with openmw. the dates can be changed to anything you desire. it doesn't care if you change the date to something that is in the future, it still works. the danger to this is>>>> ***WARNING, DO NOT change the dates on any of the main archives like morrowind tribunal or bloodmoon BSA/esm files***. that will cause a failed attempt to load correctly and the launcher will not allow any of those to be loaded out of order, in fact it will have a caution symbol next to the mods that either depend on another BSA/esm, or if say bloodmoon had a date set before morrowind, then it would not allow clicking on tribunal and it can really screw up mods when they depend on one of the main BSA files, so just don't change that. if you do then try to put the earliest dates on those main three files in order of how they should load so they load before anything else at all. or you could just date anything that has an earlier date than those three to something a year or two later than bloodmoon and tribunal. that seems to fix a lot of things when some official mods/archives that were updated have a creation date that is later than user created mods, and you want everything to load after the main three.

anyway, so now after i did that, i simply went back to the settings tab, and either run the wizard again, or import settings again, and it loads the edited morrowind ini, and it also accepts the new dates for mod file creation and logs it correctly in the config file in the openmw directory. this should be done prior to selecting any mods, in fact i just deleted all the profiles already there so i would not choose the wrong one later. once the new settings are imported, then you can go back to the data files selection tab and check all the mods you want to run and they will appear out of order in the launcher list, but will load according to creation date (as edited in wyremash) and everyone is happy. perhaps this last post (not the whole thread) should be pinned and added to the wiki for anyone who wants to use this method to order their mods if all else fails. i know it will probably be just about the only way for people using linux to get it to work, unless something changes at a later date.

[Okulo]

well, if i had known how to do that, then it probably would not be an issue, but it is kinda too late for it now. i asked how to do that in the OP. in fact i don't even know what to do step by step, so telling me what i should do, without instructions on how to do it, kind of defeats the advice. the wiki doesnt explain it, and neither do the forum topics.

Here, the wiki explains it.

[xander2077]

well, if i had known how to do that, then it probably would not be an issue, but it is kinda too late for it now. i asked how to do that in the OP. in fact i don't even know what to do step by step, so telling me what i should do, without instructions on how to do it, kind of defeats the advice. the wiki doesnt explain it, and neither do the forum topics.

Here, the wiki explains it.

it doesn't explain it step by step for linux, it just refers to the directory structure and kind of hints at what to do, but doesn't really explain it like a tutorial with a, b, c etc... in short it is worded badly and too nondescript. i want specifics, not hints. it is not an easter egg hunt.

[Antsan]

I am assuming you're on Linux:
a) Find the file `openmw.cfg`
On Linux this should be `~/config/openmw/openmw.cfg`.
b) Open that file
c) Scroll down until you find lines starting with `data=`
d) To add a mod residing in it's own folder (I assume you know how to unpack archives) add a line below the lines starting with `data=` that looks like this:

Code: Select all

data="/path/to/your/mod/folder"

A trailing `/` isn't necessary. Note that this is the path to the folder, not to any file in that folder!
e) Open OpenMW-launcher and activate the necessary ESPs
f) There is no step f). There are exactly five steps, so with our Latin alphabet this means that step e) is the last one. Step f) would after step e), which is the last one, so step f) doesn't exist, because then there would be a step after the last step, which is a contradiction with the last step being the last step.

Five steps, excluding unpacking the archive containing the mod.

The only thing missing from the wiki is the fact that you need to add the `data=` line and that can be easily inferred from the linked article and the config file itself.
Here's the important sentence in the article:

When using multiple data folders, the priority of each mod can simply be changed by moving its data= line in openmw.cfg to a different place. The last data= line has the highest priority.

This unquestionably tells us that a mod data folder has a `data=` line.
Please remember that this is a project and wiki maintained by volunteers. If you find the documentation lacking you can ask nicely or try to figure it out yourself. Even better: If you figured out how to do it ask for wiki access and correct the documentation.

[xander2077]

I am assuming you're on Linux:
a) Find the file `openmw.cfg`
On Linux this should be `~/config/openmw/openmw.cfg`.
b) Open that file
c) Scroll down until you find lines starting with `data=`
d) To add a mod residing in it's own folder (I assume you know how to unpack archives) add a line below the lines starting with `data=` that looks like this:

Code: Select all

data="/path/to/your/mod/folder"

A trailing `/` isn't necessary. Note that this is the path to the folder, not to any file in that folder!
e) Open OpenMW-launcher and activate the necessary ESPs
f) There is no step f). There are exactly five steps, so with our Latin alphabet this means that step e) is the last one. Step f) would after step e), which is the last one, so step f) doesn't exist, because then there would be a step after the last step, which is a contradiction with the last step being the last step.

Five steps, excluding unpacking the archive containing the mod.

The only thing missing from the wiki is the fact that you need to add the `data=` line and that can be easily inferred from the linked article and the config file itself.
Here's the important sentence in the article:

When using multiple data folders, the priority of each mod can simply be changed by moving its data= line in openmw.cfg to a different place. The last data= line has the highest priority.

This unquestionably tells us that a mod data folder has a `data=` line.
Please remember that this is a project and wiki maintained by volunteers. If you find the documentation lacking you can ask nicely or try to figure it out yourself. Even better: If you figured out how to do it ask for wiki access and correct the documentation.

ok, i get what you have to edit, but i dont get why an archive has to be unpacked to make this work, cant i just empty the contents of a pluginless mod into an extra folder and point the config file to that? BSA archives are designed to make it easy to install or uninstall a mod already, no need to muck things up by unpacking them in a new folder, seems convoluted to me. what i really want to know is how to use the config file the way it was intended to function, how to get the entries to stay in the file without being erased, and if i can use it on a folder of an unpacked bsa without still clicking on the bsa. seems redundant to point to a folder if i still have to click on the bsa file. the plugins i mainly use are not bsa, they may have an esp, but im not so sure its a good idea to mess with those. if you mean archives in zip or 7z or whatever, then yeah you unpack those into a folder and point the config file to those. however i have done that and the lines get deleted when i start the launcher. i go back into the file and everything i typed is gone. so something is changing it for me and it must be the launcher.

i get that the people writing the wiki are volunteers, and this is on their own time, but i also think that there is a better way to present the information. i had the same problem on gitub, because they dont have instructions posted on how to navigate their user unfriendly web pages and instructions, they let you play guessing games to find the answer. then i spent 3 days trying to get nif for blender to work only to find the link for the py script i needed to install it was broken, or the file contains html4 instead of python code, so it wont install, that is their issue not mine, as well as their convoluted instructions.

i dont know the answer or i would not be asking, therefore it would be silly to ask for permission to adjust the wiki...and i read the wiki and it didn't make sense to me that is why i asked here, because it doesn't make any sense to me, im missing key information. and frankly your instructions are ok, but still lacking in detail.

perhaps a step by step video tutorial would be better, because then people can see the steps, familiarize themselves with what they should see when they do it themselves, and someone is explaining what they are doing while they do it and why it is important, or why not to skip step 3 or whatever. that leaves no room for error, because a successful video tutorial will show everything that needs to be done, or it will make the presenter look like they dont know what they are doing if it doesn't explain anything.

i used to give classes and presentations for a living once upon a time, so i kinda know how to present information, but in this case, im lacking that information. i have asked nicely, and people either dont listen to what im saying, or they give me the wrong information or a fix for another problem entirely, or they give a pat answer that i already have in front of me in a firefox tab that tells me nothing new. kinda like asking someone where something is in the garage, and they tell you "the garage"... well duh! i want to have a eureka moment here, not scratch my head and waste more time. i've already lost days trying to figure things out, so i hope you understand why i come off as a bit irritated when im not getting the information i need to accomplish something.

anyway, if i cant get a crystal clear answer, oh well, then i will just have to do things my way instead of the openmw way and install things like they were in vanilla MW. with the data files folder and my own workaround with wyremash. it wold be nice to know how to add extra directories in the config file, but if i never figure it out, i already found a way to make my load order work. the reason i will still harp on this is because im not the only one who has asked this same question, and it would benefit the user, not just me. it makes no sense to dig through hours of pages in a cluttered forum when the information can be posted clearly in the wiki for each operating system. whether by a blog style tutorial (kinda like the tweak guides for games) or in a video tutorial that is linked in the wiki, one for each OS.

here is an example of a proper tutorial on how to install something and set it up. notice the detail and the visual aids.

https://www.veritas.com/support/en_US/a ... TECH229077

[Chris]

ok, i get what you have to edit, but i dont get why an archive has to be unpacked to make this work, cant i just empty the contents of a pluginless mod into an extra folder and point the config file to that?

That's what it says. Extract/empty the contents of mod so the data files (meshes, textures, sounds, etc) are in an extra folder, and then point a data= line to that folder.

BSA archives are designed to make it easy to install or uninstall a mod already, no need to muck things up by unpacking them in a new folder, seems convoluted to me.

BSA archives are unnecessary. They're designed to keep all the data packed in a single place, which is just what a directory does. They're also not an open format, and can't be as easily browsed like a directory or zip file. The vanilla engine didn't support multiple data directories, so they used multiple archives with a "loose files" override instead. Maybe there was an extra benefit for consoles, which needed to read from slow CDs, but there's little to no difference between a BSA archive and a directory on a PC.

If you want to use a BSA archive as-is, what you need to do is either add it to Morrowind's Data Files directory (where Morrowind.bsa, Tribunal.bsa, etc are), or add a data= line that points to the directory the BSA is in, and then add a fallback-archive= line for the BSA. Just be aware that the fallback archives always come before data directories. Files in data directories will override the same files in archives.

what i really want to know is how to use the config file the way it was intended to function, how to get the entries to stay in the file without being erased, and if i can use it on a folder of an unpacked bsa without still clicking on the bsa.

What do you mean by clicking on a BSA?

however i have done that and the lines get deleted when i start the launcher. i go back into the file and everything i typed is gone. so something is changing it for me and it must be the launcher.

I know the launcher messes up load order for esps (it's supposed to preserve order, but IIRC there's a bug that causes the list to be reordered). I've not heard of it removing data= entries, though. Might want to check https://bugs.openmw.org/ and see if there's a similar bug reported, and if not, make one and provide step-by-step instructions for reproducing it.