Data Directory

Everything about development and the OpenMW source code.
User avatar
Site Admin
Posts: 1193
Joined: 05 Aug 2011, 22:21
Location: Wroclaw, Poland

Data Directory

Post by lgromanowski » 13 Aug 2011, 21:28

Zini wrote: While considering the implementation of the editor, I encountered a problem with the data directory that we have overlooked so far.

With an installed version of OpenMW/Morrowind the data directory typically sits in a location, that does not offer write access for an ordinary user. But the editor has to write to this location and we can hardly expect the user to run the editor as root.

Therefore I propose the following changes:

We add a new command line switch to openmw: --data-local. This will point to a 2nd data directory, that the user has write access to.

This would mean:

- OpenMW will have to look for ESMs, ESPs and BSAs in two directories.
- OpenMW will have to look for resources files in two directories.
- The Launcher will have to look for ESMs and ESPs in two directories.
- The Editor will have to read ESMs, ESPs, BSAs and resources files from two directories, but has to write to only one.

Unless someone has objections against this proposal, I will add appropriate issues with a couple more details to the tracker.
pvdk wrote: What is the installation directory for the data files? Isn't it easier to set the permissions? Two data directories doesn't appeal to me that much.
Star-Demon wrote: Think of it this way - Morrowind typically installs to a protected directory (program files...). While it's easy to fix by installing to a higher directory, we have to account for that contingency.
Zini wrote: Changing the access rights for the installation directory is not trivial. With a existing MW installation, the user should have the option to simply reuse this installation for data files (without the need to make a copy).

While conventional MW can be installed anywhere, MW bought via Steam will always be in a location without write-access for an ordinary user.
ap0 wrote: A simple
will be fine under linux.
I don't know much about permissions under windows, but I think it wouldn't be hard to do a
, isn't it ?

Having 2 data directory seems to be too much for me too.
sir_herrbatka wrote: There is some sort of steam wtf? here I guess.
pvdk wrote: I had a look on some folders in Program Files under Windows 7, and they are indeed non-writable to the Users group.

Maybe we can do a copy of the Data Files to another directory whilst installing OpenMW?
Zini wrote: We are talking of a directory, that can easily contain several GB. While HD space is plenty these days, I still don't think that is a good idea. And neither is changing the rights of the default installation directory. Sounds very unclean and may not even be possible on a true multi-user system, where not every user has root access.

What we could do is, use the default location, which is completely sufficient for playing. We could instruct users, who want to do modding, to manually move the data directory to a location of their choice and adjust the openmw.cfg file accordingly.
Sounds like a support-nightmare though. I can already see the forum posts piling up asking why the editor is not working.
Star-Demon wrote:
Zini wrote:Changing the access rights for the installation directory is not trivial. With a existing MW installation, the user should have the option to simply reuse this installation for data files (without the need to make a copy).

While conventional MW can be installed anywhere, MW bought via Steam will always be in a location without write-access for an ordinary user.
Completely forgot about that! Exactly.
sir_herrbatka wrote: ok, but how these steam version "owners" installs mods? Maybe they are already copied everything and there is no real problem.
pvdk wrote: We could go for a multiple directory approach but not with a separate command-line option IMHO. We could implement supporting multiple --data arguments and multiple data= lines in the config. It also shouldn't be limited to two directories.
Zini wrote: The editor needs to know which one is the local directory, i.e. the one it is supposed to write to.
Chris wrote: Having a separate (configurable) "local" data directory would be a very nice idea. Typical program behavior would have a program's data file in /usr[/local]/share in Linux, which allows read-only access for multiple users. It also makes sure to keep the app's data files clean, so you don't have to worry about errant modifications to the original data. Having a separate local data directory in ~/.local/share, or wherever the FDO spec says, gives an easy way to see, check, and clear out modified files, which can really help with troubleshooting. No more wondering "Is this file part of vanilla Morrowind?"

It can also help modders who want to make mods that don't depend on other mods. Just change the local data directory and it simultaneously clears away all other mods (without actually deleting them) and gives a unique place for your work-in-progress to go.
Zini wrote: I didn't expected that many objections, but if anything, the discussion has shown even more clearly that we need this feature. I'll modify the original proposal to allow multiple --data entries, but we keep one --data-local to not confuse the editor. I am going to add the tasks to the tracker now.

I take the OpenMW implementation. For the sake of reuse I will put the code into the files component. Launcher and editor should make use of it.

I leave the resources file handling open for grabs. We should address the Linux path problem first though.

@pvdk: I am going to assign the launcher part to you. Should not be much work. The file component should do most of it.
pvdk wrote: One question, since the data= entries are in the openmw.cfg, what exactly is the file component going to do? I could use my settings function to do it instead, since everything is in place already. I just need to iterate over all the data= entries and call setupDataFiles(path).

Edit: question about resource handling with multiple data dirs but that's in the tracker already.
Zini wrote: When a plugin/master with the same name exists in multiple directories, the one in the directory with higher priority must be picked. And that is exactly, what I am planning to implement in the file component.
pvdk wrote: Aha I get it now :) I'll wait for you to finish.
Zini wrote: Done. The changes are in my master branch.

I also added a new command line switch --fs-strict. This will disable all Window-related case-folding (currently only implemented for esms, esps and bsas). Most likely this mode won't work with most of current master and plugin files. But it is easy to implement and offers potential performance improvements. Future TCs might want to take advantage of it.

It would be nice if the launcher could read this option from openmw.cfg and pass it on to the file collection class.
pvdk wrote: Nice, looks good. Will see if I can implement it this weekend. The fs-strict option will definitely be passed on to the file collection.
Zini wrote: Great. We are almost done, right?


is handled now?

And this:

pvdk wrote: Yeah, both tasks are resolved now.

We're almost done indeed.

Edit: question should go in the launcher topic.
Zini wrote: Oops. My fault.
best regards,