OpenMW 0.47.0

Anything related to PR, release planning and any other non-technical idea how to move the project forward should be discussed here.
Post Reply
User avatar
AnyOldName3
Posts: 2158
Joined: 26 Nov 2015, 03:25

Re: OpenMW 0.47.0

Post by AnyOldName3 »

viewtopic.php?p=70938#p70938 might be another MacOS regression.
AnyOldName3, Master of Shadows
halcyon
Posts: 14
Joined: 05 Dec 2018, 00:32

Re: OpenMW 0.47.0

Post by halcyon »

psi29a wrote: 13 Jun 2021, 09:45 The MacOS issue is the risk that is taken when you have too many loose-files (mods) that OpenMW has to open. MacOS freaks out here and doesn't handle this very well.

Try this workaround:
https://discussions.apple.com/thread/251000125
Thanks for the link. That workaround looks simple enough to implement, but since it was unnecessary in earlier versions it seems that something is less than optimal in the way 47rc2 loads mods, unless there are some compensating advantages.
User avatar
Pop000100
Posts: 82
Joined: 18 Aug 2014, 21:17
Location: Loitering around Gnisis.

Re: OpenMW 0.47.0

Post by Pop000100 »

Even though I'm not really sure how to report issues for release candidate.
Two issues:
1. #6064 is still present on RC2 for windows(i understand it might be based on a older commit)
2. RC2(windows) doesn't have Bullet compiled with multithreading support.
User avatar
akortunov
Posts: 816
Joined: 13 Mar 2017, 13:49
Location: Samara, Russian Federation

Re: OpenMW 0.47.0

Post by akortunov »

hokan wrote: 13 Jun 2021, 19:47 I'm a bit curios why OpenMW needs to hold on to so many open file descriptors.
OpenMW does not load a data for the whole game world at once, it loads data from disk on demand (e.g. when you visit a new cell and the engine needs to get data for that cell). To do not open/close hundreds of ESM/ESP files every time, there was a decision to create a reading context for every content file and keep it in array, which persists for the whole game session. As a drawback, if you want to load a thousand of plugins, you need at least a thousand of file descriptors at once (what usually not a problem).
User avatar
psi29a
Posts: 5099
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OpenMW 0.47.0

Post by psi29a »

Which gets us to the point of having to support some type of archive format for mods, such as zip.

Which should be simple enough... take your mod, zip it up. If you need to modify a file in your zip, open zip, copy file over top, close zip. I'm reasonably sure that windows supports this in explorer anyway right?

As an example, 0ad by default, uses zips for everything from their public game data and for their mods too.
As a drawback, if you want to load a thousand of plugins, you need at least a thousand of file descriptors at once (what usually not a problem).
Sadly for MacOS, it is. :(

Though it is curious that for 0.46 it wasn't a problem and for 0.47 it is... I think more testing needs to be done here.
hokan
Posts: 7
Joined: 07 Jun 2020, 17:35

Re: OpenMW 0.47.0

Post by hokan »

psi29a wrote: 14 Jun 2021, 12:04 ...
As a drawback, if you want to load a thousand of plugins, you need at least a thousand of file descriptors at once (what usually not a problem).
Sadly for MacOS, it is. :(

Though it is curious that for 0.46 it wasn't a problem and for 0.47 it is... I think more testing needs to be done here.
Perhaps something else than content/groundcover (.esps) is holding on to a sizable number of file descriptors?
I currently have 219 content entries in my 0.46 openmw.cfg file, but I've noticed the "running out of file descriptor" issue with lower mod counts (~200) when trying nightly builds in the last few months.
I'm guessing that something else is holding onto 50+ file descriptors.
User avatar
psi29a
Posts: 5099
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OpenMW 0.47.0

Post by psi29a »

We have additional caching going on that writes stuff to disk.

Something we can investigate.

Did the links work to help work-around the problem on macOS?
User avatar
AnyOldName3
Posts: 2158
Joined: 26 Nov 2015, 03:25

Re: OpenMW 0.47.0

Post by AnyOldName3 »

The way we've got things set up now, when BSAs are involved, we've got one file handle per file in the BSA instead of one per BSA (or one per BSA per thread) that has a level of indirection so other parts of the engine still see it the same as any other file. I'm not totally sure about Morrowind, but all the later games use the latter approach and it gives a measurable performance increase on Windows, where creating file handles is expensive. It doesn't need zip archive support and would immediately help the limit issue on MacOS, too.
AnyOldName3, Master of Shadows
User avatar
psi29a
Posts: 5099
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OpenMW 0.47.0

Post by psi29a »

AnyOldName3 wrote: 14 Jun 2021, 21:21 The way we've got things set up now, when BSAs are involved, we've got one file handle per file in the BSA instead of one per BSA (or one per BSA per thread) that has a level of indirection so other parts of the engine still see it the same as any other file. I'm not totally sure about Morrowind, but all the later games use the latter approach and it gives a measurable performance increase on Windows, where creating file handles is expensive. It doesn't need zip archive support and would immediately help the limit issue on MacOS, too.
If we don't have this on the tracker, then we should. An "easy" performance win.
User avatar
psi29a
Posts: 5099
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: OpenMW 0.47.0

Post by psi29a »

Another round of RCs please!
Post Reply