Cell preloading

General discussion regarding the OpenMW project.
For technical support, please use the Support subforum.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Cell preloading

Post by scrawl » 07 Feb 2016, 22:50

Just a quick heads up of a new feature I'm working on. Background preloading of cells.

My goal is to make flying around the island buttery smooth with no loading bars or frame drops whatsoever. I'm not quite at that point yet, mostly because terrain isn't preloaded yet, body parts are not preloaded yet and because I haven't profiled / optimized the code at all yet. Still, the loading times compared to the main branch should be much improved already.

Currently the preloader will activate if you go near a teleport door, or go near an exterior cell border. I'm thinking about preloading fast travel destinations as well, though of course the downside is we don't know the travel destination the player will pick, so would have to load all of them preemptively.

Feel free to try out the branch and let me know how it goes! I haven't come across any crashes or instabilities yet, which is a good sign that the new resource manager design is solid.

User avatar
Ace (SWE)
Posts: 823
Joined: 15 Aug 2011, 14:56

Re: Cell preloading

Post by Ace (SWE) » 07 Feb 2016, 23:03

I'll throw up a set of Windows builds for this tomorrow I think, I want to see this for myself.

TheMechanist
Posts: 52
Joined: 10 Jan 2014, 22:08

Re: Cell preloading

Post by TheMechanist » 07 Feb 2016, 23:59

That's awesome. This is way up my "Please-come-before-1.0-feature" list 8-)

User avatar
ElderTroll
Posts: 499
Joined: 25 Jan 2012, 07:01

Re: Cell preloading

Post by ElderTroll » 08 Feb 2016, 01:29

Nice! My 6 year old netbook is totally incapable of taking advantage of that, but really exciting stuff, scrawl. Saw that the trello board had moved finished the resource management refactoring to done. :P
“A society grows great when old men plant trees whose shade they know they shall never sit in.”

nwah
Posts: 41
Joined: 21 Nov 2013, 07:40

Re: Cell preloading

Post by nwah » 08 Feb 2016, 05:28

The mod Rise of the House Telvanni has a realm with many doors that act as portals to key destinations. The preloader might go crazy here. Maybe there should be some sort of upper bound for the amount of preloading.

SquireNed
Posts: 401
Joined: 21 Dec 2013, 22:18

Re: Cell preloading

Post by SquireNed » 08 Feb 2016, 05:49

nwah wrote:The mod Rise of the House Telvanni has a realm with many doors that act as portals to key destinations. The preloader might go crazy here. Maybe there should be some sort of upper bound for the amount of preloading.
Or maybe it'd be a perfect place to stress test the preloader. At least with vanilla Morrowind assets, I don't think there would be many issues; the rendering should be many times more impactful than the asset loading, especially since a certain number of assets will be recycled. The more preloading you do, the more assets will be recycled, at least in theory.

User avatar
Ace (SWE)
Posts: 823
Joined: 15 Aug 2011, 14:56

Re: Cell preloading

Post by Ace (SWE) » 08 Feb 2016, 14:10

Windows builds with the preloading for those that want to try;
32-bit
64-bit

Built from c805442

User avatar
Lagahan
Posts: 47
Joined: 16 Aug 2014, 11:24
Location: Ireland

Re: Cell preloading

Post by Lagahan » 08 Feb 2016, 14:41

Ace (SWE) wrote:Windows builds with the preloading for those that want to try;
32-bit
64-bit

Built from c805442
Thanks man, gotta test this out this evening!
Might install it on my NVME drive just for shits and giggles :P
Specs: Core i7 8700k @5GHz, EVGA GTX2080ti XC SLI,
32GB RAM @3.2GHz, Windows 10 1803

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Cell preloading

Post by scrawl » 08 Feb 2016, 15:46

Might install it on my NVME drive just for shits and giggles :P
Oh sure, having the game installed on an SSD definitely helps. Even on a fast drive though, the loading wasn't instantaneous, since we still need to convert assets, upload them to the GPU, etc. So the preloader should help SSD and HDD users alike.

I normally have the game on my SSD, but right now I'm using the hard drive to test a "worst case" preloading scenario. When conducting these tests it's also a good idea to clear the OS's file cache first.
The mod Rise of the House Telvanni has a realm with many doors that act as portals to key destinations. The preloader might go crazy here. Maybe there should be some sort of upper bound for the amount of preloading.
What SquireNed said, also, if the preloader does end up being overloaded, that shouldn't be much of an issue, because the main thread should continue running smoothly (i.e. we can load a cell in the main thread while the preloader still hasn't finished loading other cells in the background). The new resource system is mostly lock-free, we only need to lock when inserting a loaded item to the cache, not during the loading itself. I.e. the two threads can load at the same time without really interfering with each other.

User avatar
Lagahan
Posts: 47
Joined: 16 Aug 2014, 11:24
Location: Ireland

Re: Cell preloading

Post by Lagahan » 08 Feb 2016, 19:39

scrawl wrote:Even on a fast drive though, the loading wasn't instantaneous, since we still need to convert assets, upload them to the GPU, etc. So the preloader should help SSD and HDD users alike.
Don't think you can get much closer to instantaneous than this: :o :shock:
https://www.youtube.com/watch?v=vHB4wbl ... e=youtu.be

I take it the "ChangeCellGrid Took xx.xxx" messages are in milliseconds?

Edit: I think I'll move the files over to a hard drive and test the difference between 0.38 and this build, that NVME drive is like using a nuke to kill a fly.
Specs: Core i7 8700k @5GHz, EVGA GTX2080ti XC SLI,
32GB RAM @3.2GHz, Windows 10 1803

Post Reply