Page 1 of 10

Cell preloading

Posted: 07 Feb 2016, 22:50
by scrawl
Just a quick heads up of a new feature I'm working on. Background preloading of cells.

My goal is to make flying around the island buttery smooth with no loading bars or frame drops whatsoever. I'm not quite at that point yet, mostly because terrain isn't preloaded yet, body parts are not preloaded yet and because I haven't profiled / optimized the code at all yet. Still, the loading times compared to the main branch should be much improved already.

Currently the preloader will activate if you go near a teleport door, or go near an exterior cell border. I'm thinking about preloading fast travel destinations as well, though of course the downside is we don't know the travel destination the player will pick, so would have to load all of them preemptively.

Feel free to try out the branch and let me know how it goes! I haven't come across any crashes or instabilities yet, which is a good sign that the new resource manager design is solid.

Re: Cell preloading

Posted: 07 Feb 2016, 23:03
by Ace (SWE)
I'll throw up a set of Windows builds for this tomorrow I think, I want to see this for myself.

Re: Cell preloading

Posted: 07 Feb 2016, 23:59
by TheMechanist
That's awesome. This is way up my "Please-come-before-1.0-feature" list 8-)

Re: Cell preloading

Posted: 08 Feb 2016, 01:29
by ElderTroll
Nice! My 6 year old netbook is totally incapable of taking advantage of that, but really exciting stuff, scrawl. Saw that the trello board had moved finished the resource management refactoring to done. :P

Re: Cell preloading

Posted: 08 Feb 2016, 05:28
by nwah
The mod Rise of the House Telvanni has a realm with many doors that act as portals to key destinations. The preloader might go crazy here. Maybe there should be some sort of upper bound for the amount of preloading.

Re: Cell preloading

Posted: 08 Feb 2016, 05:49
by SquireNed
nwah wrote:The mod Rise of the House Telvanni has a realm with many doors that act as portals to key destinations. The preloader might go crazy here. Maybe there should be some sort of upper bound for the amount of preloading.
Or maybe it'd be a perfect place to stress test the preloader. At least with vanilla Morrowind assets, I don't think there would be many issues; the rendering should be many times more impactful than the asset loading, especially since a certain number of assets will be recycled. The more preloading you do, the more assets will be recycled, at least in theory.

Re: Cell preloading

Posted: 08 Feb 2016, 14:10
by Ace (SWE)
Windows builds with the preloading for those that want to try;
32-bit
64-bit

Built from c805442

Re: Cell preloading

Posted: 08 Feb 2016, 14:41
by Lagahan
Ace (SWE) wrote:Windows builds with the preloading for those that want to try;
32-bit
64-bit

Built from c805442
Thanks man, gotta test this out this evening!
Might install it on my NVME drive just for shits and giggles :P

Re: Cell preloading

Posted: 08 Feb 2016, 15:46
by scrawl
Might install it on my NVME drive just for shits and giggles :P
Oh sure, having the game installed on an SSD definitely helps. Even on a fast drive though, the loading wasn't instantaneous, since we still need to convert assets, upload them to the GPU, etc. So the preloader should help SSD and HDD users alike.

I normally have the game on my SSD, but right now I'm using the hard drive to test a "worst case" preloading scenario. When conducting these tests it's also a good idea to clear the OS's file cache first.
The mod Rise of the House Telvanni has a realm with many doors that act as portals to key destinations. The preloader might go crazy here. Maybe there should be some sort of upper bound for the amount of preloading.
What SquireNed said, also, if the preloader does end up being overloaded, that shouldn't be much of an issue, because the main thread should continue running smoothly (i.e. we can load a cell in the main thread while the preloader still hasn't finished loading other cells in the background). The new resource system is mostly lock-free, we only need to lock when inserting a loaded item to the cache, not during the loading itself. I.e. the two threads can load at the same time without really interfering with each other.

Re: Cell preloading

Posted: 08 Feb 2016, 19:39
by Lagahan
scrawl wrote:Even on a fast drive though, the loading wasn't instantaneous, since we still need to convert assets, upload them to the GPU, etc. So the preloader should help SSD and HDD users alike.
Don't think you can get much closer to instantaneous than this: :o :shock:
https://www.youtube.com/watch?v=vHB4wbl ... e=youtu.be

I take it the "ChangeCellGrid Took xx.xxx" messages are in milliseconds?

Edit: I think I'll move the files over to a hard drive and test the difference between 0.38 and this build, that NVME drive is like using a nuke to kill a fly.