Cell preloading

General discussion regarding the OpenMW project.
For technical support, please use the Support subforum.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Cell preloading

Post by scrawl » 12 Feb 2016, 01:05

Depends on packages which need a new maintainer. Depends: libjasper1 Build-Depends: libjasper-dev
Jasper tracker page: 4 security issues in sid. Two of them crashes with crafted JPEG2000 images. Not good. Why do so many packages I have installed and use depend on this library? :shock:
OSG does have a lot of dependencies, but almost all of them are for optional plugins. Moving the plugins to separate packages would be a good move, that means an apt-get install openmw wouldn't pull in so much unneeded crap.

User avatar
psi29a
Posts: 4670
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Cell preloading

Post by psi29a » 12 Feb 2016, 08:21

Scrawl, can you verify that these bugs (with patches) are fixed in 3.4? If so, that would makes the maintainer's job easier.

https://bugs.debian.org/cgi-bin/pkgrepo ... scenegraph

User avatar
psi29a
Posts: 4670
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Cell preloading

Post by psi29a » 12 Feb 2016, 09:58

What is the difference between TGA and DDS, aside from the fact that DDS is a container. Our DDS contains uncompressed data, that is sent to the GPU to render (no decompression in GPU because image is already decompressed). TGA is also uncompressed, so there is no savings in terms of bandwidth to the GPU. From this point of view, is there really a reason to use DDS?

In addition to this, can we use uncompressed in KTX instead of DDS? Or would we run into the same bug?

Update: Reading up on the Debian bugs I see this one in particular (FFMPEG 2.9 related) https://bugs.debian.org/cgi-bin/bugrepo ... bug=803849

Basically they want to move on with 3.4.0 but there are a TON of dependencies that need to be updated first then make sure rdeps are also aware that they need to transition as well. Apparently OSG is used elsewhere, which means that it could break other packages as well. That is their reason for being glacially slow. ;)

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Cell preloading

Post by scrawl » 12 Feb 2016, 14:42

What is the difference between TGA and DDS, aside from the fact that DDS is a container. Our DDS contains uncompressed data, that is sent to the GPU to render (no decompression in GPU because image is already decompressed). TGA is also uncompressed, so there is no savings in terms of bandwidth to the GPU. From this point of view, is there really a reason to use DDS?
DDS can contain pre-built mipmaps. These can potentially allow the game to load faster since we don't have to build the mip-maps at load time, but the difference is probably miniscule.

Chris
Posts: 1560
Joined: 04 Sep 2011, 08:33

Re: Cell preloading

Post by Chris » 13 Feb 2016, 06:11

Out of curiosity, is there anything in place to prevent runaway memory usage when preloading a bunch of cells (and/or cells with lots of objects in them)? How many cells does it keep preloaded before evicting unused cells (for example, if a player goes near a door or cell boundary to cause it to start preloading, then turns around and never goes to that cell)?

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Cell preloading

Post by scrawl » 13 Feb 2016, 14:34

for example, if a player goes near a door or cell boundary to cause it to start preloading, then turns around and never goes to that cell)?
By default, the assets in that cell would remain loaded for 300 seconds, then thrown out.
There is no limit on the maximum amount of cells in the cache yet, but that should be simple enough to implement. I just haven't got to it yet because the memory usage didn't seem too much of an issue with vanilla files (plus, the more cells you load, the more assets tend to get reused).

User avatar
psi29a
Posts: 4670
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Cell preloading

Post by psi29a » 13 Feb 2016, 15:07

I managed to get OSG 3.4 to build and tried getting it up to launchpad but it failed with this:

Code: Select all

────────────────────────────────────────────────────────────────────────────────
Finished at 20160213-1339
Build needed 00:33:35, 4022704k disc space
Function `xine_xmalloc_aligned' implicitly converted to pointer at /«PKGBUILDDIR»/src/osgPlugins/xine/video_out_rgb.c:2325



Our automated build log filter detected the problem(s) above that will
likely cause your package to segfault on architectures where the size of
a pointer is greater than the size of an integer, such as ia64 and amd64.

This is often due to a missing function prototype definition.

Since use of implicitly converted pointers is always fatal to the application
on ia64, they are errors.  Please correct them for your next upload.

More information can be found at:
http://wiki.debian.org/ImplicitPointerConversions
Scrawl, is there a quick fix you can think of... you know, besides disabling xine support. ;)

This probably needs to be filed as a bug upstream.

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Cell preloading

Post by scrawl » 13 Feb 2016, 15:14

I don't understand that warning.

The line in question reads:

Code: Select all

			frame->vo_frame.base[0] = (uint8_t*) xine_xmalloc_aligned(16,
							 	frame->vo_frame.pitches[0] * frame->height,
							 	&(frame->chunk[0]));
It assigns the return value of the xine_xmalloc_aligned function to a pointer. It does not assign the function itself to a pointer as your warning would suggest.

User avatar
Zini
Posts: 5536
Joined: 06 Aug 2011, 15:16

Re: Cell preloading

Post by Zini » 13 Feb 2016, 17:04

This is just a vague guess.

Maybe the compiler (or even just the error checking tool that produces this error message) does not see the function prototype of xine_xmalloc_aligned. This would then implicitly assumed to have a return type of int.

That doesn't match exactly the error description, but it would certainly not be the first misleading error message in the C and C++ space.

Does adding the line
void *xine_xmalloc_aligned(size_t alignment, size_t size, void **base) XINE_PROTECTED;
before the definition of the rgbout_update_frame_format function change anything?

User avatar
raevol
Posts: 3031
Joined: 07 Aug 2011, 01:12
Location: Caldera

Re: Cell preloading

Post by raevol » 13 Feb 2016, 18:46

scrawl, after running around with this feature a little bit, I'd just like to say thank you so much. It makes a huge difference, the game is so much more enjoyable without having that mental pause every few minutes as you wait for something to load. I especially notice it when running through a lot of interiors in cities.

Post Reply