Cell preloading

General discussion regarding the OpenMW project.
For technical support, please use the Support subforum.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Cell preloading

Post by scrawl » 14 Feb 2016, 15:57

osg can be compiled with Qt4 or Qt5. You can make the choice with the -DDESIRED_QT_VERSION=4 switch. Likewise, OpenMW accepts the -DDESIRED_QT_VERSION flag to select building with Qt4 or Qt5. You'll need to use the same flag for OSG and OpenMW.

User avatar
Zini
Posts: 5536
Joined: 06 Aug 2011, 15:16

Re: Cell preloading

Post by Zini » 14 Feb 2016, 15:59

I see. So the osg package is build against Qt5. I think that is a mistake at this point.

User avatar
psi29a
Posts: 4631
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Cell preloading

Post by psi29a » 14 Feb 2016, 16:01

I get the following cmake warnings when using Qt5, I do not get them with Qt4.

Code: Select all

[email protected]:~/Workspace/Private/OpenMW/build$ cmake ../openmw/ -DDESIRED_QT_VERSION=5
-- Configuring OpenMW...
-- Using Qt5
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.28") 
-- checking for module 'libavcodec'
--   found libavcodec, version 56.41.100
-- checking for module 'libavformat'
--   found libavformat, version 56.36.100
-- checking for module 'libavdevice'
--   found libavdevice, version 56.4.100
-- checking for module 'libavutil'
--   found libavutil, version 54.27.100
-- checking for module 'libswscale'
--   found libswscale, version 3.1.101
-- checking for module 'libpostproc'
--   found libpostproc, version 53.3.100
-- checking for module 'libswresample'
--   found libswresample, version 1.2.100
-- checking for module 'libavresample'
--   found libavresample, version 2.1.0
-- Found FFmpeg: /usr/lib/x86_64-linux-gnu/libavcodec.so;/usr/lib/x86_64-linux-gnu/libavformat.so;/usr/lib/x86_64-linux-gnu/libavutil.so;/usr/lib/x86_64-linux-gnu/libswscale.so  
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for C++ include unordered_map
-- Looking for C++ include unordered_map - not found
-- Found osgDB: /usr/lib/x86_64-linux-gnu/libosgDB.so  
-- Found osgViewer: /usr/lib/x86_64-linux-gnu/libosgViewer.so  
-- Found osgText: /usr/lib/x86_64-linux-gnu/libosgText.so  
-- Found osgGA: /usr/lib/x86_64-linux-gnu/libosgGA.so  
-- Found osgAnimation: /usr/lib/x86_64-linux-gnu/libosgAnimation.so  
-- Found osgParticle: /usr/lib/x86_64-linux-gnu/libosgParticle.so  
-- Found osgQt: /usr/lib/x86_64-linux-gnu/libosgQt.so  
-- Found osgUtil: /usr/lib/x86_64-linux-gnu/libosgUtil.so  
-- Found osgFX: /usr/lib/x86_64-linux-gnu/libosgFX.so  
-- Found osg: /usr/lib/x86_64-linux-gnu/libosg.so  
-- Found OpenThreads: /usr/lib/x86_64-linux-gnu/libOpenThreads.so  
-- Found OpenSceneGraph: /usr/lib/x86_64-linux-gnu/libosgDB.so;/usr/lib/x86_64-linux-gnu/libosgViewer.so;/usr/lib/x86_64-linux-gnu/libosgText.so;/usr/lib/x86_64-linux-gnu/libosgGA.so;/usr/lib/x86_64-linux-gnu/libosgAnimation.so;/usr/lib/x86_64-linux-gnu/libosgParticle.so;/usr/lib/x86_64-linux-gnu/libosgQt.so;/usr/lib/x86_64-linux-gnu/libosgUtil.so;/usr/lib/x86_64-linux-gnu/libosgFX.so;/usr/lib/x86_64-linux-gnu/libosg.so;/usr/lib/x86_64-linux-gnu/libOpenThreads.so (found suitable version "3.4.0", minimum required is "3.2.0") 
-- checking for one of the modules 'MYGUI;MyGUI'
--  libraries : MyGUIEngine from /usr/lib
--  includes : /usr/include/MYGUI
-- MyGUI version: 3.2.2
-- Found MyGUI: /usr/include/MYGUI  
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   system
--   filesystem
--   program_options
--   thread
-- Found SDL2: /usr/lib/x86_64-linux-gnu/libSDL2main.a;/usr/lib/x86_64-linux-gnu/libSDL2.so;-lpthread  
-- Found OpenAL: /usr/lib/x86_64-linux-gnu/libopenal.so  
-- Found Bullet: /usr/lib/x86_64-linux-gnu/libBulletCollision.so  
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libGL.so  
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.9.1") 
-- Configuring done
CMake Warning (dev) in components/CMakeLists.txt:
  Policy CMP0043 is not set: Ignore COMPILE_DEFINITIONS_<Config> properties.
  Run "cmake --help-policy CMP0043" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in components/CMakeLists.txt:
  Policy CMP0043 is not set: Ignore COMPILE_DEFINITIONS_<Config> properties.
  Run "cmake --help-policy CMP0043" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in apps/launcher/CMakeLists.txt:
  Policy CMP0043 is not set: Ignore COMPILE_DEFINITIONS_<Config> properties.
  Run "cmake --help-policy CMP0043" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in apps/launcher/CMakeLists.txt:
  Policy CMP0043 is not set: Ignore COMPILE_DEFINITIONS_<Config> properties.
  Run "cmake --help-policy CMP0043" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in apps/opencs/CMakeLists.txt:
  Policy CMP0043 is not set: Ignore COMPILE_DEFINITIONS_<Config> properties.
  Run "cmake --help-policy CMP0043" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in apps/opencs/CMakeLists.txt:
  Policy CMP0043 is not set: Ignore COMPILE_DEFINITIONS_<Config> properties.
  Run "cmake --help-policy CMP0043" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in apps/wizard/CMakeLists.txt:
  Policy CMP0043 is not set: Ignore COMPILE_DEFINITIONS_<Config> properties.
  Run "cmake --help-policy CMP0043" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in apps/wizard/CMakeLists.txt:
  Policy CMP0043 is not set: Ignore COMPILE_DEFINITIONS_<Config> properties.
  Run "cmake --help-policy CMP0043" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /home/bcurtis/Workspace/Private/OpenMW/build
Probably a idea to fix this.

User avatar
psi29a
Posts: 4631
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Cell preloading

Post by psi29a » 14 Feb 2016, 16:02

I can't upload the 3.4 travis build to PPA because we've hit our disk-space quota. I've asked for it to be raised. It is a cascading set of problems... :(

BTW: The OSG package is directly from Debian and I don't want to deviate from downstream too much.

User avatar
Zini
Posts: 5536
Joined: 06 Aug 2011, 15:16

Re: Cell preloading

Post by Zini » 14 Feb 2016, 17:36

Alight then. I removed the package and did my own OSG build. Works now.

Will merge the osg_34 branch. I guess we can survive for a while without Travis.

User avatar
psi29a
Posts: 4631
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Cell preloading

Post by psi29a » 14 Feb 2016, 21:53

Package is built and uploaded... travis should be happy now.

mym
Posts: 42
Joined: 09 Feb 2015, 13:31

Re: Cell preloading

Post by mym » 17 Feb 2016, 11:38

When preloading is enabled and once I've loaded my saved game in balmora (silt strider spot), it takes all my memory 8Go within ~10-15 seconds! I have to kill the process or it may freeze my computer.

I have a Debian 64, Jessie. Openmw on master repo up to date, scrawl OSG repo, bullet 2.83 compiled from source.
My install as lots of graphic replacers, quite high resolution images but not too much.

Here is my log up to my kill command:
Spoiler: Show

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Cell preloading

Post by scrawl » 18 Feb 2016, 17:52

When preloading is enabled and once I've loaded my saved game in balmora (silt strider spot), it takes all my memory 8Go within ~10-15 seconds! I have to kill the process or it may freeze my computer.

I have a Debian 64, Jessie. Openmw on master repo up to date, scrawl OSG repo, bullet 2.83 compiled from source.
My install as lots of graphic replacers, quite high resolution images but not too much.
I'm not overly suprised about this, I haven't really tested the preloading with lots of mods or high res texture replacers.

If you set

Code: Select all

[Cells]
preload fast travel = false
in the settings file, it won't try to preload all possible silt strider destinations. This should cut down the amount of preloaded cells.

You might still run out of memory after a while though.

I just pushed some new settings to master to help mitigate this. The new defaults are not that resource hungry so it might work out of the box. If it doesn't, just lower the settings until memory usage is to a reasonable level.

Code: Select all

# The minimum amount of cells in the preload cache before unused cells start to get thrown out (see "preload cell expiry delay").
# This value should be lower or equal to 'preload cell cache max'.
preload cell cache min = 12

# The maximum amount of cells in the preload cache. A too high value could cause you to run out of memory.
# You may need to reduce this setting when running lots of mods or high-res texture replacers.
preload cell cache max = 20

# How long to keep preloaded cells in cache after they're no longer referenced/required (in seconds)
preload cell expiry delay = 5

# How long to keep models/textures/collision shapes in cache after they're no longer referenced/required (in seconds)
cache expiry delay = 5

mym
Posts: 42
Joined: 09 Feb 2015, 13:31

Re: Cell preloading

Post by mym » 19 Feb 2016, 09:44

Disabling fast travel fixed my problem.

However I got a crash after walking 15 secs in the city:
Spoiler: Show

User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Cell preloading

Post by scrawl » 19 Feb 2016, 14:26

Should be fixed now, try again with commit 4e5462bc1977ebab9d07baf37c5b11d531686cce .

Post Reply