Unable to compile against OpenMW/osg fork in non system folder (manjaro)

Support for running, installing or compiling OpenMW

Before you submit a bug report for the first time, please read: Bug reporting guidelines
Post Reply
Naugrim
Posts: 172
Joined: 08 Jan 2016, 01:32
Location: Spain

Unable to compile against OpenMW/osg fork in non system folder (manjaro)

Post by Naugrim »

Hi,

I continue trying to get some clean compilation process, but it seems there's some issue when trying to compile against OpenMW/osg in Manjaro.
It's fine if I just have official openscenegraph package but I wanted to personally benchmark performance differences as mentioned in the compilation guide https://gitlab.com/OpenMW/openmw/-/wiki ... ment_setup.

I'd like to do the same I did for Bullet3 viewtopic.php?f=8&t=7281&p=69733&hilit=compile#p69733. Basically, compile osg in some path, do not install in system paths to avoid pollution, and compile OpenMW against such path linking statically.

But with OSG, setting paths manually does not seem to have effect. For example, add `-DOPENSCENEGRAPH_INCLUDE_DIR` and `-DOPENSCENEGRAPH_LIBRARIES` doesn't prevent cmake from complaining. Also tried adding `OSGDB_LIBRARY` and same result, cmake complains as if it was not set.
I validated the paths are correct and artifacts are present and I am simply adding extra arguments to the same script I use to build bullet, so the usual suspect should be discarded (I hope).

Code: Select all

-- Could NOT find osgDB (missing: OSGDB_LIBRARY)
-- Could NOT find osgViewer (missing: OSGVIEWER_LIBRARY OSGVIEWER_INCLUDE_DIR)
-- Could NOT find osgText (missing: OSGTEXT_LIBRARY OSGTEXT_INCLUDE_DIR)
-- Could NOT find osgGA (missing: OSGGA_LIBRARY OSGGA_INCLUDE_DIR)
-- Could NOT find osgParticle (missing: OSGPARTICLE_LIBRARY OSGPARTICLE_INCLUDE_DIR)
-- Could NOT find osgUtil (missing: OSGUTIL_LIBRARY OSGUTIL_INCLUDE_DIR)
-- Could NOT find osgFX (missing: OSGFX_LIBRARY OSGFX_INCLUDE_DIR)
-- Could NOT find osgShadow (missing: OSGSHADOW_LIBRARY OSGSHADOW_INCLUDE_DIR)
-- Could NOT find osgAnimation (missing: OSGANIMATION_LIBRARY OSGANIMATION_INCLUDE_DIR)
-- Could NOT find osg (missing: OSG_LIBRARY OSG_INCLUDE_DIR)
-- Could NOT find OpenThreads (missing: OPENTHREADS_LIBRARY OPENTHREADS_INCLUDE_DIR)
CMake Error at /usr/share/cmake-3.21/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find OpenSceneGraph (missing: OPENSCENEGRAPH_LIBRARIES
  OPENSCENEGRAPH_INCLUDE_DIR OSGDB_FOUND OSGVIEWER_FOUND OSGTEXT_FOUND
  OSGGA_FOUND OSGPARTICLE_FOUND OSGUTIL_FOUND OSGFX_FOUND OSGSHADOW_FOUND
  OSGANIMATION_FOUND OSG_FOUND OPENTHREADS_FOUND) (Required is at least
  version "3.4.0")
Call Stack (most recent call first):
  /usr/share/cmake-3.21/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.21/Modules/FindOpenSceneGraph.cmake:226 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  CMakeLists.txt:367 (find_package)
  

Any help would be greatly appreciated.

Btw, also checked the aur instructions, but those install as system files.
LoneWolf
Posts: 138
Joined: 26 Sep 2017, 19:13

Re: Unable to compile against OpenMW/osg fork in non system folder (manjaro)

Post by LoneWolf »

Naugrim wrote: I'd like to do the same I did for Bullet3 viewtopic.php?f=8&t=7281&p=69733&hilit=compile#p69733. Basically, compile osg in some path, do not install in system paths to avoid pollution, and compile OpenMW against such path linking statically.
Naugrim wrote: Btw, also checked the aur instructions, but those install as system files.
That's because pacman/makepkg job is systemwide package management.
PKGBUILDs that do anything with files in user home folders are immediately banished from AUR.

There are many openmw-specific settings for cmake, some of them allow building parts statically.
You don't seem to be aware of those settings (or where you can find info about them)

I assume you created some buildscript to set options for cmake to build openmw with ?
Please post it so we can see how you are doing things now.
Naugrim
Posts: 172
Joined: 08 Jan 2016, 01:32
Location: Spain

Re: Unable to compile against OpenMW/osg fork in non system folder (manjaro)

Post by Naugrim »

LoneWolf wrote: 27 Nov 2021, 11:41 You don't seem to be aware of those settings (or where you can find info about them)
Right on point! That's why I am thankfull of any help.

Other than the error messages I get I have been looking into `CMakeLists.txt` and the generated makefiles without much success.
I use script below which is fine when using openscenegraph from Manjaro's repos. But without it I get the error from the preivous post, but unlike what happened with bullet3, here addind for example `-DOPENSCENEGRAPH_INCLUDE_DIR=$HOME/.games/openmw_deps/osg/build/install/include \` to the build arguments doesn't fix anything, at least I would expect cmake to stop complaining about `OPENSCENEGRAPH_INCLUDE_DIR`.
That's how I could fix bullet3, working argument by argument.

Code: Select all

#!/bin/bash

build_bullet3 () {
  
  local cpu_cores=$(cat /proc/cpuinfo | grep processor | wc -l) && cpu_cores=$((cpu_cores-1))

  local BULLET3_PATH="$HOME/.games/openmw_deps/bullet3"
  local BULLET3_INSTALL="$BULLET3_PATH/build/install"

  popd
  cd $BULLET3_PATH
  # git pull --all

  rm -rf build && \
mkdir build && \
cd build && \
cmake \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_INSTALL_PREFIX=./install \
-D USE_DOUBLE_PRECISION=ON \
-D BUILD_UNIT_TESTS=OFF \
-D BUILD_CPU_DEMOS=OFF \
-D BUILD_BULLET2_DEMOS=OFF \
-D BULLET2_MULTITHREADING=ON \
-D INSTALL_LIBS=ON ..

  make -j $cpu_cores && make install
  pushd
}

build_openscenegraph () {
  
  local cpu_cores=$(cat /proc/cpuinfo | grep processor | wc -l) && cpu_cores=$((cpu_cores-1))

  local OPENSCENEGRAPH_PATH="$HOME/.games/openmw_deps/osg"
  local OPENSCENEGRAPH_INSTALL="$OPENSCENEGRAPH_PATH/build/install"

  popd
  cd $OPENSCENEGRAPH_PATH
  # git pull --all

  rm -rf build && \
mkdir build && \
cd build && \
cmake \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_INSTALL_PREFIX=./install \
-D BUILD_OSG_PLUGINS_BY_DEFAULT=0 \
-D BUILD_OSG_PLUGIN_OSG=1 \
-D BUILD_OSG_PLUGIN_DDS=1 \
-D BUILD_OSG_PLUGIN_TGA=1 \
-D BUILD_OSG_PLUGIN_BMP=1 \
-D BUILD_OSG_PLUGIN_JPEG=1 \
-D BUILD_OSG_PLUGIN_PNG=1 \
-D BUILD_OSG_PLUGIN_FREETYPE=1 \
-D BUILD_OSG_DEPRECATED_SERIALIZERS=0 ..

  make -j $cpu_cores && make install
  pushd
}

build_openmw () {

  local cpu_cores=$(cat /proc/cpuinfo | grep processor | wc -l) && cpu_cores=$((cpu_cores-1))

  local BULLET3_PATH="$HOME/.games/openmw_deps/bullet3"
  local BULLET3_INSTALL="$BULLET3_PATH/build/install"
  local OPENSCENEGRAPH_PATH="$HOME/.games/openmw_deps/osg"
  local OPENSCENEGRAPH_INSTALL="$OPENSCENEGRAPH_PATH/build/install"

  local OPENMW_PATH="$HOME/.games/openmw"

  popd
  cd $OPENMW_PATH
  git pull --all

  rm -rf build && \
mkdir build && \
cd build && \
cmake \
-DCMAKE_INSTALL_PREFIX=./install \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_WIZARD=OFF \
-DBUILD_MWINIIMPORTER=OFF \
-DBUILD_OPENCS=OFF \
-DBUILD_BSATOOL=OFF \
-DBUILD_ESMTOOL=OFF \
-DBUILD_NIFTEST=OFF \
-DBUILD_ESSIMPORTER=OFF \
-DOPENMW_LTO_BUILD=ON \
-DBULLET_INCLUDE_DIR=$BULLET3_INSTALL/include/bullet \
-DBULLET_DYNAMICS_LIBRARY=$BULLET3_INSTALL/lib/libBulletDynamics.a \
-DBULLET_COLLISION_LIBRARY=$BULLET3_INSTALL/lib/libBulletCollision.a \
-DBULLET_MATH_LIBRARY=$BULLET3_INSTALL/lib/libLinearMath.a \
-DBULLET_SOFTBODY_LIBRARY=$BULLET3_INSTALL/lib/libBulletSoftBody.a \
-DBULLET_STATIC=ON \
..

  make -j $cpu_cores

  cd $BULLET3_PATH/build && make clean

  popd
}

build_openmw_full() {
  build_bullet3
  # build_openscenegraph
  build_openmw
}
LoneWolf
Posts: 138
Joined: 26 Sep 2017, 19:13

Re: Unable to compile against OpenMW/osg fork in non system folder (manjaro)

Post by LoneWolf »

Very glad you are aware of CMakeLists.txt .

Code: Select all

$ cat CMakeLists.txt | grep OPENMW_USE_SYSTEM
option(OPENMW_USE_SYSTEM_BULLET "Use system provided bullet physics library" ON)
if(OPENMW_USE_SYSTEM_BULLET)
option(OPENMW_USE_SYSTEM_OSG "Use system provided OpenSceneGraph libraries" ON)
if(OPENMW_USE_SYSTEM_OSG)
option(OPENMW_USE_SYSTEM_MYGUI "Use system provided mygui library" ON)
if(OPENMW_USE_SYSTEM_MYGUI)
option(OPENMW_USE_SYSTEM_RECASTNAVIGATION "Use system provided recastnavigation library" OFF)
if(OPENMW_USE_SYSTEM_RECASTNAVIGATION)
option(OPENMW_USE_SYSTEM_SQLITE3 "Use system provided SQLite3 library" ON)
if(OPENMW_USE_SYSTEM_BULLET)
if(OPENMW_USE_SYSTEM_OSG)
if(OPENMW_USE_SYSTEM_MYGUI)
$
Those options determine whether openmw will use systemwide librarires or a static one created during building of openmw .
Deafult setting is to use the static openmw specific recastnavigation, and system libraries for the other stuff .

If you look further into CMakeLists.txt you'll notice that the nevessary paths , include dirs etc for the static options are set frrom it .

In other words , openmw can build static versions of bullet, the osg fork , mygui , recastnavigation and sqlite3 .
You don't need to set everything yourself but can benefit from the work openmw devs have done already.

I'm not 100% certain, but do think these options are also used to create the generic binaries in https://downloads.openmw.org/linux/generic/?C=M&O=D
Naugrim
Posts: 172
Joined: 08 Jan 2016, 01:32
Location: Spain

Re: Unable to compile against OpenMW/osg fork in non system folder (manjaro)

Post by Naugrim »

I already explored that, the issue as mentioned is that no matter how I defined the OPENSCENEGRAPH_* properties cmake doesn't seem to be able to validate them and always fails. To rewordit in a simple way, let's say it complains about X,Y,Z missing, if I set X, it still complains about it not being set.

Interestingly enough, using `OPENMW_USE_SYSTEM_OSG=OFF` builds osg locally inside `_deps` but it's not clear to me where that comes from or what version it is. Given build failed with status 500 these past days I presume it's geting it the standard OSG from GH and using some version variable to select the TAG, but could not find exactly where.

I guess I'll have to go deep into cmake provided scripts like `FindOpenSceneGraph.cmake` and `FindPackageHandleStandardArgs` to see what's failing.

EDIT: I finally found something that points to the source of the download https://gitlab.com/OpenMW/openmw/-/blob ... t#L150-156. But I the md5 does not match the tags, and the comment mentioning 23 Jan 2021 make me think it's using and outdated code.
Naugrim
Posts: 172
Joined: 08 Jan 2016, 01:32
Location: Spain

Re: Unable to compile against OpenMW/osg fork in non system folder (manjaro)

Post by Naugrim »

Naugrim wrote: 28 Nov 2021, 23:36 EDIT: I finally found something that points to the source of the download https://gitlab.com/OpenMW/openmw/-/blob ... t#L150-156. But I the md5 does not match the tags, and the comment mentioning 23 Jan 2021 make me think it's using and outdated code.
I could make it compile against the latest commit, I monkey patched the linked section. I would not consider it a good solution, but it's progress.
Also this versions does not crash openmw on close, inlike when I compile against the provided one, which displays a crash when closing the program.
Post Reply