Issues compiling under Windows, attempting to use multithreaded physics

Support for running, installing or compiling OpenMW

Before you submit a bug report for the first time, please read: Bug reporting guidelines
Post Reply
SyS
Posts: 2
Joined: 05 Mar 2021, 23:49

Issues compiling under Windows, attempting to use multithreaded physics

Post by SyS »

I just want to start off asking, is there any benefit to not having this BULLET2_MULTITHREADING on the nightly builds and releases by default? Some of us just know nothing in the first place or don't want to put up with a long process like this and, as CPUs continue to increase in thread count, we'll never be able to benefit from this. It seemingly only makes changing the async threads past 1 not work. I'm just trying to get as many frames as possible.

I was following these: http://forum.openmw.org/viewtopic.php?t=7148&p=69103, and https://wiki.openmw.org/index.php/Devel ... up#Windows

I don't know at which point I can do anything to the physics, but I followed the Windows guide until step "Run the following command: CI/before_script.msvc.sh -k -p Win64 -v 2019".
I use CI/before_script.msvc.sh -k -p Win64 -v 2019 -N because supposedly Ninja is faster to build.

I got the errors:

Code: Select all

Activating MSVC in the current shell... Error: cl (MSVC Compiler) missing.
Error: rc (MS Windows Resource Compiler) missing.
Error: mt (MS Windows Manifest Tool) missing.
Some build tools were unavailable after activating MSVC in the shell. It's likely that your Visual Studio 2019 installation needs repairing.
I had used cinst visualstudio2019buildtools hoping it would forego the need to install the full VS'19 IDE. It failed, so I installed the ~6GB C++ build tools. Then I retried using git bash. This time, it seemed to finish, but opening the folder with Cmake had it error out with the claim that it was incomplete.

Code: Select all

Configuring incomplete, errors occurred!
See also "C:/Cmake_Build/OpenMW/CMakeFiles/CMakeOutput.log".
log:

Code: Select all

The system is: Windows - 10.0.19041 - AMD64
Compiling the C compiler identification source file "CMakeCCompilerId.c" succeeded.
Compiler:  
Build flags: 
Id flags:  

The output was:
0
Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 3/5/2021 10:55:45 PM.
Project "C:\Cmake_Build\OpenMW\CMakeFiles\3.19.6\CompilerIdC\CompilerIdC.vcxproj" on node 1 (default targets).
PrepareForBuild:
  Creating directory "Debug\".
  Creating directory "Debug\CompilerIdC.tlog\".
InitializeBuildStatus:
  Creating "Debug\CompilerIdC.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX64\x64\CL.exe /c /nologo /W0 /WX- /diagnostics:column /Od /D _MBCS /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"Debug\\" /Fd"Debug\vc142.pdb" /Gd /TC /FC /errorReport:queue CMakeCCompilerId.c
  CMakeCCompilerId.c
Link:
  C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX64\x64\link.exe /ERRORREPORT:QUEUE /OUT:".\CompilerIdC.exe" /INCREMENTAL:NO /NOLOGO kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:".\CompilerIdC.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:".\CompilerIdC.lib" /MACHINE:X64 Debug\CMakeCCompilerId.obj
  CompilerIdC.vcxproj -> C:\Cmake_Build\OpenMW\CMakeFiles\3.19.6\CompilerIdC\CompilerIdC.exe
PostBuildEvent:
  for %%i in (cl.exe) do @echo CMAKE_C_COMPILER=%%~$PATH:i
  :VCEnd
  CMAKE_C_COMPILER=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\Hostx64\x64\cl.exe
FinalizeBuildStatus:
  Deleting file "Debug\CompilerIdC.tlog\unsuccessfulbuild".
  Touching "Debug\CompilerIdC.tlog\CompilerIdC.lastbuildstate".
Done Building Project "C:\Cmake_Build\OpenMW\CMakeFiles\3.19.6\CompilerIdC\CompilerIdC.vcxproj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.42


Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "CompilerIdC.exe"

Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "CompilerIdC.vcxproj"

The C compiler identification is MSVC, found in "C:/Cmake_Build/OpenMW/CMakeFiles/3.19.6/CompilerIdC/CompilerIdC.exe"

Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" succeeded.
Compiler:  
Build flags: 
Id flags:  

The output was:
0
Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 3/5/2021 10:55:47 PM.
Project "C:\Cmake_Build\OpenMW\CMakeFiles\3.19.6\CompilerIdCXX\CompilerIdCXX.vcxproj" on node 1 (default targets).
PrepareForBuild:
  Creating directory "Debug\".
  Creating directory "Debug\CompilerIdCXX.tlog\".
InitializeBuildStatus:
  Creating "Debug\CompilerIdCXX.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX64\x64\CL.exe /c /nologo /W0 /WX- /diagnostics:column /Od /D _MBCS /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"Debug\\" /Fd"Debug\vc142.pdb" /Gd /TP /FC /errorReport:queue CMakeCXXCompilerId.cpp
  CMakeCXXCompilerId.cpp
Link:
  C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX64\x64\link.exe /ERRORREPORT:QUEUE /OUT:".\CompilerIdCXX.exe" /INCREMENTAL:NO /NOLOGO kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:".\CompilerIdCXX.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:".\CompilerIdCXX.lib" /MACHINE:X64 Debug\CMakeCXXCompilerId.obj
  CompilerIdCXX.vcxproj -> C:\Cmake_Build\OpenMW\CMakeFiles\3.19.6\CompilerIdCXX\CompilerIdCXX.exe
PostBuildEvent:
  for %%i in (cl.exe) do @echo CMAKE_CXX_COMPILER=%%~$PATH:i
  :VCEnd
  CMAKE_CXX_COMPILER=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\Hostx64\x64\cl.exe
FinalizeBuildStatus:
  Deleting file "Debug\CompilerIdCXX.tlog\unsuccessfulbuild".
  Touching "Debug\CompilerIdCXX.tlog\CompilerIdCXX.lastbuildstate".
Done Building Project "C:\Cmake_Build\OpenMW\CMakeFiles\3.19.6\CompilerIdCXX\CompilerIdCXX.vcxproj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.90


Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" produced "CompilerIdCXX.exe"

Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" produced "CompilerIdCXX.vcxproj"

The CXX compiler identification is MSVC, found in "C:/Cmake_Build/OpenMW/CMakeFiles/3.19.6/CompilerIdCXX/CompilerIdCXX.exe"

Detecting C compiler ABI info compiled with the following output:
Change Dir: C:/Cmake_Build/OpenMW/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/MSBuild/Current/Bin/MSBuild.exe cmTC_e5a1c.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=16.0 /v:m && Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET Framework

Copyright (C) Microsoft Corporation. All rights reserved.



  Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29910 for x64

  Copyright (C) Microsoft Corporation.  All rights reserved.

  CMakeCCompilerABI.c

  cl /c /Zi /W3 /WX- /diagnostics:column /Od /Ob0 /D WIN32 /D _WINDOWS /D "CMAKE_INTDIR=\"Debug\"" /D _MBCS /Gm- /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"cmTC_e5a1c.dir\Debug\\" /Fd"cmTC_e5a1c.dir\Debug\vc142.pdb" /Gd /TC /errorReport:queue "C:\Program Files\CMake\share\cmake-3.19\Modules\CMakeCCompilerABI.c"

  cmTC_e5a1c.vcxproj -> C:\Cmake_Build\OpenMW\CMakeFiles\CMakeTmp\Debug\cmTC_e5a1c.exe




Detecting CXX compiler ABI info compiled with the following output:
Change Dir: C:/Cmake_Build/OpenMW/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/MSBuild/Current/Bin/MSBuild.exe cmTC_f2e21.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=16.0 /v:m && Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET Framework

Copyright (C) Microsoft Corporation. All rights reserved.



  Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29910 for x64

  Copyright (C) Microsoft Corporation.  All rights reserved.

  CMakeCXXCompilerABI.cpp

  cl /c /Zi /W3 /WX- /diagnostics:column /Od /Ob0 /D WIN32 /D _WINDOWS /D "CMAKE_INTDIR=\"Debug\"" /D _MBCS /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR /Fo"cmTC_f2e21.dir\Debug\\" /Fd"cmTC_f2e21.dir\Debug\vc142.pdb" /Gd /TP /errorReport:queue "C:\Program Files\CMake\share\cmake-3.19\Modules\CMakeCXXCompilerABI.cpp"

  cmTC_f2e21.vcxproj -> C:\Cmake_Build\OpenMW\CMakeFiles\CMakeTmp\Debug\cmTC_f2e21.exe
I tried again after downloading VS19 and the ~7GB Desktop development with C++ workload apart from the build tools. This time, git bash errored out like:

Code: Select all

- Activating MSVC in the current shell... '"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools' is not recognized as an internal or external command,
operable program or batch file.
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
I cloned the repo again but from within VS19, it gives the same error as Cmake got by itself.

Code: Select all

The system is: Windows - 10.0.19041 - AMD64
Compiling the C compiler identification source file "CMakeCCompilerId.c" succeeded.
Compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.28.29910/bin/Hostx64/x64/cl.exe 
Build flags: 
Id flags:  

The output was:
0
Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29910 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

CMakeCCompilerId.c
Microsoft (R) Incremental Linker Version 14.28.29910.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:CMakeCCompilerId.exe 
CMakeCCompilerId.obj 


Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "CMakeCCompilerId.exe"

Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "CMakeCCompilerId.obj"

The C compiler identification is MSVC, found in "C:/Users/*redacted*/source/repos/OpenMW/out/build/x64-Debug/CMakeFiles/3.19.20122902-MSVC_2/CompilerIdC/CMakeCCompilerId.exe"

Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" succeeded.
Compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.28.29910/bin/Hostx64/x64/cl.exe 
Build flags: 
Id flags:  

The output was:
0
Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29910 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

CMakeCXXCompilerId.cpp
Microsoft (R) Incremental Linker Version 14.28.29910.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:CMakeCXXCompilerId.exe 
CMakeCXXCompilerId.obj 


Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" produced "CMakeCXXCompilerId.exe"

Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" produced "CMakeCXXCompilerId.obj"

The CXX compiler identification is MSVC, found in "C:/Users/*redacted*/source/repos/OpenMW/out/build/x64-Debug/CMakeFiles/3.19.20122902-MSVC_2/CompilerIdCXX/CMakeCXXCompilerId.exe"

Detecting C compiler ABI info compiled with the following output:
Change Dir: C:/Users/*redacted*/source/repos/OpenMW/out/build/x64-Debug/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe cmTC_137bd && [1/2] Building C object CMakeFiles\cmTC_137bd.dir\CMakeCCompilerABI.c.obj

[2/2] Linking C executable cmTC_137bd.exe




Detecting CXX compiler ABI info compiled with the following output:
Change Dir: C:/Users/*redacted*/source/repos/OpenMW/out/build/x64-Debug/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe cmTC_3e415 && [1/2] Building CXX object CMakeFiles\cmTC_3e415.dir\CMakeCXXCompilerABI.cpp.obj

[2/2] Linking CXX executable cmTC_3e415.exe
Cmake alone still gives the same error.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Issues compiling under Windows, attempting to use multithreaded physics

Post by AnyOldName3 »

We know it's possible to build OpenMW for Windows with MSVC without the full Visual Studio IDE. It's possible, however, that we require packages not documented on our Wiki. The minimal set we know works are those in this list https://gitlab.com/gitlab-org/ci-cd/sha ... d-software plus the extras in our CI setup listed here https://gitlab.com/OpenMW/openmw/-/blob ... l#L128-133 as that's what our CI uses and that works. It's possible that with your testing, you've managed to get the important ones already.

As for this error:

Code: Select all

- Activating MSVC in the current shell... '"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools' is not recognized as an internal or external command,
operable program or batch file.
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
We've got a maybe-janky, maybe-dumb, maybe-genius system to detect and activate MSVC in a Bash shell, which is certainly not something Microsoft supports or recommends, but is, in fact, completely functional on my laptop, my desktop, and GitLab's CI environment. We have to do this because someone decided our prebuild script should be in Bash and to avoid porting it, we needed to make it possible to launch CMake from Bash with the MSVC environment variables all set up so it worked with Ninja. In an ideal world, CMake would have been able to handle this automatically like it can with MSBuild, but alas not.

It's pretty likely that if you just give up on Ninja and try MSBuild, your problems will magically disappear as you'll bypass the janky bit. It should be even easier to build from the command-line this way and you can check our CI script to see how, and you're likely to save more time not fighting the computer than you'd waste by not using Ninja. However, ideally we want the script to be minimally janky and work for everyone, so if you're willing to spend some time helping me track down the exact problem, that would be better for OpenMW.

Looking at how far the script got, you should have a file called ActivateMSVC.bat in your build directory. If you can upload it here (ideally as a file as it's not impossible there's an encoding issue, and copying and pasting the contents might change that) I might be able to tell what's going on from that.

Also, running the following command and posting the output here will be useful:

Code: Select all

vswhere -products '*' -version "[16,17)" -property installationPath
SyS
Posts: 2
Joined: 05 Mar 2021, 23:49

Re: Issues compiling under Windows, attempting to use multithreaded physics

Post by SyS »

OK. Does it matter where I ran the vswhere command? I presume not. I 7zipped the bat file because it wasn't a valid extension, according to the error I got.

Output:

Code: Select all

**redacted**@DESKTOP-QS9MR37 MINGW64 ~
$ **redacted**@DESKTOP-QS9MR37 MINGW64 ~
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
bash: **redacted**@DESKTOP-QS9MR37: command not found
Attachments
ActivateMSVC.7z
(276 Bytes) Downloaded 100 times
Last edited by SyS on 18 Mar 2021, 03:42, edited 1 time in total.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Issues compiling under Windows, attempting to use multithreaded physics

Post by AnyOldName3 »

Okay. I wasn't aware that it was possible to install the Visual Studio MSVC and standalone MSVC for the same year at the same time. I should be able to fix this reasonably easily now.
Post Reply