Elder-scrolls IV Oblivion

Feedback on past, current, and future development.
User avatar
psi29a
Posts: 4917
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Elder-scrolls IV Oblivion

Post by psi29a » 05 Jun 2020, 21:17

Would Alenet be interested in working with OpenMW to help Oblivion support along? :D

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Elder-scrolls IV Oblivion

Post by cc9cii » 05 Jun 2020, 22:56

cc9cii wrote:
03 Jun 2020, 22:51
I was using the ".lod" files (which contain object references but without object id, so they're like spawned objects). If the object has a low-poly version (i.e. _far.nif) that is used, and no collision shapes are created. If I remember correctly the low-poly mesh also specify low-poly textures. (just checked, they do - e.g. CastleCheydinhalKeep01_far.NIF uses Architecture\\Castle\\Cheydinhal\\CheydinhalCastleLOD01.dds)

But I noticed a few of the static object references have incorrect positions and/or scale. So I changed to another way that I will describe below.
I should note that for Morroblivion the second method didn't work, i.e. some distant statics were not included. This suggests that other Oblivion mods may have the same issue.

ponyrider0
Posts: 14
Joined: 11 Jun 2019, 23:53

Re: Elder-scrolls IV Oblivion

Post by ponyrider0 » 06 Jun 2020, 00:00

cc9cii wrote:
05 Jun 2020, 22:56
cc9cii wrote:
03 Jun 2020, 22:51
I was using the ".lod" files (which contain object references but without object id, so they're like spawned objects). If the object has a low-poly version (i.e. _far.nif) that is used, and no collision shapes are created. If I remember correctly the low-poly mesh also specify low-poly textures. (just checked, they do - e.g. CastleCheydinhalKeep01_far.NIF uses Architecture\\Castle\\Cheydinhal\\CheydinhalCastleLOD01.dds)

But I noticed a few of the static object references have incorrect positions and/or scale. So I changed to another way that I will describe below.
I should note that for Morroblivion the second method didn't work, i.e. some distant statics were not included. This suggests that other Oblivion mods may have the same issue.
Correct. The Morroblivion has a number of records where the VWD flag is enabled but there is no corresponding _far.nif file. I think these are relics from very old builds of the ESM.

Currently, we rely on TES4LODGen to generate working *.lod/*.cmp files for us. TES4LODGen will write out entries in the *.lod files for any worldspace reference ("spawned object") that links back to a base record where a corresponding *_far.nif file exists (or for a speedtree, where a corresponding \\textures\\trees\\billboards\\*.dds file exists). It does this regardless of whether or not the ESM's VWD flag is set for that record. The Oblivion engine solely relies on the *.lod/*.cmp data for rendering distant objects and ignores the VWD flags. I think VWD flags are only used by the TES4 Construction Set in its own Distant Object generation functions.

A consequence of the above Oblivion engine behavior is that the *.lod/*.cmp files must be regenerated anytime there is a change in positions of the "spawned objects" or a change in their load-order dependent FormIDs. I am able to exploit this engine behavior to render "dummy speedtree billboards" in place of non-speedtree NIF-based objects such as buildings, rocks, etc. At least in the Oblivion engine, this results in large performance gains even over equivalent pseudo-billboard meshes that contain only 4 vertices. These gains are not insignificant: there is almost no loss in FPS when rendering 500 speedtree billboards, compared to at least a 15 FPS drop when rendering 500 non-speedtree pseudo-billboard meshes with 4 vertices.
Last edited by ponyrider0 on 06 Jun 2020, 00:22, edited 1 time in total.

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Elder-scrolls IV Oblivion

Post by cc9cii » 06 Jun 2020, 00:21

cc9cii wrote:
03 Jun 2020, 22:51
Oblivion/FO3/FONV/Skyrim also has a way of indicating which cells have distant statics. From my notes/log:

Code: Select all

    Game     |  BSA              |  Folder     | filename
    ---------|-------------------|-------------|--------------
    Oblivion | Oblivion - Meshes | distantlod  | .cmp
    FO3/FONV | Fallout - Misc    | lodsettings | .dlodsettings
    Skyrim   | Skyrim - Meshes   | lodsettings | .lod
I should also note also that the FO3/FONV/Skyrim files above are nothing like the ".cmp" file in Oblivion. They are used for different purposes.

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Elder-scrolls IV Oblivion

Post by cc9cii » 10 Jun 2020, 01:01

So I had a chat with psi29a regarding NIF code. I'm assured that Zini is ok with adding new NIF support to OpenMW, so I will be pushing my experimental branch to github once I get to a wifi access point. Look for "old-ogre". I had never intended this branch to be public so it has some stuff that I'm rather embarrassed about, to be honest. But I figure hiding things won't help me pretend to be a good programmer so whatever.

It should also be noted that the code in that branch is experimental. The code is not ready for merge to my master, nor my main branch "cc9cii", not even the side branch "prepare_nif_release". There are a lot of temporary workarounds and old comments that no longer make sense. Then there are some code that really need redesign due to some fundamental issue - I have marked those with a comment block that looks like:

Code: Select all

/*
 * **************************** DO NOT USE AS IS *******************************
 *
 * Some comment.
 *
 * *****************************************************************************
 */
I had ragdoll physics working (years ago now), then broke it while re-writing major sections and never got around to finishing it. Once I crack Skyrim's skin texture shaders I will probably go back to get that working again. The only thing that wasn't working properly was scaled objects - the transformations are quite tricky due to the Havok scaling involved.

Recently I started keeping some notes because my memory is not so good - look for "extern/nibtogre/NIF-Design" - it may help whoever tries to decipher my code. (but then it may not - not all comments are kept up to date with the changes made to the code). If there are any questions I'd be happy to explain, assuming I can remember why I did things in a certain way. I think Nifskope would be a better source of info for most things since it handles NIF much better (if you can get past Qt that is).

I have a feeling that the most confusing part with the NIF code will be the resource management and not the use of Ogre API for rendering and animation.

Changes to OpenMW and OpenCS are even more experimental. The only good bit might be the load time reduction for OpenCS under Windows - the load time is almost instant for Morrowind. But I hadn't done any work in OpenCS for several years now other than occasional test compile. In OpenMW there are code blocks that don't belong there - purely because I was too lazy to properly redesign the classes (e.g. the sound/music code in MWWorld::Scene) - they need to be refactored.

All my work is done in Windows so I believe it won't even compile with gcc.

EDIT: should be there now. NOTE: it doesn't have the latest TES5 animation because I broke it while trying to get the skin shaders to work. Also missing some other test branch stuff.

1Zero
Posts: 38
Joined: 09 Feb 2020, 18:44

Re: Elder-scrolls IV Oblivion

Post by 1Zero » 10 Jun 2020, 19:18

Amazing, I was looking forward to this!
Looked through the source and have a bunch of questions but I should first attempt to build the project.
It's going to be a challenge until I actually collect all the libraries in the proper versions and get it compiled on VS 2019 Community though.

"Based on the cryptic notes on https://facegen.com/dl/sdk/doc/manual/indepth.html"

Classic Programming Humour +1.

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Elder-scrolls IV Oblivion

Post by cc9cii » 11 Jun 2020, 02:49

I'm using Ogre v1.10.11 Changeset: 12356 (e73af6d1c819) CI: enable parallel building

It's been a while since I updated Ogre but I remember that some of the later changes broke things, so it's probably best to stick to a known working version.

1Zero
Posts: 38
Joined: 09 Feb 2020, 18:44

Re: Elder-scrolls IV Oblivion

Post by 1Zero » 17 Jun 2020, 22:39

@cc9cii

Alright, I run into some problems

I don't really use CMake that often, where should I put the downloaded development libraries for CMake to find them?
Also, are there prebuild development libraries available for MyGUI? There only seems to be the option to compile them from source in Visual Studio as well?

Regarding Qt5

"To run the installer, you need a Qt Account. You can use your Qt Account credentials to sign-in to all Qt services, including the forums and wiki. If you do not already have a Qt Account, you can create one during the installation process."
https://doc.qt.io/qt-5/gettingstarted.html

I need an account for access to Qt prebuild libs? There only appears to be this large 2.4GB binary to setup the environment which actually prompts me to login with an account.

Alternatively, I would just need the Visual Studio .sln file for the project, skipping the CMake setup and add the dependencies manually.

User avatar
AnyOldName3
Posts: 1939
Joined: 26 Nov 2015, 03:25

Re: Elder-scrolls IV Oblivion

Post by AnyOldName3 » 17 Jun 2020, 23:27

If you're on Windows, then putting dependency libraries where CMake will find them is a wild ride. There's no standard lookup location, so you basically just put them wherever, then look at CMake's findLibraryName.cmake script in its modules directory to see how it looks for it, and pass it the necessary variables either on the command line or in the GUI.

As for prebuilt Qt, they recently changed it so that the official versions need an account. As the open-source-friendly version is GPL/LGPL, though, other people also provide binaries. That's what the prebuild script for upstream OpenMW uses. If you're desperate for the official binaries and don't want to give former Nokia employees access to your data, you could always make an account with Five Minute Mail or another anonymous temporary email address service.

If you've got a .sln already, though, you can probably skip CMake as its whole point is to make the solution file. Provided the solution doesn't have anything system-specific hardcoded into it, you can probably just update the paths to dependencies to refer to where you've got them on your machine.
AnyOldName3, Master of Shadows

1Zero
Posts: 38
Joined: 09 Feb 2020, 18:44

Re: Elder-scrolls IV Oblivion

Post by 1Zero » 18 Jun 2020, 00:35

AnyOldName3 wrote:
17 Jun 2020, 23:27
If you're on Windows, then putting dependency libraries where CMake will find them is a wild ride. There's no standard lookup location, so you basically just put them wherever, then look at CMake's findLibraryName.cmake script in its modules directory to see how it looks for it, and pass it the necessary variables either on the command line or in the GUI.

As for prebuilt Qt, they recently changed it so that the official versions need an account. As the open-source-friendly version is GPL/LGPL, though, other people also provide binaries. That's what the prebuild script for upstream OpenMW uses. If you're desperate for the official binaries and don't want to give former Nokia employees access to your data, you could always make an account with Five Minute Mail or another anonymous temporary email address service.

If you've got a .sln already, though, you can probably skip CMake as its whole point is to make the solution file. Provided the solution doesn't have anything system-specific hardcoded into it, you can probably just update the paths to dependencies to refer to where you've got them on your machine.
I just look for whatever version gets me to compile cc9cii's fork, I don't have any "software philosophical" preferences. Sure I could give them a burner account, but if someone has the proper dev version zipped somewhere though, feel free to link it.
Optimally, I can skip the CMake setup altogether and maybe c99cii can provide his VS solution file for now, because yes, I can simply repath all the proper include and library paths to where the libs are located on my computer. I am just eager starting to integrate the scripting environment and not so much deal with build systems at this point in time, if possible.

Post Reply