Distant Lands and OpenMW

General discussion regarding the OpenMW project.
For technical support, please use the Support subforum.
Post Reply
Time4Tea
Posts: 110
Joined: 01 Jan 2020, 00:27

Re: Distant Lands and OpenMW

Post by Time4Tea » 05 Apr 2020, 00:19

psi29a wrote:
04 Apr 2020, 16:40
Time4Tea wrote:
04 Apr 2020, 13:40
It seems like what OpenMW needs is something like a Software Engineering college student that wants to break in to coding 3D graphics for the video game industry, who could take on distant statics as a college project. It would be great experience for their resume.
Find us one... :)
If I come across anyone who fits the bill, I'll certainly try to steer them your way :)

Has anyone ever tried approaching university departments that have degree courses in 3D graphics or game development?

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Distant Lands and OpenMW

Post by cc9cii » 05 Apr 2020, 02:39

CMAugust wrote:
05 Apr 2020, 00:10
Since you touched on the subject of later games, perhaps we should discuss how that will look. For those who aren't aware, all post-Morrowind titles include LOD files pre-generated by the Construction(Creation) Set(Kit) for the engine to read. Will OpenMW eventually have the functionality to read this LOD data, or will it ignore them altogether in favor of this generic solution? A solution which, I might add, is still hypothetical. One person only has attempted it, with promising results, but it wasn't considered good enough; that person has since left, nobody has picked up where he left off, and there's no indication any of his work will be useful as a stepping-block for whoever tries next.

Be that as it may, if OpenMW will in fact be given the ability to read LOD data of later games, it might be pertinent to start researching that now - perhaps even extending it to read MGEXE distant data, if they're similar enough, or else designing a lodgen feature within OpenMW-CS. Again, if subsequent games and their construction sets have this functionality which OpenMW intends to support/copy anyway, this is a conversation that ought to happen if it hasn't already.
I've been spending the past week looking at how Oblivion does LOD. I think I got most of it figured out (fingers crossed). I think it will be possible to implement something similar for Morowind/OpenMW, as a MOD so that the vanilla game is untouched for those who don't want it. Basically:
  • the landscape is pre-calculated into low-poly mesh and texture (similar to the way MGE does, except no statics included). The low-poly mesh are "sliced" into 32x32 cell blocks (which are 16x16 Morrowind size) and I guess they get rendered based on the player's current cell. I think they take care so that when "slicing" the vertices at the boundaries are not changed (else they won't look proper next to the real terrain). These can be found in "meshes\\landscape\\lod\\" directory and the first number is the formid of the world space in decimal.
  • which static objects and trees are visible from far are pre-determined and such static objects have low-poly versions (e.g. White Gold Tower). The trees don't, but then I don't know much about speedtree. Skyrim seems to have moved away from speedtree and the trees *do* have low-poly mesh and texture. These "visible from far" objects are listed in Oblivion.esm (implied in the group hierarchy as well as sub-records in REFR). They are also listed in Oblivion - Meshes.bsa (these are the ".lod" files in "distantlod\\" directory but with world space editorid rather than formid - I think the intention is to speed things up but in my tests this might be slower?)
  • the number of cells for loading visibly distant objects are different (and greater, obviously) than the number of cells for loading "normal" objects.
  • all of the above can be put in a MOD if newer versions of ESM records are used (unfortunately this may break the savefile format, but then some complex mods break savefile anyway?)
The first 2 bullet points will require a lot of tedious work and will need volunteers/artists (choosing the right visibly distant object requires someone with an artistic vision IMHO).

Since I'm just modifying OpenMW (albeit old version) whatever I implement in the engine should be mostly applicable to the latest. I'll report back once I get a bit further (or hit a roadblock).

EDIT: bulletpoints formatting to make them more legible

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Distant Lands and OpenMW

Post by cc9cii » 05 Apr 2020, 23:05

Some more notes:
  • I have noticed that Morroblivion 0.64 already has the LOD data, including the low-poly landscape meshes. I don't know yet if they can be used as is or some tweaks needed to adjust the co-ordinates.
  • The '.cmp' files list the cell grid locations where visibly distant statics are found. Groups of 16 bit integers Y, X (not sure why they have it back to front) except the last pair which might be a some kind of end marker.
  • the '.lod' files can have one or more statics and/or trees. There are a lot of these files in Oblivion - over 9000 in world space Tamriel.
EDIT: I'm not entirely certain whether we can use Morrobilivion assets - if the Oblivion CS was used to generate Morrowind heightmap mesh do we still need a full game of Oblivion? Can we distribute (or re-distribute) the tweaked assets from Morroblivion?

User avatar
psi29a
Posts: 4917
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Distant Lands and OpenMW

Post by psi29a » 06 Apr 2020, 12:02

cc9cii wrote:
05 Apr 2020, 23:05
EDIT: I'm not entirely certain whether we can use Morrobilivion assets - if the Oblivion CS was used to generate Morrowind heightmap mesh do we still need a full game of Oblivion? Can we distribute (or re-distribute) the tweaked assets from Morroblivion?
OpenMW does not deal in content. We will not distribute nor re-distribute anything derived from Bethesda's IP.

So to answer your question, if end users want to use Morroblivion assets, go for it! If you use the Oblivion CS then you're bound by their EULA, meaning that you have (need) the full game of Oblivion.

User avatar
Jemolk
Posts: 226
Joined: 20 Mar 2017, 02:12
Location: Sadrith Mora

Re: Distant Lands and OpenMW

Post by Jemolk » 06 Apr 2020, 19:25

psi29a wrote:
06 Apr 2020, 12:02
cc9cii wrote:
05 Apr 2020, 23:05
EDIT: I'm not entirely certain whether we can use Morrobilivion assets - if the Oblivion CS was used to generate Morrowind heightmap mesh do we still need a full game of Oblivion? Can we distribute (or re-distribute) the tweaked assets from Morroblivion?
OpenMW does not deal in content. We will not distribute nor re-distribute anything derived from Bethesda's IP.

So to answer your question, if end users want to use Morroblivion assets, go for it! If you use the Oblivion CS then you're bound by their EULA, meaning that you have (need) the full game of Oblivion.
We might not want to endorse that as a project, however, for legal reasons. Not sure how liability would work here, myself, and informing people that this is an obligation without any mention of the possibility of ignoring it may fulfill our legal obligation, but I kinda feel like it's better to leave that to unaffiliated folks on the internet, given that actually enforcing the license is a physical impossibility.

User avatar
psi29a
Posts: 4917
Joined: 29 Sep 2011, 10:13
Location: Belgium
Gitlab profile: https://gitlab.com/psi29a/
Contact:

Re: Distant Lands and OpenMW

Post by psi29a » 06 Apr 2020, 23:09

There is no endorsement.

The fact is, as OpenMW evolves to support additional content, then mods like Morroblivion suddenly work by the fact that Oblivion content is supported just like Morrowind content. You wanna play Morroblivion, you gotta own Oblivion.

Again, OpenMW doesn't deal in content. That's up to modders and content creators.

User avatar
Mistahtokyo
Posts: 136
Joined: 07 Sep 2013, 18:31

Re: Distant Lands and OpenMW

Post by Mistahtokyo » 07 Apr 2020, 16:30

Seeing as cc9cii prefaced his comment by saying it'd be a MOD (in all caps, too), the discussion on OpenMW including or not including assets is irrelevant.

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Distant Lands and OpenMW

Post by cc9cii » 09 Apr 2020, 23:30

Some progress notes after playing around a bit:

If the Oblivion-like LOD was to be used for OpenMW, there are a number of things that may need to change:
  • Terrain - not sure how the latest terrain in OpenMW works, but some kind of alpha blending of textures will need to be introduced. (prob. using shaders)
  • Terrain, object and actor insertion - I suspect they are currently tied to cell insertion. If so, more fine-grain control is needed, especially statics and trees (for visibly distant rendering without physics). Most likely changes in `MWWorld::Scene` and others.
  • A mechanism to render the same object with different LOD levels. e.g. the `_far` variant of a static object. Most likely changes in `MWRender::Objects` and others.
  • A mechanism to selectively display certain statics (e.g. Ghost Fence or an Oblivion Gate) based on a quest progress or some other conditions. This will require either the NIF model to have non-standard attribute (non-standard for a vanilla Morrowind perspective, anyway) such as adding NiSwitchNode to existing NIF models. Alternatively we can achieve the same result by adding an XESP sub-record to REFR like Oblivion (this way existing NIF models won't require any changes, but it does require selective rendering as per above bullet point).
  • A set of additional config to control the distance of rendering various LOD elements, including actor fading distance, etc. (this is optional but most likely will be beneficial)
There are also some new things that will need to be introduced:
  • Add the ability to parse and load simple landscape LOD NIF models. These have only 2 blocks, NiTriStrips and NiTriStripsData. (we can probably use NiTriShape/NiTriShapeData instead for OpenMW, depends on the tool generating these)
  • A tool to generate low-poly landscape LOD mesh, if we decide not to use the Construction Set.
  • Landscape LOD meshes and textures themselves. These can be part of a MOD.
  • Low-poly models of the visibly distant statics and trees. These can be part of the same MOD. (if we want to be fancy, we can have multiple levels of complexity like Skyrim)
  • The `.cmp` and `.lod` files to indicate where to place the visibly distant statics and trees. Again part of the same MOD. These are needed so that we know where to place them *before* the cells are read from the ESM/ESP/save files. Unfortunately the formats of `.lod` files probably will need to differ from TES4 since there is no concept of FormId in OpenMW.
That's what I found out so far. There's bound to be more. The proof of concept is done, the next step is to actually implement. I think the terrain materials is the first step. Oblivion uses "splatting" of many layers of textures for LAND (not sure if Skyrim does the same) so it will be different to OpenMW.

I suspect above changes will be easier for me (in my fork) since I don't have to worry too much about the existing implementation and all the new stuff is already present in the Oblivion game files. Still, it seems quite achievable for OpenMW if we can find some devs/artists.

CMAugust
Posts: 253
Joined: 10 Jan 2016, 00:13

Re: Distant Lands and OpenMW

Post by CMAugust » 10 Apr 2020, 01:03

Thank you for your investigations so far cc9ii. Can you tell if water height is included in Oblivion LOD data? It's determined per cell in Oblivion if I understand correctly, whereas Morrowind has a single pane for the entire worldspace. It will only become necessary to support that for Oblivion, though people may want it for Morrowind before then. But I don't know how water is handled for distant detail.

User avatar
cc9cii
Posts: 511
Joined: 28 Mar 2013, 04:01

Re: Distant Lands and OpenMW

Post by cc9cii » 10 Apr 2020, 01:32

CMAugust wrote:
10 Apr 2020, 01:03
Thank you for your investigations so far cc9ii. Can you tell if water height is included in Oblivion LOD data? It's determined per cell in Oblivion if I understand correctly, whereas Morrowind has a single pane for the entire worldspace. It will only become necessary to support that for Oblivion, though people may want it for Morrowind before then. But I don't know how water is handled for distant detail.
Oblivion's CELL record has XCLW sub-record when the water height is non-zero. (it also has XCWT to indicate the type of water) Oblivion's LOD related data do not include any water height info. As far as I can tell it doesn't render water at different heights per individual CELL. I also read somewhere that transparent LOD mesh won't render objects, including any water, behind it (i.e. it isn't really transparent) - this should be easy enough to verify but no real point for us since our implementation won't be the same as vanilla engine.

EDIT: WRLD record does have flag "No LOD water" according to UESP, so maybe there's something about LOD water after all. I'll go find out which of the worlds have such flag turned on.

EDIT2: these worlds have that flag (I think this implies local water and LOD water are different - e.g. ICMarketDistrict does have local water where the Sacred Lotus plants are)

Code: Select all

no lod water TestTree
no lod water TestHopeTrees
no lod water SENSBliss
no lod water SENSCrucible
no lod water TestRegion
no lod water BrumaWorld
no lod water BravilWorld
no lod water AnvilWorld
no lod water LeyawiinWorld
no lod water CheydinhalWorld
no lod water SkingradWorld
no lod water ICTheArcaneUniversity
no lod water ICMarketDistrict
no lod water ICArboretumDistrict
no lod water ICTalosPlazaDistrict
no lod water ICElvenGardensDistrict
no lod water ICTempleDistrict
no lod water ICArenaDistrict
no lod water ICImperialPrisonDistrict
no lod water ICImperialPalace
no lod water KvatchPlaza
no lod water ChorrolWorld
no lod water AnvilCastleCourtyardWorld
no lod water ICImperialPalaceMQ16
no lod water ICTempleDistrictMQ16
no lod water SEManiaGarden
no lod water SEDementiaGarden
no lod water KvatchEast
no lod water KvatchEntrance
no lod water E3Kvatch

Post Reply