Exteriors

Everything about development and the OpenMW source code.
Locked
User avatar
lgromanowski
Site Admin
Posts: 1193
Joined: 05 Aug 2011, 22:21
Location: Wroclaw, Poland
Contact:

Exteriors

Post by lgromanowski »

Zini wrote: Since the current milestone is nearly complete, I thought we could start collecting our thoughts on the exterior cell implementation.

Here is my brain dump:

We have to options:

1. Ogre's old terrain scene manager
2. Ogre's new terrain component

We already have a prototype for option #1, which could speed up implementation a lot. But the old terrain system is no longer part of Ogre (1.7 is the last version supporting it). If we use it, we won't be able to use further Ogre upgrades.

The new terrain component is more complex, but it offers a tight integration with Ogre's paging component. Also, it is pretty much the only option, if we ever decide to modernise the landscape graphics.


Other things to consider:
- we must find out, if exterior coordinates are based on the cell- or the world-coordinate system.
- we probably will need a floating origin system (i.e. keep Ogre coordinate 0, 0, 0 always in the current cell), because we are only working with float and a float does not offer enough precision for some of the larger plugins (even default MW might cause some problems near the map border).
- we already have a suitable abstraction to distinguish between interiors and exteriors by means of the Cell/CellImp/Interior classes in openmw/mwrender. All we have to do is to add a suitable Exterior class. Object rendering code itself (in CellImp) won't be touched.
sir_herrbatka wrote: And what about multiple exteriors? How to do this? It's essential imo.

Can we add prefix to coordinates of objects in other exteriors?
fallenwizard wrote: Afaik Yacoby wrote a terrain plugin already.
Zini wrote: @sir_herrbatka: That's post 1.0 stuff. I say to that, what I say to all post 1.0 stuff at this time: *shrug*. Multiple exteriors would basically mean reworking a huge part of the engine. World-model, esm format, renderer, pretty much everything. Oh, and we would need a new editor first. I don't see that happening in the near future.

@fallenwizard: I think you are referring to the exterior rendering for the old D version. AFAIK it was really not much more than a prototype. It rendered only the landscape, probably didn't address several issues and was based on Ogre's old terrain scene manager. At least that is, what I remember. I never found the time to have a closer look at the code.
fallenwizard wrote: Yes, you are right but I am sure it's a good starting point.
sir_herrbatka wrote: hmm @zini. Can you fight some of my sentiments then?

I'm afraid that putting this essential feature will require even more reworking later. I'm not pro like you guys so I just hope that you may want to share some of your thoughts with me :-) So; can you spend a minute or two and explain this, please?
Zini wrote: It won't cause more work later, because most of the parts, that would require changing, are already implemented. We can't save any work by upgrading the world model and the other stuff now.

And even if that would not be the case, a bit more overall work would still the better choice, if it helps us to reach 1.0 faster. We won't add new features before 1.0 unless they are "drive-by" improvements, that can be done without any additional work.
Since this is a voluntary spare time project, active developers would be free to add some new features early, if it doesn't disrupt the 1.0 progress (I may decide to make use of this option at some point). Multiple exteriors would not fall into this category, because it is too big.

But the most important argument really is, that we will need a new editor, before we can add multiple exteriors or even adjust the world model in a way, that it can support multiple exteriors.
silentthief wrote: I dunno if this makes any difference, but with MW you could make an interior cell that mimics an outdoors one. They used this in the tribunal expansion. Now, one answer to multiple overland exteriors might be if the internal cells don't have the same limit on size as they did in MW (as in, you start to have problems putting anything larger than like 2-3 outside cells worth of space into an interior cell. this was why the city of mournhold was broken up into 5 interior cells).

There may be other work-arounds for anyone die hard enough to try them...

ST
Zini wrote: Interiors with the QuasiEx flag aren't really a soluation. The size limit is indeed a problem. And they can't have landscape. If we want to do this, then we have to do it properly.

After sleeping about it I came to recognise that the situation is not as bleak as I originally thought.

One of the properties of the esm format is, that records can have optional sub-records. If we add such a sub-record to each exterior-related records, we can use it to specify the exterior (with the sub-record not present the record would default to the default exterior).

The script system needs only minor modification too. Instead of adding a completely new set of instructions for multiple exteriors or modifying the exiting ones (which would break compatibility), we can simply add a SelectExterior instruction, which makes all following instructions work on the selected exterior instead of the default. The selected exterior can be stored in InterpreterContext, from where the opcode classes can pick it up.

It seems that by accident I already implemented multiple exterior support in the MWWorld::Ptr class. I was trying to solve a completely different problem, but apparently I made it more flexible than needed for the original design.

But my primary argument holds: no editor -> no multiple exteriors
sir_herrbatka wrote: Thank you zini.

I'm not sure if lack of editor is really such a problem. But It dosen't matter. It's cool that you accidently found the solution and I'm not afraid anymore so I won't bother you :D
Zini wrote:
I'm not sure if lack of editor is really such a problem
It is. Where do you propose the esm files with multiple exteriors should come from? The TES-CS can't produce them.
sir_herrbatka wrote: But there is big chance that morrowind community will create temporary solution eg: land to the other exterior mover. ;)
Zini wrote: Had a first look at the external cell records. It seems, that the co-ordinates are in world-space and not in cell-space as a reasonable person would do it. This is pretty bad, but for now we have to simply go along with it.
Zini wrote: I made a very preliminary attempt at implementing exterior cells. No landscape, only one cell rendered, not automatic cell change by moving around and probably a lot of other stuff, that isn't working yet.

The coe (center on exterior) instruction is now available from the console. Syntax:

coe x y

I will keep this in a separate branch for now (exterior). We can merge it in after the next release.
nicolay wrote: Very good. Having the game cell system (loading/unloading references, static meshes and so on) in place would be very nice to have before integrating rendering. One possible overall design is to let the terrain rendering be a separate system that is completely driven and controlled by that basic cell loading code. Ie. we could create another dummy interface to control terrain rendering, and then fill in the implementation later.
Zini wrote:
Ie. we could create another dummy interface to control terrain rendering, and then fill in the implementation later.
Actually we had such an interface for ages. The CellRender classes are divided into InteriorCellRender and ExteriorCellRender (ExteriorCellRender recently added in my branch). Both classes derive from CellRender.
CellRender has virtual show and hide functions. All we have to do is hook in the code for showing/hiding terrain there.
Zini wrote: Moving around in the exterior world works now. For sufficient small values of "works". There are several special cases not handled properly yet. We are loading only one cell instead of nine. And there is a terrible mess up with coordinates or scale or something completely different. Cells seem to render in the wrong place (probably off by a cell or two). I have no clue yet, what is going wrong.
Zini wrote: Okay, problem solved. It was an unlucky combination of two bugs that kept me stumped for quite some time. Basic exterior cell loading/unloading works now (still only one cell at this time). Will continue to improve it (no need to pull it into the main repository yet).
Zini wrote: The 3 by 3 cell area loading is working now. Performance is ... not so great. But that was to be expected. Our viewing distance is quite a bit larger than with vanilla MW and so far we have zero performance optimisations.
pvdk wrote: Loading an exterior cell fails for me with the following error:

Code: Select all

loading exterior cell '5, -11'
openmw: /usr/include/OGRE/OgreSharedPtr.h:160: T* Ogre::SharedPtr<T>::operator->() const [with T = Ogre::Skeleton]: Assertion `pRep' failed.
Zini wrote: That is not an exterior problem, but ape's creature rendering code acting up. Shouldn't crash though, if you have the latest version of my exterior branch. Both ape's and Nico's branch have fixes for this problem (they are not merged with my exterior branch yet).
pvdk wrote: Latest version. Is it possible to coc to an exterior or do I have to use doors?
Zini wrote: Oops! Looks like I overlooked coc for exteriors. Use coe x, y for now. Or doors.
Zini wrote: coc for exteriors is now implemented in my cell branch. Was actually a bit more complicated than I expected.
Zini wrote: With my latest changes merged into Nico's fork, I think I am done for now. As mentioned in the pull request, the only feature missing is to allow the player to move outside of the pre-defined cells (swim far away from the island). I think this is better implemented together with other cell handling changes, that we will have to do later (e.g. the problem of loading all cell data at startup).

Also, there is the question of performance optimisation. I think this should be postponed as well. I have a pretty good idea of what is to be done here and it would most likely interfere with the raycasting activation implementation we currently have. Therefore we should not start with optimisation, until this implementation has been replaced (with something from bullet or some other soluation).

Unless someone objects or finds other problems I haven't noticed, I would like to change the status of the roadmap entry "Exterior cells - loading, unloading and management" to "Done".
Zini wrote: Some clarification: I am not planning to pick up the terrain task. While it is true, that we had a prototype for the D version, it uses the TerrainSceneManager (IIRC). Since the TerrainSceneManager is gone with Ogre 1.8, we would be ill-advised to make use of it. Our options are to either come up with a custom implementation from scratch (lots of work and very complex, especially the LOD part) or to use Ogre's new Terrain-component (also quite a bit of work and pretty complex). I currently can't afford to put the time and brain necessary for this task into OpenMW.
sir_herrbatka wrote:
Some clarification: I am not planning to pick up the terrain task.
Ok. I should PM you. I'm stupid.
While it is true, that we had a prototype for the D version, it uses the TerrainSceneManager (IIRC). Since the TerrainSceneManager is gone with Ogre 1.8, we would be ill-advised to make use of it.
yeah, outdated.
Our options are to either come up with a custom implementation from scratch (lots of work and very complex, especially the LOD part) or to use Ogre's new Terrain-component (also quite a bit of work and pretty complex).
This is interesting. Thx for info I will edit it.
Star-Demon wrote: I think we should, if we can, set up some child scenemanagers for each region based off the basic functions of the ones that render the landscape. (although I don't know how high up scenemanager is in the rendering of stuff in our project) This would be great because as Morrowind's scope grows (IE TR and other such mods), people can build a scenemanager to reflect the weather and environment in different areas, as well as spice up existing areas.
Zini wrote: That would be very hard to impossible to do with the current structure of Ogre. Also I do not see any advantages in this approach.
sir_herrbatka wrote:
That would be very hard to impossible to do with the current structure of Ogre.
When we are here - there will be no more big changes in OGRE3D terrain in a predictable future?
Zini wrote: Assuming with "here" you mean the end of the current milestone, the answer most likely is yes. It depends a bit on how fast we are progressing with this task (once someone has picked it up). If it takes too long, we might move some parts to to later milestone (smooth texture blending might be such a sub-task). Apart from that all further improvements are post 1.0 stuff.
nicolay wrote: Some clarification on the old terrain code: It was NOT based on any external module, but rather written completely from scratch by Yacoby.

In any case, here's the current status on terrain: I've done some local (non-public) testing of the Ogre terrain system. So far results aren't fantastic, the examples I've managed to create so far crash on exit, and the documentation isn't exactly great.

So instead I've started in the other end, with the code reading landscape data from ESMs, and that's what I'll be working on for a while. The loading and pager system (responsible for loading the correct data and setting up the cell at the right time) can be made completely independent of the actual renderer, so that's the approach I'm going for ('cause I just love me some modularity.)

With this setup, it's likely that I'll make something new based on the old code and partially based on what the Ogre terrain system does behind the scenes. The reason for this is simply that we have the code already and, having worked with it quite extensively before, I understand every aspect of it.

Or I might try some more with Ogre terrains. Since the rendering system will be abstracted from the rest, the cost of making the wrong choice here should be pretty limited.
sir_herrbatka wrote: great!

Can you post link to the rss feed with commits, plz?
nicolay wrote: For the general repository you mean? It's the default rss for the page at http://github.com/korslund/openmw
sir_herrbatka wrote: great, i thought that you created another branch.
Locked