Benchmarks / Hardware Specs

[scrawl]

I figured it would be fun to redo the OpenMW vs. OpenMW-OSG benchmark from five months ago.

Build:
openmw commit 3fe38e3556d31194d8c293426b7b7e2c5dc4bd3b
scrawl/osg commit a993fc66fe47d9d95c87a661380006abb72868d1

Hardware:
i7-4790k @ 4.6 GHZ
Asus GTX 970 STRIX DC2OC

Settings:
- Object shaders = OFF
- Shadows = OFF
- Water shader = ON
- Water refraction = OFF
- Water texture size = Medium
- 1024x768 resolution

Water refraction needs to be OFF for a fair comparison, because OpenMW-Ogre3D only supported the refraction shader when object shaders are enabled - but OpenMW-OSG doesn't support object shaders yet.

OpenMW-Ogre3D 0.36.1
49 FPS


OpenMW-OSG 3fe38e3556
193 FPS


With the water shader OFF:

OpenMW-Ogre3D 0.36.1
57 FPS

OpenMW-OSG 3fe38e3556
230 FPS

Edit: I just noticed that I had antialiasing enabled in the OSG test but disabled (since it wasn't supported) in the Ogre3D test. This shouldn't make a difference in test results though since the game was still CPU limited as you can see on the profiling overlay.

I knew that we've come a long way since June but didn't expect the improvement to be so drastic. That's 400% framerate, when in June we were only at 153%.

Makes me curious: is OpenMW faster than the vanilla engine now? When I checked sometime in July or August, we were pretty close and I thought it was possible... but I no longer have Windows on my new PC to check.

If so, we would have a nice sales pitch for the release announcement.

[Spartan195]

yesterday I found this project openMw, and i was really exited to play morrowind again with this engine.

What can i say: I play it actually on windows 10, I don't had time to install it on my linux partition yet.

I use the openmw-0.36.0,

Hardware: AMD 8350 8-core 4Ghz
GTX 960 oc 4GB
16 gb ram

Aaand i have to low some details to run it at 30 fps. when i'm in the exterior sometimes reaches 45 fps, indoor reaches 150 or 200.

Is playiable this engine version (openmw-osg) in windows? or is only avaiable in linux?

[sjek]

viewtopic.php?f=20&t=1808
should be : )

edit: upgrading to openmw osg goes directly at least in linux. old setting are reserved and needed settings written to setting.cfg.
for (later than 0.36 openmw.cfg --script-run= commands goes to form script-run= or so. there was difference in platforms.)
the keys are in input_v3.xml

those + launcher.cfg to bypass install wizard should suffice.
going back to ogre needs ogre's lines in setting.cfg or crash log is given

good way seems to be backup, remove setting.cfg, load in ogre and make settings (shaders, antialising, refraction off)
try out and load in osg.
https://wiki.openmw.org/index.php?title=Paths


trying this in sadrith mora with default settings, 6666 viewing distance and same texture mods gives about
ogre: 25
osg: 55

taing grid size to 7X7
ogre: 9
osg: 35

viewing distance to 20 000
ogre: about 7
osg: 25

[SquireNed]

I figured it would be fun to redo the OpenMW vs. OpenMW-OSG benchmark from five months ago.

Build:
openmw commit 3fe38e3556d31194d8c293426b7b7e2c5dc4bd3b
scrawl/osg commit a993fc66fe47d9d95c87a661380006abb72868d1

Hardware:
i7-4790k @ 4.6 GHZ
Asus GTX 970 STRIX DC2OC

Settings:
- Object shaders = OFF
- Shadows = OFF
- Water shader = ON
- Water refraction = OFF
- Water texture size = Medium
- 1024x768 resolution

Water refraction needs to be OFF for a fair comparison, because OpenMW-Ogre3D only supported the refraction shader when object shaders are enabled - but OpenMW-OSG doesn't support object shaders yet.

OpenMW-Ogre3D 0.36.1
49 FPS


OpenMW-OSG 3fe38e3556
193 FPS


With the water shader OFF:

OpenMW-Ogre3D 0.36.1
57 FPS

OpenMW-OSG 3fe38e3556
230 FPS

Edit: I just noticed that I had antialiasing enabled in the OSG test but disabled (since it wasn't supported) in the Ogre3D test. This shouldn't make a difference in test results though since the game was still CPU limited as you can see on the profiling overlay.

I knew that we've come a long way since June but didn't expect the improvement to be so drastic. That's 400% framerate, when in June we were only at 153%.

Makes me curious: is OpenMW faster than the vanilla engine now? When I checked sometime in July or August, we were pretty close and I thought it was possible... but I no longer have Windows on my new PC to check.

If so, we would have a nice sales pitch for the release announcement.

Direct me to the OSG build you'd like me to use, and I can try to set up a comparison.
Edit: Also, are there any particular settings I should have on/off when I do the comparison, and does vanilla have a profiler like OpenMW's?

[Cramal]

Did any of you did any benchmark with Vanilla Morrowind to compare framerate?

I would be very interested to know if the new OSG implementation is faster than the Vanilla engine or not?

[scrawl]

Direct me to the OSG build you'd like me to use, and I can try to set up a comparison.

Just use Ace's latest nightly. (That doesn't yet include the optimizations I made to OSG itself, but the difference should only be 10% or so).

Edit: Also, are there any particular settings I should have on/off when I do the comparison

Make sure the framerate is set to unlimited. By default Morrowind.ini sets

Code: Select all

Max FPS=240

Disable pixel shaded water in the morrowind launcher. Set OpenMW water shader to off.

Play with the "View distance" slider until OpenMW and MW have roughly the same view distance. The maximum of the respective sliders might not be the same. Check that field of view matches up (I think it should match by default).

In OpenMW, disable antialiasing and anisotropic filtering, set texture filtering to 'bilinear'.

Use a low resolution (e.g. 1024x768). We want to benchmark CPU not GPU, because Morrowind scenes are usually CPU limited anyway, even on higher resolutions.

Use a fresh "new game" in both engines, then teleport to the benchmark location. We want to ensure that NPCs are in the default places and haven't wandered around yet. You can use "toggleAi" to prevent NPCs from wandering off screen (but ideally AI should be on in the moment you log the frame timings, since it's normally on when you play). When in the desired spot, save the game so you can go back to it.

For logging the frame times, I recommend to let the game run for 5-10 seconds, log the peak framerate, then restart and repeat. Frame rates can fluctuate a lot, so it's best to measure 10 times and then average the results.

does vanilla have a profiler like OpenMW's?

There is a "Show FPS" option in Morrowind.ini. I haven't tried it out.

[SquireNed]

Bah, accidentally closed out and lost my detailed post, so here's the specifics.

Vanilla seems to be hitting a GPU bottleneck. I got 150 frames more or less locked in from the coc vivec location, and I got up to ~375 by looking up (Morrowind and expansions loaded; I got slightly better FPS without the expansions, but I don't think we'll see too many people doing that). On OpenMW, I hit about ~200 FPS looking straight up, and ~135 looking toward vivec. There's some slight positioning stuff, and it looks like vanilla may have culled a model that OpenMW did not, though it's pretty darn identical in the comparison screenshots (which aren't playing nice).

In interior cells, however, OpenMW was running at 250! fps, while Morrowind was running at 150 fps (stable; the counter was frozen for several seconds at a time, going down to 137 for brief flashes). That 250 fps number for interiors is with 1080p and 16x antialiasing, which coincidentally gets around ~80 fps at Vivec. 270fps is the highest spike, which would seem to be my CPU bottleneck, since that was on the minimum spec interior starting cell, compared to the 150 for vanilla MW.

Interestingly, Morrowind's frame rates seemed to move in multiples of specific numbers, while OpenMW's seemed to move more flexibly. OpenMW actually beat Morrowind in the comparison screenshot I took, which showed 157 frames.

I'll leave conclusions to someone else, but I'd say that it's not too far off from Morrowind. The limit on OpenMW was certainly CPU; with some minor graphics improvements no performance was lost.

[scrawl]

Vivec should perform a lot better once I've rewritten the particle system component. I would be curious how we compare in Balmora (the spot shown on the first post's screenshot).

Even still, 135 vs 150 isn't too far off indeed. If compiled against my osg fork I think OpenMW would have exceeded 150.

Another optimization I haven't gotten around to yet are quad trees.

Then there's physics. Vanilla has an unfair advantage there because it doesn't process physics for idle actors. If you removed the object they're standing on, the actor wouldn't realize they should be falling. OpenMW process physics even for idle actors which alleviates such problems, but is more taxing on framerate. I might look into running physics in a background thread so that more CPU cores can be used.

Would also be interesting to know how the rendering of additional scene passes (e.g. reflections) compares - e.g. comparing the MGE XE water shader against OpenMW water shader. But let's leave that for another day...

[SquireNed]

Vivec should perform a lot better once I've rewritten the particle system component. I would be curious how we compare in Balmora (the spot shown on the first post's screenshot).

Even still, 135 vs 150 isn't too far off indeed. If compiled against my osg fork I think OpenMW would have exceeded 150.

Another optimization I haven't gotten around to yet are quad trees.

Then there's physics. Vanilla has an unfair advantage there because it doesn't process physics for idle actors. If you removed the object they're standing on, the actor wouldn't realize they should be falling. OpenMW process physics even for idle actors which alleviates such problems, but is more taxing on framerate. I might look into running physics in a background thread so that more CPU cores can be used.

Would also be interesting to know how the rendering of additional scene passes (e.g. reflections) compares - e.g. comparing the MGE XE water shader against OpenMW water shader. But let's leave that for another day...

One thing to remember is that OpenMW spiked up into the 150's range with regularity, while Vanilla reported 150 and occasionally 137. I don't think that that's an entirely accurate reflection of Vanilla's true speed; it might be a result of some engine error. Using threading might help the OpenMW performance somewhat, but it is probably moot until more graphics optimizations come around; I get a nice smooth 60ish in any conceivable situation with really good settings (admittedly on a FX-8350 and a GTX 970), and that's the graphics card rather than the CPU bottlenecking, probably.

[TechNoirMK]

My laptop "The Potato":

Spoiler: Show

CPU: Core i7-720QM @ 1.6 GHz
GPU: GeForce GT 240M, 1 GB VRAM
RAM: 4 GB

For the record, the FPS from Ogre come from memory, so feel free to take this post with a sack of salt.

OSG config - AA off, AF 8x, Draw Distance - Far

Interiors
Ogre - 0.36.1: 50-60 FPS
OSG - 0.37: skyhigh to 200-300 FPS

Seyda Neen
Ogre - 0.36.1: 15-20 FPS
OSG - 0.37: 60-70 FPS with water shader OFF, 40 FPS with water shader ON (close to water), 50-55 FPS with water shader ON (far from water)

Balmora
Ogre - 0.36.1: from never above 10 FPS to slow and painful death
OSG - 0.37: 55-60 FPS with water shader OFF, 30-35 FPS with water shader ON

You guys outdid yourself this time.