Software Rendering Mode for OpenMW

[KnightoftheWind]

Hey, I was wondering if the addition of a software rendering mode for OpenMW would be possible?. You might be wondering why anyone would request such a feature, well it's primarily due to my research into retro PC games and how they were developed, and I believe there are several key advantages that C-based software rendering has, over the likes of OpenGL/Vulkan and DirectX.

Nowadays, an overwhelming majority of games and applications use hardware rendering through DirectX or OpenGL, we know this. The OpenGL or DirectX API allows programmers to use the graphics card to generate the 3D images on your screen. The DirectX and OpenGL interface is supplied by the graphics drivers of your graphics cards. This method of producing visuals is called hardware acceleration, or hardware rendering. When the graphics card is not involved in the creation of the 3D picture, the procedure is called software rendering (of course).

But In the 90s, we had no 3D acceleration at all. So on 386's, 486's, and Pentium computers, the games produced the graphics by software rendering only. So games like the original Tomb Raider had to rely on the CPU, and not some expensive graphics card.

As CPUs became faster, the quality of the software rendering increased. After we reached the 166MHz Pentium 1, 640x480 became playable with a few thousand polygons. This was enough for most of the games at that time, however, to keep the rendering playable, they had no texture filtering, and very few effects. Later, the S3 Virge graphics card and the 3dfx Voodoo 1 introduced, allowing filtered textures and faster 3D performance than software rendering. Of course, to access these capabilities, developers had to rewrite the code to support these chips. First every manufacturer supported their own proprietary API, but later the interfaces was sort of standardised. OpenGL and DirectX (Direct3D) compatible drivers became widely available. In 1997, they were able to get much as double the performance compared to software rendering with these early graphics chips. They offered more fps and better graphics quality, so programmers began to switch to hardware rendering as the primary method of producing graphics.

But what is the situation now?

Currently we have three DirectX variations available on Windows. DirectX9 is mostly used for legacy titles, and uses fixed function rendering with the support for shaders. DirectX 11 supports programmable pipeline only, and DirectX 12 is designed for highly parallelised rendering. These three APIs are totally incompatible with each other, so gpu-reliant developers have to write three separate implementation for each of these APIs. If a programmer decides to write a game for DirectX12, then he must also write a separate rendering engine for DirectX9 as the older graphics chips will not support the newest DirectX 12 API. Utilizing DirectX12 and rendering a few textured triangles require an initialisation code thats approximately 1000 lines. To have a separate DirectX9 rendering engine for compatibility, thats another 500-ish lines. The story doesn't end there though – that's just for Windows compatibility.

DirectX as we know does not exist on Linux, MacOS, iOS or Android based devices. These platforms rely on OpenGL or Vulkan, except Mac. To support these platforms, you have to implement these APIs as well. Porting your graphics engine to OpenGL 1.1 will take about 500 more lines (we still speaking from a triangle based renderer that can do texturing). However, this only covers the desktop OpenGL, which does not exists on mobile phones. Mobiles have a different variation of OpenGL, which called OpenGL ES. There are two separate OpenGL ES APIs. The newer one is called OpenGL ES3, which is similar to DirectX 11, you have to write a shader based programmable pipeline to handle it. This is about 1000 lines, and its not backwards compatible with older phones. Older phones have OpenGL ES2 or OpenGL ES1.x. To create a program that can work on OpenGL ES1, you must again write a new renderer thats compatible.


Ok, now you have your renderer that has a separate code path for DirectX 11, DirectX9, OpenGL 1.1, OpenGL ES3, OpenGL ES1. The API implementation of the rendering alone grew past 5000 lines, and in theory now it is now capable of running on PCs, tablets, and phones. In theory. In reality, it will run only on your devices, as implementation of 3D APIs are broken. A code that works well on an nVidia chip, will maybe not work so well on an AMD, Samsung, or MALI chip. Some chips have no problem rendering non power of two sized textures, some may only work well with two-factor textures. Some maybe will crash if you allocate more than a few thousand of textures, maybe some will just give a white picture because you forgot to set a bit somewhere. Maybe on some configuration it will just crash your phone, so you have to check every single graphics vendor. You have to buy a couple of nVidia, AMD cards, VIA based laptops, everything from Mediatek, HiSilicon, AMlogic, Samsung, through tens of less common manufacturers, and you will spend the rest of the year by testing your engine on hundreds of chips, ensuring your game runs well on every chip, so its good for production. That's how bad it can get, depending on how widely supported you'd want your game to be.

We can easily see that these problems would not exist without hardware acceleration. As discussed previously, 3D acceleration was only born because the Pentium1 was too weak to produce "next generation graphics" at a suitable resolution. This was, however, 25 years ago. Since then, our CPUs have become more than 100x faster, so this is not an issue any more. I can play games like Quake 2 and Half-Life at very high resolutions and at high framerates, without breaking a sweat.

So the TL;DR is that the OpenMW project would heavily benefit from offering a software rendering mode. Not only does it decrease the amount of code and programming overhead necessary for the project, but it also means that OpenMW can be ported to a much wider variety of platforms. Including niche devices like embedded hardware or weaker portables.

[AnyOldName3]

Generally, OpenMW doesn't need to bother implementing different code paths for different OpenGL vendors - nearly everything we do has pretty consistent performance everywhere. Even if performance sucked so badly for a particular feature with a particular vendor, it would still be way faster than implementing the same thing ourselves in software rendering. You're effectively suggesting that because a Lamborghini goes at 202 MPH with high-octane race fuel but only 187 MPH with cheap consumer-grade fuel, we should switch to using a Little Tikes Cosy Coupe we push along with our feet (and it's probably got to be one with glue all over the wheels or with several hundred kilos of concrete blocks attached to make the numbers match).

I've no idea how you came up with the figures you've used to justify your suggestion, but they've clearly been pulled from someone's behind, whether it's yours or that of some lunatic you mistakenly thought was trustworthy.

[KnightoftheWind]

What I'm saying is that GPU utilisation isn't necessary for games like Morrowind, which didn't even look good when it released in 2002. CPUs can render everything the game has to offer and then some. Using a GPU for a game like this is like buying a six bedroom home for one person. Nice to have, but not necessary in the least.

Because it can be coded in pure C, software rendering solutions are far more portable and long-lasting. GPUs and APIs can grow to become obsolete or unwieldy over time, but CPUs?- we'll have them for the next 100 years. Even if a Mad Max-scenario occurs, we'll still have a lot more of them than we do Radeons and GeForces.

[Chris]

What I'm saying is that GPU utilisation isn't necessary for games like Morrowind, which didn't even look good when it released in 2002.

OpenMW can run Morrowind, but isn't Morrowind itself. OpenMW's rendering is more complex than Morrowind, and is likely to become more complicated as time goes on. Parallel split shadow mapping, parallax mapping, texture blending, per-pixel lighting with many lights, etc. At some point we'd also like to support PBR rendering, reflections, and more. Morrowind may not be able to use all this itself, but mods and new games made with the engine could.

I will say, software rendering can be interesting to pull off, but there's much more to it than triangle count. Texturing and lighting can become very costly when you're not doing simple vertex shading with nearest-texel single-layer texture mapping. You can maybe get something usable with highly optimized threaded pipelines, but it's not as simple as making a software polygonal renderer like the early/mid 90s. Parallelism is where you get a lot of performance benefits, something most mid-90s code rarely thought about, and which GPUs are highly optimized for. CPUs are more general-purpose, good at many things, but not great at anything specific, so using CPUs to do the work of GPUs won't be as efficient.

FWIW, Mesa has a software renderer for OpenGL, so you might be able to use that to get software rendering for OpenMW. Just don't expect it to be very fast since, as mentioned, CPUs aren't as efficient for graphics work as GPUs.

[KnightoftheWind]

What I'm saying is that GPU utilisation isn't necessary for games like Morrowind, which didn't even look good when it released in 2002.

OpenMW can run Morrowind, but isn't Morrowind itself. OpenMW's rendering is more complex than Morrowind, and is likely to become more complicated as time goes on. Parallel split shadow mapping, parallax mapping, texture blending, per-pixel lighting with many lights, etc. At some point we'd also like to support PBR rendering, reflections, and more. Morrowind may not be able to use all this itself, but mods and new games made with the engine could.

I will say, software rendering can be interesting to pull off, but there's much more to it than triangle count. Texturing and lighting can become very costly when you're not doing simple vertex shading with nearest-texel single-layer texture mapping. You can maybe get something usable with highly optimized threaded pipelines, but it's not as simple as making a software polygonal renderer like the early/mid 90s. Parallelism is where you get a lot of performance benefits, something most mid-90s code rarely thought about, and which GPUs are highly optimized for. CPUs are more general-purpose, good at many things, but not great at anything specific, so using CPUs to do the work of GPUs won't be as efficient.

FWIW, Mesa has a software renderer for OpenGL, so you might be able to use that to get software rendering for OpenMW. Just don't expect it to be very fast since, as mentioned, CPUs aren't as efficient for graphics work as GPUs.

I would be interested in seeing a fork/alternate version of OpenMW that's more of a "Chocolate Doom"-equivalent. Something that plays Morrowind as lean as possible, and with the addition of a software renderer. That way if the project gets a little too out of hand in the future, there will be a no-frills version that can be built upon and ported to a wide variety of platforms without regard for APIs and drivers that may or may not exist at that time. There are many software renderers out there, some better than others. There's "Retroquad" for the Quake engine, one that's designed to produce N64-esque visuals, and more. I don't expect high calibre 4K graphics mods through this method, but that isn't the point. We underestimate the capabilities of modern-day CPUs, but so too do we put too high a stock in modern GPUs. The graphics arms race has resulted in massive amounts of software bloat that could result in much of our entertainment history being lost in the future.

[psi29a]

You mean like this?

https://github.com/vsgopenmw-dev/vsgopenmw

[Chris]

I would be interested in seeing a fork/alternate version of OpenMW that's more of a "Chocolate Doom"-equivalent. Something that plays Morrowind as lean as possible, and with the addition of a software renderer.

That's unlikely. We don't have the original Morrowind sources to make a "lean" version that runs just like the original with minimal differences. As it is, even most Morrowind players aren't interested in sticking to pure vanilla Morrowind, given the prevalence of MCP and MGE. OpenMW is built new from the ground up, and as a result will inherently be different as it follows more modern coding practices and designs (and keeping room for future expansion for other games). The amount of work to make a software renderer capable of handling what OpenMW needs is a lot, and there aren't many people willing and able to invest such work. To say nothing of future maintenance costs to keep it working as an alternate option.

That way if the project gets a little too out of hand in the future, there will be a no-frills version that can be built upon and ported to a wide variety of platforms without regard for APIs and drivers that may or may not exist at that time.

That's the benefit of open standard APIs like OpenGL. Many platforms support it, and even for ones that don't, we know how it's supposed to work so it would be possible for enterprising individuals to make it work (see the MoltenVK project for implementing Vulkan on Metal for OSX, which doesn't support Vulkan natively, and the Zink project for implementing OpenGL on Vulkan, allowing OpenGL to work on Vulkan-capable systems that don't have native GL support, etc). This helps not just OpenMW, but all apps using the given API.

As for a no-frills version, that's what 1.0 is supposed to be. An engine written with modern methods for modern systems, to run Morrowind without (many) bells and whistles. Things like shadows and water reflections would look better because they way vanilla did them was frankly terrible and modern methods are more well known and better, but the primary focus was to get Morrowind working fully and properly without much fluff otherwise. Then after 1.0, more flavorful features would be added.

There are many software renderers out there, some better than others.

Almost all of them purpose-built for certain games or types of games. We won't be able to transplant a Quake software renderer into OpenMW because the two engines have vastly different needs. And if the renderer isn't designed to fit a known API, like OpenGL, that will also reduce the amount of available support libraries (e.g. how OpenSceneGraph presents an API to describe the scene to render at a slightly higher level, and does the rendering using OpenGL optimally for us; if we use a custom renderer, we can't use OSG, thus need to do more to replicate its functionality for a different rendering API).

We underestimate the capabilities of modern-day CPUs, but so too do we put too high a stock in modern GPUs.

Maybe so, but writing a well-featured renderer written for modern CPUs isn't an easy task, it requires specialized skills. Not something we really have, nor is there an apparent need for one when OpenGL and Vulkan work well enough.

[AnyOldName3]

You actually get people writing software renderers for GPUs these days now compute shaders and other forms of general-purpose GPU compute are available. That's far less monumentally dumb than expecting a CPU-based one to give acceptable speed. If you go off the raw number of operations a second GPUs can do versus the graphics accelerators of the 90s (and this is ignoring things like them being integer operations in the 90s), you're looking at a gain from 12,000,000 to 84,600,000,000,000, a factor of over seven million. If we pretend your figure of a 100 times performance increase of CPUs since the original Pentium (which is a massive underestimate) is accurate, it's easy to see that a hundred is a lot less than seven million. It's also easy to see that if you use a more accurate figure of about two thousand, it's still way less than seven million.