Sound Improvements

[Chris]

I'm interested in redesigning/improving the sound system. Now that OpenMW is a bit more capable (moving about in the world with sound emitters, sound activators, etc), there are some things that could be done to make the engine's use of OpenAL be more efficient and less error-prone.

I was talking with werdanith on IRC, but one of the stumbling blocks is working through all the various layers. There's the mwsound subsystem, openengine, mangle, and the output and input backends. According to their (outdated) pages on the wiki, openengine and mangle are considered abandoned and not likely to get more improvements.

So considering the improvements and interface redesigning I want to do, what would be the best way to go about it? If openengine and mangle are abandoned, should I move/merge the relevant sound code into mwsound? Should I keep the code where it is and work on stuff there?

[werdanith]

I'd like to add here that the sound backend may be considered good enough for now, but it will need some restructuring eventually if we want proper attenuation with proper distance cutoff, as well as the ability to alter the volume of specific sound types, the ability to have fade in and fade out for effects like rain and a superior loading of the stream buffer and of resources. Also, I don't know whether the situation is optimal for supporting moving sounds and the sound of water.

Up until know I've been refactoring the frontend of sound management, but the backend, all the Open Engine/Mangle/OpenAL stuff are still a mystery to me.

[Zini]

So considering the improvements and interface redesigning I want to do, what would be the best way to go about it? If openengine and mangle are abandoned, should I move/merge the relevant sound code into mwsound? Should I keep the code where it is and work on stuff there?

interesting question.

Sound is one of the few areas where mangle is actually useful. When the sound part of mangle was written, we couldn't find a sound library that works equally well for everyone, so we have three options now that can be chosen from by setting a cmake variable. I would suggest to keep it like that.

OpenEngine is a different question. I see no problem with integrating libs/openengine/sound into apps/openmw/mwsound.

Integrating the openengine soundmanager class into the mwsound soundmanager class is again a different question. In general these *manager classes in apps/openmw/mw* are meant to be an implementation of the facade design pattern, i.e. they glue together the various parts of the mw* subsystem and provide a single point of access for other mw* subsystems and the Engine class.
In its current state the mwsound soundmanager class is not a great facade, because it does all the work by itself instead of delegating; resulting in a very bloated class. Maybe another refactoring effort would be a good idea; during which the openengine code could be integrated into mwsound.

A reasonable well written example of the facade pattern can be found in mwrender.

But I don't know all that much about our current sound implementation, so I will leave these design decisions to you. Just don't make mwsound soundmanager any more bloated than it already is.

btw. if you want to work on OpenMW, we still need you to register on the bug tracker, so we can officially assign tasks to you. Since you probably know best, you should also enter these tasks into the issue tracker (and assign them to yourself). There is a checklist on the wiki for new contributors

[Chris]

Sound is one of the few areas where mangle is actually useful. When the sound part of mangle was written, we couldn't find a sound library that works equally well for everyone, so we have three options now that can be chosen from by setting a cmake variable. I would suggest to keep it like that.

As far as I can see, the only useful part of Mangle's audio portion is the decoder abstraction, but that's really nothing more than a base class with one of a few possible derivations for implementations (so far as I can tell).

My preliminary thoughts would be to move the openengine and backend-inspecific mangle sound code to mwsound (or scrap them entirely, as their ideas are all pretty much abandoned anyway, and (re)build from what's there in mwsound). The decoder and output abstractions are something to keep, so you can still have different options by just setting a cmake variable (or maybe even multiple options that can be enabled at once and change implementations at run-time), but I think the interfaces can be done much closer to the actual game engine and reflect what the engine is trying to accomplish, instead of acting as generic decoder/3D audio APIs.

btw. if you want to work on OpenMW, we still need you to register on the bug tracker, so we can officially assign tasks to you. Since you probably know best, you should also enter these tasks into the issue tracker (and assign them to yourself). There is a checklist on the wiki for new contributors

I'm on the bug tracker already, here. I'm not sure what to put for a task, though, other than a nebulous 'rewrite/refactor the sound code'.

[Zini]

My preliminary thoughts would be to move the openengine and backend-inspecific mangle sound code to mwsound (or scrap them entirely, as their ideas are all pretty much abandoned anyway, and (re)build from what's there in mwsound). The decoder and output abstractions are something to keep, so you can still have different options by just setting a cmake variable (or maybe even multiple options that can be enabled at once and change implementations at run-time), but I think the interfaces can be done much closer to the actual game engine and reflect what the engine is trying to accomplish, instead of acting as generic decoder/3D audio APIs.

That sounds reasonable.

I'm not sure what to put for a task, though, other than a nebulous 'rewrite/refactor the sound code'.

Good enough. At least we have it on record then.

I assume this is something for the 0.14.0 release?

Edit: Just in case you missed it, we recently merged the submodules into the main repository. Therefore you can work on mangle and OpenEngine directly in your OpenMW fork and don't need to bother with the submodule repositories anymore.

[Chris]

I assume this is something for the 0.14.0 release?

Probably, if you were hoping the have 0.13.0 out relatively soon. It'll take me a bit to get (re)acquainted with the code, then to find something that works, then make it work.

Edit: Just in case you missed it, we recently merged the submodules into the main repository. Therefore you can work on mangle and OpenEngine directly in your OpenMW fork and don't need to bother with the submodule repositories anymore.

Yeah, that's actually what drew me back to start hacking on OpenMW again. Modifying and maintaining those submodules was rather bothersome.

EDIT:
Created task.

[Chris]

I have a preliminary attempt ready for perusal.

https://github.com/ChrisKCat/openmw/tree/sound-rewrite

The Mangle and OpenEngine sound code is gone, and it's all built up from MWSound. OpenAL and libsndfile+mpg123 are reimplemented and sectioned off into their own source modules and accessed through virtual base classes, so alternatives can be added easily enough. There's still more work to be done on the interfaces and implementations, which I'll be getting to, but they do everything they're required to right now.

At this point I'm looking for feedback, to make sure it's moving in a good direction and to see if there's anything I should change. In particular it doesn't seem right that I should be using cout for "logging".

P.S. The FFmpeg decoder is stubbed but not yet reimplemented. It seems I can't build with support for it since the FindFFMPEG script is failing, due to it needing a non-existant ffmpeg-config program. I can try using cmake's pkg_config module to look for ffmpeg before falling back to the Find script, or I can try to get rid of the need of the ffmpeg-config program.

P.P.S. Support for Audiere (input) was removed. This task and the fact that it's not in Gentoo's repo makes it seem to be a lib that's not well supported anymore. If someone really cares it can be added back, but all that there really was left of it was the cmake option.

[Zini]

Will have a look once the release preparations are under way.

I agree about Audiere.

About the logging: We never managed to establish a proper logging mechanism and actually we aren't do much logging at all. At this point it is probably a bit late to hack it in.

[Ace (SWE)]

Unfortunately I disagree with removing Audiere support, both ffmpeg and mpg123+libsndfile need a Cygwin and MingW install to properly compile and use in Windows.

[Zini]

Didn't know that.