Music Player

Everything about development and the OpenMW source code.
Locked
User avatar
lgromanowski
Site Admin
Posts: 1150
Joined: 05 Aug 2011, 22:21
Location: Wroclaw, Poland
Github profile: https://github.com/lgromanowski
Contact:

Music Player

Post by lgromanowski » 13 Aug 2011, 20:45

jhooks1 wrote: The next thing on the roadmap is a music player. What exactly does this mean? Is it switching from regular morrowind music to battle music/other morrowind music? Or does it mean a player for your own mp3s?
Zini wrote: Basically the music player should start playing a random title from the Music/Explore folder. And when this title has finished, it should pick up another one.

But there are a few things to consider:

1. We have a script command to play music. This should stop whatever is playing currently and instead play the track specified in the script command.
When the title has finished playing, the music player should return to its regular mode.

2. There are a few special situations, that require interrupting whatever is currently played and play a different title instead (e.g. battle begins, level up). I suggest we leave these out for now, since the corresponding events haven't been implemented yet.

3. I think regions have special sound tracks too. But I am not 100% sure how they work. You probably will have to do a bit of research first.

Playing music is already implemented (apps/openmw/mwsound/soundmanager.hpp: SoundManager::streamMusic). So the only thing left to implement would be checking when a track has finished playing and also handling special situations, that require playing a different track.
Guest wrote: I played through Morrowind a little bit today going to different locations to see if there is region based music. In this quick search I couldn't find region based music, but I imagine there may be region based music for certain mods.
jhooks1 wrote: Above post was me.
jhooks1 wrote: I familiarized myself with the "streammusic" command. I have noticed that right now openmw allows multiple files to play at the same time. The first thing to do will be to make the old mp3 stop when a newer one is loaded.
Zini wrote:
Guest wrote:I played through Morrowind a little bit today going to different locations to see if there is region based music. In this quick search I couldn't find region based music, but I imagine there may be region based music for certain mods.
Don't remember any either. But there is a entry for sound in the region data structure. This may not be full music tracks, but additional sound effects, that are randomly played in parallel to the music. This should be handled by the music player too.
Ace (SWE) wrote: I would also imagine that the music player would smooth between the tracks instead of just switching them, a smooth crossfade for a tenth of a second would be enough to make it virtually unnoticeable. Not sure if the Morrowind one does this but otherwise switches would sound horrible.

Might not need to be a requirement but perhaps something to think about for later?
sir_herrbatka wrote: every explore track in mw starts with silence and ends with silence

OTOH there is fade out effect when battle music begins and ends - so we need to add this along with battle music.
jhooks1 wrote: I got the player to stop the current song when using streammusic, so now only one song can play at a time.
jhooks1 wrote: A random title mx_explore_1 through mx_explore_7 now plays on startup.
Mordicus wrote: Hello,

No, there is no region based music in Morrowind at all. Morrowind use 2 kinds of music randomly: music for exploration and music for battle. There is a "explore" and a "battle" directory in the DataFiles\Music, and MW will randomly play any of those mp3 being in these folders, whatever is the filename.
For ambient sounds it's a bit different: there are several regional ambient sound registered inside the Morrowind.ini file.

Mods adding musics usually start a script to play a specific mp3 added by the mod, and rarely use a random command to play any mp3. Musics played by script can be put anywhere, even in a new created path (ie: Music\Special\Mymusic.mp3). But, for example, Morrowind cannot randomly play a list of music being in a new folder, since Streammusic need a specific file name. With streammusic you can start any MP3 file, but as soon as it's finished, Morrowind will come back to default directories and play explore/battle music, excepted if you used a particular script to play another name.mp3 from a specific folder.

The original streammusic function was buggy and it was fixed by Morrowind Code Patch. Something a little bit annoying with the battle/explore music system, is that you hear battle music as soon as you are attacked by any hostile creature even if you did not saw it yet. So there is almost no surprising attack. Most sound command are broken; playsound3D do not play any 3D sound, etc. and the pitch value (volume and pitch can be set separately for each sound) does not work.

Hope it may help.
jhooks1 wrote: Player is working good now, when one title ends, the next is picked randomly and played. The titles have fadeouts built-in (previously mentioned in this thread), so the transitions are smooth. I will try to commit my code to git soon.
jhooks1 wrote: code is up on git
Zini wrote: I have a few comments:

- You are assuming a fixed set of explore music files. I am not totally sure, what MW is doing here, but IMHO it would be better to list all music files in the Explore directory (once) and then choose among them. We are using boost::filesystem already anyway, so this would not require any additional support libraries.

- The line

Code: Select all

MWWorld::Environment test = mEnvironment;
is completely pointless and slightly expensive. You can use mEnvironment directly.

- You shouldn't write text to cout when playing a track. I think we had something like this before, but the whole music playing construct at this point was only for debugging purpose.

- I think music should not stop, when someone is saying something (SoundManager::say)

- Your changes to startRandomTitle look a bit odd. I guess some people are more strict about this than others, but why use C-methods to generate a file name? Also, the rand usage is suboptimal. Depending on the quality of the random number generator using modulo can result in a very bad random number distribution.
sir_herrbatka wrote: Well, zini - I'm sure that MW just plays tracks from explore folder the way you wrote. You can drop any *.mp3 file there and it will be played in game no matter what file name is.
jhooks1 wrote: Zini, please advise me on the proper way of generating a random int. I tried another way, which I commented out, that I had crashing problems with. Also, I left the music cout in as I find it very useful in debugging.

The new code is up on git, the player searches the explore directory for all mp3 files. The MP3 lookup is done on startup, but could easily be changed to be done every time a random title needs to be played.

I am not sure why the say function would stop the music, and I haven't tested it yet.


EDIT: The code is a little messy. Also, there might be a better way to do this than storing all the mp3 file references in a list.
Zini wrote:

Code: Select all

I am not sure why the say function would stop the music, and I haven't tested it yet.
I didn't test it either. I reasoned this behaviour based on your commit, but now I can't see anything of it in the code either. Maybe I have just misread it.

I can't test your changes easily, because my current Morrowind setup is based on Redemption, which does massive modifications to the data directory. Don't have the time to work around that right now, so I will assume that your changes do work correctly. The code does look right as far as I can see.
Zini, please advise me on the proper way of generating a random int. I tried another way, which I commented out, that I had crashing problems with
Well, a slightly better method is to normalize the random number first (reduce it to the interval [0, 1) by dividing it through RAND_MAX) and then multiply it with the integer that specifies the ranges.

Maybe we should consider to use a more advanced random number generator. The standard C RNG isn't that great after all. Boost has something nice to offer in this department. But we can worry about that later. For now the modulo can stay (my last comment was meant more as a side note; sorry for not making that clear).

Edit: Spelling and a minor correction
Zini wrote: Regarding your edit:

Well, the code indention isn't so great, but that is true for a lot of OpenMW source files (a bit of cleanup would be welcome though).

I would use a vector instead of a list. Also there is no point in having a nFiles variable. Just use the size member function of the container. But the basic concept of your code looks right.
jhooks1 wrote: New version with code cleanup and changes (vectors, nFiles) will be up soon.

Region sound ids, chance, and priority can be viewed in the TES construction set by going to world->regions...
I am trying to figure out how to read these variables in openmw. Right now we can get the current cells region, but as far as I can see we can't access the region sounds. I am guessing new code will need to be written for reading these values from the esm file?
Zini wrote: Can't follow you. SoundRef has a member variable named sound, that contains the ID of the sound. What else do you need?
Chris wrote: FWIW, I don't see any reason why the regional sounds would be tied to the music player. If it's anything like Oblivion, anyway, they're normal short sound effects that are played at random times while the player is in the region. As such, they should be handled by the sound system as any other sound effect, so they can be cached and not waste IO resources to stream them from disk and decode each time.
Zini wrote: I think the distinction between music and sound effects is a rather arbitrary one. Any caching behaviour should ideally handled by the resources system based on usage patterns, size and available resources. Ogre offers something in this area and IIRC Nicolay way planning to handle all resources through the Ogre resources system. I don't know if this is implemented for sound and music though.

Anyway, no one said anything about streaming region sounds from disc. "Music Player" is really just an umbrella term for some sound related task. There is no "Music Player" class. Everything sound and music related is handled by the SoundManager class.

It is arguable if the region sounds fit into the "Music Player" category, but I suggest we include it anyway, because this is the last piece of sound-implementation, that is not bound to objects. It would be a good thing to get it out of the way.
athile wrote: Morrowind's options let the user control "master" volume, "voice" volume, "effects" volume, "footsteps" volume, and "music" volume independently. The sound system will need at least that minor bit of type information about each sound it is playing.
Zini wrote: Does our mission statement include replicating the configuration options of vanilla MW? IMHO it doesn't, but that might be arguable.
Anyway, most of the original options sounds reasonable. At least we should have master, voice, effects and music (footsteps I would see as a part of effects). These four options can easily supported with our current SoundManager class, because we already have sounds separated into voice, effects and music.

An addition to my previous post: Apparently we have not covered all non-object related sound effects. Even with region sounds, we are still missing weather sounds. But we can't do these at this point, because we don't have weather yet. I still think doing region sounds as part of 0.09 would be a good idea.
jhooks1 wrote: When I run the following code:

ESM::Region test;

test.load(mEnvironment.mWorld->getEsmReader());


I get ERROR: ESM Error: Expected subrecord FNAM but got PGRD
File: Morrowind.esm
Record: CELL
Subrecord: PGRD
Offset: 0x385bd92
Zini wrote: Sorry, I still don't understand. All the loading is already handled by OpenMW. You are not supposed to load records manually.
jhooks1 wrote: Oh, for some reason I thought I would need to reload Region data. Nevermind.
Vance987 wrote: Okay... I'm guessing that everybody forgot about what should be implemented into the music player which makes me sorta mad. Please go to this topic.

<FIXME - OLD LINK> viewtopic.php?f=3&t=11&start=10

I think it should be diverse as ever... and not just be a cheaply made randomizer. I'd like to play my death metal while fighting dagoth ur... or one of his minions, and play melodies while strolling in the grazelands for example. And there ARE regional music files, just look up on youtube. :roll: There's hundreds.

And I'm not saying that it should bog down the game either! There can be simple *instances* that are compressed enough to be streamlined through the music player for special occasions. I'm not saying take every little instance out there and stick a music file on it, cuz that's bogging down. And we never talked about what happens if the user deletes that certain music file that should be played. If that happens... by all means, randomize it.

And please, let it be user custimizable. :twisted:

Sorry if I'm sounding rude, I'm just in a rush cuz I'm on one of my school's computers. :mrgreen:
Zini wrote: That is post 1.0 stuff. We are not doing that yet. We will investigate how to improve music handling sometimes after the 1.0 release (among others by harvesting the Feature Request section in the forum and the Wishlist in the Wiki for ideas).
Vance987 wrote: Oh ok :roll: I just saw posts saying that it'll bog down the game and I'm like nu'uh! But ok that's fine. As long as its awesome after 1.0 v, I'm okay with the basics until then. :3
Chris wrote:
Zini wrote:Any caching behaviour should ideally handled by the resources system based on usage patterns, size and available resources.
Not sure I quite agree with that. Simply keeping the bare resource cached won't be very useful.. the OS will cache files into memory for you anyway, and will be much better about clearing stale cache entries out. Instead, a cached resource needs to be kept in the end-point, where the applicable subsystem can use it efficiently (a sound in an OpenAL buffer and an image in an Ogre texture, for instance). That's the whole point of caching resources, after all, to use the resources quickly and efficiently.

It would be quite inefficient if the OS had the file cached, you had the bare resource cached, the decoded resource cached, and the end-point resource cached, when it's the end-point resource that will be the most important part. Eg, if the OpenAL buffer isn't there, that would mean it's not cached and you've got work to do regardless of how your caching scheme works.

As for determining whether or not to cache a resource, the size isn't a very good determining factor given the different types of compression (ogg, mp3, flac, pcm, etc) and quality (44khz, 22khz, 11khz, 8-bit, 16-bit, etc). By the time the resource is decompressed, if it's decompressed, the actual sizes can be quite different. As well, a resource being small in size/length doesn't mean it will be used again any time soon, and vice versa.

IMO, for sounds, it may not be a bad idea to separate them into different categories: voice (not cached), misc. effects (cached), footsteps (cached, low playback priority), and music (not cached, high playback priority). That would clearly define the usage patterns for any kind of sound used, and provide a means to handle the different volume controls.
Zini wrote: Does not convince me. The right thing to do here is to go for a simple resources management strategy and only improve it, if we have profiling data, that clearly indicates a bottleneck. Everything else is just premature optimisation.
Star-Demon wrote:
Zini wrote:Does not convince me.
Okay? It seems very reasonable that dividing up and classifying regarding what data we need and don't need to a reasonable amount of sophistication is a good idea, and laying it out ahead of time seems like a good investment...

What are the points of concern that cause you not to be convinced? What needs to be resolved for you to be convinced? We can't answer you, otherwise...
The right thing to do here is to go for a simple resources management strategy and only improve it, if we have profiling data, that clearly indicates a bottleneck. Everything else is just premature optimisation.
Improving stuff helps us prioritize concerning existing and future tasks, but we're not shoving the music system in everyone's face to code, right?

If it was interfering with something else to be done in the immediate (Nothing in this project has to be done time sensitively.), then I'd say lay the groundwork for the sophisticated design and fill it in later to satisfy the way it should operate fully-logically. Functionally correct - Logically propped up.
Chris wrote:
Zini wrote:Does not convince me. The right thing to do here is to go for a simple resources management strategy and only improve it, if we have profiling data, that clearly indicates a bottleneck. Everything else is just premature optimisation.
True, 'premature optimization is the root of all evil', but the corollary is 'do it right the first time'. Again, the whole point of caching resources is to be able to use them quickly and efficiently. Caching the bare resource will not be efficient, especially when the OS will be caching the bare file for us anyway. The right thing to do is to cache it where it can be used efficiently, which is in the end point. Caching it anywhere else is just going to be wasteful.

Besides which, I don't think attempting to guess caching needs by analyzing resource size and usage frequency is 'simple resource management', compared to specifying caching needs manually based on what the resource is needed for. Especially considering the former will need some time (the resource will need to be used multiple times) before any such logic can determine "okay, this resource should probably be cached." It will be both more complex and slower to cache, resulting in higher resource IO and more potential for bugs.

EDIT:
If it was interfering with something else to be done in the immediate (Nothing in this project has to be done time sensitively.), then I'd say lay the groundwork for the sophisticated design and fill it in later to satisfy the way it should operate fully-logically.
Many projects die because they don't lay the ground work for a proper design, leaving the code a mess and making it nearly impossible to fix properly later on. Especially something as big as a full game, you're not going to want to go back and rewrite it all from the ground up. The code that's written now will be replaced piece-by-piece, so it's critical that those pieces are designed well from the outset so they may stick around for a while and support improvements in the surrounding code.
Zini wrote:
Okay? It seems very reasonable that dividing up and classifying regarding what data we need and don't need to a reasonable amount of sophistication is a good idea, and laying it out ahead of time seems like a good investment...

What are the points of concern that cause you not to be convinced? What needs to be resolved for you to be convinced? We can't answer you, otherwise...
This whole discussion is about reducing IO bottlenecks. To convince me that we need to do something about it would require, that we have an IO bottleneck. Which we probably don't have at this stage and maybe never will. Even if reloading sound effects everytime and handling them in another naive way is relative slow, it might not cause an overall slowdown. Or maybe it will. But until we see that it does, there is little point in putting additional manpower into fixing it.
Zini wrote:
Many projects die because they don't lay the ground work for a proper design, leaving the code a mess and making it nearly impossible to fix properly later on. Especially something as big as a full game, you're not going to want to go back and rewrite it all from the ground up. The code that's written now will be replaced piece-by-piece, so it's critical that those pieces are designed well from the outset so they may stick around for a while and support improvements in the surrounding code.
Probably true, but what does this have to do with OpenMW? Do you mean to say our code is a mess? The dependencies in apps/openmw got a little bit out of hand, but we will sort that out eventually and it isn't that bad in the first place.
We also have a design flaw in the ESM implementation (which originated from the very beginning of the project). This one is too hard to fix, because it would indeed mean a huge rewrite. But we have workarounds in place, that make sure this flaw has very little impact on the code base as a whole.
Apart from these two problems I don't see anything messy about our code.
Chris wrote:
Zini wrote:Probably true, but what does this have to do with OpenMW? Do you mean to say our code is a mess?
No, I mean you don't want to do a knowingly sub-par implementation that doesn't take into account a proper implementation, otherwise it will become a mess. Eg, if you don't account for caching audio resources into OpenAL buffers(*),whether or not it actually does, then other code written in support of the audio subsystem/resources will not worry about that possibility. Thus, because the supporting code doesn't take that possibility into account, it becomes that much harder to update the code to actually cache audio resources in OpenAL. The same goes for any other resource type.

(*) Properly using OpenAL actually necessitates caching sounds into buffers and reusing them. You should not assume copying data into an OpenAL buffer will be a cheap operation.. the data may need to be sent to hardware or a remote process, and/or require further conversion. Source and buffer objects are separate for a reason, and you'll be causing more work for yourself by trying to tie them tightly together.
Even if reloading sound effects everytime and handling them in another naive way is relative slow, it might not cause an overall slowdown.
No one raindrop believes itself responsible for the flood. A little naivite here, a little there... and it will add up. If you want to do a naive implementation, you still need a proper design to work on so improvements won't break anything.
Zini wrote: Well, I have no clue about OpenAL. Never worked with it. But that doesn't change the fact, that this whole discussion is pointless. Basically we have two options:

a) We do now some preparations for optimisations later in the development process (the groundwork, interfaces and such).
Later we will either see, that these optimisations are needed or that they are not needed. If they are needed, our preparations will either work out or they will not work out. Until we have proper data, we can't say for sure how to best separate sounds into categories with different optimisation strategies.

b) We don't do anything at all right now.
Later we will either see, that optimisations are not needed or that they are needed. If they are needed, we will have the data to create an efficient optimisation strategy. And that will cost us at most marginally more than doing it right now (as in option a), because OpenMW has a decently engineered structure, that is easily maintainable.

Really, that is a complete no-brainer.
jhooks1 wrote: I committed some new code to git. I have the region sounds and chances being printed out. Now it is time to work on actually playing these sounds.
jhooks1 wrote: Also I am trying to figure out how to handle these sounds. The way I was thinking of was to pick a random regional sound based on chance every 10 seconds or so.
Chris wrote:
Really, that is a complete no-brainer.
I can pretty much guarantee you that caching and sharing sound resources in the audio end-point will be warranted, and it's a complete no-brainer strategy besides. But even if it only results in 164.7 FPS instead of 164.5 FPS, it's still The Right Thing To Do(tm)(c)(r).

Plus, different OpenAL implementations may behave differently, so even if the loss is negligible to some doesn't mean it won't be more noticable to others (see the reason behind the various Quiet Feet mods; they remove the footstep sounds of various NPCs/critters because they caused slowdown for some people, which was caused by poor caching as a result of DirectSound's inability to properly cache sounds.. every time a sound played, it had to be reloaded into the device).
Star-Demon wrote: Chris, the situation feeling otherwise - I have to say I appreciate the explanations and agree.

If it really isn't a big deal, then I think we should be sure that, when it is time to make it work properly, that it's already ready and pretty much done before we do it. That requires more up front.
Zini wrote: For the region record my documentation says the following:
SNAM = Sound Record
byte SoundName[32] (lots of extra junk beyond string?)
byte Chance
Multiple records with the order determining the sound priority
Obviously you have to consider the Chance entry. Not so sure about the "Multiple records with the order determining the sound priority" part.
Picking a sound every 10 seconds sounds reasonable, but if we want to mimic the MW behaviour closely, some more tests with the original game and the CS might be in order.
Zini wrote: Finally had a look at your recent commit. Sorry, but there are multiple problems.

Code: Select all

ESM::ESMReader getEsmReader();
Ptr::CellStore getMCurrentCell();
ESM::Region getCurrentRegion();
bool getIsExterior();
void setIsExterior(bool set);
World should not have these functions.

- You can determine the exterior state by testing the current cell. There is a flag for it. Manually keeping track of it with an extra flag is error-prone and pointless.

- I don't think you should expose the ESMReader

- getMCurrentCell copies a data structure of substantial size. Also, this function is not needed at all. The current cell is the cell, that the player is in. This (or actually the matching cell store) can be retrieved from the player object: mWorld->getPlayerPos().getPlayer().getCell()

- likewise the current region can be determined from the current cell

The interface could use a slight cleanup and a bit higher abstraction (we have these on the roadmap already), but blowing up the already large World interface is not the answer.
jhooks1 wrote: The current region name can be determined from the cell, but the actual ESM::Region cannot AFAIK.

Those other functions were in there when I was trying different things brainstorming, I will remove them.
Zini wrote:
The current region name can be determined from the cell, but the actual ESM::Region cannot AFAIK.
Huh? If you have the name (actually the internal ID) of a record, you can get the record. That is what the ESMStore is for. There is even a find function.
jhooks1 wrote: Ah ok, thanks, I will get this all fixed ASAP.
jhooks1 wrote: Made suggested changes and committed to git. I really should have studied the framework more before than trying to rig up an immediate solution.
jhooks1 wrote: New code is up on git. A random sound from the region sound list is picked every 10 seconds. It is picked based on chance and played. I am trying to make them play louder, at the moment they can be quite hard to hear.
Zini wrote: Finally found the time to look at your code.

You have a syntax error in engine.hpp. The line where you declare the MP3Lookup function is wrong.
error: extra qualification â??OMW::Engine::â?? on member â??MP3Lookupâ??
After fixing it your code kinda worked. I see the log messages, but I don't hear anything. That might be the volume problem you mentioned. I didn't test it extensively.

Furthermore I have two requests:

- please use 4 spaces instead of the tabs

- could you move the functions MP3Lookup and startRandomTitle from Engine to MWSound::SoundManager please? Engine is meant only as a small front-end to the various sub-systems and these two functions are entirely sound specific (I guess you can call MP3Lookup in the SoundManager constructor).
jhooks1 wrote: New code is up with mp3 functions in the SoundManager. I changed the SoundManager constructor arguments and the MP3Lookup() now requires a path argument.


To hear the sounds I had to put my ear right up to my speakers. Turning down the music volume in the code also helps. Using a higher volume argument for playSound() doesn't seem to make any difference for regional sounds, nor does modifying playSound() internally.


I will try to remember not to use tabs from now on.


EDIT: At this time I am wondering if using a different sound library would solve the problem, I am using Audiere.
tombofsoldier wrote:
jhooks1 wrote:New code is up on git. A random sound from the region sound list is picked every 10 seconds. It is picked based on chance and played. I am trying to make them play louder, at the moment they can be quite hard to hear.
Randomly varying the time between sounds, and how loud the sounds play would be an improvement. Though of course I'm not sure what Morrowind actually does with these sounds to begin with.
Zini wrote: The new code looks good. Audiere shouldn't be the problem, because it works for everything else. It would be rather strange if a sound library fails only at something as specific as region sounds.

@tombofsoldier: Researching what MW does here would be the right course of action. btw. that is another of those tasks a non-coder could do. Any volunteers?
Rhys wrote: The morrowind ini has entries in weather section:
Minimum Time Between Environmental Sounds=1.0
Maximum Time Between Environmental Sounds=5.0
Could this be related?
Zini wrote: Looks like it. Thanks!

@jhooks1: I suggest you add two new command line options for this purpose (--env-sound-min, --env-sound-max ?), that default to the values given in the post above. We can make these options hidden later, so that they will be read only from our openmw.cfg file usually (the command line options and the cfg file are unified though boost::program_options in OpenMW)..
sir_herrbatka wrote: @rhys
that was funny :D Seems that you solved our problem! ;-)
best regards,
Lukasz

Locked

Who is online

Users browsing this forum: No registered users and 1 guest