NLSOUND Music Implementation Notes

There are two possible output paths for music. One is through the NeL Sound library driver, the other is through the OS provided multimedia libraries. The first method allows to place music in the in-game 3d scene, the second allows to use many media formats, and allows OS provided systems to interfere with the music volume to support applications which require the music to be turned off during voice calls, or while other multimedia applications are running with higher priority output.

We can assume that music placed in the in-game 3d scene will only consist of content that is placed there by the developers. As such, there is control over the formats that need to be supported. It is preferred here to only support a single format, more specifically Ogg Vorbis. The decoding will be implemented on the level of the NeL Sound library, and will be played back through the low-level library driver using the stream sources. Stream sources are not and will not be implemented for the FMod library driver, and as such the FMod library driver is henceforth considered deprecated. This will be called the Stream Sound feature. Feature #795.

Background music, as well as music played by the user in the ingame music player, can be considered not to require any in-game 3d scene placement. It would also be preferable that this music does not interfere with any music the player has running in an external media player, and therefore it would be useful to make use of OS provided media facilities for playback of these media files. This will also ensure that upmixing of the music files is done for us, as well as allow any file format supported by the system to be used. In environments where this capability is not available, or for which we have not implemented this yet, this will be routed through the Stream Sound system. Music can be assigned different levels of importance, such as background and user-played, in order to ensure that only one music file is played back. This feature will be called the Music system.

The current music implementation will be largely removed in favour of this new system. The Stream Sound feature shall be implemented first, followed by the Music features which initially will run on only top of this system.

The Music handling system can be provided either by the loaded sound library driver, such as FMod, when this is available in the used sound library, or by NeL Sound when it can be implemented using OS specific functionality, such as Windows Media Foundation and GStreamer. The implementation of this is not a priority.

Implementation Stage One

The stream sound feature has been implemented with the following limitations. It has only been tested for stereo streams, played back at position 0, 0, 0 with relative mode enabled. Positioning behaviour of stereo streams in other situations is undefined. It is not known if the implementation correctly handles mono streams, and whether these are positioned correctly. The music system remains the legacy system, and has been extended to allow playback directly through the stream sound feature, so that a per-driver implementation of ogg streaming is not necessary.

Status: Applied to default branch of repository in changeset c424f6edad82.
“Merged: From sound_dev to default. (Summary: Implemented Stream Sound and Group Controller feature. Various performance improvements and bug fixes in the sound code. Fixed sanity of IThread start and isRunning, added setPriority)”

Implementation Stage Two

Define positioning behaviour for stereo streams and sounds. Ensure positioning works correctly for mono streams.

Status: Todo (High Priority)

Implementation Stage Three

Reinvent the music channel system. Support for OS, and possibly hardware, provided multimedia playback facilities for music channels.

Status: Todo