Author Topic: Matrix mixing mono channels into stereo mixer plays only in one ear  (Read 1036 times)

rileythefox

  • Posts: 30
I'm dealing with OGG streams with N number of channels and attempting to apply a matrix to alter the panning of each channel using BASSmix. Each channel in the OGG represents a mono audio source for a collection of instrument stems in a song. The panning data for each channel is fetched from an external metadata file.

The current setup involves creating a stream from memory for the whole OGG with all channels, and then creating splitter streams for each channel, resulting in a splitter streams with 1 channel. I then plug each splitter stream into a mixer and attempt to modify the matrix for each channel. A problem occurs with some OGGs however where the panning does not seem to work as intended. A few users have made me aware that some OGGs play only in one ear, even though the matrix should be altering it so that some channels play in the left and some channels play in the right.

After gaining some affected files I noticed that all of them have <= 8 channels. Usually, these audio files can contain around ~15 channels. As a test, I added an extra silent channel to an affected OGG file to make the total channel count 9 and updated the panning metadata to account for the extra channel. Upon testing the new file, all the channels play in the correct left and right stereo channels.

One of the affected files (8 channels) have all channels silent except the last 2 (there is a reason why, but it isn't relevant here). All of the channels including the last 2 are split, added to the mixer and matrixed. Yet the last 2 channels of the original source only play in the right ear. The expected result is that the last 2 channels should play in the left and right respectively.

The mixer is created with a hardcoded number of 2 channels and upon creating each splitter stream I verified that each splitter stream contains 1 channel. The only stream in this process with more than 2 channels is the source OGG itself, with <= 8 channels.

Here are the matrixes for each channel in an affected OGG:
Code: [Select]
Channel 1
{ {1}, {0} }

Channel 2
{ {0}, {1} }

Channel 3
{ {1}, {0} }

Channel 4
{ {0}, {1} }

Channel 5
{ {1}, {0} }

Channel 6
{ {0}, {1} }

Channel 7
{ {1}, {0} }

Channel 8
{ {0}, {1} }

Here is some sample code of the current setup. Note this is written in C# using the ManagedBass wrapper library. The code is not 100% accurate to our setup, as we have made some classes to wrap around BASS handles to manage freeing streams automatically. However I have wrote the example so that it would have identical functionality.

Code: [Select]
byte[] oggArray = LoadOggFileIntoMemory();
int oggStreamHandle = Bass.CreateStream(oggArray, 0, oggArray.Length, BassFlags.Prescan | BassFlags.Decode | BassFlags.AsyncFile);

int mixer = BassMix.CreateMixerStream(44100, 2, BassFlags.Default);

int[] channelMap = new int[2];
channelMap[1] = -1;

// channelCount is fetched from a metadata file
for(int i =0; i < channelCount; i++)
{
    // For each iteration, we get 1 channel in the splitstream using the channel map
    channelMap[0] = i;

    // Single channel from multi-channel ogg
    int splitHandle = BassMix.CreateSplitStream(oggStreamHandle, BassFlags.Decode | BassFlags.SplitPosition, channelMap);

    // This might seem strange but we need dual streams with one for reverbing
    int main = BassMix.CreateSplitStream(splitHandle, BassFlags.Decode | BassFlags.SplitPosition, null);
    int reverbSplit = BassMix.CreateSplitStream(splitHandle, BassFlags.Decode | BassFlags.SplitPosition, null);

    // Sometimes need to alter tempo of stream, so we use BassFX streams
    streamHandle = BassFx.TempoCreate(main, flags);
    reverbStreamHandle = BassFx.TempoCreate(reverbSplit, flags);

    // Apply a compressor to balance stem volume
    Bass.ChannelSetFX(streamHandle, EffectType.Compressor, 1);
    Bass.ChannelSetFX(reverbStreamHandle, EffectType.Compressor, 1);

    // Removed setting FX params and other attributes like volume

    BassMix.MixerAddChannel(mixer, streamHandle, BassFlags.MixerChanMatrix);
    BassMix.MixerAddChannel(mixer, reverbStreamHandle, BassFlags.MixerChanMatrix);

    float[,] channelPanVol = (matrix data for this channel fetched from metadata, reference matrixes earlier in post)

    BassMix.ChannelSetMatrix(streamHandle, channelPanVol);
    BassMix.ChannelSetMatrix(reverbStreamHandle, channelPanVol);
}

Bass.ChannelPlay(mixer, false);

It may seem weird creating 2 splitstreams from a split stream. The reason we do is because we have essentially duplicated streams and apply reverb to one for some effects mixing. It does work and I have verified the channel counts for these splitstreams (should be 1) are correct.

When attempting to play this mixer, the 8 channels can only be heard in the right stereo channel, despite the matrix I posted above showing 4 mixer channels for the left stereo channel and 4 for the right. I'm not sure what the problem is here, as we have verified numerous times that our matrices are correct. The issue ONLY occurs with ogg files with <= 8 channels. Adding a 9th channel immediately fixes the problem. We did some further testing with a 6 channel ogg (which also has the issue) and shuffled the channels in the ogg around. Given the same scenario as before, 2 channels with sound and the rest are silent (and the exact same matrix but for 6 channels), shifting the order of the channels has different effects.

Placing the 2 channels with sound on in channels 1 and 2 causes them to play in the left ear. If the channels are in 3 and 4 they play centred. If they are in 5 and 6 they are in the right ear again.

My best guess is that BASS is assuming a 5.1 speaker setup when loading a 6 channel ogg, and a 7.1 speaker setup when loading an 8 channel ogg. This would be fine if it weren't for the mixer being forced to 2 channels and the splitter streams having 1 channel each. I would assume that matrixing each splitstream into standard stereo panning would make them follow that and not assume a setup of 5.1/7.1. Is there a problem with this setup or could this be an issue with how splitstreams assume speaker configurations?

These are the BASS versions we are working with:
BASS: 2.4.17.0
BASSmix: 2.4.12.0
BASS_FX: 2.4.12.6

Ian @ un4seen

  • Administrator
  • Posts: 25882
My best guess is that BASS is assuming a 5.1 speaker setup when loading a 6 channel ogg, and a 7.1 speaker setup when loading an 8 channel ogg.

BASS does indeed reorder the channels to match its (and Windows') channel ordering for up to 8 channels, which is as follows:

3 channels = front left, front right, center.
4 channels = front left, front right, rear left, rear right.
5 channels = front left, front right, center, rear left, rear right.
6 channels = front left, front right, center, LFE, rear left, rear right.
8 channels = front left, front right, center, LFE, rear left, rear right, side left, side right.

While Vorbis's channel order is as follows:

3 channels = front left, center, front right
4 channels = front left, front right, rear left, rear right
5 channels = front left, center, front right, rear left, rear right
6 channels = front left, center, front right, rear left, rear right, LFE
8 channels = front left, center, front right, rear left, rear right, side left, side right, LFE

The center and LFE channels are in different positions. There isn't currently any way to disable the reordering, but perhaps you can undo it in your matrix?

If the channel splitters are being plugged into the same mixer then it seems to me like they aren't really needed, and you could just plug the original multi-channel stream into the mixer and use one big matrix to enable (and reorder) the channels? For example, your 8 matrices above could be replaced with one like this:

Code: [Select]
{
{1, 0, 1, 0, 1, 0, 1, 0}, // L1, L2, L3, L4, L5, L6, L7, L8
{0, 1, 0, 1, 0, 1, 0, 1} // R1, R2, R3, R4, R5, R6, R7, R8
}

After reordering, it could look like this:

Code: [Select]
{
{1, 1, 0, 0, 1, 0, 1, 0}, // L1, L3, L2, L8, L4, L5, L6, L7
{0, 0, 1, 1, 0, 1, 0, 1} // R1, R3, R2, R8, R4, R5, R6, R7
}

rileythefox

  • Posts: 30
Hi, thank you for your response!

If the channel splitters are being plugged into the same mixer then it seems to me like they aren't really needed, and you could just plug the original multi-channel stream into the mixer and use one big matrix to enable (and reorder) the channels?

We can't do this as we need effect control using BASS_FX for each individual channel. Each channel (or in some cases numerous channels) as part of the original multi-channel stream are one instrument. Some of these instruments need effects like reverb while others don't. We treat each channel as its own effective stream by using splitstreams to be able to apply things to it while leaving others untouched.

One example is we have an unprocessed splitstream and a reverb splitstream. At most times the reverb splitstream is muted, but in certain conditions we will set the volume so it can be heard (the reverb stream has only wet signal) at the same time as the unprocessed stream. This approach gives a far better listening experience than applying the affect to the original stream.

We would ideally need some way for BASS to ignore channel re-ordering/assuming speaker configurations and let us handle channel mappings and matrices.

Ian @ un4seen

  • Administrator
  • Posts: 25882
We would ideally need some way for BASS to ignore channel re-ordering/assuming speaker configurations and let us handle channel mappings and matrices.

I'll look into it and then hopefully come back with an update for you to try, probably next week. There is the BASS_DEVICE_NOSPEAKER option to disable reording for the output device, which could perhaps be extended to disable reordering by the decoder too, or perhaps it'd be better to add a new BASS_ChannelSetAttribute option for it.

rileythefox

  • Posts: 30
I'll look into it and then hopefully come back with an update for you to try, probably next week. There is the BASS_DEVICE_NOSPEAKER option to disable reording for the output device, which could perhaps be extended to disable reordering by the decoder too, or perhaps it'd be better to add a new BASS_ChannelSetAttribute option for it.

Could it not be considered a bug that BASS is reordering channels when the mixer is stereo and the splitter streams being plugged into it are 1 channel each? The original multi-channel stream is never used besides being the source for the splitters, so I was confused as to why BASS would treat the splitters this way if they are mono streams.

I look forward to trying the update, been trying to figure out this issue for over a week now! ;D

Ian @ un4seen

  • Administrator
  • Posts: 25882
The reordering is done by the Vorbis decoder, so the data has already been reordered when BASSmix sees it. The channel order is defined in the Vorbis spec, so I don't think it can be ignored by default; it would need to be via some option.

Have you considered using Opus encoding instead? That does include an unidentified channel mapping option (ie. not based on speakers), so your case could be supported as standard then.

rileythefox

  • Posts: 30
Have you considered using Opus encoding instead? That does include an unidentified channel mapping option (ie. not based on speakers), so your case could be supported as standard then.

We need to support a variety of formats including ogg, opus, mp3, wav etc. The typical format of these files with many channels is vorbis and we cannot change this as the user of the software opens the files in the software.

Ian @ un4seen

  • Administrator
  • Posts: 25882
Is the user creating the Ogg Vorbis files themselves, eg. from WAVs? If so, the channel reordering should actually result in the same order as their original file, ie. the Vorbis encoder reorders the original data, and then BASS reverses that. But I notice that BASS's reordering of 7/8 channel Vorbis files (the order I posted above) is currently incorrect because it hadn't been updated since 7/8 channel ordering was added to the Vorbis spec. Perhaps that's what's causing your issue? Here's an update with corrected reordering for you to try:

   www.un4seen.com/stuff/bass.zip

Let me know whether it helps.

rileythefox

  • Posts: 30
Is the user creating the Ogg Vorbis files themselves, eg. from WAVs?

The Ogg Vorbis files which have these high channel counts and are affected are ripped from old Xbox 360 games released from 2008-2010. We have no way of "fixing up" or reordering the channels in the file, we just need to be able to support what is inputted into the software.

I tried the BASS update you provided and it didn't change anything. I also tried initialising BASS with the BASS_DEVICE_NOSPEAKER flag to see if that would help, but no effect there either.

If you could try that new attribute you mentioned to disable reordering, that could be worth a shot.

Ian @ un4seen

  • Administrator
  • Posts: 25882
OK, here's something for you to try:

   www.un4seen.com/stuff/bass.zip

There was a choice between using a flag or an attribute for the new option, and I've gone with a new flag because that allows the option to be set during stream creation instead of only after. This is the new flag:

Code: [Select]
#define BASS_STREAM_NOREORDER 0x30000000 // don't reorder channels to match speakers

Note that's a combination of the existing BASS_SPEAKER_LEFT (0x10000000) and BASS_SPEAKER_RIGHT (0x20000000) flags, as flag bits are in short supply. I don't think that will be a problem because SPEAKER flags only apply to stereo streams, so there shouldn't be any reason to use both at the same time, but if it does turn out to be a problem then it can be changed to an attribute before the next BASS release. Let me know how you get on with it. It currently only applies to Vorbis streams, but could also be applied to other formats that have a different channel order to BASS, eg. Opus and ALAC.

rileythefox

  • Posts: 30
Tried the update and after applying the new flag it worked immediately! Will do further testing to make sure all affected files that we know about now work properly.

When do you think this fix would get released as part of the main BASS download? We're a free open source game and all contributors need to download BASS in order to compile, so we setup scripts which contributors run which downloads all the libraries required from this website. So we need everyone to be downloading this "fixed" version. Thanks!

Ian @ un4seen

  • Administrator
  • Posts: 25882
Good to hear that the update is working well for you so far. I think it needs a bit of time to see if any problems arise (eg. with the flag conflict I mentioned) before it's confirmed in a release, but it is a "release" build so it should be fine to use in your game if you wish in the meantime.

rileythefox

  • Posts: 30
Good to hear that the update is working well for you so far. I think it needs a bit of time to see if any problems arise (eg. with the flag conflict I mentioned) before it's confirmed in a release, but it is a "release" build so it should be fine to use in your game if you wish in the meantime.

Are you able to provide downloads for Mac and Linux too? Our game is cross platform so we need the update for those. Thanks!

Ian @ un4seen

  • Administrator
  • Posts: 25882
I'm now looking into possibly reclaiming one of the old BASS_SAMPLE_FX/SOFTWARE/VAM flag bits (unused since BASS 2.4.16) for this, instead of using the BASS_SPEAKER_LEFT+RIGHT combination, as a single bit is preferable. Once that's decided, I'll post updates for those other platforms too.

Ian @ un4seen

  • Administrator
  • Posts: 25882
The new option has now been expanded to also apply to samples (BASS_SampleLoad), so it's been renamed to BASS_SAMPLE_NOREORDER, and its value has changed (recycling the old BASS_SAMPLE_VAM flag):

Code: [Select]
#define BASS_SAMPLE_NOREORDER 64 // don't reorder channels to match speakers

Here are the updates:

   www.un4seen.com/stuff/bass.zip
   www.un4seen.com/stuff/bass-linux.zip
   www.un4seen.com/stuff/bass-osx.zip

Let me know if you have any trouble with them. Note it isn't possible to toggle the BASS_SAMPLE_NOREORDER flag on samples (via BASS_ChannelFlags) but it is on streams.

rileythefox

  • Posts: 30
The update seems to be working well. No problems reported so far and the flag hasn't affected anything else.