Author Topic: Can't create stream from multichannel M4A file - but Audacity can open it???  (Read 2175 times)

trojannemo

  • Posts: 143
Are you creating the files yourself, and if so, do you really need them to be MP4 Opus instead of regular Ogg Opus? Ogg Opus can handle 10 channels, and will already work with BASSOPUS.

These are files used in the video game Fortnite Festival. They are indeed streamed to the players but it is possible to download them in chunks and they create this output file when put together. I don't have a say in the format of the files, if I did I would have just chosen ogg which is my preferred format.

Ian @ un4seen

  • Administrator
  • Posts: 26035
btw, I send you the "10 mono channels" wav file as well (included in the zip folder)...

While

   BASS_ChannelGetInfo(hChannel, &info);

show me the right amount of channels ... 0x00a = 10 channels,

   BASS_ASIO_ChannelEnableBASS(FALSE, 0, hChannel, TRUE)

in my case fails.

   DWORD dwError = BASS_ASIO_ErrorGetCode();

returns 0x12. Can you tell me what the Error Code 0x12 means since not every error code
was handled by a #define BASS_ERROR_ ... in bass.h or bassasio.h

Any ideas about that? Is it possible that this error was caused in my case by the fact that my ASIO Device only has 8 Output channels?
And if so, is there a prefered way to map individual channels (in my case the channels 3-10 = 8 channels) to my 8 outputs of my ASIO device?

Error code 0x12 (18) is BASS_ERROR_NOCHAN, which means the device does not have enough channels to accommodate the BASS channel, ie. it doesn't have 10 channels in this case. You could use a mixer to get around that, ie. create an 8-channel mixer (with BASS_Mixer_StreamCreate) for BASS_ASIO_ChannelEnableBASS and plug the 10-channel file into that. You could also use matrix-mixing (BASS_MIXER_CHAN_MATRIX) if you would like the extra 2 channels to be heard on any of the other channels, or if you want to rearrange them in any other way.

PS: Is there a way to donate you for all your work on bass.dll?

There isn't a donate option, but thanks for the offer! If you insist, you could purchase a Shareware licence. Perhaps it'll even come in useful in future :)

mhuellwegen

  • Posts: 20
These are files used in the video game Fortnite Festival. They are indeed streamed to the players but it is possible to download them in chunks and they create this output file when put together. I don't have a say in the format of the files, if I did I would have just chosen ogg which is my preferred format.
@trojannemo:

Ahh, okay, now it makes sense to me as well that your audio is "opus" audio in "ogg" container since "opus" is ideal for real-time communication and streaming.

In my case internet live streaming is in fact NOT involved.

I have hundreds of ACC encoded MP4 files on a hard drive, all with a 2 channel stereo default mix and with optional 4 more separate subs as separate stereo channels of audio to play with (e.g. make the musik like an acapella version for some secons during playback, like it was done regular in a dj set for mixing).

mhuellwegen

  • Posts: 20
Error code 0x12 (18) is BASS_ERROR_NOCHAN, which means the device does not have enough channels to accommodate the BASS channel.
Okay, that's what I had already thought...

Ian, can you add "BASS_ERROR_NOCHAN" and perhaps other missing error codes to bass.h and bassasio.h as well, since it makes debugging much easier?

mhuellwegen

  • Posts: 20
Okay, my fault:

#define BASS_ERROR_NOCHAN   18   // can't get a free channel

is already defined.

mhuellwegen

  • Posts: 20
You could use a mixer to get around that (10 channels but 8 outputs only), ie. create an 8-channel mixer (with BASS_Mixer_StreamCreate) for BASS_ASIO_ChannelEnableBASS and plug the 10-channel file into that.
You could also use matrix-mixing (BASS_MIXER_CHAN_MATRIX)...
@Ian Thanx, BASS_MIXER_CHAN_MATRIX now working well for me with a 10 channel WAV file mapped to 8 ASIO outputs.

So in my special case the last remaining problem is the "fragmented" MP4 file that BASS can load successfully but will only grab and play channel 1+2 => the 1st stereo goup marked as "default".

Does this mean that it is a "demux of mp4/ogg" and not a "decoding of aac/opus" problem?

Regards,
Martin

trojannemo

  • Posts: 143
For my part i'm giving up on this. Bass isn't ready to do what I need it to do. If I have to do half the work with the cutting edge release of Bass and the other half with a third party library that I don't know how to use, I might as well use ffmpeg which I do know how to use and works lightning fast.

Code: [Select]
public static int ProcessFortniteFile(string file)
{
    var bin = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location) + "\\";
    var ffmpeg = bin + "ffmpeg.exe";
    if (!File.Exists(ffmpeg))
    {
        Console.WriteLine("ffmpeg.exe is missing and is required ... process aborted");
        return 0;
    }
    var opus = bin + "temp.opus";     
    if (File.Exists(opus))
    {
        File.Delete(opus);
    }
    var arg = "-i \"" + file + "\" -codec copy \"" + opus + "\"";
    var app = new ProcessStartInfo
    {
        CreateNoWindow = true,
        RedirectStandardOutput = true,
        UseShellExecute = false,
        FileName = ffmpeg,
        Arguments = arg,
        WorkingDirectory = bin
    };
    var process = Process.Start(app);
    do
    {
        //
    } while (!process.HasExited);
    process.Dispose();
    if (!File.Exists(opus))
    {
        return 0;
    }           
    var stream = BassOpus.BASS_OPUS_StreamCreateFile(opus, 0L, File.ReadAllBytes(opus).Length, BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT);
    File.Delete(opus);
    return stream;
}

This is the entire code and it integrates into my existing workflow perfectly.
I hope one day I can just narrow it down to one BASS_StreamCreateFile call but until then I'll rely on ffmpeg.

mhuellwegen

  • Posts: 20
Code: [Select]
public static int ProcessFortniteFile(string file)
@trojannemo
Can you and would you like to send me a fortnite file so i can have a look on it as well?

trojannemo

  • Posts: 143
There's a link to a zip file with some of these fortnite files inside in reply #7 on this thread on the first page.

mhuellwegen

  • Posts: 20
There's a link to a zip file with some of these fortnite files inside in reply #7 on this thread on the first page.
Got the zip file, will look into the five m4a Files tomorrow.

trojannemo

  • Posts: 143
Thanks.

If you can get it down to just Bass or Bass + integrated code where I manipulate the input data, that's worth it. If the only way to make it work is to integrate another program or library, it's probably not worth it for me unless it's a very lightweight dll and easy to integrate.

Ian @ un4seen

  • Administrator
  • Posts: 26035
So in my special case the last remaining problem is the "fragmented" MP4 file that BASS can load successfully but will only grab and play channel 1+2 => the 1st stereo goup marked as "default".

Does this mean that it is a "demux of mp4/ogg" and not a "decoding of aac/opus" problem?

Yes, your case is more of a demuxing issue, in that BASS (and BASS_AAC) currently always plays the first audio track in an MP4 file and not any other tracks that there may be. Your example file isn't actually an fMP4 (that was just trojannemo's files), but fMP4 is supported by Media Foundation, so the file would still be playable with BASS if it was fMP4.

The good news is that the various OS decoders (on Windows/macOS/iOS/Android) used by BASS for MP4/AAC support (without BASS_AAC) do appear to allow the track to be specified, at least in theory - I haven't confirmed it's working yet. It seems like your example MP4 file may have the same sound in all 5 tracks? If so, can you provide a file with different sound in each track? That'll be better for testing purposes.

For my part i'm giving up on this. Bass isn't ready to do what I need it to do. If I have to do half the work with the cutting edge release of Bass and the other half with a third party library that I don't know how to use, I might as well use ffmpeg which I do know how to use and works lightning fast.
...
I hope one day I can just narrow it down to one BASS_StreamCreateFile call but until then I'll rely on ffmpeg.

To be honest, if you don't want to bother with MP4 parsing and you already have another working solution, then I think it would be best to stick with that for now. There will hopefully be a BASSMP4 add-on (for more flexible MP4 parsing/demuxing) at some point, but I can't say when that will be.

trojannemo

  • Posts: 143
There will hopefully be a BASSMP4 add-on (for more flexible MP4 parsing/demuxing) at some point, but I can't say when that will be.

I will wait patiently  ;D

trojannemo

  • Posts: 143
I saw you issued the update for BASSOpus that supports raw opus. Still not an easy implementation with my fragmented files?

trojannemo

  • Posts: 143
I've made a lot of progress on this. Got MP4/Opus parsing going on, my test file I have over 9,000 samples and I know the offset and size of each sample, plus I have its data. At first I wrote it all to file and it's almost 1:1 in file size with the original m4a file. I tried feeding the whole thing to the new bassopus with StreamCreate and no luck. I'm now trying to feed it sample by sample and I'm getting a memory violation exception at the BASS_OPUS_StreamCreate step. Any ideas what it could be?

trojannemo

  • Posts: 143
Ignore this. I have a different approach that is almost working. See below.
« Last Edit: 5 Aug '24 - 01:45 by trojannemo »

trojannemo

  • Posts: 143
Ian,

I need your in-depth knowledge of BASS to come in and help take my code to the finish line. I have literally spent three straight days trying all kinds of things.
Here's where the code stands now:

Code: [Select]
case ".m4a":
    isM4A = true;
    //default to Apple ALAC codec first
    BassStream = BassAlac.BASS_ALAC_StreamCreateFile(input_file, 0L, File.ReadAllBytes(input_file).Length, BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT);
    if (BassStream != 0) break;
    //if that failed then try AAC codec second
    BassStream = BassAac.BASS_AAC_StreamCreateFile(input_file, 0L, File.ReadAllBytes(input_file).Length, BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT);
    if (BassStream != 0) break;
    //if that failed then try MP4 third
    BassStream = BassAac.BASS_MP4_StreamCreateFile(input_file, 0L, File.ReadAllBytes(input_file).Length, BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT);
    if (BassStream != 0) break;
    //if that failed then this might be a Fortnite .m4a Opus file, let's try to work with it
    BassStream = ProcessFortniteFile(input_file);
    isFnF = BassStream != 0;
    break;

Code: [Select]
.....
ExtractOpusDataFromMp4(file, opusData);
opusSamples = opusData.ToArray();
return FortniteFestivalRawToBassStream();

At this point, I have a byte array of raw Opus data samples. Each sample is preceeded by a 4 byte size identifier so we don't have to guess how much to read from the byte array. I have confirmed the data is correct and the sample sizes are correct. This particular test file is 9932 samples 12.xxMB large.

Code: [Select]
private static int FortniteFestivalRawToBassStream()
{
    BASS_OPUS_HEAD head = new BASS_OPUS_HEAD
    {
        version = 1,
        channels = 10,
        preskip = 312,
        inputrate = 44100,
        gain = 0,
        mapping = 255,
        streams = 5,
        coupled = 5,
        chanmap = new byte[] { }
    };
    head.chanmap = new byte[255];
    for (int i = 0; i < head.channels; i++)
    {
        head.chanmap[i] = (byte)i;
    }

    int streamHandle = BASS_OPUS_StreamCreate(ref head, (int)(BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT), STREAMPROC_PUSH, IntPtr.Zero);

    if (streamHandle == 0)
    {
        Console.WriteLine("Failed to create Opus stream. Error: " + Bass.BASS_ErrorGetCode());
        return 0;
    }

    // Push Opus packets to the stream
    PushOpusPacketsToStream(streamHandle);

    return streamHandle;
}

Code: [Select]
private static void PushOpusPacketsToStream(int streamHandle)
{
    GCHandle handle = GCHandle.Alloc(opusSamples, GCHandleType.Pinned);
    try
    {
        IntPtr pointer = handle.AddrOfPinnedObject();
        int offset = 0;
                       
        while (offset < opusSamples.Length)
        {
            if (offset + 4 > opusSamples.Length)
            {
                Console.WriteLine("Not enough data for size identifier.");
                break;
            }

            int packetSize = BitConverter.ToInt32(opusSamples, offset);
            offset += 4;

            if (packetSize <= 0 || offset + packetSize > opusSamples.Length)
            {
                Console.WriteLine("Invalid or incomplete packet data.");
                break;
            }

            int result = BASS_OPUS_StreamPutData(streamHandle, IntPtr.Add(pointer, offset), packetSize);

            if (result == -1)
            {
                Console.WriteLine("Failed to push data. Error: " + Bass.BASS_ErrorGetCode());
                break;
            }
            offset += packetSize;
        }

        // Confirm all data is pushed before ending the stream
        if (offset >= opusSamples.Length)
        {
            BASS_OPUS_StreamPutData(streamHandle, IntPtr.Zero, (int)BASSStreamProc.BASS_STREAMPROC_END);
        }
        else
        {
            Console.WriteLine("Stream ended prematurely. Offset: " + offset);
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine("Error: " + ex.Message);
    }
    finally
    {
        handle.Free();
    }
}

So this pushes all the Opus samples into the stream, which is returned into the original function so now BassStream is streamHandle.
BassStream is fed into a BassMixer that does a few things, including downmixing and setting a channel matrix.

Then I create an encoder (depends on the user but my test case is MP3) and I use the following to write to file:

Code: [Select]
const int MaxBufferSize = 1048576; // 1MB, adjust based on available memory and requirements
while (true)
{
    var buffer = new byte[MaxBufferSize];
    var c = Bass.BASS_ChannelGetData(BassMixer, buffer, buffer.Length);
    if (c < 0) break;
}

This whole thing "works" in the sense that there is no error and valid audio data is written to file. But only 44 seconds of audio is written, then the audio ends abruptly.
Obviously I'm missing something but I don't know what. Hopefully you can find the obvious mistake.

Thanks in advance.

EDIT: After further debugging, same file, I can tell you that I am pushing 12585050 bytes into the channel, but the channel is only writing 8581439 bytes to file. These numbers are consistent from run to run.
« Last Edit: 5 Aug '24 - 03:55 by trojannemo »

Ian @ un4seen

  • Administrator
  • Posts: 26035
It sounds like you've made good progress.

Regarding playback stopping early, perhaps there's a problem in the parsing when moving from one data fragment to the next? If you try processing just one fragment, does playback have the same length? Btw, I presume you're reading the sample sizes from the file's "trun" atoms/boxes?

Regarding the BASS_OPUS_HEAD values to pass to BASS_OPUS_StreamCreate, you can read that from the file's "dOps" atom/box instead of hardcoding it. Note MP4 has big-endian values, so you'll need to swap the byte order before passing to BASS_OPUS_StreamCreate. There is some info on this and other MP4 Opus things here:

   https://opus-codec.org/docs/opus_in_isobmff.html

The "Movie Fragments" part applies in your case.

trojannemo

  • Posts: 143
Are you saying that you dont think the problem is with my BASS specific code?

I'll look at your suggestions tonight...but I was pretty convinced I was misusing BASS and that's why the audio files are always truncated at different lengths per file but the same place each time I run the code in any given file (ie file 1 always ends at 44 seconds, file 2 at 28 seconds, etc)

trojannemo

  • Posts: 143
I don't get it. Even chatGPT is going around in circles trying to figure it out.

Quote
Input File: badguy.m4a
Input Extension: .m4a
Input Audio Channels: 10
Input Audio Frequency: 48000
Output File: C:\Users\Yemil-PC\source\repos\AudioConverter\AudioConverter\bin\Debug\converted\badguy.opus
Output Format: opus
Output Audio Channels: 10
Output Audio Frequency: 48000
Bitrate/Quality: 256
BassStream Value: -2147483647

Total bytes pushed: 12585050
Total packets processed: 9932

Read 85814400 total bytes from stream


Processing complete.

Success


That's the output from the program (minus metadata stuff I cut out).
I can output 10 to 10 channels from my raw data to opus or ogg if I do channelgetdata on BassStream, or 10 to 2 channels if I run it on BassMixer. Always works, no matter the output format. But it's always 44 seconds long. Even though it's saying it read enough for about 8MB worth of data yet it outputs a fraction of that to file. The audio that is there is perfect, so why would the rest of it be damaged? I have 12MB worth of audio data from the m4a file. I need to clear my head  :-\

trojannemo

  • Posts: 143
FIXED IT AND IT WORKS BEAUTIFULLY NOW. THANK YOU ALL FOR LETTING ME RANT/VENT ON THE FORUM.

The problem was in the parsing of the opus data from the m4a file after all. I revisited that, found the file had a mdat and a MDaT block, and from there we were able to fix things. Now it reads and writes to file correctly, even while going through a downmixing mixer.

Bass is one hell of a system. Thank you Ian.

Ian @ un4seen

  • Administrator
  • Posts: 26035
FIXED IT AND IT WORKS BEAUTIFULLY NOW.

Excellent!

STEMS (as MP4 files, also known with .m4a file extension) are often used by DJs because they have separate tracks for drums, bass, vocals and instruments (channels 3-10) in addition to the fully mixed stereo track (channels 1 and 2).

This stem stuff should probably be a separate thread, so I have now split some of these posts (where the subject wasn't mixed) into a new thread, and also posted a BASS_AAC update for you to try there:

   www.un4seen.com/forum/?topic=20474.0

trojannemo

  • Posts: 143
Now here's a follow up.

Currently I can read the m4a file, use the new bass_opus_streamcreate function and create a working stream that i can push the data to, and feed that into my mixer, downmix (or not) and encode to any of 5 different output formats.

But I want the option to save the data straight to file without any reencoding, like ffmpeg does. I.e. I want the original 12MB data + the opus headers and ogg pages that opus_streamcreate should be adding to the file.

When I try outputting the stream's buffer in the same loop I use to write the mixer to file, but instead do it on the stream itself, i get a 363MB headerless file on my hard drive.

So, is there a way to save the opus data with header and ogg pages without reencoding?

Ian @ un4seen

  • Administrator
  • Posts: 26035
BASS_OPUS_StreamCreate doesn't actually add any Ogg wrapping to the provided Opus data but rather passes it straight to the Opus decoder. Similarly, BASS_OPUS_StreamCreateFile/etc extracts the Opus data from the Ogg wrapping before passing it to the Opus decoder.

To create an Ogg Opus file, you would use the libogg library. Basically call ogg_stream_packetin for each Opus packet, eg. at the same time as (or instead of) BASS_OPUS_StreamPutData. Note that the 1st packet needs to be an "Identification Header" (containing the BASS_OPUS_HEAD values), and the 2nd packet needs to be a "Comment Header". The details can be found here:

   https://datatracker.ietf.org/doc/html/rfc7845.html#section-5

trojannemo

  • Posts: 143
Is there no way to accomplish this with bass? I got out of using ffmpeg I dont want to have to fall back to another library or program.