Topic: Can't create stream from multichannel M4A file - but Audacity can open it???

[Ian @ un4seen]

No, I'm afraid not, but libogg is quite a small library, so it won't bloat your app much. Note that libogg is the official library (from Xiph) for dealing with Ogg containers. It's officially distributed only in source form, but you can get a pre-compiled DLL here (eg. "ogg.dll" from the "ogg vorbis dlls using libvorbis v1.3.7 + aoTuVb6.03" downloads):

   https://www.rarewares.org/ogg-libraries.php

And here's the API docs:

   https://xiph.org/ogg/doc/libogg/

[trojannemo]

Thanks. I'll look into this tonight.

[trojannemo]

I gave up on that. But thanks for the link. Now I have another question.

Given that I have captured the data in the Opus stream from your new function, how can I get that into a usable stream that I can play from?

For example, after  I push the opus data into the array, I can convert it and export it out to file by starting an encoder and feeding the data to it, like so:

Code: [Select]

BassEnc_Opus.BASS_Encode_OPUS_StartFile(fnfStream, "--vbr --music", BASSEncode.BASS_ENCODE_DEFAULT | BASSEncode.BASS_ENCODE_AUTOFREE, tempFile);

while (true)
{
    var buffer = new byte[20000];
    var c = Bass.BASS_ChannelGetData(fnfStream, buffer, buffer.Length);
    if (c <= 0) break;
}

That creates a temporary file that I can then read back and play from normally. But that's doing double the work and limits me to hdd write speed vs ram speed.

I tried just assigning fnfStream to BassStream and running my standard player code on BassStream but it fails. I then tried to run the while (true) loop without an encoder, and playback still fails.
Is it possible to do this with the raw opus data at all?

[Ian @ un4seen]

Do you still want to create an Ogg Opus file from the Opus packets that you've extracted from an MP4 Opus file? If so, I still think using libogg is probably the way to go. The code could look something like this:

Code: [Select]

ogg_stream_state state;
ogg_packet packet;
ogg_page page;

ogg_stream_init(&state, rand());

// OpusHead packet
packet.packet = opushead; // "OpusHead" + the MP4's "dOps" atom/box (note byte order)
packet.bytes = opushead_size;
packet.b_o_s = 1;
packet.e_o_s = 0;
packet.granulepos = 0;
packet.packetno = 0;
ogg_stream_packetin(&state, &packet); // feed-in the packet
ogg_stream_flush(&state, &page); // get back an Ogg page
WritePage(&page); // write the page

// OpusTags packet
packet.packet = "OpusTags\0\0"; // no tags
packet.bytes = 10;
packet.b_o_s = 0;
packet.packetno++;
ogg_stream_packetin(&state, &packet); // feed-in the packet
ogg_stream_flush(&state, &page); // get back an Ogg page
WritePage(&page); // write the page

// Opus packets
while (1) {
// get next opus packet here
packet.packet = opuspacket;
packet.bytes = opuspacket_size;
if (lastpacket) packet.e_o_s = 1;
packet.granulepos += opuspacket_decodedsamples;
packet.packetno++;
ogg_stream_packetin(&state, &packet); // feed-in the packet
if (lastpacket) {
ogg_stream_flush(&state, &page)) // get back an Ogg page
WritePage(&page); // write the page
break; // done
}
if (ogg_stream_pageout(&state, &page)) // get back an Ogg page
WritePage(&page); // write the page
}

...

void WritePage(ogg_page *page)
{
fwrite(page->header, page->header_len, 1, outfile);
fwrite(page->body, page->body_len, 1, outfile);
}

Please see the libogg docs for details on the functions/structs used.

[trojannemo]

Sorry if I wasn't clearer. I already am able to create any kind of output file with my code which results in an int stream that I can then feed to any encoder and run the while() loop for channelgetdata to pull the data I pushed earlier in the code. That works and it gives me an opus/ogg/mp3/flac/wav file on disc.

What I'm trying to do is PLAY that int stream, and it's not working. So i'm forced to write it out to file, read back in with a normal createstreamfile function, then play that. I'm looking for a way to avoid the writing to file and subsequent reading from file, as it is not efficient.

[Ian @ un4seen]

Please clarify how you're creating the "int stream". If it's the "fnfStream" handle in the code above and you would like to play that, then you should remove the BASS_STREAM_DECODE flag from it and call BASS_ChannelPlay on it (instead of BASS_ChannelGetData in a loop).

[trojannemo]

I'll post the whole code below. What you suggested did not work.

Code: [Select]

public int m4aToBassStream(string m4aFilePath, int channels, bool doDecode)
        {
            var opusData = ExtractOpusDataFromMP4(m4aFilePath);
            if (opusData.Count() == 0) return 0;
            opusSamples = opusData.ToArray();
            channelCount = channels;
            return FortniteFestivalToBassStream(doDecode);
        }


private static List<byte> ExtractOpusDataFromMP4(string m4aFilePath)
{
    List<byte> opusData = new List<byte>();
    opusPackets = new List<byte[]>();
.....extract mp4 data containing opus packets......
    return opusData;
}


private static int FortniteFestivalToBassStream(bool doDecode)
{
    try
    {
        BASS_OPUS_HEAD head = new BASS_OPUS_HEAD
        {
            version = 1,
            channels = (byte)channelCount,
            preskip = 312,
            inputrate = 48000,
            gain = 0,
            mapping = channelCount == 2 ? (byte)0 : (byte)255,
            streams = (byte)(channelCount / 2),
            coupled = (byte)(channelCount / 2),
            chanmap = new byte[] { }
        };
        head.chanmap = new byte[255];
        for (int i = 0; i < head.channels; i++)
        {
            head.chanmap[i] = (byte)i;
        }
        int flags = doDecode ? (int)(BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT) : (int)BASSFlag.BASS_SAMPLE_FLOAT;
        int streamHandle = BASS_OPUS_StreamCreate(ref head, (uint)flags, STREAMPROC_PUSH, IntPtr.Zero);

        if (streamHandle == 0)
        {
            MessageBox.Show("Failed to create Opus stream. Error: " + Bass.BASS_ErrorGetCode());
            return 0;
        }

        PushOpusPacketsToStream(streamHandle);
        return streamHandle;
    }
    catch (Exception e)
    {
        MessageBox.Show("Error creating stream handle: " + e.Message + "\nBASS says: " + Bass.BASS_ErrorGetCode());
        return 0;
    }
}


private static void PushOpusPacketsToStream(int streamHandle)
{
    GCHandle handle = GCHandle.Alloc(opusSamples, GCHandleType.Pinned);
    try
    {
        IntPtr pointer = handle.AddrOfPinnedObject();
        int offset = 0;
        int packetCount = 0;
        int totalBytesPushed = 0;

        while (offset < opusSamples.Length)
        {
            // Check for size identifier
            if (offset + 4 > opusSamples.Length)
            {
                MessageBox.Show("Not enough data for size identifier.");
                break;
            }

            int packetSize = BitConverter.ToInt32(opusSamples, offset);
            offset += 4;

            if (packetSize <= 0 || offset + packetSize > opusSamples.Length)
            {
                MessageBox.Show("Invalid or incomplete packet data.");
                break;
            }

            int result = BASS_OPUS_StreamPutData(streamHandle, IntPtr.Add(pointer, offset), packetSize);
            if (result == -1)
            {
                MessageBox.Show("Failed to push data. Error: " + Bass.BASS_ErrorGetCode());
                break;
            }

            offset += packetSize;
            totalBytesPushed += packetSize;
            packetCount++;

            //MessageBox.Show($"Pushed packet {packetCount}: {packetSize} bytes");
        }

        //MessageBox.Show($"Total bytes pushed: {totalBytesPushed}");
        //MessageBox.Show($"Total packets processed: {packetCount}");

        // Signal end of stream
        BASS_OPUS_StreamPutData(streamHandle, IntPtr.Zero, (int)BASSStreamProc.BASS_STREAMPROC_END);
    }
    catch (Exception ex)
    {
        MessageBox.Show("Error: " + ex.Message);
    }
    finally
    {
        handle.Free();
    }
}

this all works for writing the opus data to file, but not to play the channel containing the opus data. even if I don't use the decode flag.

Code: [Select]

BassStream = fnfParser.m4aToBassStream(m4aFilePath, 10, false);//always 10 channels, no preview allowed here
if (BassStream == 0)
{
    MessageBox.Show("File '" + m4aFilePath + "' is not a valid input file", Text, MessageBoxButtons.OK, MessageBoxIcon.Exclamation);
    return;
}

whether with decode or not, BassStream has a valid result, not 0.

Code: [Select]

BassEnc_Opus.BASS_Encode_OPUS_StartFile(fnfStream, "--vbr --music", BASSEncode.BASS_ENCODE_DEFAULT | BASSEncode.BASS_ENCODE_AUTOFREE, tempFile);
while (true)
{
    var buffer = new byte[20000];
    var c = Bass.BASS_ChannelGetData(fnfStream, buffer, buffer.Length);
    if (c <= 0) break;
}
Bass.BASS_StreamFree(fnfStream);

BassStream = BassOpus.BASS_OPUS_StreamCreateFile(tempFile, 0L, File.ReadAllBytes(tempFile).Length, BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT);
if (BassStream == 0)
{
    MessageBox.Show("That is not a valid .m4a input file", Text, MessageBoxButtons.OK, MessageBoxIcon.Exclamation);
    File.Delete(tempFile);
    return;
}

this writes an opus file to hard drive just fine. but then I'm writing out to then read back in.

skipping the code above and and not using the decode flag:

Code: [Select]

var channel_info = Bass.BASS_ChannelGetInfo(BassStream);

results in a null channel_info.

Code: [Select]

var len = Bass.BASS_ChannelGetLength(BassStream);

results in a -1 length.

I don't know why this is happening. Is it a byproduct of it taking in raw opus data? Or something I'm missing about how BASS works internally?

[Ian @ un4seen]

A stream created with BASS_OPUS_StreamCreate has an indeterminate length (like BASS_StreamCreate with PCM), so BASS_ChannelGetLength will indeed fail (return -1) with BASS_ERROR_NOTAVAIL then. That won't prevent playback though. Is playback also not working (eg. BASS_ChannelPlay failing) or is it just length retrieval that's not working?

[trojannemo]

I think it might be the indeterminate length then.

I have always relied on channel_info to determine the original frequency and not resample while playing. Fine, for these files it's always 48000 so i can hard code that in.
I also rely on the length for the seeking and progress bar animations. Although I suppose I can sacrifice those features for immediate playback vs 5-10 seconds of waiting to start playback.

But it's still not working, and I think it's the indeterminate length.

Code: [Select]

else if ((isM4A))
{
    BassMixer = BassMix.BASS_Mixer_StreamCreate(48000, 2, BASSFlag.BASS_MIXER_END);
}
BassMix.BASS_Mixer_StreamAddChannel(BassMixer, BassStream, BASSFlag.BASS_MIXER_MATRIX);

var matrix = new float[2, isM4A ? 10 : channel_info.chans];
var splitter = new MoggSplitter();
matrix = splitter.GetChannelMatrix(Parser.Songs[0], isM4A ? 10 : channel_info.chans, GetStemsToPlay());
BassMix.BASS_Mixer_ChannelSetMatrix(BassStream, matrix);
Bass.BASS_ChannelPlay(BassMixer, false);

At this point Bass returns BASS OK.

Then my Playback Timer kicks in which keeps track of everything. Including the following:

Code: [Select]

if (Bass.BASS_ChannelIsActive(BassMixer) == BASSActive.BASS_ACTIVE_PLAYING)
{
    // the stream is still playing...
    var pos = Bass.BASS_ChannelGetPosition(BassStream); // position in bytes
    PlaybackSeconds = Bass.BASS_ChannelBytes2Seconds(BassStream, pos); // the elapsed time length
[do stuff here]
}
else
{
    PlaybackTimer.Enabled = false;
}

It's never finding that the channel is active playing so it always goes to stopping the timer.
what (else) am I missing here?

[trojannemo]

Further info after combing through the documentation.

Internally, a mixer will use the BASS_ChannelGetData function to get data from its source channels. That means that the source channels must be decoding channels (not using a RECORDPROC in the case of a recording channel).

As you might remember, I work with multichannel files (these are 10 channels) so I need a mixer and I use a matrix. If I remove the decoding flag, then it doesn't play in the mixer. If I leave the decoding flag and I remove a lot of my functionality (showing the length of the track, an animation showing the progress of the track, the ability to manually seek in either direction by dragging the dot on my progress line, etc, etc) then it plays the audio. But without knowing the length of the audio I can't calculate where in relation to the length the current position is or the desired place on the track, etc. In short, it kills a lot of functionality of the program.

Is there no way to get the same behavior I've always gotten but with StreamCreateFile?

[Ian @ un4seen]

Yes, if you're playing the stream through a mixer then it should indeed be a decoding channel, ie. BASS_STREAM_DECODE flag set. As well as the length being unavailable when using BASS_OPUS_StreamCreate, note that rewinding is also impossible because the data is discarded after use (so it can't be played again).

If you would like to be able to get the length and seek, you could achieve that by decoding the whole stream to memory (with BASS_ChannelGetData), putting a WAV header in front of it, and passing that to BASS_StreamCreateFile (with mem=true), but of course that isn't very memory-efficient. If the streams aren't very long then perhaps that won't be a problem.

[trojannemo]

thanks for the tip, but i'm going to stick with file based processing.

I tried what you suggested and it right away ran out of memory. these are 10 channel 48000hz songs that can vary from 1 minute 10+ minutes, so file sizes are in the hundreds of megabytes. .NET did not like that very much. Reverted back to writing WAV to file and reading it into BASS and it worked. There's just a 3-5 second delay to start. I'll have to live with that. Thanks for all the help.