Topic: Drawing Live Waveforms for compressed HTTP audio files

[BaseHead]

Hey All and Ian!

First some back story:
We have streaming audio via StreamCreateURL for ages and also drawing hi-res waveforms locally.
When uploading file to our server we create tiny little .wf files for the pre-drawn waveforms and upload them to the server that we download when we call any file from our servers  to be played and it's been working great for years.

but now....
We have a few situations working with other companies that they are not gonna upload our .wf waveform cache files in our format to their servers but have allowed us to have access to stream their files and we need a way to draw waveforms as the audio stream is playing and buffering live, but course at the faster speed of the buffering instead of playback.   
Waiting till it buffers 100% before drawing just won't cut it also. 


My coder said...if we streamed WAV files it would be less headache but we need to draw OGG, FLAC and MP3 also and he says it gets difficult messy with compressed audio when I asked why don't we just draw _data as it comes in the code snippet below.

Code: [Select]

try
 {
     // increase the data buffer as needed
     if (_data == null || _data.Length < length)
         _data = new byte[length];
     iCloudFileLength += length;

     Marshal.Copy(buffer, _data, 0, length);  // copy from managed to unmanaged memory
     _fs.Write(_data, 0, length);   // write to file                   

 }
Any tips for this or example bits anyone have doing the same function.?
Thx so much!

Steve T.

[Ian @ un4seen]

If I understand correctly, you need to access a stream's decoded data as it plays? If so, a DSP function (via BASS_ChannelSetDSP) seems like the answer - that'll always give you decoded PCM data regardless of what the stream's original format is. If you don't need all data rather just pieces of current data then BASS_ChannelGetData may be another option.

[BaseHead]

Hey Ian!
Sort of....
It will be playing the stream in real time but we want to draw ahead of where it's playing and instead draw from the amount looked ahead and downloaded already via the DownloadProc

Is sent him your response and this is his.
Looks like his is still stuck   


======FROM MY CODER======

Hi
I have a problem with WF drawing of cloud audio file
When create a stream with BASS_StreamCreateUrl() function, WF data taken from the stream is not correct sometimes.
So I  am going to draw with the current downloaded buffer through some steps in DOWNLOADProc function to advance the performance of WF drawing
I have to extract audio data part from current downloaded buffer and draw WF with the audio data.
so I tried to create a stream with current downloaded buffer and get channel data from the stream.

My code is as follows.

private FileStream _fs = null;
int downStream = 0;

public void MyDownloadProc(IntPtr buffer, int length, IntPtr user)
{
    if(_fs  == null)
    {
        _fs = File.OpenWrite(cloudTempFullPath);
        downStream = Bass.BASS_StreamCreate(sr, channels, BASSFlag.BASS_SAMPLE_FLOAT|BASSFlag.BASS_STREAM_DECODE, null, IntPtr.Zero);
    if (buffer == IntPtr.Zero)
    {
        ...
    }
    else
    {
        byte[] data = new byte[length];
        Marshal.Copy(buffer, data, 0, length);
        _fs.Write(data, 0, length);   // write to file
                   
        float[] floatArray = new float[data.Length / 4]; // Assuming each float is 4 bytes

        for (int i = 0; i < data.Length; i += 4)
        {
             floatArray[i / 4] = BitConverter.ToSingle(data, i);
         }

         Bass.BASS_StreamPutData(downStream, floatArray, floatArray.Length);
         GenerateWaveFormWithStream(downStream);
    }
}

public void GenerateWaveformWithStream(downStrream)
{
    streamLength = (long)Bass.BASS_ChannelGetLength(downStream);
   
    ... // some codes for drawing
   
    if(downStream !=0)
    {
         Bass.BASS_StreamFree(downStream);
         downStream = 0;
    }
}


but the streamLength =-1 here.
could you let me know what the resolution is? is it possible to do like this?
Thanks.

[Ian @ un4seen]

It looks like the issue there is that it's assuming the data received by a DOWNLOADPROC is PCM, but it most likely isn't - it's still in the original format, eg. MP3 or whatever. If you want to separately decode the data received by a DOWNLOADPROC then you should use BASS_StreamCreateFileUser (or perhaps BASS_StreamCreateFile with mem=true) instead of BASS_StreamCreate (which only deals in PCM). Also note that with formats that have headers (inc. OGG and FLAC) the individual blocks of data won't be decodable by themselves - the headers (received earlier) is needed.

How far ahead of playback do you need to draw? If it's no more than 5 seconds then another/simpler option could be to use a larger playback buffer (via BASS_CONFIG_BUFFER), so that data gets decoded further in advance, and use BASS_ChannelGetData to fetch data from it.

[BaseHead]

We definitely want it to draw way more ahead then 5 seconds so I will show him the first part of your response.
If a 5 min file take about 7 seconds to Download a FLAC file to 100% I want the waveform to also draw at the same rate as the download of 7 seconds and see it growing as it's grabbing pieces of it.   

I will let you know if it he is still stuck.
thx man!
Steve

[Ian @ un4seen]

In that case, I would suggest using BASS_StreamCreateFileUser with STREAMFILE_BUFFERPUSH. Your DOWNLOADPROC would create the stream/decoder in the first call, and the subsequent calls would just feed more data to it via BASS_StreamPutFileData. Note that the first call may also need to use BASS_StreamPutFileData after BASS_StreamCreateFileUser, to feed any remaining data that wasn't needed to create the stream.

[BaseHead]

===New Response from my Coder===

I tried to create the stream using BASS_StreamCreateFileUser, but it can't create it properly.

Code: [Select]

public int downStream = 0;
int MyFileProcUserRead(IntPtr buffer, int length, IntPtr user)
{
    try
    {
        int bytesread = _data.Length;
        Marshal.Copy(_data, 0, buffer, bytesread);
        return bytesread;
    }
    catch
    {
        return 0;
    }
}

public void StreamDownloadProc(IntPtr buffer, int length, IntPtr user)  //Might need to move start of Proc somewhere else
{
    ....
    BASS_FILEPROCS fileprocs = new BASS_FILEPROCS(null, null, MyFileProcUserRead, null);
    downStream = Bass.BASS_StreamCreateFileUser(BASSStreamSystem.STREAMFILE_BUFFERPUSH, BASSFlag.BASS_DEFAULT, fileprocs, IntPtr.Zero);
    BASSError err = Bass.BASS_ErrorGetCode();
}

Here downStream is still zero and returns BASS_ERROR_ILLPARAM

Any ideas?
Thx!

[Ian @ un4seen]

You will need to provide FILECLOSEPROC and FILELENPROC functions too. If the file length is unknown then the FILELENPROC can just return 0.

Note the FILEREADPROC shouldn't ever return more data than is requested in the "length" parameter. It should also keep a read pointer and advance that each call, so that it doesn't return the same data again.

[Jin]

Hi, I am Jin, the coder working on this task for Steve.
I succeed to create a stream as follows, but can't get the length of the stream.
In the below code, streamLength = Bass.BASS_ChannelGetLength(downStream) returns -1 now.
please let me know what is wrong.
Thanks.

byte[] _data;
FileStream _fs = null;
public int downStream = 0;
bool decoderInitialized = false;

BlockingCollection<byte[]> _dataQueue = new BlockingCollection<byte[]>(); // For thread-safe queue operations
int MyFileProcUserRead(IntPtr buffer, int length, IntPtr user)
{
    int totalBytesCopied = 0;
    while (totalBytesCopied < length)
    {
        byte[] data = null;
        if (_dataQueue.Count < 1)
            return 0;

        try
        {
            data = _dataQueue.Take(); // Wait until data is available
        }
        catch (InvalidOperationException) { break; }

        if (data != null && data.Length > 0)
        {
            int bytesToCopy = Math.Min(length - totalBytesCopied, data.Length);
            Marshal.Copy(data, 0, buffer + totalBytesCopied, bytesToCopy);
            totalBytesCopied += bytesToCopy;

            if (bytesToCopy < data.Length) // If there's remaining data, add it back to the queue
            {
                byte[] remainingData = new byte[data.Length - bytesToCopy];
                Array.Copy(data, bytesToCopy, remainingData, 0, data.Length - bytesToCopy);
                _dataQueue.Add(remainingData);
            }
        }
        else { break; }

        // Check if the decoder has been initialized and return if any amount of data has been read
        if (totalBytesCopied > 0 && decoderInitialized)
        { break; }
    }

    if (totalBytesCopied > 0)
    {
        return totalBytesCopied; // Return the number of bytes read
    }
    else { return 0; }
}
void MyFileProcUserClose(IntPtr user)
{
    Console.WriteLine("StreamCreateFileUserClosed");
}

long MyFileProcUserLength(IntPtr user)
{
    return 0L; // indeterminate length
}

public void StreamDownloadProc(IntPtr buffer, int length, IntPtr user)  //Might need to move start of Proc somewhere else
{
    if(_fs == null)
    {
         decoderInitialized = false;
    }
    if (buffer == IntPtr.Zero) //FYI....fires faster then CloudProgress start on short files
    {
        // finished downloading
        _fs.Close();
        _fs = null;
    }
    else
    {
        Marshal.Copy(buffer, _data, 0, length);  // copy from managed to unmanaged memory
        _fs.Write(_data, 0, length);   // write to file             
        _dataQueue.Add(_data);

        if (!decoderInitialized)
        {
           BASS_FILEPROCS fileprocs = new BASS_FILEPROCS(MyFileProcUserClose, MyFileProcUserLength, MyFileProcUserRead, null);
           downStream = Bass.BASS_StreamCreateFileUser(BASSStreamSystem.STREAMFILE_BUFFERPUSH, BASSFlag.BASS_DEFAULT, fileprocs, IntPtr.Zero);
           BASSError err = Bass.BASS_ErrorGetCode(); // returns BASS_OK now
      if (downStream != 0)
            decoderInitialized = true;
        }
   else
        {
            if (downStream != 0)
            {
                int _len = Bass.BASS_StreamPutFileData(downStream, _data, _data.Length);
                long streamLength = (long)Bass.BASS_ChannelGetLength(downStream);
            }
   }
    }
}

[Ian @ un4seen]

Depending of the file format, it may not be possible to get the stream's length without knowing the filesize, ie. returned by the FILELENPROC. If the original URL you're streaming (with the DOWNLOADPROC on it) is a file then you would usually be able to get its size from BASS_StreamGetFilePosition with mode=BASS_FILEPOS_SIZE, something like this:

Code: [Select]

long MyFileProcUserLength(IntPtr user)
{
    return Bass.BASS_StreamGetFilePosition(origstream, BASSStreamFilePosition.BASS_FILEPOS_SIZE);
}


[Jin]

This MyFileProcUserLength does not help me.
I tried like that, but I couldn't get the stream length still.
Also I failed to get data from downStream.

Code: [Select]

    BASS_FILEPROCS fileprocs = new BASS_FILEPROCS(MyFileProcUserClose, MyFileProcUserLength, MyFileProcUserRead, null);
    downStream = Bass.BASS_StreamCreateFileUser(BASSStreamSystem.STREAMFILE_BUFFERPUSH, BASSFlag.BASS_DEFAULT, fileprocs, IntPtr.Zero);
    if (downStream != 0)
    {
        int _len = Bass.BASS_StreamPutFileData(downStream, _data, _data.Length);

        short[] mem = new short[100];
        int byteGotten = Bass.BASS_ChannelGetData(downStream, mem, 200);
    }
Here byteGotten is 0.
Could you give me an example related to BASS_StreamCreateFileUser and BASS_FILEPROCS please?
Thanks.

[Ian @ un4seen]

This MyFileProcUserLength does not help me.
I tried like that, but I couldn't get the stream length still.

Please confirm what value the BASS_StreamGetFilePosition call is returning, and what the file format is.

Also I failed to get data from downStream.

Code: [Select]

    BASS_FILEPROCS fileprocs = new BASS_FILEPROCS(MyFileProcUserClose, MyFileProcUserLength, MyFileProcUserRead, null);
    downStream = Bass.BASS_StreamCreateFileUser(BASSStreamSystem.STREAMFILE_BUFFERPUSH, BASSFlag.BASS_DEFAULT, fileprocs, IntPtr.Zero);
    if (downStream != 0)
    {
        int _len = Bass.BASS_StreamPutFileData(downStream, _data, _data.Length);

        short[] mem = new short[100];
        int byteGotten = Bass.BASS_ChannelGetData(downStream, mem, 200);
    }
Here byteGotten is 0.

It's possible that BASS_ChannelGetData may sometimes return 0 if the decoder needs more file data first, ie. from the next BASS_StreamPutFileData call.

Did BASS_StreamCreateFileUser succeed, ie. "downStream" isn't 0? If so, the next thing is to check is that you only pass to BASS_StreamPutFileData data that the FILEREADPROC (MyFileProcUserRead) didn't already provide.

[Jin]

Please confirm what value the BASS_StreamGetFilePosition call is returning, and what the file format is.

the file format is .ogg and the above function returns -1 usually and returns the correct length(>0) very occasionally

Code: [Select]

var origStream = Bass.BASS_StreamCreateURL(path, 0, BASSFlag.BASS_STREAM_DECODE, StreamDownloadProc, new IntPtr(RandomNumber()));
...
public void StreamDownloadProc(IntPtr buffer, int length, IntPtr user)  //Might need to move start of Proc somewhere else
{
    ...
    BASS_FILEPROCS fileprocs = new BASS_FILEPROCS(MyFileProcUserClose, MyFileProcUserLength, MyFileProcUserRead, null);
    downStream = Bass.BASS_StreamCreateFileUser(BASSStreamSystem.STREAMFILE_BUFFERPUSH, BASSFlag.BASS_DEFAULT, fileprocs, IntPtr.Zero);
    ...
}
...
long MyFileProcUserLength(IntPtr user)
{
    return Bass.BASS_StreamGetFilePosition(origStream, BASSStreamFilePosition.BASS_FILEPOS_SIZE);
}
when MyFileProcUserLength is called, origStream is 0 usually because it is not created yet.

Did BASS_StreamCreateFileUser succeed, ie. "downStream" isn't 0? If so, the next thing is to check is that you only pass to BASS_StreamPutFileData data that the FILEREADPROC (MyFileProcUserRead) didn't already provide.

yes, the downloadStream is not 0 definitely.
and I only pass the first downloaded buffer to FileReadPROC and then pass the 2nd, 3rd ... downloaded buffers to BASS_StreamPutFileData.
so I only pass to BASS_StreamPutFileData data that the FILEREADPROC (MyFileProcUserRead) didn't already provide.

Code: [Select]

int _len = Bass.BASS_StreamPutFileData(downStream, _data, _data.Length);

this function returns correct value.
Here _data is downloaded buffer when StreamDownloadProc is called.
let me know your opinion.
Thanks for your help.

PS: Is it correct if I think FileReadPROC is for reading file header and BASS_StreamPutFileData is for putting the remainder of file into a stream?

[Ian @ un4seen]

when MyFileProcUserLength is called, origStream is 0 usually because it is not created yet.

Oh right, that's true. You'll need to get the filesize from the HTTP "Content-Length" header instead in this case. If you include the BASS_STREAM_STATUS flag in your BASS_StreamCreateURL call then the DOWNLOADPROC will first receive the HTTP headers, which you can extract the "Content-Length" header value from. I'm not a .Net user myself but in C/C++ it could look like this:

Code: [Select]

void CALLBACK DownloadProc(const void *buffer, DWORD length, void *user)
{
if (buffer && !length) { // got headers
const char *p = (const char*)buffer;
while (*p) {
if (!strnicmp(p, "Content-Length:", 15)) { // found the "Content-Length" header
filesize = atoi(p + 15); // get its value
break;
}
p += strlen(p) + 1;
}
}
...


...I only pass the first downloaded buffer to FileReadPROC and then pass the 2nd, 3rd ... downloaded buffers to BASS_StreamPutFileData.
so I only pass to BASS_StreamPutFileData data that the FILEREADPROC (MyFileProcUserRead) didn't already provide.

Note that BASS_StreamCreateFileUser might not request all of the data that you have via your FILEREADPROC, in which case you should provide the remainder via BASS_StreamPutFileData.

PS: Is it correct if I think FileReadPROC is for reading file header and BASS_StreamPutFileData is for putting the remainder of file into a stream?

Pretty much, yes. Some initial audio data may be read from the FILEREADPROC too, but it won't be called again after BASS_StreamCreateFileUser returns (when STREAMFILE_BUFFERPUSH is used).

[Jin]

I received the HTTP header with the code you sent, but it is only "HTTP/1.1 200 OK".
the header does not contain "Content-length".
Anyway I got the filesize from db file and MyFileProcUserLength returns total file size currently.

Code: [Select]

public void StreamDownloadProc(IntPtr buffer, int length, IntPtr user)  //Might need to move start of Proc somewhere else
{
    ...
    int _len = Bass.BASS_StreamPutFileData(downStream, _data, _data.Length);
    long streamLength = 0;
    streamLength = (long)Bass.BASS_ChannelGetLength(downStream);

    short[] mem = new short[100];
    int byteGotten = Bass.BASS_ChannelGetData(downStream, mem, 200);
    ...
}
Now the streamLength is not -1 and I think BASS_ChannelGetLength returns the correct data length.
But the byteGotten is still 0.
let me know what the reason is.
Thanks.

[Ian @ un4seen]

I received the HTTP header with the code you sent, but it is only "HTTP/1.1 200 OK".

The HTTP headers are in a series of null-terminated strings, the first of which will contain the status code (as above). So it looks like you only checked the first string. BASS.Net has a Utils.IntPtrToArrayNullTermUtf8 method that you can use to access all of them. But if you already have the filesize another way, then no need for this and the BASS_STREAM_STATUS flag.

Now the streamLength is not -1 and I think BASS_ChannelGetLength returns the correct data length.
But the byteGotten is still 0.
let me know what the reason is.

Is it always 0, or only on the first call after BASS_StreamCreateFileUser? Note that 0 from BASS_ChannelGetData isn't an error (-1 is), it just means that there's no data available currently.

[Jin]

Is it always 0, or only on the first call after BASS_StreamCreateFileUser? Note that 0 from BASS_ChannelGetData isn't an error (-1 is), it just means that there's no data available currently.

Yes, it is always 0.

Code: [Select]

    int _len = Bass.BASS_StreamPutFileData(downStream, _data, _data.Length);
    short[] mem = new short[100];
    int byteGotten = Bass.BASS_ChannelGetData(downStream, mem, 200);
Here _len >0, but byteGotten is always 0 while downloading.
I tried to do it with wav or flac instead of ogg, but the same.

[Ian @ un4seen]

Ah! I just noticed the BASS_STREAM_DECODE flag isn't used in the BASS_StreamCreateFileUser call. Adding that will make BASS_ChannelGetData decode rather than just copy from a (empty) playback buffer.

[Jin]

great!!! At last I got the data with BASS_ChannelGetData from downStream after add BASS_STREAM_DECODE
thanks for your help.

[Jin]

I tried with the below function to get the correct length of original http file in MyFileReadProc()

Code: [Select]

static IEnumerable<string> ExtractMultiString(IntPtr ptr)
{
    while (true)
    {
        string str = Marshal.PtrToStringUTF8(ptr);
        if (str.Length == 0)
            break;
        yield return str;
        ptr = new IntPtr(ptr.ToInt64() + (str.Length + 1 /* char \0 */) * sizeof(char));
    }
}

But the result string array is as follows.

Code: [Select]

HTTP/1.1 200 OK
56982
t-Type: audio/ogg
, 21 Sep 2021 03:19:54 GMT
"0x8D97CAEAE5FC8BB"
e-Blob/1.0 Microsoft-HTTPAPI/2.0
4e-5f0a-9d7b75000000
-04
-meta-Mtime: 2020-10-26T17:07:55.671000000Z
4 GMT
ease-status: unlocked
lable
lob-type: BlockBlob
 
ms-server-encrypted: true
ers: x-ms-request-id,Server,x-ms-version,x-ms-meta-Mtime,Content-Type,Content-Encoding,Content-Language,Cache-Control,Last-Modified,ETag,x-ms-creation-time,x-ms-lease-status,x-ms-lease-state,x-ms-blob-type,Content-Disposition,x-ms-server-encrypted,Accept-Ranges,Content-Length,Date,Transfer-Encoding
do you think this result is correct?
"Content-Length"is in, but the value is not in this string.
thanks

[Ian @ un4seen]

Those strings don't look right. I believe a "char" is 2 bytes in .Net, while the HTTP headers are 1 byte per character, so try removing the *sizeof(char) part and see if the strings look better then.