Topic: Recording and serving flac stream with varying sample rates

[soundgals]

I’m using BASS_RecordStart and feeding the resulting recording’s handle to BASS_Encode_FLAC_Start to create a flac encoded handle. I then serve this via BASS_Encode_ServerInit.

All is working fine. My question though is regarding the sample rate I feed into BASS_RecordStart’s freq parameter. Normally I would use the current rate of the device I’m using for recording; but I have cases where this can vary while the recording/streaming process is taking place.

If I set it to the lowest sample rate I’’ll encounter, the quality of incoming audio at higher sample rates will be compromised when streamed via my app. I’ve tried stopping and restarting my app whenever the device sample rate changes; but this makes for a bad user experience.

So instead, I’m thinking of setting the freq parameter to the highest sample rate I’m likely to encounter (192000).

Can you tell me what the effect of this will be? Will all incoming audio be automatically re-sampled to 192000, or do I need to use BASSMIX to achieve that?

[Ian @ un4seen]

Higher sample rates will result in higher bitrates, especially with a lossless format like FLAC, so I don't think you should use a higher sample rate than necessary. I also don't think it's generally a good idea for internet streams to change format mid-stream, as it's a bad user experience like you said. So I would suggest picking a sample rate (eg. 44100 or 48000) and setting your recording device to use that (eg. in the Sound control panel on Windows or Audio MIDI Setup app on macOS). The device's rate shouldn't change by itself, but if it does happen to be different to the BASS_RecordStart call's rate then the data will be resampled accordingly.

[soundgals]

Thanks for clarifying that Ian.

I will follow your advice and pick a sample rate to use for all incoming audio.

Just out of interest can you share with me how the resampling is carried out, when it takes place? In other words do you use an external re-sampling library, or your own re-sampling methods?

[Ian @ un4seen]

BASS always handles playback resampling itself, but recording resampling varies across platforms. On Windows, BASS will handle it when using WASAPI. On macOS, the OS's AudioConverter functions are used. In all other cases, it's currently left to the OS, ie. BASS will simply request a rate (as in the BASS_RecordStart call) and let the OS resample if needed. If you have a particular resampler that you would like to use, then you could use the device's native rate in the BASS_RecordStart call and then use the resampler on the recorded data.

[soundgals]

Thanks again Ian. Of course re-samplers vary greatly in quality. One that's considered one of the best and incorporated into a number of apps is R8Brain https://github.com/avaneev/r8brain-free-src. I will consider incorporating this into my app in the way you suggested.

Have you ever considered incorporating a re-sampler of this quality into BASS itself, for whenever re-sampling needs to be performed?

That would be wonderful!

[soundgals]

Well, I believe I have successfully incorporated the r8Brain up-sampler library into my Swift project.

If you have a particular resampler that you would like to use, then you could use the device's native rate in the BASS_RecordStart call and then use the resampler on the recorded data

I'm just not sure how to go about this.

The Swift code to call the r8Brain re-sampling process is as follows…

Code: [Select]

             
                        let wrapper = ResamplerWrapper(inputRate: inputRate, outputRate: outputRate, taps: Int32(taps))

                        wrapper.resample(&input, &output, input.count)
… where, obviously inputRate will be the device's native rate I set in BASS_RecordStart. The outputRate will be my desired target rate and taps determines the re-sampling quality

The line

Code: [Select]

wrapper.resample(&input, &output, input.count) will obviously need to be called incrementally as more data becomes available.

I have my RecordProc callback and I'm assuming the wrapper.resample code needs to go in there.

Here's the callback code

Code: [Select]

func myRecordProc(handle: HRECORD, buffer: UnsafeRawPointer, length: DWORD, user: UnsafeMutableRawPointer?) -> Bool {
   
            BASS_StreamPutData(playchan, buffer, length)
    }
   
        return true
}
Can you show me an example of how to achieve this?

C code will be fine, I should be able to translate it to Swift.

Thanks in advance.

[soundgals]

I believe I might be making some further progress in resampling the data using R8Brain.

Here's my modified RecordProc…

Code: [Select]

func myRecordProc(handle: HRECORD, buffer: UnsafeRawPointer, length: DWORD, user: UnsafeMutableRawPointer?) -> Bool {
    let floatPointer = buffer.bindMemory(to: Float.self, capacity: Int(length))
    let mutableFloatPointer = UnsafeMutablePointer(mutating: floatPointer)
    var output = [Float()]
    let inputRate: Double = 44100
    let outputRate: Double = 192000
    let taps: Int = 16
    let wrapper = ResamplerWrapper(inputRate: inputRate, outputRate: outputRate, taps: Int32(taps))

    wrapper!.resample(mutableFloatPointer, &output, Int32(length))
    let mutableBuffer = UnsafeMutableRawPointer(mutating: buffer)
    let byteCount = Int32(length) * Int32(MemoryLayout<Float>.stride)
    output.withUnsafeBufferPointer { bufferPointer in
        mutableBuffer.copyMemory(from: bufferPointer.baseAddress!, byteCount: Int(byteCount))
    }
}
InputRate, OutputRate and Taps are all hard coded here just as a test.

I believe this should result in incoming buffer being modified with the re-sampled data from R8Brain.

What worries me is that being up-sampled data in this case, the length of the buffer will no longer be large enough to hold the resampled data.

I need this data to be further encoded to flac using the encoder with the --ogg flag, the served via BASS_Encode_ServerInit.

I may be going about this completely the wrong way and would really appreciate some advice, as I lack confidence in this area. Thanks

[Ian @ un4seen]

I have my RecordProc callback and I'm assuming the wrapper.resample code needs to go in there.

That sounds right. Does the resampler have the option of you feeding it input data and then requesting output data, rather than both in the same call? If so, you could do something like this:

Code: [Select]

BOOL CALLBACK RecordProc(HRECORD handle, const void *buffer, DWORD length, void *user)
{
int inlen = length / sizeof(float); // convert to samples
AddResamplerInput(buffer, inlen); // feed input data to resampler

while (1) {
float output[10000];
int outlen = GetResamplerOutput(output, 10000); // get resampler output
if (!outlen) break; // no more
BASS_StreamPutData(pushstream, output, outlen * sizeof(float)); // send it to push stream
}

return true; // continue recording
}

If the resampler needs you to feed input and receive output in a single call then you'll need to allocate a buffer for the output, something like this:

Code: [Select]

BOOL CALLBACK RecordProc(HRECORD handle, const void *buffer, DWORD length, void *user)
{
int inlen = length / sizeof(float); // convert to samples
int outlen = GetOutputAmountFromInputAmount(inlen); // get amount of output from amount of input
float *output = new float[outlne]; // allocate a buffer that size
int got = Resample(buffer, inlen, output, outlen); // resample
BASS_StreamPutData(pushstream, output, got * sizeof(float)); // send output to push stream
delete[] output;

return true; // continue recording
}


[soundgals]

Does the resampler have the option of you feeding it input data and then requesting output data, rather than both in the same call?

I believe it has to be both in the same call. So I need to use your second example.

I'm trying to relate your code to the Swift code I've already created in myRecordProc.

Assuming my

Code: [Select]

var output = [Float()] float array is equivalent to your

Code: [Select]

float *output = new float[outlne];; where I'm stuck is how to determine the length for the re-sampled data in the output buffer.

Where does the function

Code: [Select]

GetOutputAmountFromInputAmount come from? I'm assuming I would have to create this?

Also there is no return value from wrapper.resample… it should replace the buffer received in the callback with the output buffer data. At least that's my (admittedly vague) understanding.

So BASS_StreamPutData would just be passed the output buffer for its second parameter.

I also don't understand where the pushStream handle comes from.

I have my handle from BASS_RecordStart, which I'm calling "rChannel" and the handle from BASS_Encode_FLAC_Start, which I'm calling "encoder".

If you can relate your examples as closely as possible to my code, it will help me piece it together more easily. At least that's what I hope.

My aim now is to turn this into a new feature of my app. So to use R8Brain for high quality up-sampling, whenever desired by the user.

Apologies for all my confusion.

[Ian @ un4seen]

Assuming my

Code: [Select]

var output = [Float()] float array is equivalent to your

Code: [Select]

float *output = new float[outlne];; where I'm stuck is how to determine the length for the re-sampled data in the output buffer.

Where does the function

Code: [Select]

GetOutputAmountFromInputAmount come from? I'm assuming I would have to create this?

The resampler would hopefully have a function that tells you how much output it'll produce for a given amount of input. If not, you can calculate a value based on the sample rates. But looking at your earlier "wrapper.resample" call, there's no "output amount" parameter, so I'm guessing that "output" is some class that contains an amount value as well as the data? In which case, probably no need to calculate or allocate anything yourself.

I also don't understand where the pushStream handle comes from.

I have my handle from BASS_RecordStart, which I'm calling "rChannel" and the handle from BASS_Encode_FLAC_Start, which I'm calling "encoder".

I just included a BASS_StreamPutData call because the code you posted above did so. But it sounds like a "dummy" stream may be better for you:

Code: [Select]

dummy = BASS_StreamCreate(outrate, outchans, BASS_SAMPLE_FLOAT | BASS_STREAM_DECODE, STREAMPROC_DUMMY, 0); // create dummy stream with same format as resampler output
encoder = BASS_Encode_FLAC_Start(dummy, ...); // set FLAC encoder on it

...

BOOL CALLBACK RecordProc(HRECORD handle, const void *buffer, DWORD length, void *user)
{
inlen = length / sizeof(float); // convert to samples
int outlen = GetOutputAmountFromInputAmount(inlen); // get amount of output from amount of input
float *output = new float[outlne]; // allocate a buffer that size
int c = Resample(buffer, inlen, output, outlen); // resample
BASS_Encode_Write(encoder, output, c * sizeof(float)); // send output to FLAC encoder
delete[] output;

return true; // continue recording
}


If you can relate your examples as closely as possible to my code, it will help me piece it together more easily. At least that's what I hope.

I'm not at all fimiliar with Swift, so unfortunately I'm unable to help with that, but hopefully the above gives some ideas.

[soundgals]

Thanks for your patience Ian.

But looking at your earlier "wrapper.resample" call, there's no "output amount" parameter, so I'm guessing that "output" is some class that contains an amount value as well as the data?

Yes, hopefully I can figure out how to get the size of the resampled data from the class.

I just included a BASS_StreamPutData call because the code you posted above did so. But it sounds like a "dummy" stream may be better for you:

You're quite right, of course, I did include that in an earlier example of the code; but that was for a secondary case to the most important one.

To be clear what I'm trying to achieve here is…

1/ Record the incoming data.

2/ Upsample the data as it comes in within the record proc callback

3/ Encode the data to flac

4/ Serve it to an external player

Without the up-sampling it hasn't been necessary to do anything to the data within the record proc call back for all this to work. In fact all I've been doing is returning "True"

So do I even need a "dummy" stream"?

[Ian @ un4seen]

You will need a stream (with the same format as the resampler output) to host the encoder, ie. to use in the BASS_Encode_FLAC_Start call. If you want to play the resampled data as well as encode it then a "push" stream (and BASS_StreamPutData) would be a good choice, but if you only want to encode it then a "dummy" stream (and BASS_Encode_Write) will be a bit simpler and more efficient.

[soundgals]

Thanks Ian, that makes sense.

Unfortunately my code for interfacing Swift to R8Brain is not working when I try some simple examples outside of using it with BASS. Until I get some basic test examples working, I'm getting ahead of myself in trying to use it with BASS.

If anyone has had any luck interfacing Swift to the R8Brain C++ re-sampler library, and can offer some tips, I'll be very grateful.

[soundgals]

I found and corrected some errors in my code to call the R8 Brain re-sampler library from Swift.

When you said…

a "dummy" stream (and BASS_Encode_Write) will be a bit simpler and more efficient

Do this also go in the my myRecordProc callback along with the calls to the re-sampler?

If so what do I use here as the handle to BASS_Encode_Write?

Thanks

[Ian @ un4seen]

Yes, the BASS_Encode_Write call would go in your RECORDPROC callback function. You would pass the resampled data to it, along with the handle that you got from BASS_Encode_FLAC_Start (can also use the dummy stream handle instead). Please see the last code snippet that I posted above:

   www.un4seen.com/forum/?topic=20462.msg143403#msg143403

[soundgals]

Ok, that's clear now. Thanks again Ian.

I still have problems getting the re-sampler library to work from Swift. Hopefully I'll eventually get it working, then I'll be able to try this.