Topic: Converting DSD to PCM with BASS_ChannelGetData

[soundgals]

This is re-visiting my earlier post https://www.un4seen.com/forum/?topic=20260.msg141722#msg141722 from November last year.

I thought I had this working perfectly; but I obviously only tested on fairly short tracks.

I am able to take an original DSD (DSF or DFF) url, extract the data and convert it to a PCM stream at whatever sample rate I choose. Using first; BASS_DSD_StreamCreateURL then repeated calls to BASS_ChannelGetData until the length of the data has been fetched into a buffer.

I then convert the buffer to raw PCM data and further convert this to the data equivalent of a WAV file, by adding the WAV header, with a function I wrote for that purpose.

The resulting WAV data plays back perfectly; but only up to a certain point. That point depends, obviously, on the length of the track; but also on the sample rate I choose to use for the conversion to PCM.

For example if I choose a sample rate of 88200, I will get the first 6.23 minutes played back and then silence for any track which is longer than the 6.23 minutes. If I double the sample rate to 176400, I will get half that amount, about 3:13 minutes played, followed by silence.

I've been looking into this for a long time and just can't see what's going wrong, as all the data appears to be fetched into the buffer correctly.

Here's the Swift code I'm using…

Code: [Select]

func getDSDToPCMData(theURL:String, device:Int32, freq:Double = 88200, checkFormat:Bool = false)->Data{
        var info = BASS_INFO()
        BASS_GetInfo(&info)
        var trackData:Data? = nil
        let stream = BASS_DSD_StreamCreateURL(theURL, 0, DWORD(BASS_SAMPLE_FLOAT | BASS_STREAM_DECODE), nil, nil, DWORD(freq))
        var channelInfo = BASS_CHANNELINFO()
        BASS_ChannelGetInfo(stream, &channelInfo)
        if (info.freq != channelInfo.freq){ // the rates don't match
            BASS_Init(device, channelInfo.freq, DWORD(BASS_DEVICE_FREQ | BASS_DEVICE_REINIT), nil, nil)
        }
        print("Freq", channelInfo.freq)
        var len:QWORD = BASS_ChannelGetLength(stream, DWORD(BASS_POS_BYTE))
        let bufferSize = len
        let buffer = UnsafeMutablePointer<UInt32>.allocate(capacity: Int(bufferSize))
                defer {
                    buffer.deallocate()
                }
        var done:QWORD  = 0
        while (done < len) {
            let got = BASS_ChannelGetData(DWORD(stream), buffer + UnsafeMutableRawPointer.Stride(done), 0xFFFFFFF | DWORD(BASS_DATA_FFT_COMPLEX | BASS_DATA_FFT_INDIVIDUAL | BASS_DATA_FFT_REMOVEDC))
            if (Int(got) == -1) { // error/end
                    len = done
                    break
                }
            done += UInt64(got)
        }
        trackData = Data(bytes: buffer, count: Int(len))
        trackData = regularData(bitDepth: 24, outSR: freq, audioData: trackData!, formatCheck: checkFormat)
        return trackData!
       
    }

Any help will be much appreciated, as I seem so close to achieving what I want.

Thanks in advance.

[Ian @ un4seen]

The BASS_Init "freq" parameter doesn't affect DSD decoding (or decoding in general), so you can remove that BASS_Init (with BASS_DEVICE_REINIT) call. You can also remove the BASS_DATA_FFT_COMPLEX + BASS_DATA_FFT_INDIVIDUAL + BASS_DATA_FFT_REMOVEDC flags from your BASS_ChannelGetData call, as they only apply when getting FFT data. You should also free the stream (BASS_StreamFree) after you're done with it.

Regarding the silence problem, to narrow down what's causing that, please try also setting a BASSenc add-on WAV writer on the DSD stream and then check if the written file has the same problem. You can do so like this:

Code: [Select]

BASS_Encode_Start(stream, "bassenc.wav", BASS_ENCODE_PCM | BASS_ENCODE_AUTOFREE, 0, 0);


[soundgals]

Thanks Ian, I've cleaned up the code to remove the redundant DWORD(BASS_DATA_FFT_COMPLEX | BASS_DATA_FFT_INDIVIDUAL | BASS_DATA_FFT_REMOVEDC flags, as well as the redundant if (info.freq != channelInfo.freq){ // the rates don't match
            BASS_Init(device, channelInfo.freq, DWORD(BASS_DEVICE_FREQ | BASS_DEVICE_REINIT), nil, nil)
        }

Call.

I'm a bit confused though, when you say,

The BASS_Init "freq" parameter doesn't affect DSD decoding (or decoding in general

…

… I believed the decoding took place as a result of this line…

let stream = BASS_DSD_StreamCreateURL(theURL, 0, DWORD(BASS_SAMPLE_FLOAT | BASS_STREAM_DECODE), nil, nil, DWORD(freq))

… where the pcm freq I want the DSD to be decoded to is specified. I can just leave this at 0 and the freq specified in BASS_Config will be used as in…

BASS_SetConfig(
                           DWORD(BASS_CONFIG_DSD_FREQ),
                           DWORD(88200)
                       )
In fact I call this in the calling function, because I want to change the PCM frequency, depending on the sample rate of the original DSD. If it were a DSD128, I may prefer 176400 as the PCM rate, for example.

… or have I completely mis-understood the process?

I will try your BASS_Encode suggestion to write an encoded WAV file to disc from an original PCM.

[soundgals]

I think I was incorrecly assuming that…
BASS_SetConfig(
                           DWORD(BASS_CONFIG_DSD_FREQ),
                           DWORD(88200)
                       )

Is equivalent to BASS_Init(freq), which, of course, it's not. Also it's working up to the point where longer tracks become silent. So the process I'm using itself must be correct.

[soundgals]

Unfortunately, adding this line…

Code: [Select]

BASS_Encode_Start(stream, "bassenc.wav", BASS_ENCODE_PCM | BASS_ENCODE_AUTOFREE, 0, 0);
… as you suggested for writing to a file, fails to return a handle. It just returns "0"

Here is how I've written it in Swift following the code I posted yesterday…

Code: [Select]

var encoder = BASS_Encode_Start(stream, "~/Documents/output.wav", DWORD(BASS_ENCODE_PCM | BASS_ENCODE_AUTOFREE), nil, nil)
        if encoder == 0{
            print("Can't start the encoder")
        }
Any ideas why this is not returning the handle?

[Ian @ un4seen]

I'm a bit confused though, when you say,

The BASS_Init "freq" parameter doesn't affect DSD decoding (or decoding in general

…

… I believed the decoding took place as a result of this line…

let stream = BASS_DSD_StreamCreateURL(theURL, 0, DWORD(BASS_SAMPLE_FLOAT | BASS_STREAM_DECODE), nil, nil, DWORD(freq))

… where the pcm freq I want the DSD to be decoded to is specified. I can just leave this at 0 and the freq specified in BASS_Config will be used as in…

BASS_SetConfig(
                           DWORD(BASS_CONFIG_DSD_FREQ),
                           DWORD(88200)
                       )
In fact I call this in the calling function, because I want to change the PCM frequency, depending on the sample rate of the original DSD. If it were a DSD128, I may prefer 176400 as the PCM rate, for example.

Yes, the BASS_DSD_StreamCreateURL "freq" parameter (not the BASS_Init "freq" parameter) will determine the DSD stream's PCM conversion rate, and if that's 0 then the BASS_CONFIG_DSD_FREQ setting is used. Unfortunately, it isn't possible to change an existing DSD stream's conversion rate, so you would need to recreate the stream with a different "freq" parameter to do that.

Unfortunately, adding this line…

Code: [Select]

BASS_Encode_Start(stream, "bassenc.wav", BASS_ENCODE_PCM | BASS_ENCODE_AUTOFREE, 0, 0);
… as you suggested for writing to a file, fails to return a handle. It just returns "0"

Please call BASS_ErrorGetCode after that to get the error code.

[soundgals]

Ok, that's giving me error code 33 (BASS_ERROR_CREATE)

[Ian @ un4seen]

I don't think "~" is valid outside of a shell? Please try a path without that.

[soundgals]

Thanks; but I already tried without the tilde character. There is obviously a problem with where the system thinks I'm trying to write the file too. I've tried the same code that I use elsewhere in my app, to write to the user's documents folder; but for some reason that code is not working within this function. I tried writing to my app's bundle resources. This time I got a handle correctly returned by BASS_Encode_Start; but no file was written.

I'm trying to figure it out.

[Ian @ un4seen]

Is BASS_Encode_Start creating an empty file or no file at all? If the call succeeded then there should at least be an empty file. Data will be written to the file as its decoded/processed by the source stream, so you will still need the BASS_ChannelGetData loop.

If it still doesn't seem to be working, please try building and running the CONVERT example included in the BASSenc package to see if you have a problem with that too. If you copy the BASSDSD library into the same folder, it can be used to convert DSD files, so you can check if you get the silence with that too.

[soundgals]

Thanks Ian.

BASS_Encode_Start will not create a file for me. I get the error code 33, if the file doesn't exist.

If I create a file manually, with the same path as the one used in the BASS_Encode_Start call, I get error 2, to indicate the file could not be opened.

I can create and write to files on MacOS, using the usual foundation calls. So the problem seems to be limited to BASS_Encode.

I realised I still need the get_Channel_Data call, to get the data to write to the file. I was adding the BASS_Encode_Start call after that.

I'll try running the example you suggested.

[soundgals]

please try building and running the CONVERT example included in the BASSenc package to see if you have a problem with that too

I tried and I'm getting an error message with that too. "Couldn't start recording". I'm not sure if I'm going about it the right way though. If I tried adding the DSD library it crashed. So I tried a conversion from WAV to MP3.

[Ian @ un4seen]

"Couldn't start recording" sounds a lot like you're building the RECTEST example

Please try the CONVERT example instead, as it doesn't require recording and includes support for add-ons (eg. BASSDSD). Note it should to be run in a Terminal window, ie. open a Terminal window in the build folder.

[soundgals]

I knew I must be doing something obviously wrong. I opened the bassenc.xcodeproj and expected to be able to run all the examples from there.  I can't see how to choose "convert" from the interface. Or is that a command line tool only?

Just wondering also if anyone has tested using BASS_Encode_Start to write to a file on MacOS? It doesn't appear to be possible.

Something else I'd like to try is writing to STDOUT instead of a file and extracting the data from there. Is that possible, and if so, could you help me figure out how to do it?

Thanks again.

[Ian @ un4seen]

Yes, the CONVERT example is a command-line tool (you give it the source filename in the command-line), so you need to run it in a Terminal window (right-click on the build folder in Finder and choose "New Terminal at Folder"). That example (and BASS_Encode_Start in general) is working fine here, so will hopefully do so there too. Remember to put libbassdsd.dylib in the same folder to enable DSD support.

[soundgals]

Thanks Ian. Can you tell me if you're running it on Apple Silicon, or an Intel Mac? Also which version of xCode? If it needs to be built for Rosetta, I can't do that with xCode 15. I think the last version that supported Rosetta was 14.2 or 14.3

[soundgals]

At the moment, whatever I try to do with the command line tool "convert", I get the following response on my MacBook Air M1:

"bad CPU type in executable: convert"

I will try on my wife's Intel Mac over the weekend.

Can you just give me an example of the command line to enter in the terminal, to convert a dsf file to wav or point me to the documentation. It's not obvious to me from looking at the code.

[Ian @ un4seen]

At the moment, whatever I try to do with the command line tool "convert", I get the following response on my MacBook Air M1:

"bad CPU type in executable: convert"

That sounds like the arm64 architecture wasn't included in the build. What do you see if you run "file convert" in the same Terminal window? The example Xcode project should automatically include "arm64" on such a system, but you could try disabling the "Build Active Architecture Only" setting if not.

Can you just give me an example of the command line to enter in the terminal, to convert a dsf file to wav or point me to the documentation. It's not obvious to me from looking at the code.

You only need to provide a filename to have it converted to WAV, so something like this:

Code: [Select]

./convert filename.dsf


[soundgals]

Thanks Ian. I tried disabling "Build Active Architecture Only"; but that didn't help. I think there's something fundamental I'm missing on how to build and test the BASSEnc examples.

The only app that does not show in red in xCode is the Cast.app and that is the only one that builds. See the attached screenshot.

[Ian @ un4seen]

I believe that red colouring just means that those files don't exist, ie. they haven't been built yet. Do you have "convert" selected as the build scheme in Xcode, as described here?

   https://developer.apple.com/documentation/xcode/building-and-running-an-app

If so and it's failing to build then please post the error message from Xcode to have a look at.

[soundgals]

Thanks Ian. Of course I realise those targets in Red haven't been built. Funnily enough, although I've built projects containing multiple targets before, I've never had a situation where they didn't all build automatically and I had to select one in the build scheme, before it would build.

When I do that, unfortunately "convert" crashes.

I get the feeling it still doesn't want to build on Apple Silicon architecture. Do you know for sure if convert should build on Apple Silicon, or would it have to run under Rosetta?

I believe Apple stopped supporting builds for Rosetta after version 14.3 of xCode and I'm on 15.x.x.

Perhaps that's the problem?

[soundgals]

… wait a minute, I am finally getting somewhere here. Despite the crash, convert did build. I needed to go to the location where the target was built and manually add the "libbassdsd.dylib" alongside it there. I needed to go through Apple's security checks, before it would run "libbassdsd.dylib" code to convert form DSF to wav. I also needed to run convert using "sudo" in the terminal.

It has produced a wav file, and that wav file plays to the end. Unlike the wav file produced when I used "BASS_ChannelGetData" in my code.

The differences between the two WAVs, is that my code produces 88.2/24 bit data, whereas the convert command is converting to only 16bit.

Could that be a clue?

If so, is there a way to specify that a 24bit wav file shouild be produced, using the convert command, for a further more, like with like, comparison?

Actually with my own code I produce 32f wav data, as I had problems trying to produce a 24bit wav.

[Ian @ un4seen]

Good to see that progress is being made. The CONVERT example doesn't include a 24-bit WAV option, but that can be done by adding the BASS_SAMPLE_FLOAT and BASS_ENCODE_FP_24BIT flags to these lines:

Code: [Select]

...
chan = BASS_StreamCreateFile(FALSE, argv[filep], 0, 0, BASS_STREAM_DECODE | BASS_SAMPLE_FLOAT);
...
encoder = BASS_Encode_Start(chan, files[enc], BASS_ENCODE_PCM | BASS_ENCODE_FP_24BIT, NULL, 0);
...

I suspect the issue you're having with your code is in how the floating-point sample data from BASS_ChannelGetData is converted to 24-bit, which I guess is done in the "regularData" function?

[soundgals]

Thanks again Ian,

Adding the BASS_SAMPLE_FLOAT and BASS_ENCODE_FP_24BIT flags, has correctly produced a 24bit file which plays to the end.

Now I need to achieve the same in my code, to obtain the 24bit conversion in a data variable.

You were correct, in my code the sample data from BASS_ChannelGetData is converted to 24-bit, or rather 32-bit (padded with zeros) in the regularData function.

Actually, it would be good if I could get the converted data in flac format. I see there's an option for this with convert. So I basically would like to harness the power of the "convert" command to convert DSF to flac; but again in the form of a data variable rather than a file.

Depending on the sample rate I use to convert DSD to PCM and the size of the original file, I could find myself going over the 4gb WAV limit, whereas flac has no such limit.

[Ian @ un4seen]

You can use the BASSenc_FLAC add-on to encode to FLAC without writing a file. The function of interest is BASS_Encode_FLAC_Start, which takes a callback function that receives the encoded data. Note FLAC doesn't support floating-point, so you would need to use the BASS_ENCODE_FP_24BIT flag in that call (like in the BASS_Encode_Start call above).