Author Topic: Bass FFT on Linux-ARM64 Not Working  (Read 567 times)

digitalhigh

  • Posts: 14
Bass FFT on Linux-ARM64 Not Working
« on: 18 Feb '22 - 20:18 »
Sorry for the vague subject, but I swear, I'll provide more information. :P

So, I've got this project, "Glimmr". It takes and reads audio or video data and then uses that information to drive various lighting devices.

Glimmr is written in dotnet, and I'm using the ManagedBass wrapper.

For the most part, I'm using Bass to grab audio from a USB HDMI input device, but also potentially any device the user selects.

From there, do a FFT on every update of the audio data, do some magic to come up with color values, and spit them out to the lights.

This works fine on both Windows and Linux-ARM.

But, since porting my app to run on ARMF Raspbian (x64), I'm discovering that something is just...wrong. I can use arecord to capture the audio stream and verify that it's getting into the pi and not distorted or anything. But when I try using Bass to do the FFT and generate colors, it seems like it's just getting random noise. Some FFT amplitudes will come back as all "max" - even when the audio is stopped. The end result is the lights just randomly flickering on and off, versus a sweet sweet light show.

Edit: I can do Bass.GetRecordDeviceInfo and view that my selected device is enabled and get no errors as well...

My code is here:

https://github.com/d8ahazard/glimmr/blob/master/src/Glimmr/Models/ColorSource/Audio/AudioStream.cs

I know it's a long shot, but if anybody out there knows what the heck I'm doing wrong or how I can make this work right, I'd be so happy. I'm now on like day 3 of trying to sort this, and I'm out of ideas.

I've tried changing the sample rate. Anything other than 48000 does *nothing*. I've tried changing the channel count (stereo vs. mono), the FFT rate/size, adding various run and config flags, changing the period with which the callback fires...nothing seems to help, and everything on Windoze just works. The end result on the Pi 64 is just random, flickering colors.

Tia.

« Last Edit: 18 Feb '22 - 20:38 by digitalhigh »

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #1 on: 22 Feb '22 - 12:07 »
To perhaps narrow down what/where the problem is, please see if you can reproduce it with either the SPECTRUM or LIVESPEC examples included in the BASS package. You can build them by extracting the Linux BASS package into a folder, extracting the ARM Linux BASS package into the same folder (overwriting makefile.in), and then opening the folder in Terminal and entering "make spectrum livespec".

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #2 on: 22 Feb '22 - 14:33 »
To perhaps narrow down what/where the problem is, please see if you can reproduce it with either the SPECTRUM or LIVESPEC examples included in the BASS package. You can build them by extracting the Linux BASS package into a folder, extracting the ARM Linux BASS package into the same folder (overwriting makefile.in), and then opening the folder in Terminal and entering "make spectrum livespec".

I'll give that a try today when time allows, report back.

In the meanwhile, one thing I'm noticing is that if I check the data_available amount after grabbing FFT data, it still seems to go up much faster than I can read it. Do you have any advice on making the record buffer as "realtime" as possible?

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #3 on: 22 Feb '22 - 17:12 »
You only really need to be concerned with the amount of buffered data when not using a RECORDPROC callback function. That's because the BASS_ChannelGetData calls remove data from the buffer then, so that each byte is processed exactly once. When a RECORDPROC is used, the RECORDPROC receives the data for processing, and BASS_ChannelGetData can simply always use the most recent data (even if that means receiving some data multiple times or not at all).

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #4 on: 23 Feb '22 - 17:46 »
You only really need to be concerned with the amount of buffered data when not using a RECORDPROC callback function. That's because the BASS_ChannelGetData calls remove data from the buffer then, so that each byte is processed exactly once. When a RECORDPROC is used, the RECORDPROC receives the data for processing, and BASS_ChannelGetData can simply always use the most recent data (even if that means receiving some data multiple times or not at all).

So, I think I'm starting to sort this out, based on the spectrum sample apps...albeit slowly.

A few more questions, and hopefully I'll be in better shape:

Looking at the documentation for the various flags that can/should be set for RecordInit and ChannelGetData is a bit tough to decipher. I see mention that for recordInit, if I want to use the Float param, I also need to use the Decode flag if not on windows. Is that correct?

Assuming then that everything is floatwise from init, do I also need to do FFT1024 | Float in ChannelGetData?

Once I have those two sorted...my next big issue is just trying to convert the FFT amplitudes to a human-readable number...ideally within a 0-255 range.

One of the things I was seeing initially with the "flashing" seems to be that sometimes, the FFT data returns all float.NaN or float.Infinity values. Can you explain this behavior? I started using Bass.ChannelGetLevel to determine if the overall volume is < 1000 and only trying to process then, but I feel like I'm ultimately missing something as to how I should be doing this on a "frame-by-frame" basis in realtime.

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #5 on: 24 Feb '22 - 14:36 »
Looking at the documentation for the various flags that can/should be set for RecordInit and ChannelGetData is a bit tough to decipher. I see mention that for recordInit, if I want to use the Float param, I also need to use the Decode flag if not on windows. Is that correct?

BASS_RecordInit doesn't take any flags, so I assume you mean BASS_RecordStart? If so, that also doesn't use the BASS_STREAM_DECODE flag. The BASS_STREAM_DECODE flag only applies to playback streams/etc, but the recording equivalent is to not use a RECORDPROC callback, ie. in both cases the channel's data is processed with BASS_ChannelGetData. It doesn't affect the availability of floating-point support in either case.

Assuming then that everything is floatwise from init, do I also need to do FFT1024 | Float in ChannelGetData?

FFT data is always floating-point regardless of the source channel's format. Except on old ARM without FPU (ie. the "softfp" version), where the BASS_DATA_FIXED flag can be used for 8.24 fixed-point instead.

One of the things I was seeing initially with the "flashing" seems to be that sometimes, the FFT data returns all float.NaN or float.Infinity values. Can you explain this behavior?

No, that shouldn't happen. Were those values before or after you had done some processing of the data? If unsure, please check before that processing, to see whether it may be introducing them. Do you ever see these values from the "hardfp" BASS version, or only the "aarch64" version?

Btw, I think there's a problem with the FftIndex2Frequency function in the code you posted. The "length" value appears to be half the FFT size, so it should be like this:

Code: [Select]
return 0.5f * index * sampleRate / length;

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #6 on: 24 Feb '22 - 15:19 »
BASS_RecordInit doesn't take any flags, so I assume you mean BASS_RecordStart? If so, that also doesn't use the BASS_STREAM_DECODE flag. The BASS_STREAM_DECODE flag only applies to playback streams/etc, but the recording equivalent is to not use a RECORDPROC callback, ie. in both cases the channel's data is processed with BASS_ChannelGetData. It doesn't affect the availability of floating-point support in either case.

FFT data is always floating-point regardless of the source channel's format. Except on old ARM without FPU (ie. the "softfp" version), where the BASS_DATA_FIXED flag can be used for 8.24 fixed-point instead.

Noted. I'll discard the FLOAT flags and the DECODE flag on recordStart and ChannelGetData.

No, that shouldn't happen. Were those values before or after you had done some processing of the data? If unsure, please check before that processing, to see whether it may be introducing them. Do you ever see these values from the "hardfp" BASS version, or only the "aarch64" version?

I don't currently have a non-x64 Pi OS flashed - that's on my list of stuff to do today. However, I'm like 95% sure this never happened on the hardfp version, nor does it happen on my windows test(s). It only seems to be happening on aarch64.

Btw, I think there's a problem with the FftIndex2Frequency function in the code you posted. The "length" value appears to be half the FFT size, so it should be like this:

Code: [Select]
return 0.5f * index * sampleRate / length;

Thanks, I'll fix this up.


ALSO, I was *finally* able to get Xming working so I could run the livespec test application.

I had to update the input device to "1", (My USB capture device); as well, I adjusted the sample rate to 48000 and channels to 2, as that's the only setting I can use in aRecord without it spitting out an error.

Even after making these adjustments (and trying it with the default settings), the only thing the livespec window ever shows is "Make some noise". I tried decreasing the threshold of the quiet check from 1000 to 500, and still no change.


So, now I'm just even more confused. At the very least, if I do the "quiet check" in my code at the 1k level, I can definitely detect "something" when I start/stop playback...but the livespec app doesn't even get that far?

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #7 on: 25 Feb '22 - 00:06 »
@Ian - One more update.

Rebuilt on non-x64 raspbian, made the recommended tweaks, everything is working more or less as expected.

On the x64 version, it's what I originally described. Incredibly low amplitudes or none at all, and the INFINITY/NaN values persist.


So, on one had, for now at least, I'll need to revert my "main" target platform to x86 ARM on the pi.

On the other, I've more or less improved my LED visualization 100% versus what I was doing before, and I'm definitely more versed in how Bass works.



Let me know if there's anything I can do to help with the x64 version. It would definitely be ideal to figure out what's wrong and how to fix it. ;)

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #8 on: 25 Feb '22 - 17:46 »
Ah! I see what it is now. There's an incorrect "rbit" instruction in the FFT initialization in the aarch64 Linux version (other platforms/archs are unaffected). Here's an update to fix that:

   www.un4seen.com/stuff/libbass.so

Let me know if you do still see the problem happening.

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #9 on: 25 Feb '22 - 20:01 »
Ah! I see what it is now. There's an incorrect "rbit" instruction in the FFT initialization in the aarch64 Linux version (other platforms/archs are unaffected). Here's an update to fix that:

   www.un4seen.com/stuff/libbass.so

Let me know if you do still see the problem happening.

Swapped out the lib, tested...same thing, more or less. Still working fine with the same code on the x86 version...

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #10 on: 28 Feb '22 - 12:11 »
That's strange. Please check the BASS_GetVersion return value (it should be 0x02041011) to confirm that the updated library is being loaded, as I'm not seeing any problem here when using it with the SPECTRUM example. Is the SPECTRUM example working OK with it there?

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #11 on: 1 Mar '22 - 16:22 »
That's strange. Please check the BASS_GetVersion return value (it should be 0x02041011) to confirm that the updated library is being loaded, as I'm not seeing any problem here when using it with the SPECTRUM example. Is the SPECTRUM example working OK with it there?

I'm getting this version number:

Using bassver: 2.4.16.17

Edit: x86 reports  Using bassver: 2.4.15.0
« Last Edit: 1 Mar '22 - 16:28 by digitalhigh »

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #12 on: 1 Mar '22 - 17:08 »
That is the correct version number, so it looks like the update is being loaded. Are you sure you're still getting Infinity/NaN values in the FFT data, or is it a different issue now?

Please try running the SPECTRUM example and see if that works. If you're having trouble building it then perhaps you don't have gtk2 installed - you can install that by running: sudo apt-get install libgtk2.0-dev

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #13 on: 1 Mar '22 - 18:02 »
That is the correct version number, so it looks like the update is being loaded. Are you sure you're still getting Infinity/NaN values in the FFT data, or is it a different issue now?

Please try running the SPECTRUM example and see if that works. If you're having trouble building it then perhaps you don't have gtk2 installed - you can install that by running: sudo apt-get install libgtk2.0-dev

I will check. I pulled back logging from the audio stuff temporarily so I wasn't filling up massive logs of FFT values.

I had the spectrum compiled and running before - I just deleted the directory recently to ensure I only had *one* libbass.so anywhere in the system. Building again shouldn't be an issue.

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #14 on: 1 Mar '22 - 20:56 »
That is the correct version number, so it looks like the update is being loaded. Are you sure you're still getting Infinity/NaN values in the FFT data, or is it a different issue now?

Please try running the SPECTRUM example and see if that works. If you're having trouble building it then perhaps you don't have gtk2 installed - you can install that by running: sudo apt-get install libgtk2.0-dev

No infinity/NaN values now, just...nothing. It doesn't seem to be grabbing the input. Or, more specifically, channel.getlevel is returning 0, and the FFT values are also always 0.

Same with the livespec - no visualizations, only the prompt to "make some noise".

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #15 on: 2 Mar '22 - 12:21 »
That sounds like the issue is in the recording, ie. it's capturing nothing/silence. To confirm that the FFT is fine now, please try running the SPECTRUM example rather than the LIVESPEC example, as it gets the sound from a file rather than a recording device.

If the SPECTRUM example is working fine then next try building and running the DEVLIST example to see what recording devices BASS is detecting. Do you see your target device in the list? If so, check that you're using its device number in your BASS_RecordInit call.

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #16 on: 2 Mar '22 - 13:41 »
That sounds like the issue is in the recording, ie. it's capturing nothing/silence. To confirm that the FFT is fine now, please try running the SPECTRUM example rather than the LIVESPEC example, as it gets the sound from a file rather than a recording device.

If the SPECTRUM example is working fine then next try building and running the DEVLIST example to see what recording devices BASS is detecting. Do you see your target device in the list? If so, check that you're using its device number in your BASS_RecordInit call.

I would have to agree with you there. I'll build and test the spectrum app with a file shortly, same with DEVLIST.

However, in my code, I'm already enumerating every available/enabled audio device in realtime using BassRecordDeviceInfo whenever I refresh system data, so I'm pretty confident that the device is detected, and I'm using the right device number (1) in both my code's RecordInit call, as well as in livespec.



And, again, I've tested direct capture from the sound card using Arecord, which works as expected.

Will test with Spectrum and devlist and report back, just for sanity purposes...but I feel like if the channel level isn't detecting anything, the problem is with the lib getting the stream now...

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #17 on: 3 Mar '22 - 13:48 »
That sounds like the issue is in the recording, ie. it's capturing nothing/silence. To confirm that the FFT is fine now, please try running the SPECTRUM example rather than the LIVESPEC example, as it gets the sound from a file rather than a recording device.

If the SPECTRUM example is working fine then next try building and running the DEVLIST example to see what recording devices BASS is detecting. Do you see your target device in the list? If so, check that you're using its device number in your BASS_RecordInit call.

SPECTRUM works now, so does DEVLIST. Livespec using the target device "1", does not.

Input Devices
0: Default
        driver: default
        type: Unknown
        flags: enabled default (3)
1: MS2109: USB Audio
        driver: hw:3,0
        type: Unknown
        flags: enabled (1)

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #18 on: 3 Mar '22 - 15:46 »
OK. Next please try building and running the RECTEST example, and see if you can capture sound with that in any of the available sample formats. Also note whether the position display is advancing, ie. whether it's capturing anything.

Please also post the output from the working arecord after adding "--dump-hw-params" to its command-line.

digitalhigh

  • Posts: 14
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #19 on: 3 Mar '22 - 16:23 »
OK. Next please try building and running the RECTEST example, and see if you can capture sound with that in any of the available sample formats. Also note whether the position display is advancing, ie. whether it's capturing anything.

Please also post the output from the working arecord after adding "--dump-hw-params" to its command-line.

It's...all just working now.

IDK what is going on. I had originally blacklisted the default onboard sound card. Un-did that in order to test the spectrum app and rebooted.

Tried to test arecord for you, and noticed I had no sound. Went into alsamixer, enabled "capture" for the USB device. Arecord worked. Livepec now works. And my app now works.

So, I guess now my task is to figure out what the necessary combination of parameters is on the pi itself, and how to make that the "default" settings on a new system.

Sorry for taking up so much of your time with an issue that now appears to be just me being dumb.

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #20 on: 4 Mar '22 - 14:20 »
All's well that ends well :)

recliq

  • Guest
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #21 on: 21 May '22 - 09:04 »
Hi there,

I'm reviving this old thread because I had the same problem: No usable values in FFT bins...
I used the libbass.so for AARCH64 from the provided package here: http://www.un4seen.com/files/bass24-linux-arm.zip
The Version it reports is: 0x02041007

After I replaced it with the version from this thread everything works as expected.
The version this library reports is: 0x02041011

I used the library on a Raspberry Pi 3B+ running Debian 11 (bullseye) with Kernel  5.15.32-v8+ aarch64.

I think the ARM package on the main download site should be updated.  ;)

Anyways, awesome library!

Ian @ un4seen

  • Administrator
  • Posts: 24418
Re: Bass FFT on Linux-ARM64 Not Working
« Reply #22 on: 23 May '22 - 13:29 »
There should be a new BASS release coming soon (on all platforms), but I have now moved the update posted above into the ARM Linux package for the meantime.