Topic: Occasional garbled audio on iOS

[saga]

We've encountered some really odd behaviour with the audio output of our app on iOS. I'm not really sure how we could only notice now, because I can even reproduce it with older versions of our app, so maybe the problem started happening with an iOS update (although my iPad is on 15.4.1, so I didn't update it very recently at least). Updating BASS and BASSMIX to the latest "stuff" versions also didn't help.

Anyway, on to what is actually happening: Once our app has been outputting sound for a while (it could be anything from 10 to 40 minutes), the output becomes garbled - it sounds like there are dropouts and earlier audio frames are being repeated. So for example if there is some speech signal, you can hear the words being played at normal speed, but with a lot of crackling, and then half a second later or so you hear the same word again, still garbled. After about half a minute or so the audio output becomes stable again.

Has anyone else encountered something similar? Based on the error description, can you identify what the issue could be, Ian?

[Ian @ un4seen]

Does the problem depend on what you're playing (eg. a type of stream), or does it happen regardless?

To narrow-down what/where the problem is, please try writing the output to a WAV file (using BASS_Encode_Start + BASS_ENCODE_PCM) and see if the problem is present in that. If you're playing a single stream then you can set the WAV writer on that. If that WAV sounds fine, or if you're playing multiple streams simultaneously and the problem is affecting all equally (ie. not any individually), then set a WAV writer on the BASS output mix like this:

Code: [Select]

mixstream = BASS_StreamCreate(0, 0, 0, STREAMPROC_DEVICE, 0); // get output mix stream
wavwriter = BASS_Encode_Start(mixstream, wavfilename, BASS_ENCODE_PCM | BASS_ENCODE_QUEUE, 0, 0); // set WAV writer on it


[saga]

We're mixing everything into one global mixer, everything coming out of that mixer is affected. I'll try to create a WAV recording as suggested.

[saga]

As expected, the issue is not in the encoded data. I have uploaded an example recording to the FTP as ios-output-issue.zip. "bass output.flac" shows what's being played, and "speaker output.flac" is what I recorded with a microphone from the iPad's speaker.

[Ian @ un4seen]

Was the "bass output.flac" recording taken from the mixer or STREAMPROC_DEVICE stream? If the former, please try getting a recording form the latter too. The STREAMPROC_DEVICE stream recording will show what's being given to iOS, so if the problem isn't present in that then it would seem to be introduced afterwards by iOS. In that case, does it go away if you call BASS_Stop/Pause and BASS_Start again? If that doesn't fix it, does BASS_Init with BASS_DEVICE_REINIT help? Also, does it only happen when using the "voice" output device and/or a specific BASS_CONFIG_IOS_SESSION setting?

[saga]

- The recording was done with the STREAMPROC_DEVICE code you provided above.
- BASS_Pause/BASS_Start does not appear to help, either.
- It doesn't matter if the Default or Voice output device is used.
- I haven't been able to do the BASS_DEVICE_REINIT test yet, that will follow next, together with removing other iOS session parameters.

It could be coincidence, but the moment I switched to the iOS task switcher, the audio immediately stabilized again in my latest test.

[saga]

Update: BASS_DEVICE_REINIT also didn't help. But once again the artifacts went away immediately when I "grabbed" the application with the 4 finger gesture to show the app overview. Really strange.

[Ian @ un4seen]

Strange indeed. If you set a DSPPROC callback on the DEVICE stream (via BASS_ChannelSetDSP), and have that log the "length" parameter and a timestamp, does that show the "length" parameter and/or callback interval changing when the problem begins and ends?

Also, are you able to reproduce the problem on multiple devices, and if so, are they running the same iOS version? Are there any devices that you've tried where it isn't happening, and are they running a different iOS version?

[saga]

We have been able the reproduce the problem on iPads of various generations. At the very least iOS 15.4 and 15.5 are affected. A colleague tested on iOS 14.8 for a while and didn't encounter the issue, but since it is rather random in nature, that doesn't have to mean anything. I haven't been able to test without the session flags yet, but I will try to do that tomorrow and also check the callback lengths.

Now that I toyed around with the 4 finger gesture to open the app overview, I also noticed that audio becomes unstable in a similar way when switching to other apps (such as the iOS control panel). It's nowhere near as extreme, so it's hard to tell if it's just some missing frames or if they are reordered similarly to what can be heard in the recording.

[saga]

I was now also able to confirm that removing our custom BASS_CONFIG_IOS_SESSION config didn't help. I wasn't able to reproduce the issue yet with the DSP callback length logging, but that will hopefully happen tomorrow.

[saga]

Strange indeed. If you set a DSPPROC callback on the DEVICE stream (via BASS_ChannelSetDSP), and have that log the "length" parameter and a timestamp, does that show the "length" parameter and/or callback interval changing when the problem begins and ends?

The DSP callback length stays at 8192 at all times.

[Ian @ un4seen]

OK. Did the time interval between callbacks stay the same too?

Have you so far only reproduced the problem in your main app? If so, can you try reproducing it in a simpler test app, to see if UI/other stuff may be triggering it somehow? For example, does it happen if you just open and play a file in a loop (with BASS_SAMPLE_LOOP set) without any UI updates/etc? If a simple test isn't affected, you could then try temporarily disabling stuff in your main app to hopefully find what's triggering the problem.

I think it's unlikely to make a difference, but I'll also send you the latest dynamic library BASS build to try just in case. BASS and the add-ons will be switching from static to dynamic libraries on iOS (as on the other platforms) at the next BASS release.

[saga]

Trying to make sense of this again, as it's still happening in our app. I created a small test app which plays a continuous sine sweep, and BASS is set up in a similar way as in our app. While I cannot reproduce the "garbled" audio where it sounds like the order of some audio buffers is swapped randomly, I can observe on my iPad that it is really easy to interrupt the audio: If I switch away from the app, say, the iPad settings app, just randomly scrolling around in the settings and opening some settings views causes pops and dropouts to be heard regularly. I don't notice this when e.g. playing some audio via the web browser and then doing the same. It's almost like the audio thread doesn't have real-time priority, so whenever there is a bit of load on the CPU, it isn't able to update the audio quickly enough. I can observe that our app mostly has the garbled audio issue when the CPU load is high, so in that regard it's very similar to the test app I have attached.
Do you observe the same issue when running the attached app?

[Ian @ un4seen]

I haven't tried running that code yet, but I notice you're setting BASS_CONFIG_BUFFER the same as BASS_CONFIG_DEV_BUFFER. Instead of doing that, I would suggest disabling playback buffering (set BASS_ATTRIB_BUFFER=0) on the stream. That will mean the stream's processing is in sync with the final output processing, which should reduce the chance of buffer underruns, as well as minimizing latency.

[saga]

While setting BASS_ATTRIB_BUFFER=0 in the demo code seems to improve things, I noticed that we're already doing that on our global mixer stream in the app, and just removing the BASS_CONFIG_BUFFER setting doesn't seem to improve the situation. Interestingly, after trying this modification of the demo app, I did get some heavily garbled audio for a second when launching the settings app, but I was not able to reproduce it again!

However, I have just found another interesting data point in our app: We're setting the BASS_ATTRIB_MIXER_THREADS to 2, and forcing it to 1 appears to avoid the garbling. Another threading issue?

[Ian @ un4seen]

At first glance, I can't see a reason for BASS_ATTRIB_MIXER_THREADS>1 to cause such a problem, but I'll keep looking! Please let me know if you find the problem does actually happen even with BASS_ATTRIB_MIXER_THREADS=1 in the meantime. Also, in case it gives a clue, are you ever changing the BASS_ATTRIB_MIXER_THREADS setting during playback or only before playback?

[saga]

I'll keep investigating next week. For now, I can tell that the BASS_ATTRIB_MIXER_THREADS setting is only touched when creating the mixer and then left as-is.

[saga]

I don't have any concrete results yet, but can confirm once more that the audio isn't garbled when I write out the mixer's audio data, however the audio is definitely not continuous either (which might explain why we hear some buffers twice - maybe the OS is just playing the same data twice because the buffers haven't been updated in time). Do you see any reason why setting BASS_ATTRIB_MIXER_THREADS to 2 instead of 1 could amplify this situation? The only thing I could imagine right now is that the mixer threads don't run at a high-enough priority and thus having more of them makes the problem more pronounced when the CPU is already under load. I read somewhere that iOS doesn't yield to lower-priority threads if a high-priority thread is available, but that information is several years old, so I don't know how relevant it still is.

[Ian @ un4seen]

Good timing. I was just about to suggest setting a WAV writer on the mixer to see if the problem is present in the mixer output

I haven't been able to find an explanation for your issue in the code, so I suspect it is an update cycle taking too long due to lack of CPU, like you say. When playback buffering is disabled, the mixer's main processing thread will be a system audio thread with high priority, but the extra mixing threads (with BASS_ATTRIB_MIXER_THREADS>1) currently run at normal priority. Here's an update for you to try, which gives those threads a bit higher priority:

   www.un4seen.com/stuff/bassmix-ios.zip

Let me know whether it helps.

[saga]

The audio is definitely much better with this version! There are still a few dropouts while the app is under heavy load, but at least they don't end up being played back twice.

[Ian @ un4seen]

Good to see that helped. Here's another update to try, which raises the thread priority even higher (to same priority as BASS's update thread(s)) to see if that helps even more:

   www.un4seen.com/stuff/bassmix-ios.zip

[saga]

I don't think I can notice any changes with this version, neither good nor bad, but I did just notice that I was doing the tests with a Debug build of our app, and with a Release build the remaining dropouts are completely gone.

[Ian @ un4seen]

Nice. Are the dropouts gone now when using the previous BASSmix update (2.4.12.5) too?

[saga]

Yup, in a release build I cannot tell the two versions apart. Still, giving the mixing threads the same priority as update threads does seem like a logical thing to do, so I guess it makes sense to keep the changes from .6?

[Ian @ un4seen]

Yeah, on balance it's possibly best to leave it as it is now. One concern though is that it could perhaps cause issues elsewhere if multiple threads are used on non-realtime mixers, eg. mixing to file as quickly as possible (in a BASS_ChannelGetData loop) might hog the CPU in that case. A decoding mixer (with BASS_STREAM_DECODE set) could perhaps be treated differently, but BASSmix can't be sure that it won't ultimately be feeding output (eg. ASIO).