Author Topic: Implementing Sidetone  (Read 1166 times)

chuckd

  • Posts: 67
Implementing Sidetone
« on: 10 Jul '14 - 04:05 »
We've tried to implement Sidetone (feeding back real time mic audio to the earphones) by using an audio stream with no buffering. Every time our record callback is called, we immediately push the packet (adjusted in volume first), to the audio stream for playback.  This is being done on IOS.  We are also implementing the callback time parameter at 5 ms, which is about as fast as IOS will let us go without problems.

However, that ugly word 'latency' creeps in here, and it's fairly noticeable.

I've seen some comments around about Core Audio being able to implement a fairly real-time Sidetone path.  The cell phone function of IOS devices seem to use this feature. Do you know if this is possible? And if so, any way to get BASS to tie in?  Perhaps this is a function of the internal voice processing unit?

Ian @ un4seen

  • Administrator
  • Posts: 20400
Re: Implementing Sidetone
« Reply #1 on: 10 Jul '14 - 14:03 »
How much latency are you getting? One possible cause of latency is the output "push" stream having too much buffered data. For example, if the recording is started some time before the output is, then there may be a build-up of data before the output starts. You can check how data the push stream has buffered in the BASS_StreamPutData return value. The playback buffer level (BASS_ChannelGetData + BASS_DATA_AVAILABLE) would normally be added to that, but that's not necessary if playback buffering is disabled (via BASS_ATTRIB_NOBUFFER), which it should be in this case.

Do you need to call BASS_ChannelGetData/Level on the recording channel, eg. for a level display? If not, you may get lower latency by dropping the RECORDPROC function and replacing the push stream with a normal custom stream with a STREAMPROC function that fetches the data from the recording channel. For example, something like this...

Code: [Select]
BASS_SetConfig(BASS_CONFIG_DEV_BUFFER, 5); // request a 5ms device buffer (affects recording device too on iOS)
BASS_Init(2, 44100, 0, 0, 0); // initialize "voice" output
BASS_RecordInit(-1); // initialize recording
BASS_RECORDINFO ri;
BASS_RecordGetInfo(&ri); // get recording device info (for sample rate)
input=BASS_RecordStart(ri.freq, 1, BASS_RECORD_PAUSE, 0, 0); // setup recording (paused) without a RECORDPROC
output=BASS_StreamCreate(ri.freq, 1, 0, StreamProc, (void*)input); // create output stream
BASS_ChannelSetAttribute(output, BASS_ATTRIB_NOBUFFER, 1); // disable playback buffering on it
BASS_ChannelSetAttribute(output, BASS_ATTRIB_VOL, 0.05); // set its volume level
BASS_ChannelPlay(input, FALSE); // start the recording
BASS_ChannelPlay(output, FALSE); // start the output

...

DWORD StreamProc(HSTREAM handle, void *buffer, DWORD length, void *user)
{
DWORD input=(DWORD)user;
DWORD got=BASS_ChannelGetData(input, buffer, length); // get data from the recording
if (got==(DWORD)-1) return BASS_STREAMPROC_END; // failed, end the stream
return got;
}

chuckd

  • Posts: 67
Re: Implementing Sidetone
« Reply #2 on: 10 Jul '14 - 17:45 »
I'm pretty much doing the same thing you're showing, except with a record proc, and I'm pushing to the playback stream in the record proc.  It is called at different intervals with either 946 bytes (473 samples) or 1102 bytes (551 samples) at 44100.  That's at least 10.7 ms or 12.4 ms.  We measured the actual mic-to-earpiece latency, and it's about 28ms.  That's enough to drive 'some' people a bit crazy.

I know there's a slight latency in the phone itself on playback, so this seems about normal.  This is on an iPhone 5, by the way.

I didn't know if the Core Audio loopback with the voice unit was real or people are just making that up.  I've seen a few claims of less than 1ms latency using this mode.  I figured if anybody knew that this existed, it would certainly be you!


Ian @ un4seen

  • Administrator
  • Posts: 20400
Re: Implementing Sidetone
« Reply #3 on: 11 Jul '14 - 16:30 »
No, I'm afraid I'm not aware of any way to get as little as 1ms latency. Where did you see that mentioned? My guess is that "Core Audio loopback" would involve forwarding the data from the input side of the unit to the output side, like you are already doing? Removing the RECORDPROC (as in the code above) should reduce the latency a bit when doing that, as the output then takes the data directly from the recording buffer (in the STREAMPROC). A RECORDPROC can introduce a bit of extra latency because it is called asynchronously (each RECORDPROC has its own thread).

chuckd

  • Posts: 67
Re: Implementing Sidetone
« Reply #4 on: 11 Jul '14 - 18:23 »
I can't seem to find the web reference I found, but in the book 'Learning Core Audio: A Hands-On Guide to Audio Programming for Mac and IOS', it mentioned that when using audio units, no perceivable latency is noticed (end of chapter 7).  They're stressing the advantage of audio unit programming vs buffered/queued concepts.

I just glazed over this, since I really don't want to dig down into Core Audio programming if I can help it.  The other references I had seen mentioned that you could implement the audio loopback in an audio unit with absolutely no noticeable delay.  Admittedly, I haven't looked into how this is exactly accomplished.