Topic: getting started with live effects (iPhone)

[arlomedia]

Hi folks,

I'm just getting started using BASS for an iPhone application that will add effects to real-time input and send it straight to the output (no file reading or saving). I already have a Remote I/O audio unit (aurio) set up that reads audio input and sends it to output. I was hoping to just hook into that data and process it as it comes through. Can I do this with BASS, or will I have to rework the application to use BASS's input and/or output?

If the latter, do you know of any sample code for something like this? I saw a reference to a "LIVEFX" sample on this forum, but the post was two years old and I don't see that in the code download now. Otherwise I'm trying to pull bits of the livespec, rectest and fxtest sample projects.

Thanks,
-Arlo

[arlomedia]

Okay, I just found the livefx sample in the Windows download ... it's not included in the Mac download I was looking at earlier.

I'm still curious whether I can/should use the Mac's Audio Units for I/O, or whether I should just use BASS for the whole thing? I'll try to get livefx up and running in the meantime.

[Ian @ un4seen]

It would probably be simpler to have BASS handle the input and output (eg. as in the LIVEFX example), but if you have already implemented that yourself, then you could keep that code and just have BASS apply the FX to it. That can be done with a "dummy" stream. The BASS calls could look something like this...

Code: [Select]

BASS_Init(0, 44100, 0, NULL, NULL); // initalize "no sound" device

...

fxstream=BASS_StreamCreate(44100, 2, BASS_STREAM_DECODE, STREAMPROC_DUMMY, NULL); // create dummy stream (adjust sample format to match data)
BASS_ChannelSetFX(fxstream, ...); // set FX on it

...

BASS_ChannelGetData(fxstream, data, length); // feed sample data through dummy stream to apply FX


[arlomedia]

I was able to set up an effects stream as described to manipulate the Audio Unit data as it comes through. However, I don't think I'm feeding the right data into the 2nd and 3rd arguments -- I'm just getting electronic noise back. Apple's data is packaged in an AudioBufferList structure and I haven't been able to pick that apart to give BASS_ChannelGetData what it needs. If anyone knows how to do this, please help! Otherwise I will post some code when I figure it out.

[arlomedia]

It turns out I was intercepting the right data after all. The problem was that I was testing with the BASS_FX_DX8_PARAMEQ effect, and that effect gives me nothing but static with the default settings (I'll look into that as a separate issue).

More to the point of this thread, here's what I found with modifying live I/O. I first tried handling the entire audio functionality with BASS, which was very easy to set up once I found the LIVEFX example in the Windows download. (I had to remove the DirectX version check and maybe another tweak or two and then the code was compatible.) However, on an iPhone that gave me about 1/2 second latency between input and output, which was a problem for my application. Ian said he would look into some ideas for reducing the latency.

Meanwhile, I changed my app to handle the audio I/O using Remote I/O, as in the aurioTouch sample project from Apple. I then set up a BASS dummy stream as Ian described above, and applied effects to it as in the LIVEFX sample. Finally, I added this line of code to the Remote I/O callback function, which provides the ioData and inNumberFrames variables:

BASS_ChannelGetData(THIS->fxstream, ioData->mBuffers[0].mData, ioData->mBuffers[0].mDataByteSize);

The first thing that confused me about this approach is what argument is the input and what is the output, or whether I should be using the result of the function call as output. As I learned, the second argument is both the input and the output ... if you set it to your input data, that just gets modified without explicitly putting it anywhere.

The other source of confusion was the looping in the callback function. It turns out that ioData contains multiple buffers -and- multiple frames. I found that the buffers represent channels while the frames represent samples. I didn't need to loop through the buffers because my application was mono, and I didn't need to loop through the frames because I could hand them all to BASS at once. So in the end this was deceptively simple.

Now I just need to get the effects to sound right...

[Darren]

I too am using mono samples, but am unable to set the PARAMEQ Parameters on a channel.

The code below is a snippet from my init method. The AU callback also has
BASS_ChannelGetData(THIS->fxstream, ioData->mBuffers[0].mData, ioData->mBuffers[0].mDataByteSize);

==
BASS_Init(0, 22050, 0, NULL, NULL); // initalize "no sound" device
fxstream=BASS_StreamCreate(22050, 2, BASS_STREAM_DECODE, STREAMPROC_DUMMY, NULL); // create dummy stream (adjust sample format to match data)
BASS_ChannelSetFX(fxstream, BASS_FX_DX8_PARAMEQ, 1); // set FX on it , 3rd param is the priority

BASS_DX8_PARAMEQ lowstop = { 500.0, 36.0, -15.0 };
   
BOOL bassError;
bassError = BASS_FXSetParameters(fxstream, &lowstop);
===

Everything compiles, but the FXSetParameters call returns false, and there is no discernible effect applied to the sound.

Any suggestions?

[Ian @ un4seen]

The BASS_FXSetParameters "handle" parameter should be the return value from the BASS_ChannelSetFX call. Also, the dummy stream's sample format should match the data that you will be feeding through it, eg. mono in this case. So your code should probably be modified something like this...

Code: [Select]

BASS_Init(0, 22050, 0, NULL, NULL); // initalize "no sound" device
fxstream=BASS_StreamCreate(22050, 1, BASS_STREAM_DECODE, STREAMPROC_DUMMY, NULL); // create dummy stream (adjust sample format to match data)
eqfx=BASS_ChannelSetFX(fxstream, BASS_FX_DX8_PARAMEQ, 1); // set FX on it , 3rd param is the priority

BASS_DX8_PARAMEQ lowstop = { 500.0, 36.0, -15.0 };
   
BOOL bassError;
bassError = BASS_FXSetParameters(eqfx, &lowstop);


[Darren]

Getting closer to a solution, but now I can't seem to find a combination of parameters that make the output NOT sound like it's buried in noise.

I've been experimenting with the echo effect, and though it is very plain to tell than an echo has been applied to the stream, BASS has also introduced a TON of noise as well. So much so, that the output is unusable.

For the PARAMEQ, setting the fGain to 0, effectively stops BASS from modifying the sound at all. But gain increments as small as 0.001 reintroduce noise into the output.

   fxstream=BASS_StreamCreate(22050, 1, BASS_STREAM_DECODE, STREAMPROC_DUMMY, NULL);

   HFX eqfx = BASS_ChannelSetFX(fxstream, BASS_FX_DX8_PARAMEQ, 1);
   BASS_DX8_PARAMEQ lowstop;   
   lowstop.fBandwidth = 12.0f;
   lowstop.fCenter = 440.0f;
   lowstop.fGain = -0.0f;
   BASS_FXSetParameters(eqfx, &lowstop);

   HFX echoFX = BASS_ChannelSetFX(fxstream, BASS_FX_DX8_ECHO, 2);
   BASS_DX8_ECHO echo1;
   echo1.fFeedback = 50.0;
   echo1.fWetDryMix = 50.0;
   echo1.lPanDelay = 0;
   echo1.fLeftDelay = 500.0;
   echo1.fRightDelay = 500.0;
   BASS_FXSetParameters(echoFX, &echo1);

Am I going in the right direction?

[arlomedia]

Darren, I had this problem too. As Ian pointed out to me, the default aurio code from Apple uses 8.24 format audio, while BASS uses 16-bit audio. I had to dig into the Apple code and find the audio unit setup and the stream description and change mBitsPerChannel to 16. This is also where you would change your sampling rate to 22050 if you haven't already. Anyway, once I got the two audio formats to match, the effects worked great for me.

If you're using the aurioTouch code, I can post more specific info.

[darren]

Thanks for the great tips so far.
Yes, arlomedia, I would love to see the changes you've discovered to be necessary to get the aurioTouch code working.
I've built my solution, in part, based on that sample, so should be able to look at your changes and adapt them to my own code.

I have already mode changes to get the formats in synch, to the point where now I have the ECHO effect working brilliantly, but have still had no joy with the PARAMEQ. Perhaps the latter is more sensitive to the formatting...?

Darren

[arlomedia]

In my case the echo and a couple other effects worked fine with no formatting changes -- I don't know why -- but other effects just produced static until I changed the remote I/O unit's stream description. In the aurioTouch project, this is configured in the CAStreamBasicDescription.h file in a SetAUCanonical function. I added a new function for 16-bit audio:

Code: [Select]

void Set16Bit(UInt32 nChannels, bool interleaved)
{
audioFormat.mSampleRate = 44100.00;
mFormatID = kAudioFormatLinearPCM;
mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
mChannelsPerFrame = nChannels;
mBitsPerChannel = 16;
mFramesPerPacket = 1;
if (interleaved) {
mBytesPerPacket = 2 * nChannels;
mBytesPerFrame = 2 * nChannels;
} else {
mBytesPerPacket = 2;
mBytesPerFrame = 2;
mFormatFlags |= kAudioFormatFlagIsNonInterleaved;
}
}

Then I went to aurio_helper.cpp and edited the SetupRemoteIO function to use my new Set16Bit function instead of the default SetAUCanonical function. That should solve it for you, but if not, let's compare our callback functions and see if I'm forgetting any changes I made there.

[darren]

Here's the audio format I apply to all 3 of the audio units I use (AU3DMixerEmbedded, MultiChannelMixer, and RemoteIO)

   // Describe format
   audioFormat.mSampleRate         = 22050.00;
   audioFormat.mFormatID         = kAudioFormatLinearPCM;
   audioFormat.mFormatFlags      = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
   audioFormat.mFramesPerPacket   = 1;
   audioFormat.mChannelsPerFrame   = 2;
   audioFormat.mBitsPerChannel      = 16;
   audioFormat.mBytesPerPacket      = 4;
   audioFormat.mBytesPerFrame      = 4;

This did indeed work for me. Now all I have to do is sit down with a graphic EQ and figure out the ranges I'll put into my presets for killBass, killMids, killHigh

Thanks for all the help!!

[arlomedia]

Now all I have to do is sit down with a graphic EQ and figure out the ranges I'll put into my presets for killBass, killMids, killHigh

Hopefully that will be the easy part!

[wiseguybec]

Hello everyone,

am trying to achieve a pitch shift on a recording voice and play it back instantaneosly. To avoid audio feedback I must try to catch the input via Audio Units from the mic, apply the pitch shift via BASS_TempoCreate and play back using BASS. Or could I play back via audio units?

Thanks to this post I have tried to wire my working BASS code into the aurioTouch example. But since I am actually trying to play the sound, I need the -1 in BASS_Init, correct? I.e. BASS_Init(-1, 44100, 0, NULL, NULL). I was not allowed to insert this code in front of the aurioTouch audiosession init code, this caused an audio session error. So I insert it afterwards. BASS_getErrorCode throws no errors.

I then create the stream with

        fxstream=BASS_StreamCreate(44100, 1,BASS_STREAM_DECODE,STREAMPROC_PUSH,0);
        BASS_ChannelSetAttribute(fxstream, BASS_ATTRIB_NOBUFFER,0);
        BASS_ChannelPlay(fxstream,FALSE);

BASS_ChannelSetAttribute fails with error code 37 (BASS_ERROR_NOTAVAIL).
BASS_ChannelSetAttribute fails with error code 38 (BASS_ERROR_DECODE).

In the aurioTouch callback I inserted
    BASS_StreamPutData(fxstream,ioData->mBuffers[0].mData, ioData->mBuffers[0].mDataByteSize);

(fxstream is declared in the actual .mm file, so no need for "THIS->"). This call throws no errors. But a
BASS_ChannelGetPosition(fxstream,BASS_POS_BYTE);
returns 0.

And my app is silent.

How did you guys ever get aurioTouch to play the fxstream?

Thanks for any clues!

[arlomedia]

"Since I am actually trying to play the sound, I need the -1 in BASS_Init, correct?" I'm playing the live sound and I'm using 0 for the first parameter of BASS_Init.

This is the sequence I'm following to set everything up:

- Initialize, configure and activate the audio session.
- Set up the RemoteIO audio unit.
- Initialize BASS, create a stream and set the effects parameters.
- Start the audio unit.

Then in the RemoteIO callback function, I use the BASS_ChannelGetData method to modify the audio samples as they come through. I'm doing that after AudioUnitRender, and I commented out the DC rejection filter code there because it is designed for the 8.24 format and BASS uses 16-bit format as mentioned above.

Hope that helps...