Topic: Getting fundamental Frequency from Music Instruments

[gnag]

Has anyone experience with getting the fundamental Frequency ("Musical Note") from a Music Instrument which has Harmonics?

By just looking at FFT you will get multiple Peaks with almost the same Amplitude switching Positions over time.
One of these Peaks is the fundamental Frequency (F), the other Peaks are Harmonics (Frequency: F*2,F*3,...) so they are just Multiples, I am interested in the fundamental Frequency.

Here are some Resources I found but was not able to implement. Currently I am using AutoCorrelation to find the Fundamental which takes a lot of CPU Power, is more complicated to understand so it is an overkill for the Task I want to complete.

https://github.com/Notnasiul/R2D2-Processing-Pitch/blob/FFT/PitchProject/PitchDetectorHPS.pde
https://ccrma.stanford.edu/~pdelac/154/m154paper.htm
http://stackoverflow.com/questions/435533/detecting-the-fundamental-frequency

Can someone follow the Instructions to show me how this should/could be implemented or give me Hints what they are doing there?

[Ian @ un4seen]

Here's something you can try. It's a variation of the HPS method that works back from the higher harmonics to find the fundamental frequency, for greater frequency resolution. For example, if the 5th harmonic is found at bin 31 then you can say the fundamental is at bin 6.2 (31/5), while you would only get bin 6 if looking for the fundamental directly.

Code: [Select]

float fft[1024];
BASS_ChannelGetData(handle, fft, BASS_DATA_FFT2048); // get the FFT data (2048 sample)
// find the peak (5th harmonic)
float peak=0;
int peaki=0;
for (int a=5*2; a<1023; a++) {
float p=fft[a]*fft[a*4/5]*fft[a*3/5]*fft[a*2/5]*fft[a/5]; // product of this bin and 4 lower harmonics
if (peak<p) { // found a new peak
peak=p;
peaki=a;
}
}
// get the frequency
float freq=(float)peaki*samplerate/2048/5; // translate the peak bin to Hz and divide by 5 to get fundamental

Interpolation can also be used to finetune the result.

Code: [Select]

// get the frequency
float freq=0;
if (peaki) {
float y0=20*log10(fft[peaki-1]), // amplitude of peak-1 bin in dB
y1=20*log10(fft[peaki]), // amplitude of peak bin in dB
y2=20*log10(fft[peaki+1]); // amplitude of peak+1 bin in dB
freq=peaki+0.5*(y0-y2)/(y0-2*y1+y2); // interpolate with neighbouring bins to finetune the peak location
freq=freq*samplerate/2048/5; // translate it to Hz and divide by 5 to get fundamental
}


[gnag]

Thanks for your interesting Answer, my Application has a Recording Callback which copies all available Sample Bytes from the Buffer (using Marshall.Copy) to a Processing Function (AutoCorrelation currently, the Function receives the PCM Float Samples and returns the Fundamental Frequency).

I chose not to use a Timer because I dont want Information to be lost or to have the Application running at a slow/fixed Speed, I want all Data which is in the Buffer to be processed whenever it arrives.

I am afraid that if I switch from PCM Samples to FFT Data (using the ChannelGetData Call) that I can no longer ensure that everything is processed when the Callback is called, I would need to set a fixed FFT Size whether there would be more Data in the Recording Buffer.

Is there any way to directly apply FFT on the Buffered Data to not loose Information like I can with my Sample Processing Function?

I was thinking of making a second Stream (Decoding Stream) which could be fed with the Samples arriving at the Callback and be processed "as fast as possible" in another Thread, but again I would need a fixed interval and if its running to fast it would waste CPU Power and just loop until new Data comes in.

[Ian @ un4seen]

It is possible to apply FFT processing to buffered data via a "push" stream (with the same sample format), ie. feed the sample data to the push stream via BASS_StreamPutData and get back the FFT data from BASS_ChannelGetData. For example, you could do something like this...

Code: [Select]

fftstream=BASS_StreamCreate(rate, chans, BASS_STREAM_DECODE, STREAMPROC_PUSH, 0); // create a push stream with same format as recording
fftneed=2048*chans*2; // amount of data required for 2048 sample FFT

...

BOOL CALLBACK RecordProc(HRECORD handle, void *buf, DWORD len, void *user)
{
int got=BASS_StreamPutData(fftstream, buf, len); // feed the recorded data to the push stream
while (got>=fftneed) { // got enough data to process
float fft[1024];
BASS_ChannelGetData(fftstream, fft, BASS_DATA_FFT2048);
// process the FFT data here
got-=fftneed;
}
return TRUE;
}


[gnag]

Thank you for your Reply and the nice Code, I needed some Days to understand how and why the Harmonic Product Spectrum actually works and some time to try out your Replies and find out how to use them in my C# Program, experiment with your Version and another Version of HPS I found here at Github.

I can confirm the Version of the "Harmonic Product Spectrum" Algorithm you posted to be working and useful for Pitch Detection of Audio Input with Harmonics !

[teq]

Hi there!
Get get this code works.

Code: [Select]

Bass.BASS_RecordInit(-1);
_recHandle = Bass.BASS_RecordStart(44100, 1, BASSFlag.BASS_RECORD_PAUSE, MyRecording, IntPtr.Zero);
_encHandle = BassEnc_Flac.BASS_Encode_FLAC_StartFile(_recHandle, "-4",
                                                                     BASSEncode.BASS_ENCODE_AUTOFREE,
                                                                     FileManager.Instance.GetRecordedFilePath(model));
Bass.BASS_ChannelPlay(_recHandle, false);
int channels = 1;
recordedTunerStream = Bass.BASS_StreamCreate(44100, channels, BASSFlag.BASS_STREAM_DECODE, STREAMPROC_PUSH, IntPtr.Zero);
fftneed = 2048 * channels * 2;


private int STREAMPROC_PUSH(int handle, IntPtr buffer, int length, IntPtr user)
{
  return 1;
}

private bool MyRecording(int handle, IntPtr buffer, int length, IntPtr user)
    {
        int got = Bass.BASS_StreamPutData(recordedTunerStream, buffer, length);
        if (got < 0)
        {
            Debug.Log("error got = " + Bass.BASS_ErrorGetCode());
        }
       
        while (got >= fftneed)
        {
            // got enough data to process
            float[] fft = new float[1024];
            Bass.BASS_ChannelGetData(recordedTunerStream, fft, (int) BASSData.BASS_DATA_FFT2048);
            // process the FFT data here

            float peak = 0;
            int peaki = 0;
            for (int a = 5 * 2; a < 1023; a++)
            {
                float p = fft[a] * fft[a * 4 / 5] * fft[a * 3 / 5] * fft[a * 2 / 5] * fft[a / 5]; // product of this bin and 4 lower harmonics
                if (peak < p)
                {
                    // found a new peak
                    peak = p;
                    peaki = a;
                }
            }

// get the frequency
            float freq = (float) peaki * 44100 / 2048 / 5; // translate the peak bin to Hz and divide by 5 to get fundamental

            Debug.Log("MyRecording freq = " + freq);
           
            got -= fftneed;
        }
       
        return true;
    }

After

Code: [Select]

int got = Bass.BASS_StreamPutData(recordedTunerStream, buffer, length);i got an error

BASS_ERROR_NOTAVAIL

[Ian @ un4seen]

BASS_ERROR_NOTAVAIL indicates that the "recordedTunerStream" handle is not a push stream. How have you defined the "STREAMPROC_PUSH" value that you're using in the BASS_StreamCreate call? If you're using BASS.Net, you can use "BASSStreamProc.STREAMPROC_PUSH" there.

[teq]

BASS_ERROR_NOTAVAIL indicates that the "recordedTunerStream" handle is not a push stream. How have you defined the "STREAMPROC_PUSH" value that you're using in the BASS_StreamCreate call? If you're using BASS.Net, you can use "BASSStreamProc.STREAMPROC_PUSH" there.

It's defined like method

Code: [Select]

private int STREAMPROC_PUSH(int handle, IntPtr buffer, int length, IntPtr user)
    {
        return 1;
    }

In your case it shows an error

[Ian @ un4seen]

I'm not a .Net user myself, so I'm not sure about that, but BASS.Net also provides a BASS_StreamCreatePush function that you could use instead:

Code: [Select]

recordedTunerStream = Bass.BASS_StreamCreatePush(44100, channels, BASSFlag.BASS_STREAM_DECODE, IntPtr.Zero);