Kyle Pickle and Brian Dao's EEC172 Final Project
Using the power of the Fast Fourier Transform and Hooktheory's API we created ReChorder, a device that uses a microphone and ADC to detect and record chords from instruments such as a piano and guitar or even full songs. Once it detects three chords, you will get song recommendations on an OLED display based on the chord progression it has recorded in your jam session.
Brian’s been playing guitar for 7 years.
“I’ve always wanted an easy way to pick out the chords of a song by ear but it’s sometimes hard to do.
If you’re tone deaf and sometimes when you’re in a different tuning on the guitar, it’s hard to know what chord you are playing.
So why not make a device that can detect the chords for you?”
With this project he wanted to combine his passion for guitar playing with his interest for embedded systems and software.
Kyle's work with embedded systems began with his job at the CLTC, where he works with building controllers and their various APIs and software.
He may have only taken Piano I in high school, but it was his idea to communicate with the HookTheory database to look up chord progressions.
"It's fun and rewarding when you forget about the 10+ hours you spent debugging the html requests."
● Program will listen to what chords you play after authenticating with HookTheory API
● Once three chords have been detected you will get song recommendations based on the chord progression recorded
● Press SW2 to delete the previously detected chord
● Press SW3 to start a new ReChording
● Can detect twelve basic major and minor chords
● Returns up to 4 songs and artists per request
We utilized a Fast Fourier Transform (FFT) algorithm from the CMSIS DSP Software Library to transform audio data as a time-domain signal into its frequency-domain representation.
According to Nyquist’s Theorem, we must sample twice as fast as the highest frequency we want to measure.
Keeping this in mind, we use a 16 kHz sampling rate, 2048 samples.
The higher the number of samples, the greater the frequency resolution and accuracy of the chord recognition but it comes at the cost of efficiency.
After collecting the 10-bit ADC samples into a buffer using ISR timer interrupts, we run the FFT to get the real and complex components.
We use an FFT magnitude function in order to get the magnitude of the real and complex parts into one final destination buffer.
This function utilizes the Pythagorean Theorem to get the magnitude.
Our frequency resolution comes out to about 16k / 2048 = 7.8125 Hz.
This means that every tick in the horizontal axis of the FFT is equal to tick * 7.8125 Hz.
An FFT of 1024 Samples Plotted During the Debugging Phase
We then analyze the spectrum of the audio signal at different frequencies.
1) There are 12 notes [C,C#,D,D#,E,F,F#,G,G#,A,A#,B] and each note has 9 octaves so there are 12*9 = 108 frequencies we need to collect the energy of from the FFT
2) Create a PCP (pitch class profile of the FFT) array of 12 where each element is the total aggregate strength of the note. Loop over all 9 octave frequencies of each note and add the FFT strength together to aggregate the strength of the note
- The resulting array looks something like [3782, 2393, 2990, 2498, 3489, 2085, 2957, 3868, 2085, 2974, 2756, 2946]
- The 3 strongest are replaced with 1's and the rest are 0's
- For example: [1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0] is a C chord
3) After the Pitch Class Profile array has been defined for the samples, we convert the array to binary so that we can index into a Chord Library array. If a match has been found then it will be printed onto the OLED display
Our program starts its connect with HookTheory's REST API by sending an authentication POST request with our username and password. Everything is connected over HTTPS, which requires an adequate root certificate and TLS connection. Once accepted, it responds with an authentication key “activkey” that is used to retrieve song data from the database. We then query a request with the key and the chord progression formatted in the GET header, and it responds with a json-formatted list of the first 20 songs that match the progression, in order of popularity. We iterate through the first 4 items in this list, displaying them on the OLED display.
The CMSIS library is massive, especially when compared to the measly 250kB of memory offered by the TI Launchpad.
We had to selectively cut and paste headers, code, and coefficient tables to utilize as much functionality with as little space as possible.
This resulted in a lot of errors and time spent configuring our project’s optimization and memory settings.
Once the repository worked the shrunken-down version of the FFT library still wasn’t nearly as precise as we had hoped.
It still picks up a lot of noise that’s hard to predict or compensate for, and Brian’s board could hardly detect anything until he changed his optimization settings.
Still, we tuned it the best we could and it works within a fair degree of accuracy.
We had a lot of challenges getting HookTheory to work with our project, as it is a simple API used with simple scripting languages like Python, never with C.
We encountered many errors that would result from the smallest mistakes, with little to no hint as to where the error was actually coming from.
Kyle spent more time than he’d like to admit with a “SL_ESECCLOSED” error that resulted from forgetting to create a new tls connection when pulling song data from a different source.