Software research engineer Joren Sixx has come up with a unique way to control a computer.
He has created a device that allows you to control your computer's cursor using a flute rather than a traditional mouse. The project is called PiPePoPo.
Joren uses an Arduino Nano RP2040 Soppesti development board with an RP2040 chip for the project to listen to the flute and process the sound, then sends the appropriate commands to control the computer mouse cursor. He also created a Chrome browser extension that allows the flutist to move a virtual cursor rather than the actual computer cursor in the browser.
The most challenging aspect of this project was determining the frequency of the audio, since sound is not just a single wave, but a combination of many different frequencies with different amplitudes. In order to find the fundamental frequency, a highly optimized library implementing the YIN algorithm was included.
Once the frequency was found, the next task of RP2040 Connect was to move the mouse according to pitch. Sixx configured 4 different mutually exclusive ranges in which the cursor position changes by a given number of pixels left, right, up, or down. The fifth, higher range, causes a click event to be sent before the cool-down timer starts, which helps prevent over-clicking.
For more information about the PiPePoPo project: https://0110.be/posts/Making_a_flute_controlled_mouse.
Joren uses an Arduino Nano RP2040 Soppesti development board with an RP2040 chip for the project to listen to the flute and process the sound, then sends the appropriate commands to control the computer mouse cursor. He also created a Chrome browser extension that allows the flutist to move a virtual cursor rather than the actual computer cursor in the browser.
The most challenging aspect of this project was determining the frequency of the audio, since sound is not just a single wave, but a combination of many different frequencies with different amplitudes. In order to find the fundamental frequency, a highly optimized library implementing the YIN algorithm was included.
Once the frequency was found, the next task of RP2040 Connect was to move the mouse according to pitch. Sixx configured 4 different mutually exclusive ranges in which the cursor position changes by a given number of pixels left, right, up, or down. The fifth, higher range, causes a click event to be sent before the cool-down timer starts, which helps prevent over-clicking.
For more information about the PiPePoPo project: https://0110.be/posts/Making_a_flute_controlled_mouse.