< prev  |  next >

Controlling FM Synthesis with ML5 and PoseNet

Note: this sketch uses your computer or phone's camera. If prompted, plase click "allow" for it to work. (No images are captured or uploaded.) 1) click the Carrier button, then 2) click the Modulator button to start the sound. Move your right hand and wrist around the screen to affect the sound wave.
I'm utilizing ML5.js and the PoseNet model to track the position your right wrist. This data then feeds into an FM Synthesizer and modulates a carrier frequency. Move your wrist, (arm & hand) up or down to increase or decrease the depth of modulator (from -150 to +150). Move it from left to right to increase the frequency (far left is 0, far right is 75) of modulator. Hint: more extreme effects are heard near the corners.
In this sketch the carrier is a sine wave and the modulator is a sawtooth wave.