Air Piano

About the project

Air Piano is a computer vision application that runs entirely in the browser. The webcam captures movement without sending images to a server, and MediaPipe identifies the 21 landmarks on each hand. Motion stabilization, calibration, hysteresis, and independent state machines interpret every finger and convert gestures into musical events. Tone.js produces the audio, and the architecture separates vision, gestures, musical mapping, synthesis, and Canvas rendering. Automated tests cover jitter, tracking loss, multiple hands, keyboard paging, and note control.

Technologies

• Computer Vision
• TypeScript
• MediaPipe
• Tone.js
• Canvas API
• Vitest