The curator's reviewConduct a full orchestra through your webcam. Wave your arms and a pose-estimation model translates your flailing into tempo, volume and section cues for a virtual ensemble.
Semi-Conductor puts a full string orchestra behind your webcam and hands you the baton. Wave your arms and a pose-estimation model (PoseNet, running entirely in the browser, 2018) reads your gestures: tempo from your beat, volume from the size of your movements, section cues from where you point. The orchestra - a sampled ensemble playing Eine kleine Nachtmusik - obeys instantly.
The obedience is the comedy and the point. The orchestra follows your arms with terrifying sincerity, which means it also faithfully performs your sneeze, your phone-answering gesture, and your attempt to push the cat off the desk. There is no smoothing layer between you and the players; you are simply, suddenly, responsible for them.
It is also quietly one of the best demonstrations of on-device ML ever shipped: skeletal tracking from a laptop camera, no install, no server, 2018. The same PoseNet lineage went on to power a whole genre of movement interfaces, but nothing since has made the capability feel this ceremonial.
Everyone deserves one minute of this power - the specific feeling of raising both arms and hearing the strings swell to meet you. My kingdom for arms. Stand up, allow the camera, and make the second violins regret nothing.
