Over the last few months, I had the great opportunity to work together with the fantastic electronic music artists MiesFM and SpaceVentura as well as dancer Elena Nijsen on a next-generation audiovisual performance. As a team, we collaborated with De Basis Nijmegen to showcase the latest version of our AI music project for the Kunstnacht 2023. Over the course of the evening of September 16th, we gave three performances at the Stadsschouwburg in front of a full audience.
In this project, we track two dancers on stage with a camera and use computer vision to get real-time pose estimations. We can track different body parts of the dancers in 3D space and use their relative positions to control music parameters in Ableton Live. The music production software then sends out a series of signals to influence the drum patterns and filters of digital synthesizers thereby changing the music as it is being played. This creates a feedback loop between the performers on stage dancing to the music and the music creation itself. We also show the pose estimation projected onto a screen above the dancers. The code for the project is a work in progress and is available on GitHub.
The night was a full success as we got great feedback from the audience as well as the organizers. We are excited to see where we can take this technology in the future. Stay tuned for more!
Leave a Reply