Qosmo, Inc., who advances human creativity with the help of AI, publicly releases today "AI DJ Project#2 Ubiquitous Rhythm" an evolution of the AI DJ Project, which explores possibilities of music creation under human-AI collaboration. Video recording of the performance given by Nao Tokui (CEO, Qosmo, Inc.) at Creativity4Better organized by IAA between Oct 27-28 is now available from the following link along with the behind-the-scenes interview of Nao who explains the technical workings of the innovative new AI music generation system.
What is the AI DJ Project?
The initial version of AI DJ, developed by Nao Tokui around 2016, was in a "back-to-back" format playing tracks in turn between a human DJ and an AI system based on deep-learning algorithms. With exceedingly positive reception by the audience, the AI DJ Project was performed at numerous locations around the world including Google I/O in the USA in 2019.
AI DJ Project#2 extends the horizon of human-AI interaction by using generative AI technology and taking the performance into improvisational challenges. Rhythm patterns and baselines are generated on-the-fly, combined with a loop selection mechanism which selects melodies from a custom library. The MIDI sequences generated by the AI models are fed into analog synthesizer modules to produce sound. Behaviors of the AI are modulated by feedback from human input through a turntable and a mixer, forming a mutually collaborative system of real-time music production.
Some of these AI models have been made publicly available as Ableton Live plugins as a result of research conducted by Nao Tokui in recent years. For more technical information about this project, see the project page below.
Performance Video and Behind-the-Ccenes Videos
The recorded performance, world premiered during Creativity4Better organized by IAA between Oct 27-28th, utilized AR (Augmented Reality) technology with help from the Dentsu Craft Tokyo technical team. Virtual interfaces displayed in front of the performer visualize sequences generated and loops selected by the AI models.
In addition to the main performance digest video, Nao Tokui explained the technical details of the system and added commentary to the entire performance in the behind-the-scenes video produced specially for this general release.
Comments by Nao Tokui (System Developer and Performer)
"In this update, I've achieved what has previously been seen as impossible by a human DJ alone, composing music in real-time during the performance with the help of AI. This extends my search for new forms of collaborative human-AI expression significantly and also challenges the current definition of DJ performance, which conventionally plays pre-recorded tracks.
I felt challenged to face unforeseen situations during the performance as if I was playing with an unknown alien creature.
Projects like this one embody our mission at Qosmo, Inc. to strive and develop forms of expression unknown to us previously. I hope to make advances like this in the future broadly in creativity fields including and extending beyond art and music." CEO, Qosmo, Inc. / Associate Professor, Keio Univ./ Dentsu Craft Tokyo, Head of Technology
Based on his Ph.D. research at Tokyo Univ., Nao releases some of his first music albums and works in collaboration with the renowned Hip-Hop producer, Nujabes (RIP). In 2009, he founded Qosmo, Inc. in order to extend human creativity in collaboration with AI. His works have been exhibited at MoMA New York, Barbican Center London, NTT InterCommunication Center, Ars Electronica etc. In Jan, 2021, he published a book based on his research titled "AI for creativity - never ending story of machines and creativity".
He continues to be active in the global multi-disciplinary intersections between technology and creativity as an artist, a researcher and a DJ.
Source: https://qosmo.jp/