IntelliSynth

IntelliSynth was the bachelor thesis of my ITS bachelor at FH Salzburg: an interactive musician powered by AI, so a human and an AI can play and compose music together. A human plays a few notes on a piano, the AI (which has been listening) takes over and continues the music, then the human takes over again, and so on.

The setup is an e-piano, a Linux PC running IntelliSynth (Python) with musicautobot, a pre-trained transformer model that continues MIDI music, and a Raspberry Pi running Zynthian, a synthesizer that turns the MIDI back into piano sound.

My bachelor course has two theses, the first in a team of two, the second alone. IntelliSynth was the topic for both. In the first, together with Florian Hinterberger, we built the prototype: I integrated musicautobot and the Zynthian output, Florian handled the MIDI interface and file format.

The prototype had a flaw: the prediction isn’t realtime, so you had to wait for the model before it played, which hurts making music together. My second thesis removed that wait using semaphores and a pipe. The prediction now starts in its own process while the user is still playing, and the user’s last few notes serve as buffer time, so the prediction delay goes unnoticed.