Mountains instead of an office. At 2,315 m in the Austrian Alps, Heidi joined the Music & AI Hackathon: 24 hours of building, far too little sleep and one of the most unusual working environments of her 30+ hackathons.
Team: David Smith, Nelli Funk, Sara Rutz, Berkay Cengiz, Zardasht Ilyas and Heidi — engineers and artists thrown at the same problem.
How do you make music creation more accessible and intuitive, especially for people with disabilities?
The idea started with a single image: someone standing in front of a painting in a museum, unable to fully experience its colours, shapes or subject because of a visual impairment. What if the painting could be heard instead of seen?
Synesthesia is a picture-to-music translation tool. It divides an artwork into a grid and reads each section for its colour palette, composition, objects (via AI detection) and even the mood of the scene. Each signal is mapped to chords, melodies and soundscapes. With a Chordcat controller, you move across the painting section by section, listen to how each part translates into sound, save the melodies you like and build your own composition by remixing the mappings or swapping instruments.
The project continues as an ongoing project: see Synesthesia – The Art of Hearing Pictures.
Sitting in a hotel lobby at 6 am, deep in signal data, trying to get the last piece working before the deadline. The team was so heads-down that it never managed a proper team photo.
To Jan Bendick, who helped untangle some of the trickiest parts, and to the organisers Florian Bauer, Nobuko Okazaki, Michael Otter and Seiichiro Matsumura for bringing so many different minds together at the top of a mountain.
Go from a human insight to a working prototype in 24 hours, work across engineering and art, and design technology for accessibility.
Mountains, minimal sleep and a working prototype by the end.