Musical Vision: turning images into music

Musical Vision is a highly flexible, interactive and bio-inspired sonification tool that translates color images into music by mimicking the human visual system. Above all else, Musical Vision is an extremely flexible system that the user can interactively configure to convert an image into either a few seconds or a several minutes long musical piece. Thus, it can be used with trans-artistic purposes like the conversion of a painting into music, for augmenting vision with music, or for learning musical skills such as sol-fa.

This work has been recently accepted for publication at the Journal on Multimodal User Interfaces, and it is co-authored by former student Antonio Polo and GTM researcher Xavier Sevillano. In the paper, the authors present a pilot user study demonstrating that, after a short training process, users with some musical education are capable of accurately "drawing by ear" simple images from no other stimuli than the music generated by Musical Vision. Click if you want to hear Times Square and Van Gogh's painting Bank of the Oise at Auvers converted into music by Musical Vision!

Polo, A. & Sevillano, X. (2018) Musical Vision: an interactive bio-inspired sonification tool to convert images into music. Journal on Multimodal User Interfaces. DOI: 10.1007/s12193-018-0280-4. View-only version of the paper here.