Conservatorium van Amsterdam

The use of computer vision to control the live electronics set: a light and colour tracking approach in Max/MSP

Computer vision is one of the technological achievements that seems to have the possibility of reshaping the way live electronic music is performed. It is unveiling new expressive and gesture based possibilities that can surpass physical limitations of the performer controlling the live electronic set, as well as bringing the musician a step closer from playing along to playing with the machine. This paper focus on providing the musician that doesn’t need to have the instrument in both hands, such as pianists or harpists, a way of controlling the live electronic set without the big hassle of taking the hands to far away from where they traditionally produce sounds to reach another type of physical controller, and thus also facilitating the returning to the initial point. The gesture based nature of this type of control also allows a more human like musical response from the electronic counterpart, that can be used where the fader/knob linear response doesn’t make sense, musically speaking. The system makes use of a webcam and color tracking of LED lights put on a glove (see image 1.). Even though the live video input must be treated using the camera settings before being processed in the computer, all the programming is done in Max/MSP. The objective is to retrieve the position of a moving light in a cartesian referencial and associating the coordinates values to controllable parameters in electronic music, converting these values to MIDI data. Different colours act as different MIDI controller numbers. Control over stage lights and camera settings such as saturation, white balance, sensor sensibility or ISO, exposure or shutter speed and focus are essential to obtain satisfying results, as is the use of HSL color space instead of RGB.

Keywords: computer vision performance

Biography

Diogo Carriço is a Portuguese pianist and electronic music artist. His background education in classical music led him to study in institutions as Porto Music Consevatory, Porto Catholic Culture Centre (organ), School for the Applied Arts of Castelo Branco (under the guidance of Luisa Tender), and Centro Superior Katarina Gurska in Madrid (under the guidance of Javier Negrín). Diogo is currently a student of the Live Electronics master programme at Conservatorium van Amsterdam, where he is researching about the usage of computer vision in performances with live electronic music.