Machine Learning’s Potentials in Composition, Performance and Reception of Contemporary Music: Multimodal Analysis of
Recent Mixed Music Works
As a highly structural form of art, music composition leads to numerous attempts to use automatic generation methods, mostly derived from symbolic artificial intelligence (AI) techniques1 [1]. Such methods are used by composers for the idiosyncratic extension of their creative processes, the creation of compositional tools, the implementation of musical theories and/or specific styles, or the programmatic modeling of cognitive theories [2].
Over the past decade, new composer-centered [3] research-creation projects based specifically on machine learning systems2 have been launched (Bresson & al. 2018; Sturm & Ben-Tal 2021; Anders 2022). These systems have rarely been used by composers as a tool designed to assist the composer [4]. They differ from symbolic AI systems based on a complex system of rules for inference and production of structure and/or musical material. Machine learning systems consist of a multitude of interconnected procedural units that interact with each other [5]. Inspired by the functioning of the human brain, they are capable of learning, being trained, generalizing and abstracting contradictory data. Thus, the importance of the “memory” of such systems and its impact on compositional processes raises a lot of new questions.
Commissioned by Ircam in 2019, Norwegian composer Anders Vinjar’s electroacoustic work Sonata IV3 incorporates machine learning techniques, which Vinjar adapted to his creative process and concept. The work’s second section, “Largo”, features a non-fixed layer modified before each performance using according to Vinjar recorded sounds “related to events shaping world history in significant ways, either contemporary or historically”. For example, for a recent performance (2022), the composer used recordings of the current conflicts in Ukraine. The notion of memory is of crucial importance in Sonata IV, both in the method of generating musical material through machine learning techniques and in the concept of the piece itself, which is updated for each performance. The work invites us, then, to consider the ways in which the encounter between art and technology can both document and constantly reactivate our collective history.
According to Roger Dannenberg, the literature dedicated to the study of the use of AI and machine learning methods in musical composition focuses on the description and programming of tools and software intended for creation, but rarely on the composer’s intentions [6]. Based on analysis of the compositional process and interviews with the composer, this paper will therefore focus on the ways in which Vinjar implements various machine learning techniques in his compositional process and how he links them with the conceptual project of his Sonata IV. We aim to shed light on the motivations underlying the use of these tools and on the ways in which they shape the aesthetics and structure of the work, thus contributing to a better understanding of the dynamics between composers and technology in contemporary musical composition.
[1] Such as generative grammars, evolutionary algorithms or cellular automata.
[2] Programs with the ability to learn, with or without explicit information given (corresponding respectively to supervised or unsupervised learning methods).
[3] Follow this link to listen to Sonata IV: https://soundcloud.com/anderstracks/sonata-iv-binaural-version.
Keywords: machine learning, composition, creative process
Biography
Florian Iochem is PhD candidate in musicology at the University of Strasbourg and Aix-Marseille University. His research interests range from 20th- and 21st-century music historiography to computer-assisted musical analysis, analysis of composer archives, psychology of music perception and musical aesthetics. As part of his PhD dissertation (co-directed by Anne-Sylvie Barthel-Calvet and Vincent Tiffon), he is particularly interested in the impact of the emergence of artificial intelligence in mixed music and the potential paradigmatic break that this implies. As student member of the international ACTOR project (Analysis, Creation and Teaching of Orchestration), he took part as a research assistant in the APCOR (Analysis of Creative Practices in Orchestration, dir. Nathalie Hérold and Gilbert Nouno) and is a member of the East Asian Music research group (dir. Robert Hasegawa). Since 2022, he has been responsible for the administrative management of the French Society for Music Analysis.
University of Strasbourg, France – florian.iochem@etu.unistra.fr