[ISEA2020] Paper: Ronald Boersen, Aaron Liu-Rosenbaum, Kivanç Tatar & Philippe Pasquier — Chatterbox: an interactive system of gibberish agents


Keywords: Translanguaging, Paralanguage, Multi-agent systems, Artistic research, Interactive art, Sound art, Generative art.

We present the interactive multi-agent system Chatterbox, as part of the sound art installation Translanguaging, exploring the notion of translanguaging as a mediation of multilingual and intercultural communication. We discuss the act of languaging as a dual process comprising both semantic language communication, as well as paralanguage that relates to the affective, personal, and cultural aspects related to translanguaging. Through the creation of the Chatterbox agent, generating gibberish vocal streams devoid of semantic content, we aim at highlighting the paralinguistic dimension of languaging.
The agent model comprises a kind of gradient map, clustering a segmented corpus of vocal sounds in the latent space of a self-organized map, according to its paralinguistic fingerprint. We utilize Factor Oracles for the creative generation of novel utterances of paralanguaging gibberish by the agent. Incorporating simple subsumption architecture inspired rules, we further moderate the interaction between the gibberish agents, creating rich and complex multi-agent behavior in “paralanguaging discussion”. We outline the artistic and technical considerations in developing our Chatterbox agent throughout the paper. We share several observations made throughout the process of creating the Chatterbox agent, highlighting some of the connections between the notion of (trans)languaging and the implementation of our model.

  • Aaron Liu-Rosenbaum, Faculté de musique, Université Laval, Québec, QC, Canada. What new forms of sonic creation and research are possible in the digital era and what is their impact on our ways of communicating, our identity, and how we navigate our ever-increasingly technologized world? This is the question that underlies the research and creative work of Aaron Liu-Rosenbaum, professor, composer, and music technologist at Laval University, where he serves as Director of the Certificate programme in Digital Audio Production. A polyvalent researcher, Liu-Rosenbaum is a member of the the Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT) at McGill University, as well as an affiliate researcher at the Laboratory for New Technologies of Image, Sound, and Stage (LANTISS) at Laval University.
  • Ronald Boersen (NL/CA), School of Interactive Arts & Technology, Simon Fraser University, Surrey, BC, Canada. Ronald Boersen, born 1980 in the Netherlands, is internationally active as a composer, improviser, and sound artist. His general interests are centered around the question “What is musical expression in performance-ship?” Within this context, he aims to formulate concept frameworks and analogous algorithms exploring issues such as musical gesture, intuition, directionality, and interactivity. His work has been performed internationally in the Netherlands, Belgium, Germany, Austria, Switzerland, Israel, Sweden, the United States, and Canada. Ronald is currently enrolled as a Ph.D. student in the School of Interactive Arts & Technology at Simon Fraser University, where his research interests include multi-agent systems in musical free-improvisation, musical gestures as enactive embodiment, and cognitive modeling in machine learning. He has completed both a Bachelor’s and a Master’s degree in composition and live-electronics at the Institute of Sonology (NL), as well as a BMus in viola at the Royal Conservatoire The Hague (NL).
  • Philippe Pasquier (FR/CA), School of Interactive Arts & Technology, Simon Fraser University, Surrey, BC, Canada. Philippe Pasquier researches the theory and practice of Creative AI, focussing on creative processes and generative systems. He is a scientist specialized in artificial intelligence, a multidisciplinary artist, an educator, and a community builder. His contributions range from theoretical research in multi-agent systems, computational creativity, and machine learning, to applied artistic research and practice in digital art, computer music, and generative art. Philippe is an associate professor at Simon Fraser University’s School for Interactive Arts and Technology, in Vancouver, where he directs the Metacreation Lab for Creative AI. newforms.digital
  • Kivanç Tatar (TR/CA), School of Interactive Arts & Technology, Simon Fraser University, Surrey, BC, Canada. kivanctatar.com