[ISEA2015] Paper: Jason Long – Augmenting Virtual Worlds with Musical Robotics

Abstract (Long paper)

Keywords: Virtual Reality, Video Games, Music, Sound Effects, Musical Robotics, Solenoid, MIDI.

This paper introduces the concept of augmenting the experience of interacting with virtual worlds by making use of musical robotics systems. By creating or interpreting real-time control data from the music and sound-effect channels of interactive software while it is being used, signals for controlling robotic musical instruments and other acoustic sound-creating devices can be generated. These acoustic instruments add a physicality to virtual environments by bringing previously virtual sound into the real world. A proof of concept is described, making use of a set of custom-built robotic pitched and non-pitched percussion instruments playing in conjunction with a well-known vintage videogame. The system is presented as an installation of kinetic art and sound for participants to experience. The design of the robotic sound-objects, their control systems and the software used is described and the paper concludes with an outline of future work and a summary of some of the many potential applications for this technology.

  • Jason Long is a composer and sound artist from Christchurch, New Zealand. He completed his undergraduate study there at the University of Canterbury, and the Utrecht School of the Arts, the Netherlands. He was subsequently awarded a Japanese Government scholarship to undertake a Master degree at the Tokyo University of the Arts where he designed and constructed an ensemble of robotic musical instruments. With a number of his pieces being performed internationally at festivals such as the ISCM, ACL, and ICMC and a slew of music released internationally in the form of vinyl records, CDs and digital
    distribution, Jason is currently pursuing a PhD at Victoria University in Wellington, conducting research in the fields of musical robotics and live electronic music.

Full text (PDF) p. 518-523