[ISEA2016] Artists Talk: Sungeun An, Juyoung Lee, Sanghoon Shim & Ji-Hyun Lee — Playlight: Enriching Music Experiences through Interactive Lighting Based on Motion

Artists Statement

The paper presents “PlayLight”, a system that allows people who attend concerts or live music events to control lighting with musical gestures.

Interactive lighting concepts have been widely used to please the eyes and enhance the mood at concerts, karaoke bars, and other live music events. [1][2] Going beyond the visual effect, what if lighting could be used to enhance the social and cultural aspect of music? We look into body motions and explore how they can become a medium for enriching music experiences through interactive lighting. Consider certain kinds of behaviors that typical people engage in. People can tap their foot rhythmically in response to a musical stimulus.
It is common to see attendees at the concerts holding bright mobile phone displays and waving to participate in the music and the mood. Such body motions contribute and lead people to experience and express music socially and emotionally. Many researchers and designers have explored professional side (performer) of musical behaviors for sound and gesture mapping in live music environment. [3] We investigate music-related ancillary movements of the typical users engaged in the music, not the performer.
In this work, we present PlayLight, an interactive system that captures hand gestures of users while engaging in the music environment using smartphones, affecting the intensity and color of virtual projected lights. The prototype system includes two main concepts, musical gestures and lightings, which both contribute to the enjoyment of music.

For euphoria to happen, the gestures that come naturally during euphoria are used for input types. Observations with six participants were done in a karaoke room that reflects the context of live music environments and used to identify possible hand-gestures to sense ultimately for controlling light. We want participant’s actions to get animmediate and identifiable response. The participants’ actions should be immediately identifiable from changing the color and the intensity of light. Each hand gestures inform different lighting mappings.

The test trials showed the combination of body motions and interactive lighting proved to be an appropriate medium for musically coherent and visually satisfying experiences. We are still in the process of understanding audience interaction and developing practical interactive systems for live events. Some of the biggest challenges are reconciling the effect of lots of audience members interacting at once and letting them operating the system without their attention being distracted from live music or social interactions. These issues should be further studied for visually more pleasing lighting designs to create better music experiences.


  1. iMakeStuff, “Hue Disco”, http://www.vandenbrakel.nl/app/hue-disco/
  2. Num3ricalAudio, “Ambify”, getambify.com/mac.html
  3. DG Paine, “Gesture and musical interaction: interactive engagement through dynamic morphology”, Proceedings of the 2004 conference on New interfaces for musical expression, 80-2004.
  4. Ryan Ulyate and David Bianciardi, “The Interactive Dance Club: Avoiding Chaos in a Multi-Participant Environment”, Computer Music Journal 26, 3, 40–49. 2002.
  • Sungeun An, Juyoung Lee, Sanghoon Shim, Ji-Hyun Lee, Graduate School of Culture Technology, KAIST, South Korea