Abstract (Short paper)

Keywords: Taiko, Machine Learning, Dance, Choreography, Expressive Movement Recognition, Sound Recognition, Movement Classification, Musical Gesture, Interactive Installation, Kinesthetic Awareness, Visualization.

We describe our approach to collecting, analyzing and visualizing expressive movement data to support the creation of an interdisciplinary performance and installation work, 3 Movements in Translation. We seek to understand how three perspectives (the performer, the audience, and the machine) can inform one another to create a cross-cultural performance that allows a broad audience to kinesthetically engage and empathize with expressive features of taiko performance. Taiko is a Japanese artistic practice that combines stylized movement with drumming technique. We share initial results in machinelearning analysis of taiko sound and movement. These results are used, in combination with a performer’s perspective, to inform artistic visualizations of important expressive features within taiko. Through this process, we explore how multiple perspectives of taiko can inform the translation process between the performer, the audience and the machine.

• Shannon Cuykendall, PhD Student, Simon Fraser University, Surrey, BC, Canada shannoncuykendall.wordpress.com
• Yawen Wang, Composer, Music Director, Interdisciplinary Artist, Asian Rain Werk inter/arts, Vancouver, BC, Canada. Described by Colin Thomas of Vancouver’s Georgia Straight as “a dancer at the keyboard”, Yawen Wang is an all-terrain interdisciplinary performance artist crossing over music, dance and theatre. A genre-defying artist, Yawen has created a body of work in the realms of music composition, gallery installation, performance art, musical theatre, music for dance, radio play, and film. Her representative works include Urinetown, The Musical! (Jessie Richardson Award, Best Production, 2007), Made in China (CBC/BBC Radio, 2004), Excursion (Jessie Nomination, Outstanding Original Musical, 1998), The Peach Project (cross-cultural opera, 2001) and The Difficult Mountain Paths of Szechwan (National Taiwan Symphony New Composition winner, 1992). In recent years, she had been commissioned a number of Podplays by the Neworld Theatre, Playwrights Theatre Centre and the Gateway Theatre. She was the recipient of the explorPERFORMANCE Award 2005, and has been a frequent nominee for various awards, including The Future Generation’s Millennium Award (Canada Council for the Arts, 2000) and the AMSSA Riasat Ali Khan Diversity Award (2005), which recognize and celebrate her contribution in the Arts, Culture and Diversity. Yawen thrives on the idea of polymath and polyrhythm : besides counting musically, she also counts numerically as a tax accountant.
• Philippe Pasquier (FR/CA) is Associate Professor and Graduate Program Chair at Simon Fraser University’s School of Interactive Arts and Technology. He is both a scientist specialized in artificial intelligence and a multi-disciplinary artist. His contributions range from theoretical research in artificial intelligence, multi-agent systems and machine learning to applied artistic research and practice in digital art, computer music, and generative art. Philippe is the Chair and investigator of the AAAI series of international workshop on Musical Metacreation (MUME), the MUME-WE concerts series and the International workshop on Movement and Computation (MOCO). He has coauthored over 100 peer-reviewed contributions.
• Thecla Schiphorst is Associate Director and Associate Professor in the School of Interactive Arts and Technology at Simon Fraser University in Vancouver, Canada. Her background in dance and computing form the basis for her research in embodied interaction, focusing on movement knowledge representation, tangible and wearable technologies, media and digital art, and the aesthetics of interaction. Schiphorst is the recipient of the 1998 PetroCanada Award in New Media, a biennial award presented to a Canadian Artist for their contribution to innovation in art & technology in Canada. Her media art installations have been exhibited internationally in Europe, Canada, the United States and Asia in many venues including Ars Electronica, the Dutch Electronic Art Festival (DEAF), Future Physical, Siggraph, the Wexner Centre for the Arts, the Canadian Cultural Centre in Paris, and the London ICA.
• Michael Junokas, Research Assistant, Illinois Informatics Institute, Champaign-Urbana, USA. Michael J Junokas develops innovative, multi-platform systems that have the ability to gather, interpret, process, and control signals in live artistic performance. Through the exploration of these systems, he hopes to create immersive technological environments artists can use for their own creative pursuits.  junokas.wordpress.com
• Kyungho Lee, Mohammad Amanzadeh, David Kim Tcheng & Guy Garnett, University of Illinois,  Urbana-Champaign, IL, USA

Full text (PDF)  p. 346-349