Abstract

Keywords: Data Visualization, Hashtag wars, Twitter, Brazil Presidential Elections, Social Media, Assembly of God, trolls, bots, Political Crystals, Silicon Valley, Digital Fabrication, Altered Reality, Virtual Reality, Digital Fabrication, Parametric Design, Morphogenetic Design, Algorithmic Design.

This paper presents the concept of Embodied Sonic Meditation (ESM) and its proof-of-concept art installation entitled “Resonance of the Heart.” ESM artistically explores the theories of “embodied cognition” and “deep listening.” The goal of this artistic practice is to improve laypersons’ comprehension of the relationship between body gestures, sounds, and visuals. To practice this approach, we designed and built a real-time audio-visual interactive system. This system uses an infrared sensing device and touchless hand gestures to produce various sonic and visual results. An artificial neural network was implemented to track and estimate the performer’s subtle hand gestures using the infrared sensing device’s output. Six sound filtering techniques were implemented to simultaneously process audio based on the gesture. Selected Mudra hand gestures were mapped to seven 4-dimensional Buddhabrot fractal deformations in real-time. This project was applied in both college teaching and public art installation. It connects Eastern philosophy to cognitive science and mindfulness practice. It augments multidimensional spaces, art forms, and human cognitive feedback. It disrupts the boundary between cultural identities, machine intelligence, and universal human meaning.

• Jiayue Cecilia Wu Originally from Beijing, China, Dr. Wu is a scholar, composer, multimedia performer, and audio engineer. She earned her Bachelors of Science degree in Design and Engineering in 2000. In 2013, she obtained her Master’s degreein Music, Science and Technology from Stanford University. In 2018, She obtained her Ph.D. in Media Arts and Technology from University of California Santa Barbara, where she studied music composition and media arts with Dr. Curtis Roads. As an audio engineer, she received a grant award from Audio Engineering Society. As a musician, she received an award from the California State Assembly for being a positive role model in sharing Chinese culture. As a multimedia artist, she received the “Young Alumni Arts Project Grant Award” from Stanford University. As a scholar, she has been awarded a multi-year UC Central Campus Fellowship and a National Academy of Sciences Sackler Fellowship. Currently, Dr. Wu is an Assistant Professor at the University of Colorado Denver’s College of Arts and Media.
• Donghao Ren is a Ph.D. candidate in the Department of Computer Science at the University of California, Santa Barbara, USA. He received a B.S. degree from Peking University, China. Donghao’s primary research focus is on the design and implementation of user interfaces for visualization construction. He also works on designing visualizations for machine learning, as well as visualizations in virtual and augmented reality. His recent work “Charticulator” (see charticulator.com) won the honorable mention award at InfoVis 2018, the most selective venue for information visualization. Find more information about Donghao at donghaoren.org.

Full text (PDF) p. 247-254