[ISEA2020] Paper: Erin M. Gee, Alex M. Lee & Sofian Audry — Playing with Emotions: Biosignal-based Control in Virtual Reality Game Project H.E.A.R.T.

Abstract (short paper)

Keywords: biometrics, militainment, interactivity virtual reality, affective computing, video games, avatars, vocaloids

Project H.E.A.R.T.is an interactive. virtual reality-based installation which probes the ‘militainment’ video game genre as a satirical starting point for exploring pop culture, mediated violence, empathy, automation. and quantification of emotional engagement. Project H.E.A.R.T. invites the viewer to place their fingers on a custom biodata gathering device, and then summon their enthusiasm to engage a holographic pop star as a form of ‘combat therapy’. The emotional labor of the viewer is quantified through biosensing technologies, giving the viewer indirect emotional control over artificial agents. The narrative of the work implicates the viewer in military violence through their affective participation in entertainment technologies, highlighted in VR through the voyeuristic gaze.
The incorporation of physiological metrics into artificial agents fosters a liminal situation between the VR user’s corporeal body and virtual space. In an era of self-help apps and biometric devices for controlling one’s emotional state, Project H.E.A.R.T. embodies a dystopic and satiristic interpretation of these technologies, as well as the limits of technologically mediated empathy and sincerity often championed as the new frontier for interaction in virtual contexts.

  • Erin Gee is a Canadian artist who uses sound and technology to transduce the invisible, embodied aspects of affect, communication, and presence. After earning an MFA in Studio Arts from Concordia University, Gee was an Assistant Professor at Concordia University (2015-2017) teaching Sound Production, Gender and Technology Studies, and Sound Studies. She is currently a doctoral student in Composition et Création Sonore at Université de Montréal, articulating feminist materialist compositional methods in biofeedback music. Her work in neural networks, choral composition, ASMR, virtual reality, networked music performance, and robotics have been shown in solo exhibition at MacKenzie Art Gallery (Regina) and in group exhibitions at Darling Foundry (Montréal), Toronto Biennale, NRW Forum (Düsseldorf), and Ars Electronica. Gee is currently artist in residence at the Institut quantique de l’Université de Sherbrooke in partnership with Sporobole artist-run centre.
  • Alex M. Lee is an artist who utilizes 3D animation, video game engines, virtual/augmented/immersive reality platforms, and the potential of simulation technologies in order to visualize and complicate our perception of time, space, image and light – culling from concepts within science, science fiction, physics, philosophy, and modernity. He received his BFA (2005) and MFA (2009) from the School of the Art Institute of Chicago. Born in South Korea, Lee was raised in the USA and is associate professor in the Digital Arts & Sciences Program at Clarkson University (NY, USA). Lee has exhibited internationally in North America and Asia. Selected exhibitions include: Mio Photo, Osaka, Japan; Daegu Art Factory, Daegu, Korea; The Gene Siskel Film Center, Chicago, Illinois; Eyebeam: Center for Art & Technology, New York, NY; Gallery DOS, Seoul, Korea; Museum of Contemporary Art, Montreal, QC. His work has been published in articles covering art, science, and culture including: Metaverse Creativity, Canadian Art, and Routledge Press.
  • Sofian Audry is an artist, scholar, Professor of Interactive Media within the School of Media at the University of Quebec in Montreal (UQAM). His work is inspired from visual art, artificial intelligence, artificial life, biology and cognitive sciences. His computational artistic practice branches through multiple media including robotics, interactive installations, immersive environments, physical computing interventions, internet art, and electronic literature. Audry studied computer science and mathematics at University of Montreal (BSc, 2001) where he completed a master in machine learning (MSc, 2003); following which he obtained a master in communication (interactive media) at UQÀM (MA, 2010). His PhD is in Humanities from Concordia University (2016). In 2017 he was a Postdoctoral Fellow at the Massachusetts Institute of Technology. His work and research have been presented in multiple international events and venues such as Ars Electronica, Barbican, Centre Pompidou, Club Transmediale, Dutch Design Week, Festival Elektra, International Digital Arts Biennale, International Symposium on Electronic Art, LABoral, La Gaîté Lyrique, Marrakech Biennale, Nuit Blanche Paris, Society for Arts and Technology, V2 Institute for Unstable Media, Muffathalle Munich and the Vitra Design Museum.