[ISEA2019] Paper: Sihwa Park — Multimodal Data Portrait for Representing Mobile Phone Use Behavior

Abstract

Keywords: Self-representation, quantified self, mobile phone use behavior, data art, multimodal data representation, visualization, sonification, indexicality, heatmap, WebSocket, WebAudio

This paper presents BeHAVE, a web-based audiovisual piece that explores a way to reveal my mobile phone use behavior through multimodal data representation, considering the concept of indexicality in data visualization and sonification. It visualizes the spatiality and overall trend of the records of my mobile phone use as a geographical heatmap visualization and a heatmap chart. On top of that, BeHAVE presents a mode for temporal data exploration to make a year of data perceivable in a short period and represent the temporality of data. Based on a
microsound synthesis technique, it also sonifies data to simultaneously evoke visual and auditory perception in this mode. As a way of indexical visualization, BeHAVE also suggests an
approach that represents data through mobile phones simultaneously by using WebSocket. Ultimately, BeHAVE attempts to not only improve the perception of self-tracking data but also
arouse aesthetic enjoyment through a multimodal data portrait as a means of self-representation.

  • Sihwa Park is a sound interaction designer, media artist, and developer. His interest mainly lies in data-driven audiovisual arts. He also seeks to create new musical experiences by embracing advanced technologies in making a novel musical interface/interaction. He is currently a Ph.D. candidate in Media Arts and Technology at the University of California, Santa Barbara, Santa Barbara, CA, USA

Full text (PDF) p.  36-43