[ISEA2023] Paper: Pierre Yat Siu Shum & Tobias Klein — VR Calligraphy: Transposing Chinese calligraphy as choreographed movements into whole-body performances in VR


Full Paper. Theme Immersion(s) Sub theme Symbiotic Imaginaries

Current research in the area of digitizing Chinese calligraphy is primarily concerned with computer graphics (CG), calligraphy ink/brush simulation, human-computer interaction (HCI), and automatic calligraphy generation. In all above research fields, calligraphy is mainly expressed and represented in two dimensions as traditional ink on paper. Even within recent contemporary calligraphic movements, there is little exploration into a true spatialization of calligraphy beyond the flat medium.

This research articulates an evolution of Chinese calligraphy, from a physical two-dimensionality into a multidimensional digital form via choreographed whole-body movements using VR (virtual reality) interfaces. We present a series of experiments to test the transposition, creation, and presentation of the essential qualities of calligraphy into a VR format. The aim is, through disseminating new virtual phenomenology in calligraphy, to contribute to and extend the existing writing on the relation between calligraphy and technology as an art entering into immersive environments and interactive virtual spaces. Thus, the research interrogates the dimensionality of calligraphy to synthesize the notion of shufa (书法, the way of writing; the Chinese word for calligraphy) from the perspective of a choreographed performance within VR.

  • Pierre Shum (岑逸少) is currently a PhD student at School of Creative Media at The City University of Hong Kong, and a recipient of the prestigious Hong Kong PhD Fellowship Scheme (HKPFS). Born in Hong Kong and educated in the UK, his work is cross-cultural and multi-disciplinary, with experience in architecture, interior design, curation, urban planning, product design, graphic design, and art education. The current PhD research explores the possible evolutions of Chinese calligraphy into multi-dimensional digital media. The research seeks to distill the essential qualities of this traditional heritage, then expand the realization of calligraphy in various experimental augmentations with technology, and examine how the application of such technology could affect the discourse and future of Chinese calligraphy. Pierre holds a Master of Arts from the Royal College of Art, UK and a Bachelor of Architecture from the University of Bath (UK).
  • Tobias Klein (簡鳴謙) (DE/HK) is a trained Architect and Artist. His work articulates a syncretism of contemporary CAD/CAM technologies with site and culturally-specific design narratives, intuitive non-linear design processes, and historical cultural references. Through his works and writings, he established the notion of Digital Craft as an operational synthesis between digital and physical materials and tools as poetic (Poïesis) and technical (Technê) expressions, opposing a traditional dualistic separation of digital workflows and analogue making and material understanding. His works have been internationally exhibited at venues such as the London Science Museum, the V&A, the Venice Architectural Biennale, Ars Electronica, the Science Gallery (Melbourne), the container (Tokyo), the Bellevue Arts Museum, the MoCA Taipei, the Museum of Moscow, the Museum of Vancouver and at Art Basel Hong Kong. In 2020, The University Museum and Art Gallery, Hong Kong, opened a retrospective exhibition of his works from the past 15 years, accompanied by a publication with the same title as the exhibition, Metamorphosis or Confrontation. He holds a PhD from RMIT, a Master of Architecture and Diploma in Architecture with Distinction from UCL, Bartlett School of Architecture and a German Bachelor equivalent in Architecture from the RWTH Aachen. Before coming to Hong Kong he ran a postgraduate studio at the Architectural Association and the Royal College of Art in London, UK. https://www.scm.cityu.edu.hk/people/klein-tobias