[ISEA2024] Paper: Elke Reinhuber, Benjamin Seide & Ross A. Williams — Different Layers of Reality. A Retrospective Evaluation on Diverse Possibilities for digitally preserving memories of the Yunnan Garden as an Immersive Experience

Abstract

Keywords:
Digital Ecologies, Digital Preservation, 360-degree Video, Immersive Experience, Virtual environments, Photogrammetry, Interactions, Human and Non-human Agents, Gaussian Splatting

This paper offers an in-depth evaluation and future outlook of the representation of Singapore’s Yunnan Garden, with a central focus on various methods to create an immersive experience. The methods explored include 360-degree video, photogrammetric capture, and detailed photorealistic VR reconstruction. We critically assess their effectiveness and potential in the preservation of cultural and natural heritage in the digital world. The incorporation of interactive elements and the inclusion of motion capture of performers are also thoroughly examined. Challenges faced during implementation are discussed and solutions proposed for future attempts. The paper concludes with an exploration of the current approach to working with Gaussian Splatting as a future outlook. The research goes beyond the objective of preservation by reconstructing the former garden into a virtual cultural heritage application. It delves into diverse narrative and aesthetic experiments, aiming to create artistic interpretations that engage the audience. This endeavour challenged the project team to explore novel visual and aural forms of representation.

The authors currently teach and research in Media Art, Benjamin Seide and Ross Williams at ADM, School of Art, Design and Media, NTU Singapore, Elke Reinhuber at SCM School of Creative Media, City University of Hong Kong. With their experience and specific expertise in the areas of sound design (Williams), computer animation and visual effects (Seide) as well as camera and concept (Reinhuber), they explore the fascination and possibilities of immersive media from different points of view, especially in regard to representations of culturally relevant subjects.

  • Elke Reinhuber is a media artist, educator, researcher and Associate Professor at SCM City University of Hong Kong. In her work she explores different modes of presentation and strategies of storytelling to emphasise the parallel existence of multiple truths of the here and now, anchored in expanded photography and spanning into several disciplines such as time based media, immersive environments, Augmented and Virtual Reality as well as performance. Her award winning artistic research was presented internationally, at conferences, exhibitions and festivals. www.eer.de
  • Benjamin Seide teaches and researches in Media Art at ADM, School of Art, Design and Media at NTU Singapore. With his experience and expertise in the areas of computer animation and visual effects, he explores the possibilities of immersive media, especially in regard to representations of culture and heritage. www.ataribaby.de.
  • Ross Williams, ADM, School of Art, Design and Media, NTU Singapore