Abstract
Mixed reality art installation often contain complex interactivity and participatory requirements intermeshed with simulated synaesthetic and kinesthetic elements in real time that cannot be perceived by passive observation. These works can be approached conceptually as experimental creative laboratories exploring principles concerning enactive perception, embodied cognition, cross‑modal sensing and multi‑modal interactivity. Within these ‘laboratories’ the need for a complex conceptual framework that can be used to provide multiple interdisciplinary entry points and understandings is required. It is in the act of freely crossing and negotiating disciplinary boundaries when assessing these artistic multisensory environments that the practice becomes more deeply considered.
These works are sensing perceptual events (‘a speculative event’) that test definitions regarding embodied and extended cognition and modes of being across nonhuman and human form. It brings all forms into a relational practice based on process, iteration, encounter and exchange. Massumi states: “The form of the object is the way a whole set of active, embodied potentials appear in present experience: how vision can relay into kinesthesia or the sense of movement, how kinesthesia can relay into touch.” (1)
Raewyn Turner and Brian Harris (Aotearoa/New Zealand) are artists skilled in mixed reality installation who play with cross‑modal sensing and enactive perception. Turner and Harris have been creating a series of ‘backyard’ experiments that contemplate changes in the environment and atmosphere. Their recent works Moss (2014), Verdugos (2014) and Snatch (2013) will be examined. Moss takes data from real‑time ‘touching’ and remediates it into an auditory composition. As Turner explains: “When the moss is stroked it responds with piano notes. Data is captured from the audience’s touch, amplified and fed through an algorithm to produce a cultural sound. The hand on the land produces cultural artifacts” (2) Verdugos is created from pre‑bud fruit saplings wound with copper wire and is intended as an ‘atmosphere sifter’ receiving short‑wave radio signals.
In addition to the sensory and environmental concerns, these works reflect on decision making in a time of data‑driven reasoning. Their works scutinise the ownership and ‘truth values’ of data sets. They question data capture and support the debate posed by Gavin Starks: “Whose data and whose algorithms or interpretation are we trusting? Who wrote the algorithm? Who wrote the code? What are the unexpected consequences of combining different data?’ If our decisions are shaped by these data and interpretations what can we employ to see the bigger picture, how can we see past the algorithm?” (3)
References:
- Brian Massumi, “The Thinking‑Feeling of What happens. An interview with Brian Massumi”, in Interact or Die (Rotterdam: V2_publishing/NAi Publishers, 2007) 75‑79
- Raewyn Turner, personal correspondence to author 24th September 2014
- Gavin Starks, the CEO of the ODI (Open Data Institute) http://theodi.org/culture
- Deborah Lawler-Dormer, University of Auckland, New Zealand