[ISEA2011] Paper: Brigitta Zics – Eye Gaze as a Vehicle for Aesthetic Interaction: Affective Visualisation for Immersive User Experience


This paper will explore potential applications of aesthetic interaction with a particular perspective on emerging technologies that is inclusive of the user’s cognitive properties.  It investigates the philosophical concept of affection which in this  paper will be described  as the interplay between technological effect and affective human response. This paper argues that understanding such an interrelationship between technological effect and affective responses implies an enhanced affective intelligence of the system with a potential control of the user’s immersive experience.

In this investigation affection is applied through aesthetic interaction. This implies that there is a semantic framework developed  that is based on the affective capacity of audiovisual elements  (or “meta-meaning” ) rather than other coded references. As such the designer of such an affective system uses the semantics of technological effect to anticipate affective responses of the user in order to trigger the cognitive state of immersion. Such immersive experiences will be referred to here as a cognitive flow of the user with productive and potentially novel states of consciousness.

Applying the concept of affection in practice, the paper introduces the affective visualisation of “Mind Cupola”  which applies eye tracking technology to evaluate and respond to the changes in the  cognitive profile of the person. The paper highlights the development process of this affective visualisation and explains the technological aspects of the cognitively inclusive interaction design. It provides a conceptual overview of the interaction architecture including the system flow of the stages of ” limbo”, “meditative”, “chaos”, “eye-message board” and “golden state – nirvana”. It describes how eye tracking facilitates an aesthetic interaction through direct feedback and indirect effects to guide its user toward a productive immersive state. The paper concludes with the evaluation of this affective system regarding the concept of affection and how users reflected on the   degree of immersion and their quality of experiences.

  • Dr. Brigitta Zics is an artist and researcher based in Culture Lab Newcastle University, UK. Her main interest is the investigation into the potential of technology as a philosophical tool and aesthetic ecology in relation to human experiences. Her recent research subjects include experimental data visualisation,  bio-signal interaction & affective environments, application of swarm theory, and aesthetic ecologies that facilitate immersive experiences. Her artistic work and research have been broadly exhibited and published in journals such as Leonardo (MIT), Journal of Visual Art Practice (Intellect),  in books such as New Realities: Being Syncretic book (Springer) and festivals such as Siggraph. Other affiliations include Visiting Fellow at Transtechnology Research (University of Plymouth), Senior Lecturer at the University of Wales, panel member of the Leonardo Reviews and Research Advisor in the Doctoral School at the Hungarian University of Fine Art.

Full text (PDF) p. 2666-2671