Abstract

Alongside changing processes and theoretical frameworks for making art, artist-researchers working with digital technology are extending the physical interface by incorporating gestural control into their work, looking at ways to read human actions and ways to exploit the ways in which people understand the world through their body. Applications of gestural controllers, such as these, can be seen in a wide variety of presentation formats, from interactive installation and performance to gaming. Such technologies allow for the design of novel interactive experiences but challenges still remain in designing controllers that support expressivity, meaningful interaction and intuitive control.

Embodied Cognition is an area of cognitive science that focusses on perception, cognition and action as profoundly shaped by the human experience of having a body and living in a physical environment and in a culture. It offers a new way and an alternative approach that complements current initiatives in the design of interactive technologies and gestural controllers. Embodied Cognition offers a novel way to analyse the complex interactions between user and technology in terms of the fundamental categories of embodied existence.

Image Schemata are fundamental categories that are neurally encoded, pre-conceptual symbols that are recruited from experiences of bodily movement and perceptual interaction, including the body’s interaction with its environment in terms of spatial relations, perception of force and magnitude. These categories are involved in organizing mental representations into meaningful coherent units and are implicated in the formation of new concepts. This paper looks at the design of meaningful controllers that depend upon empirical knowledge of fundamental categories in terms of how the body interacts in and understands its environment. It proposes that these structures may be used in order to identify correlations between gesture and other phenomena such as sound and colour. Such correlations are the necessary building blocks of a grounded cross-modal mapping schema on which to base the design of controllers that allow for meaningful gestural interaction with music and image.

In doing so, it presents an interdisciplinary approach to the design of interfaces to digital technology, one that can have considerable impact in the arts and technology domain.

• Mark Stephen Linnane is a video artist, creative technologist and researcher who makes work for dance, theatre and music. His work has been shown in the USA, Holland, Canada, Bulgaria, Germany and Italy. He lectures at Trinity College, Dublin, Ireland, where he is is currently pursuing PhD research in performance technology. Previous work includes Cortisol/Retinol (2008-2010), a music/visual performance, funded by the Arts Council; Walk Don’t Run (2008), a dance film in association with Catapult Dance Company, commissioned by Project Arts Centre and funded by the Arts Council and The Weathering (2004), a performance film for Donnacha Dennehy’s music which was premiered at Lincoln Centre, New York in 2004.  vimeo.com/marklinnane
• Prof. Linda Doyle is Director of `CTVR/the telecommunications research centre’ and a faculty member in the School of Engineering in Trinity College, University of Dublin, Ireland. CTVR is a national centre comprising six different universities and involving over 100 researchers. Prof. Doyle’s research focuses on wireless networks, reconfigurable systems and spectrum management regimes. Prof. Doyle is an international leader in cognitive radio research and her group has built an international reputation in experimental cognitive radio work and shown how these new kinds of communications systems can strongly impact on society. In addition to mainstream telecommunications research Prof. Doyle has a significant interest in art and technology and has been involved in a wide range of collaborative interdisciplinary projects in the last decade. One of her current projects is focused on the 1953 film by Ray and Charles Eames- ‘A Communications Primer’.
• Dermot Furlong  [ISEA2011 provided no biographical information]

Full text (PDF) p. 1551-1556