[ISEA2020] Workshop: Eunsu Kang, Jean Oh & Robert Twomey — Measuring Computational Creativity: Collaboratively Designing Metrics for Evaluating Creative Machines

Workshop Statement

How do we make a creative machine? Creativity involves “a multitude of definitions, conceptualizations, domains, disciplines that bear on its study, empirical methods, and levels of analysis, as well as research orientations that are both basic and applied – and applied in varied contexts.” This workshop follows a series of studies conducted in the classroom setting at CMU and UC San Diego. To begin, participants were asked to provide a definition of creativity specific to machines. Then more questions emerged: Can we computationally model ambiguity? Would a novelty search result in valuable discoveries? Where is the threshold between randomness and creativity? How do we evaluate the creativity of an algorithm? To answer these questions, participants developed sets of criteria to assess their own and peer groups’ creative AI and ML projects. Although such a human-centred method is subjective, participants found that the exercise helped them to better describe and interpret dimensions of algorithmic creativity.

This half-day workshop extends these methods and engages a broader arts and machine learning community to collaboratively define quantitative metrics assessing the creativity of algorithms and machines. This workshop is a first attempt to establish evaluation metrics for the area of creative AI. Maximum of participants: 30, Duration: 4 hours.

  • Eunsu Kang is a Korean media artist making interactive art installations and performances. She is also a researcher on the possibility of creative AI and an educator teaching art-making using machine learning methods. Her career started as a self-taught video artist in Seoul, Korea. Having over 100 exhibitions and constantly studying new technologies for two decades, her works have transformed into interactive and interdisciplinary art projects. She has won the Korean National Grant for Arts three times. Her works have been invited to exhibitions around the world and presented at conferences such as ACM, ICMC, ISEA, SIGGRAPH Asia and NeurIPS. A couple of years ago she left her tenured art professorship to focus on research at the intersection of art and machine learning. Most recently she taught Art and Machine Learning and Creative AI courses at the Machine Learning department of Carnegie Mellon University, USA.
  • Jean Oh is a faculty member at the Robotics Institute at Carnegie Mellon University, USA. Jean is passionate about creating persistent robots that can co-exist and collaborate with humans in shared environments, continuously learning to improve themselves over time through training, exploration, and interactions.Jean heads an interdisciplinary research group, Bot Intelligence Group (BIG); her team has won two Best Paper Awards in Cognitive Robotics at IEEE International Conference on Robotics and Automation (ICRA) in 2015 and 2018 for the works on following natural language directions in unknown environments and socially-compliant robot navigation in human crowds, in 2015 and 2018, respectively. Jean received her Ph.D. in Language and Information Technologies at Carnegie Mellon University, M.S. in Computer Science at Columbia University, and B.S. in Biotechnology at Yonsei University in South Korea.
  • Robert Twomey is an artist and engineer exploring the poetic intersection of human and machine perception. He has presented his work at SIGGRAPH, the Museum of Contemporary Art San Diego, and his research has been supported by Microsoft, Amazon, and NVIDIA. Twomey received his BS from Yale (USA) with majors in Art and Biomedical Engineering, his MFA from UC San Diego, and his PhD in DXARTS from the University of Washington. He is a postdoctoral with the Arthur C. Clarke Center for Human Imagination at UC San Diego, and an instructor in Data Science, Electrical Engineering, and Visual Arts.