[ISEA2019] Paper: Songwei Ge, Austin Dill, Eunsu Kang, Chun-Liang Li, Lingyao Zhang Manzil Zaheer & Barnabas Poczos — Developing Creative AI to Generate Sculptural Objects

Abstract

Keywords: Partitioned DeepDream, Amalgamated DeepDream, 3D, Point Cloud, Sculpture, Art, Interactive Installation, Creative AI, Machine Learning

We explore the intersection of human and machine creativity by generating sculptural objects through machine learning. This research raises questions about both the technical details of automatic art generation and the interaction between AI and people, as both artists and the audience of art. We introduce two algorithms for generating 3D point clouds and then discuss their actualization as sculpture and incorporation into a holistic art installation. Specifically, the Amalgamated Deep Dream (ADD) algorithm solves the sparsity problem caused by the naive DeepDreaminspired approach and generates creative and printable point clouds. The Partitioned DeepDream (PDD) algorithm further allows us to explore more diverse 3D object creation by combining point cloud clustering algorithms and ADD.

  • Songwei Ge is a Masters student in the Computational Biology Department at Carnegie Mellon University, USA.
  • Austin Dill is a Masters student in the Machine Learning Department at Carnegie Mellon University, USA.
  • Chun-Liang Li is a PhD candidate in the Machine Learning Department of Carnegie Mellon University, USA. He received IBM Ph.D. Fellowship in 2018 and was the Best Student Paper runner-up at the International Joint Conference on Artificial Intelligence (IJCAI) in 2017. His research interest is on deep generative models from theories to practical applications.
  • Dr. Eunsu Kang is a Korean media artist who creates interactive audiovisual installations and AI artworks. Her current research is focused on creative AI and artistic expressions generated by Machine Learning algorithms. Creating interdisciplinary projects, her signature has been seamless integration of art disciplines and innovative techniques. Her work has been invited to numerous places around the world including Korea, Japan, China, Switzerland, Sweden, France, Germany, and the US. All ten of her solo shows, consisting of individual or collaborative projects, were invited or awarded. She has won the Korean National Grant for Arts three times. Her researches have been presented at prestigious conferences including ACM, ICMC, ISEA, andNeurIPS. Kang earned her Ph.D. in Digital Arts and Experimental Media from DXARTS at the University of Washington. She received an MA in Media Arts and Technology from UCSB and an MFA from the Ewha Womans University. She had been a tenured art professor at the University of Akron for nine years and is currently a Visiting Professor with emphasis on Art and Machine Learning at the School of Computer Science, Carnegie Mellon University.
  • Lingyao Zhang earned her Master’s degree in the Machine Learning Department at Carnegie Mellon University, USA
  • Manzil Zaheer earned his Ph.D. degree in Machine Learning from the School of Computer Science at Carnegie Mellon University, USA, under the able guidance of Prof Barnabas Poczos, Prof Ruslan Salakhutdinov, and Prof Alexander Smola. He is the winner of Oracle Fellowship in 2015. His research interests broadly lie in representation learning. He is interested in developing large-scale inference algorithms for representation learning, both discrete ones using graphical models and continuous with deep networks, for all kinds of data. He enjoys learning and implementing complicated statistical inference, data-parallelism, and algorithms in a simple way.
  • Dr. Barnabs Pczos is an associate professor in the Machine Learning Department at the School of Computer Science, Carnegie Mellon University, USA. His research interests lie in the theoretical questions of statistics and their applications to machine learning. Currently he is developing machine learning methods for advancing automated discovery and efficient data processing in applied sciences including healthsciences, neuroscience, bioinformatics, cosmology, agriculture, robotics, civil engineering, and material sciences. His results have been published in top machine learning journals and conference proceedings, and he is the co-author of 100+ peer reviewed papers. He has been a PI or co-Investigator on 15+ federal and non-federal grants. Dr. Poczos is a member of the Auton Lab in the School of Computer Science. He is a recipient of the Yahoo! ACE award. In 2001 he earned his M.Sc. in applied mathematics at Eotvos Lorand University in Budapest, Hungary. In 2007 he obtained his Ph.D. in computer science from the same university. From 2007-2010 he was a postdoctoral fellow in the RLAI group at University of Alberta, then he moved to Pittsburgh where he was a postdoctoral fellow in the Auton Lab at Carnegie Mellon from 2010-2012.

Full text (PDF) p. 225-232