[ISEA2023] Paper: Ziwei Wu, Danlu Fei, Xinyu Ma, Mingming Fan & Kang Zhang — NER: Physical-Virtual Multimodal Generative NFT with a Rarity Model


Full Paper Theme: NFT – Blockchains Subtheme: Symbiotic Organizations

Nowadays, NFTs are increasingly emerging into public view. It is necessary to consider NFT as a sustainable trading model for media arts with the audience interaction as the symbiosis community.

This paper presents a design architecture of a Multimodal NFT with generative patterns triggered by sensors. By designing various 3D mesh attributes, such as eyes pattern and fur color of a robin bird, the authors generate a series of different NFT 3D artworks. We also demonstrate our experience with the generative system together with the rarity scoring model that could be used to evaluate the heterogeneity of NFT collections.

  • Ziwei Wu is a media artist and researcher who was born in Shenzhen, China. She is a Ph.D. student in Computational Media and Arts (CMA) at HKUST. Her artworks are mainly based on biology, science, and the influence on society by using a range of media. She is a Lumen prize winner; Batsford prize winner. Her research was published in the SIGGRAPH Art Program and EVA London. She has exhibited at international venues, including CYFEST in Southeastern Europe, Norwegian BioArt Arena (NOBA)-Vitenparken in Norwegian, Watermans Gallery and Cello Factory in London, NeurIPS AI Art Gallery, ArtMachine2 in Run Run Shaw Creative Media Centre in Hong Kong and so on.
  • Danlu Fei is a game developer and researcher. She owns a bachelor degree in Computer Science – Computer game and graduated with a 3.97/4.0 GPA from McGill University, Canada. She worked as a gameplay programmer intern at several well-known game companies, including Ubisoft, Behaviour Interactive, and Netease. She is pursuing her Ph.D degree in Computational Media and Arts (CMA) at HKUST. Her research interest is mainly around procedural content generation (PCG) in games, especially AI-based PCG. Her research was published in the CHI Play conference proceedings.
  • Xinyu Ma is an art practitioner residing in Shenzhen and Hangzhou, China, who holds a bachelor’s degree in sculpture and a master’s in fine arts. Currently, he is pursuing a master of philosophy degree at the Hong Kong University of Science and Technology (Guangzhou). His work focuses on experimenting with various forms of public system structures, investigating their modes of operation in an individual, fictional, and fictitious way. In addition, he also researches the topics of the public sphere, philosophy of technology, urban space, community, play, and political aesthetics.
  • Mingming Fan is an Assistant Professor in the Computational Media and Arts (CMA) Thrust at The Hong Kong University of Science and Technology (Guangzhou) and in the Division of Integrative Systems and Design at The Hong Kong University of Science and Technology. Dr. Fan is the director of the Accessible and Pervasive User EXperience (APEX) Group, which conducts research at the intersection of Human-Computer Interaction (HCI), AI, and Aging & Accessibility. Dr. Fan’s research has been published in top-tier venues in HCI (e.g., CHI, TOCHI) and received many paper awards, including the best paper award at CHI and the best paper honorable mention award at UbiComp. He is a Siebel Scholar. https://www.mingmingfan.com
  • Kang Zhang is Professor of Computational Media and Arts, Hong Kong University of Science and Technology (Guangzhou) and of Division of Emerging Interdisciplinary Areas at the Hong Kong University of Science and Technology, and Professor Emeritus of Computer Science at the University of Texas at Dallas. He was a Fulbright Distinguished Chair and ACM Distinguished Speaker, and held academic positions in USA, Czech Republic, Australia, and UK. Dr. Zhang has delivered keynotes in computer science, architecture, art, design, and psychology areas, and won numerous art and design awards. Interdisciplinary research topics of computational aesthetics and aesthetic computing, and their differences and relationships. https://cma.hkust-gz.edu.cn/people/kang-zhang