The eyes of the robot collect up snippets of video and transmit them to the body when the buses they are on come within range of an appropriate wifi node.
The body sits in an art gallery. On the wall is a projection of its stream-of-consciousness editing, and on a separate screen are finished films which it completes nightly. Video received from the eyes is split into individual frames, which are analysed and stored. The output of the robot is made from arrangements frames and false memories that the robot forms from its favourite images.
The Filmmaking Robot was first exhibited in Wellington, New Zealand, in July 2004, and shown again in Auckland, July 2005. In each case the eyes were installed on local buses.
The eyes consist of a small computer with a camera and wifi card. Whenever the bus is on they are collecting video, and when an
appropriate access node is detected, the video is transferred to the body.
For about 12 hours a day the robot dwells on its memories, recalling well liked or pertinent imagery, and attempting to connect it through a sequence of frames, which is shown as video. Each frame is analysed to give 20 numbers summarising its qualities, and these numbers are treated as coordinates in a 20 dimensional space. Proximity in this space is used to determine similarity for the purposes of creating a video sequence. These numbers are also feed to neural networks which have been trained to like selections of fine art and images identified as well-composed by human subjects, and the response of these networks determines whether the robot likes an image.
At the end of the day, it looks over its production, and picks out a few minutes to release as a finished film. This is made available via http, and is usually burnt onto DVD and put on a screen in the gallery.
If it finds itself using an image too often, or finds a lonely image in a sparse part of the 20 dimensional space, it creates false memories based on that image. These confabulations are created by applying simple filters, such as blurring, recolouring, sharpening; by zooming, distorting or mirroring; and by blending different images together. This is intended to help the robot develop a full visual sense, beyond and between what it has seen.
When the robot’s memory is full (at 5,000,000 images, which has not happened yet), it will begin discarding the pictures it least likes or least needs, just as it reproduces the images it most likes and needs. This means that in its mature stage, the robot will produce more and more characteristic and stylised work.
The Filmmaking Robot retains memory between showings, so in Auckland it incorporated Wellington imagery, and in San Jose it will mix New Zealand images in with local footage.
Despite its name, the robot avoids autonomous movement.
- Douglas Bagnall, New Zealand halo.gen.nz/robot