[ISEA2015] Artist Statement: Ben Lacker — New Interfaces for Data-Driven Musical Interaction

Artist Statement

Keywords: New Musical Interfaces, Web Audio, Music Information Retrieval, Computer Music, Algorithmic Composition

The availability of advanced audio analysis software opens up new ways of interacting with music that fall in between traditional concepts of composing, remixing, and listening. Here I present my ongoing work in this domain, focusing on two recent projects that use such data in the service of web browser based musical interfaces: Jazz Drum Machine, which uses detailed beat-timing data to create idiosyncratic drumming patterns; and Venetian Spiral, which explores mathematical properties of medieval lute music by algorithmically reordering its notes.

Jazz Drum Machine
Jazz Drum Machine is a web application that asks users to select from several recordings of drum solos by famous jazz drummers. It then uses the Echo Nest Remix API2 to analyze the selected recording, split it up into individual beats, and divide the beats into clusters based on their timbres. The clusters correspond loosely to different types of drum sounds, such as loud snare sounds, soft snare sounds, kick drum and cymbal hits, low tom sounds, and high tom sounds. The application then uses the Web Audio API to play back a one-measure loop, at the original song’s tempo, with controls for how many beats from each cluster of sounds are heard in the loop. Each sound is played back at its original place within the measure. For instance, if the control corresponding to the group of “soft snare sounds” is turned up halfway, half of the beats containing such sounds will be played back in the loop. Furthermore, if these beats tend to fall on the second and fourth beats of the measure in the original recording, they will be heard on those same beats in the resulting loop. By manipulating the controls while the loop plays, the user can change the drum beat, adding more of certain sounds and less of others, while always maintaining the rhythmic nuances of the original song.

Venetian Spiral
Venetian Spiral is another web application that uses audio analysis and the Web Audio API to turn an existing piece of music into a sort of instrument. It links the notes of a piece of medieval lute music to forms in a simple, algorithmically driven animation. The animation is based on phyllotaxis, the mathematical formula that describes the arrangement of leaves on a plant stem. In place of leaves, this animation consists of a series of dots, one for each note in the lute composition, initially arranged in a spiral. The Python package Librosa4 was used to detect the onset times of each note in the composition. Each note is mapped to one dot in the animation; the dots rotate in polar coordinate space, and as each dot passes the polar axis, the corresponding note is played. As the animation progresses, the notes from the lute composition are heard in various orders and combinations, creating musical patterns that reflect both the visual patterns displayed in the browser and the original sequence of notes in the composition. These patterns shift and evolve to create a generative, ever-changing composition that continues indefinitely. echonest.github.io/remix github.com/bmcfee/librosa