Ross Bencina / Somaya Langley / Danielle Wilde > Gesture~Sound: Residency Report, July 9-30, 2007

During July this year we spent 3 weeks at STEIM with the goal of exploring and experimenting with new methods for controlling and performing computerised sound using whole-body gesture. Each of us came to the residency with different expertise, approaches and expectations. We aimed to learn from each other in a workshop type environment, as well as creating new ideas for performance. One guiding theme was Ross’ interest in creating systems which support a kind of kinaesthetic-auditory synaesthesia, where human body motion is mapped into sound in such a way that sound production becomes an inherent and unavoidable consequence of moving the body. Our approach was multifaceted and reflected the diversity of our interests. Considerations included: physicality in the space, sonic and compositional form, structure and aesthetics, conceptual semantics, sensor technologies and applications. These concerns were used as the basis for devising experiments, some of which were undertaken without interactive technology. For example in the early phases of the residency we experimented with movement-only composition, and later, some of the sound mappings were prototyped by improvising movement to either pre-recorded sound or live voice.

We focused on two sensor technologies: 3-axis accelerometers (deployed using 7 Nintendo Wiimotes), and a custom wireless ultrasonic range finding system which we developed to measure the distance between performers. Both sensor systems fed a computer running max/msp software which outputted Open Sound Control Protocol which was further processed in a custom version of AudioMulch using the Lua scripting language to specify the mappings between sensor data and sound. This approach was quite successful, although some technical limitations were encountered (for example we had some difficulty using six Wiimotes at once, and three stopped working over the course of the residency).

The residency had a duration of three weeks. The intended schedule was to complete the development of hardware and software systems in the first week in an artist’s workshop, and then to work in the large performance studio experimenting with the system for the following two weeks. In reality it was necessary to continue to work on technology development concurrently with performance and movement work throughout the duration of the residency. Overall we could have used more time, since the ultrasound system only became functional in the final week.

On the final Friday of the residency we invited peers to a work in progress presentation. Approximately 8 people attended the session including sound artists from the Amsterdam experimental music scene, Australian sound artists/producers and non-locals from the USA and Berlin. During the presentation we demonstrated a representative sample of the range of experiments we had undertaken. Feedback was positive, and people were generally impressed with the musicality of the experiments. Overall we found that the musicians in the audience were less interested in performances with strong conceptual or theatrical basis if the sound itself was too simple, or too predictable. It is clear that further work remains to fully comprehend the feedback and outcomes we produced.

We intend to further document this work on the web, including:

– Ross Bencina ( Somaya Langley ( and Danielle Wilde (


Comments are closed.