My two week stay at STEIM was to enable me to work on mapping strategies for the Novint Falcon – a haptic USB device – that would help me consider how it might be used in a musical context.

The Falcon allows you to move it’s grip around in three dimensions, but what makes it particularly interesting is that it can also exert fairly powerful forces back on your hand. This makes the mapping layer in the software rather more involved than it is in non-haptic devices as the data generated by the controller can influence both sound and physical forces.

I have chosen to deal with this problem by creating phyisical metaphors for the control – where the controller affects a simulated physical system. I have therefore been primarily concerned with two mapping layers: from the controller to the physical model and from the model to the forces on the controller. The ones I have developed so far are:

– A wheel that can be turned. The momentum of the wheel can be felt as resistance if you push in the opposite direction or as an assisting force if you push in the same direction. With certain settings, once some momentum has been generated you can leave the controller to turn itself due to the forces on it. See the video examples below.

– A 3D grid that locks your hand into discreet cells. The force with which this happens can be varied to make it easier or harder to move between cells. The 3 dimensions of the grid can be also be individually altered (n x m x p). See the video examples below.

– A string model. 4 virtual strings are set up and positioned along the z axis. When they are struck, the falcon exerts a force away from the string bouncing your hand back.

I now hope to develop some instruments based on these models, which will mean grappling with a third mapping layer: from the model to the sound engine. It would also be interesting to try alternative mapping setups, such as analysing the audio produced and using that to determine the forces on the controller.

Video examples:

– 3D grid demo:

– 3D grid demo 2 (with simple audio mapping):

– Winding wheel example:

________________

On another note, I have also begun developing a patch that allows you to ‘feel’ a waveform by running your hand along it, and having your hand displaced vertically by the amplitude of the audio file at that point (this idea was suggested by Michael Young). This can also be expanded by allowing you to scan for different frequencies. For example, if set to 100 Hz it displaces your hand according to the amplitude of the sound file within a certain bandwidth around that frequency. As there are two dimensions to play with – y and z if the waveform is spread over the x-axis – two different frequencies can be scanned at once.

________________

I was also happy to present some of my past work on the Feedback Joypad instrument and give a short performance with it. More info at tommudd.co.uk/projects/joypad.htm

or for more info on me generally:

tommudd.co.uk

In July I spent two weeks at STEIM developing some early concepts for a new sound-theatre work called BOTCH.  The work will premiere in New York City in 2013 so there’s a lot of time to experiment with different ideas.  Basically I’m interested in finding interesting relationships between people and machines that highlight our contemporary co-dependence while finding what is is distinct about the human half of the equation.  It seemed to me that working with STEIM technology would be a good way to experiment with this since the institution is so involved with interface.

(more…)

My name is Freeka Tet and I am currently active in different projects aka Sgure, Gazormass, Chlorgeschlecht, and MAxibacon.
First I want to say thanks to all the Steim gang, especially Jun, Frank, and Taku.
Second I want to apology for my english ( yes i am french )

Ok so basically I was resident at STEIM begining of April 2010 for 2 things, hacking the Sixaxis, and build a glove with different sensors on it to replace the Wiimote I am currently using during my performance.

(more…)

The work to date on my project is documented here: http://cec.concordia.ca/econtact/12_3/scott_wigi.html

As usual I am very grateful to everyone at STEIM for this continuing residency. It makes things possible in days which would take months and other things which would never happen at all without STEIM’s support and influence.

On this visit we replaced one of the Wiimotes was using with a replacement handset and an arduino board: it has more and better sensors, a more reliable , tiny rechargeable batteries, it is more ergonomic. It took a couple of days to convert/recreate the programming. We ran out of time on this before thinking creatively about what new possibilities having two of these instead. But yet again I have the feeling it is again opening up the scope of what is possible. Next time we will replace the second handset reprogram JunXion.

Unanswered questions arising:

Could the XY (and potentially Z) sensors from the Lightning be replaced? Hack the existing transmitter/receivers or replace them. I would like a more compact system.

Visual feedback? It would be nice to have the current LiSa buffer represented in text form on an LCD. I also wonder if some of the controllers could be represented. As the system gets more complex I sometimes need to check the status of 4 different continuous controllers – I’m not sure how this could be done – perhaps a camera and screen could be combined in some way?

Related point: Ideally it would be nice to have LiSa/junXion running in an independent box, like an i-phone which could also function as infra-red camera and display- and get rid of the laptop completely – I guess the technology is not there for that yet but it is an interesting long term goal.

Vibration feedback to give an indication of when the edges of spatial regions are close.

Onwards and upwards!

STEIM has been collaborating for a while now with artist Richard Scott on the development of his electronic instrument, the WiGi. Richard wrote an interesting article on the instrument.

The author and WiGi at the Next Festival Bratislava 2009. Photo © Judith Goodman.

Getting WiGi with It:

Performing and Programming with an Infrared Gestural Instrument: A Case Study

by Richard Scott

Abstract

I offer an account of working with the Buchla Lightning MIDI controller and of the expansion of the instrument into the “WiGi” system, using Nintendo Wii-motes, JunXion and LiSa software, and made with the support and collaboration of STEIM (Studio for Electro-Instrumental Music). I will emphasise my background as an instrumentalist and consider some of its practical consequences for the project. I outline the various stages of development which have led to an expanded, fully functional performing instrument, and highlight some of the questions and decisions which occurred over a period of five years. I consider some æsthetic and other questions which arise from the process, some limitations of the system and my plans for its future development. Finally, I outline the various functions of the current instrument, the issues of mapping, scaling, buffers and illustrate my use of it.

You can read the whole story here.

RETINAn is a 15 minute performance that includes piano in a setting that immerse it with live video and electronics and putting the classical idiom into modern context.

It is an artistic study in the kinetic relationships between sound, image, motion and space. It takes its inspiration from the early cinematic practices of Étienne-Jules Marey, whose work in studying and photographing movement revolutionized the way we visualize time and motion, and puts them in the contemporary perspective.

The main question of this project is the ontology of the cinematic praxis: what is contained on the audiovisual material that allows the cinematic experience to be constructed? How is the narrative material developing? Is modern human differs in attention? How to present the material to the modern audience?

The soundscape of sober electronics promotes the image and brings the sound and time dimension to the visual aspect. The nostalgic and familiar sounds of piano bridge the modern and old sounds from the early cinematic world into the modern world.

Our goal is to explore the audiovisual phenomena form the perspective of emergence much like the image emerges onto the eye’s retina. The spectator is provided with an audio-visual objects that are observed by the virtual camera’s eye. This observation gives a dynamic perspective in time and space and allows the creation of narrative of sound, image, motion and space as an individual experience.

The inspirations of the project are the early cinematic practices of Étienne-Jules Marey and the Russian filmmaker Dziga Vertov. The deep technical limitations they faced and the creative solutions they came up with are a start point for the creative techniques that are used in this project.

We wanted to work in a supportive and stimulating environment, and STEIM was a perfect place for that. The best aspect was the support from Nico and his attentiveness to our needs.

The project was presented at STEIM, as well as during ‘SPOR’ festival in Aarhus (Denmark).

The subsequent performances are going to happen during Gaudeamus festival this year on September 10th and during Musica Electronica Nova Festival in Poland in May 2011.

Here are some pictures from the Antwerp show of Disco Debris. Unlike the Amiens installation which was up for 3 months, and had to be self-maintaining, i.e I couldn’t use wireless headphones because they would have required too much caretaking, the Antwerp show was just one day as part of Champ d’Action’s Time Canvas event. For this one I had the chance to expirement with how the piece worked with headphones, with an LED on the headphones (thanks Jun !).

While the tracking was more sensitive in the dark gallery space in Antwerp, because the light source was more focused (as opposed to the 4 theatre lights in Amiens ). The visual impact of the space was less impressive and therefore in some way there was less dialogue between the space and the sound.

In both cases one thing I’ve realised from the installation is to keep things very simple. In the art world as opposed to the music world, an audience is not used to sound abstraction, and need as much help as possible in terms of representational material. One also does not have a captive audience, and what that means for the interactivity is that  one has to look for the most transparent cause and effects of the movement while maintaining some kind of narrativite engagement to keep the audience in the space.

this was a real eye-opener for me about translating ideas from the concert hall to the gallery..

Disco Debris – an interactive sound installation

The public enters one person at a time into a dark neutral space in which they experience the sensation of literally walking through debris of frozen sound. One stumbles on a landscape of frozen voices, barely recognizable shards of 1970’s pop music, static bird song, broken pulses of disco music reduced to an almost Geiger like clicking, resonances of invisible spaces. These imaginary spaces are mapped onto a topography of intersecting voices and sounds, which slowly transform over time.

Technically this is achieved by using a video camera which tracks the movements of the audience so that the XY position is mapped directly onto the frozen moment of the sound heard back in the space or in the headphones. The sound is granulated and the position of the person in the space determines which moment of time is heard. (more…)

The first performance with our new limb-mounted Wiimotes instrument which we’re calling ‘Gaia’ took place in the Blue Room of the Conservatorium van Amsterdam last Friday 7th May. Heartfelt thanks are due to Jos Zwannenburg for going the extra mile and making this possible. Videos of the 3 pieces have now been uploaded to YouTube:

Threshold 1:

http://www.youtube.com/watch?v=XTJzSWHL1rc

Threshold 2:

http://www.youtube.com/watch?v=mJl3lxdj9WM

Threshold 3:

http://www.youtube.com/watch?v=I-NtGf1zjlk

(more…)