My two week stay at STEIM was to enable me to work on mapping strategies for the Novint Falcon – a haptic USB device – that would help me consider how it might be used in a musical context.
The Falcon allows you to move it’s grip around in three dimensions, but what makes it particularly interesting is that it can also exert fairly powerful forces back on your hand. This makes the mapping layer in the software rather more involved than it is in non-haptic devices as the data generated by the controller can influence both sound and physical forces.
I have chosen to deal with this problem by creating phyisical metaphors for the control – where the controller affects a simulated physical system. I have therefore been primarily concerned with two mapping layers: from the controller to the physical model and from the model to the forces on the controller. The ones I have developed so far are:
– A wheel that can be turned. The momentum of the wheel can be felt as resistance if you push in the opposite direction or as an assisting force if you push in the same direction. With certain settings, once some momentum has been generated you can leave the controller to turn itself due to the forces on it. See the video examples below.
– A 3D grid that locks your hand into discreet cells. The force with which this happens can be varied to make it easier or harder to move between cells. The 3 dimensions of the grid can be also be individually altered (n x m x p). See the video examples below.
– A string model. 4 virtual strings are set up and positioned along the z axis. When they are struck, the falcon exerts a force away from the string bouncing your hand back.
I now hope to develop some instruments based on these models, which will mean grappling with a third mapping layer: from the model to the sound engine. It would also be interesting to try alternative mapping setups, such as analysing the audio produced and using that to determine the forces on the controller.
Video examples:
– 3D grid demo:
– 3D grid demo 2 (with simple audio mapping):
– Winding wheel example:
________________
On another note, I have also begun developing a patch that allows you to ‘feel’ a waveform by running your hand along it, and having your hand displaced vertically by the amplitude of the audio file at that point (this idea was suggested by Michael Young). This can also be expanded by allowing you to scan for different frequencies. For example, if set to 100 Hz it displaces your hand according to the amplitude of the sound file within a certain bandwidth around that frequency. As there are two dimensions to play with – y and z if the waveform is spread over the x-axis – two different frequencies can be scanned at once.
________________
I was also happy to present some of my past work on the Feedback Joypad instrument and give a short performance with it. More info at tommudd.co.uk/projects/joypad.htm
My name is Freeka Tet and I am currently active in different projects aka Sgure, Gazormass, Chlorgeschlecht, and MAxibacon.
First I want to say thanks to all the Steim gang, especially Jun, Frank, and Taku.
Second I want to apology for my english ( yes i am french )
Ok so basically I was resident at STEIM begining of April 2010 for 2 things, hacking the Sixaxis, and build a glove with different sensors on it to replace the Wiimote I am currently using during my performance.
As usual I am very grateful to everyone at STEIM for this continuing residency. It makes things possible in days which would take months and other things which would never happen at all without STEIM’s support and influence.
On this visit we replaced one of the Wiimotes was using with a replacement handset and an arduino board: it has more and better sensors, a more reliable , tiny rechargeable batteries, it is more ergonomic. It took a couple of days to convert/recreate the programming. We ran out of time on this before thinking creatively about what new possibilities having two of these instead. But yet again I have the feeling it is again opening up the scope of what is possible. Next time we will replace the second handset reprogram JunXion.
Unanswered questions arising:
Could the XY (and potentially Z) sensors from the Lightning be replaced? Hack the existing transmitter/receivers or replace them. I would like a more compact system.
Visual feedback? It would be nice to have the current LiSa buffer represented in text form on an LCD. I also wonder if some of the controllers could be represented. As the system gets more complex I sometimes need to check the status of 4 different continuous controllers – I’m not sure how this could be done – perhaps a camera and screen could be combined in some way?
Related point: Ideally it would be nice to have LiSa/junXion running in an independent box, like an i-phone which could also function as infra-red camera and display- and get rid of the laptop completely – I guess the technology is not there for that yet but it is an interesting long term goal.
Vibration feedback to give an indication of when the edges of spatial regions are close.
STEIM has been collaborating for a while now with artist Richard Scott on the development of his electronic instrument, the WiGi. Richard wrote an interesting article on the instrument.
Performing and Programming with an Infrared Gestural Instrument: A Case Study
by Richard Scott
Abstract
I offer an account of working with the Buchla Lightning MIDI controller and of the expansion of the instrument into the “WiGi” system, using Nintendo Wii-motes, JunXion and LiSa software, and made with the support and collaboration of STEIM (Studio for Electro-Instrumental Music). I will emphasise my background as an instrumentalist and consider some of its practical consequences for the project. I outline the various stages of development which have led to an expanded, fully functional performing instrument, and highlight some of the questions and decisions which occurred over a period of five years. I consider some æsthetic and other questions which arise from the process, some limitations of the system and my plans for its future development. Finally, I outline the various functions of the current instrument, the issues of mapping, scaling, buffers and illustrate my use of it.
“The fairy tale of Sensitive to Pleasure tells about an artist that has created her piece for her own physical pleasure. I act as myself, the artist, dealing with the ambiguity of being in love with my own piece, a creature inside an ambisonics cube capable of revealing the sound of her body when in contact with other human beings. I created it to be pleased with but, as the creature has human-like thinking behaviors, my ideal lover has left me behind. I’m standing still outside the ambisonics cube. I become the doorkeeper of the interaction between the visitor and her. I am condemned to see a visitor entering alone and having pleasure with her without looking at it and what I get back is a very physical pain, which might be also pleasant for me; this is part of the piece. This is a reflection of an event that everybody can recognizes. And beside this human life aspect I like to transport this feeling into the artworld. Of course I’m not speaking of a lover but about my artwork. This is basically the underlying idea behind this piece.”
Below are some photos, more info on the dedicated blog: stp.nimk.nl.
Looking for ways to play with audio software and my cello simultaneously without losing touch of my instrument I have developed “Fello” at STEIM since 2005.
In the recent version this system consists of one Nunchuck sensor (a part of the WiiMote) attached to the frog of my bow, one MIDI controller and one computer. The sensor’s movement data is translated to MIDI through junXion and allows for manipulation of an audio delay setup in Ableton Live (using six copies of the fantastic OhmBoyz Delay) and LiSa.
The general approach is to record the cello sound into the buffer and play it back immediately, while the x and y movements of the bow control delay feedback and time.
In my studio compositions I have always looked out for sounds like drum rolls, made of any other sound material. Playing sounds like drawing dots and lines, for the creation of beat structures. With “Fello” these kinds sounds can be played directly. It’s fun and inspiration!
Additionally to the accelerometer sensor on the bow I use a BCR2000 MIDI controller plus two foot switches to choose the midi channel(s) on which the bow data shall be sent. The function of the bow interface can change through a quick touch of a button or pedal.
Last year I have played “Fello” in the dance performance “()else” by Victoria Hauke from Hamburg, which has taken me even to a dance festival in India in November. There I met with DJ Prashanth from Hyderabad, who plays traditional Indian percussion sounds through a turntable/ laptop setup. For late 2010 we are planning a tour through India and Sri Lanka with support of the Goethe Institute.
As preparation for this tour, I spent some time re-working my junXion configuration. It’s clean and stable now, simpler than before but with a few more options: Robert helped me taking a closer look at LiSa and using it additionally to Ableton, which works well and sounds really good. Thanks Robert!
Byung-Jun Kwon made a backup sensor for me, very beautiful! Jun is great support. His advice is to switch to another sensor-system in the near future, as the WiiMote’s accelerometer can be unreliable when too much bluetooth is around. We’ll hopefully do this in summer.
For the last three days of my residency, Axel Kochmeier (DJ akaak) joined me in the studio.
We will play a duo show called “Sutsche & Fello” in Berlin’s awesome Berghain Club on April 14th. He plays and mixes Techno Maxis pitched down to 33rpm. Here’s a video snippet of our rehearsal:
And here’s a two minute track which I composed only with “Fello” and a metal pipe:
Emotion Light, work in progress, (c) Adinda van 't Klooster, 2009
Project team:
Artist: Adinda van ‘t Klooster (concept, sculptural design, porcelain model, electro acoustic composition and project management)
Hardware design and advice: Ben Knapp, Ken Brown, Marc Boon, Bob Young, Belvin Ho
Software: Vincent Akkermans, Ken Brown, Miguel Angel Ortiz-Perez & Nick Ward
Rapid Prototyping: advice: Alan Stafford, AMAP, University of Sunderland
3d modelling: Neil Milburn, Iain Barrett, Dave Knapton, AMAP, University of Sunderland
The residency at STEIM was made possible by a small grant for the arts from the Arts Council England.
STEIM RESIDENCY 7-22 April 2009
Adinda van ‘t Klooster is an international artist and currently an AHRC funded PhD researcher at CRUMB, Sunderland University. She creates responsive artworks using sensors, light and sound. Her contextual research focuses on artworks which use biofeedback in particular.http://www.axisweb.org/seCVPG.aspx?ARTISTID=8405
The aim of the residency at STEIM was to further develop the first prototype ‘Emotion Light’: a sculptural light which uses biofeedback technology to visualize the holder’s physiological state in changing light patterns emerging from a portable, wireless sculpture. (more…)
i had two very fruitful weeks of isolated, concentrated, fantastically experimental time in steim’s studio II (composition). for the last 3 years i have lived in new jersey, in the u.s., and have had access to well stocked recording and electronics studios. this year my wife and i are living in paris, our apartment is rather small. my steim residency was for me an opportunity to catch up on my last 6 months of ideas, and assemble parts for my next 6 months of projects.
on my arrival, i re-arranged the entire room and created a very nice u-shaped workbench. i set up my current performance equipment on one table, a soldering workbench on another, a ‘dirty table’ behind me and a small table for my computer to my right. this let me (with the help of a rather incredible swivel-chair) let my mind flit from idea to idea without clearing space to work and in-so-doing wasting time.
i have been using a 3 x 3 matrix switcher/mixer in my live performances for about a year, it uses ‘competition rated gold-microswitch arcade buttons’ which you’d find in an console game like galaga or space invaders:
the benefits to these buttons is that they act extremely fast, they are ‘user-interface’ friendly and they are bright red (which, next to a ‘blue-led’ is the second coolest thing you can use in an interface).
the downside is that they are noisy (they click loudly when pressed), they are momentary (i can’t find a state i like and leave it without the use of something heavy), and they are huge… here’s the inside:
the third thing, is that this particular mixer, has some minimal cross-talk. i use it for feedback, so this is a good thing.
i set myself to building a new matrix switcher that:
a) had on-off-(on) toggle switches to allow the ’saving of states’
b) was MUCH smaller
c) and had worse cross-talk
The new matrix mixer:
is 3″ x 3″ with a matrix of 3 x 3 (nkk) on-off-(on) toggles with switchcraft jacks. the wiring on the inside is a bit different then the original switcher (i’d open it and take a picture, but i fear the pandora within). as such it is not much of a switch but a non-linear feedback matrix. the inputs in each row are wired to each other as well as the outputs. i used some very nice metal film resistors for that ‘hi-fi’ edge over the generic carbon-film resistors of it’s predecessor.
the main difference is in the size and play-ability. the small size is more portable for sure, but it is VERY different to play than the old red-button switcher in-so-far-as it i can’t memorize hand patterns to bring back ‘certain’ feedback areas. i can however leave the setting ’set’ using the fixed toggles. this lets me use the momentary toggles to perform-from a setting which previously required a number of heavy objects to hold down buttons or a poly-dactyl amount of fingers.
the next stage will be to replace the switches entirely with digital switches and variable digital resistors in the place of the summing resistors so that the 100% analogue routing and feedback can be controlled VERY slowly via computer. this is, however, another project all together which will most likely involve a ouiji board to ask david tudor for advice. so far the digital resistors that i’ve tried make a clicking or stepping between values… any thoughts on this would be most welcomed.
On december 2007, I started to research in STEIM the possibilities to build a midi-ratchet, to eventually make a project with Electronic Hammer. During summer 2008 I put myself the task to build the ratchet from zero for both hardwood & hardware (using an arduino BT) and necessary programing.
With the wonderful cratfmanship of Sean Little (sglittle@gmail.com) the woodwork of a sturdy but yet elegant ratchet came to life. For playing purposes I redesigned the instrument: A second handle was added, (see picture 1a) so that the percusionist can create leverage with both hands, and therefore be able to play controlled rhythms rather than only swirling the thing around. I had 4 of them made.
Next was the turn of the hardware. Because of the rotating nature of the ratchet, the logical way to go was to make a wireless connection to avoid cable entanglement. The arduino BT came out to be the best option, even though thanks the bluetooth the circuit is a bit pricey (90 euros).
)
03
06
09
12
15
18
21
24
27
30
