My two week stay at STEIM was to enable me to work on mapping strategies for the Novint Falcon – a haptic USB device – that would help me consider how it might be used in a musical context.

The Falcon allows you to move it’s grip around in three dimensions, but what makes it particularly interesting is that it can also exert fairly powerful forces back on your hand. This makes the mapping layer in the software rather more involved than it is in non-haptic devices as the data generated by the controller can influence both sound and physical forces.

I have chosen to deal with this problem by creating phyisical metaphors for the control – where the controller affects a simulated physical system. I have therefore been primarily concerned with two mapping layers: from the controller to the physical model and from the model to the forces on the controller. The ones I have developed so far are:

– A wheel that can be turned. The momentum of the wheel can be felt as resistance if you push in the opposite direction or as an assisting force if you push in the same direction. With certain settings, once some momentum has been generated you can leave the controller to turn itself due to the forces on it. See the video examples below.

– A 3D grid that locks your hand into discreet cells. The force with which this happens can be varied to make it easier or harder to move between cells. The 3 dimensions of the grid can be also be individually altered (n x m x p). See the video examples below.

– A string model. 4 virtual strings are set up and positioned along the z axis. When they are struck, the falcon exerts a force away from the string bouncing your hand back.

I now hope to develop some instruments based on these models, which will mean grappling with a third mapping layer: from the model to the sound engine. It would also be interesting to try alternative mapping setups, such as analysing the audio produced and using that to determine the forces on the controller.

Video examples:

– 3D grid demo:

– 3D grid demo 2 (with simple audio mapping):

– Winding wheel example:

________________

On another note, I have also begun developing a patch that allows you to ‘feel’ a waveform by running your hand along it, and having your hand displaced vertically by the amplitude of the audio file at that point (this idea was suggested by Michael Young). This can also be expanded by allowing you to scan for different frequencies. For example, if set to 100 Hz it displaces your hand according to the amplitude of the sound file within a certain bandwidth around that frequency. As there are two dimensions to play with – y and z if the waveform is spread over the x-axis – two different frequencies can be scanned at once.

________________

I was also happy to present some of my past work on the Feedback Joypad instrument and give a short performance with it. More info at tommudd.co.uk/projects/joypad.htm

or for more info on me generally:

tommudd.co.uk

After Orientation at STEIM I was full of inspiration and impressions. Thinking and planning new musical interface got the new contures and perspectives what was the very encouraging impulse for me. Some small but very important “pushes” and ideas was the main gains of Orientation for me. Few words are sometimes enough…
Recently I have to finish some opened projects to be able to start work on the new “finger drum” interaface with which I would like to keep in touch with STEIM.
Actually I am “installation transporter”… Changing location of the project www.sonicave.com ( already finished) and removing sound bridge called “Loveka”

This piece will be installed at city park and will be completely energy independent – using the solar power.
(Project is also part of the SolArs2010 comptetion). It is project based on something that I call “basic interactive” concept – limited and very simple input and output configuration.
I am very intersted how long it will survive…

more later…

Blog post no.1!!! Back in Amsterdam, tonight we’re giving the first performance using our recently developed ‘Gaia’ instrument, at the Music Conservatory. We’ve made three short pieces which form interludes in Kingston University London’s choral mixed-media piece taking place simultaneously here and at Kingston (and live online).

Development of the ‘Gaia’ project started in January, which has so far involved two short but fruitful trips to STEIM and many hours in my London workspace. In the February visit I had very useful mentoring and advice on both conceptual and practical issues from Robert, Daniel and Takuro. During the second visit in April I worked with Frank to really get up to speed with JunXion, finding out how to effectively condition and interpret Nina’s martial arts based movements via 4 Wii-motes (one per limb!) using Ableton Live as the audio engine. During a STEIM lunch I had further orientation from Kristina regarding the methodology of motion-sensing and the very pertinent advice “not to let her (Nina) become just an operator” which we’ve taken on board!

It’s been quite a steep learning curve and there have many technical and procedural issues to address, not least the reliability of Bluetooth.. I’m going to post some more detail soon about the development stages, retrospectively, as well as keeping the blog up to date about where things go from here. We’re looking at tonight as a great chance to get a first public airing on a project which feels full of potential. We’re over here with Ben Horwood from Kingston University whose doing the live feed tonight. Just going over to the Conservatorium now having checked out the wiis and weblink over breakfast in the STEIM guesthouse!!

More soon…
Walter

An interactive music area is a place where sound induces sound in specific rhythms and harmonic relations. The system indicates the difference between silence and sound and shows the pitch in colours. Due to the use of pentatonic scales and measured repetitions a visitor/user is likely to experience his or her own input as music …
The interactive system is based on the Jazzperiments Jam technology partly developed at STEIM in October 2007. The installation on the picture – Interactive Music Area 51.180012,4.414798 – is a ‘mobile’ version, set up near the Yayoi sculpture of Corey McCorkle in Middelheim Museum on 7 January 2010 (sound of wind, birds and distant airplanes).

During Time Canvas 2010 (7 March) in M HKA Antwerp a larger installation will be set up: Interactive Music Area 51.211140,4.389929.

Finally got to work on my blog.

It was great to get a hands on approach with the technology available at Steim and it was awe inspiring to learn of just a fraction of the possibilities on offer with interface technology. In between recording my work, I want to develop my idea of a a musical interface using a carved wooden chessboard as a controller. I felt that with using current console technology was too obvious. Why not make something elegant and simple which would engage with the full gestures of the body. I need to think about sensors. It could be scaled for portability and can be carried anywhere.

The movement of each specific (vertical, horizontal, sliding) piece will determine the outcome of the output signal. I have a long way to go with the idea, but I am willing to research its potential. I think such a set up would be perfect for the new versions of Lisa and Junxion at Steim. First of all, I have to think about concrete elements in hardware/software electronics. This blog will keep a record of my progress.

Jazzperiments JAM (STEIM residency project in 2007) is an applet you use while playing an instrument. Doing so, it isn’t hard to notice that a standard user interface of a laptop with a mouse or touchpad is not ideal in that situation (neither on pc nor on Apple). This is even more obvious during a performance. To avoid the delays and distractions caused by looking on the screen and moving the mouse, a graphical user interface with the following extras was developed:
1. Every control on the screen has a number between 01 and 99, preferably in logical groups. All numbers are shown on the controls when you hit the [/]-key.
2. All lists are numbered, all sliders and switches have a numeric value.
3. If you type the number of the control, it is selected.
4. If you double click a control, it is selected.
5. You can change the setting of a selected control with the [+] or [-] keys.
6. You can change the setting of a selected control with the scroll wheel regardless where the mouse cursor is.
7. All numbers and values as presented on screen can also be used in macro’s.
8. You can set a control by typing the number, a value and [Enter].
For example: you type: 2050[Enter]; on the screen appears 20=50 (the =-sign is added automatically); in Jazzperiments Jam control 20 is set to 50, which means the velocity of voice 2 is set to 50%.
9. You can even play notes with the numeric keys ([0] = C … [9] = Bb, [*] = B; [+] and [-] set the octaves). Especially vocalists appreciate this feature.

These additions allow a player to control a complex program (with upto 99 controls) using only a (wireless) numeric keypad. A normal keyboard is only needed for text input and a mouse isn’t needed at all, but of course using a mouse or touchpad is still possible as always.
The system proves to be especially handy to change fixed setting (like changing an instrument or switching on an effect) without even looking at the screen, while continuous or interactive settings are changed with more intuitive controllers (like a microphone in the case of Jazzperiments JAM).
For Jazzperiments JAM this live music user interface is programmed in Java, but the principles can be applied in almost any development tool and for many musical applications.
The system can be tested on http://www.jazzperiments.com, click JAM to start the Java applet. All suggestions for improvements of this experimental application are certainly appreciated and questions sent to info@jazzperiments.com will be answered with pleasure.

Teun de Lange, Antwerp

Microphone and numeric keypad setup

“Noise Trumpet,” as some of my friends call it, is using extended techniques on the trumpet to make it sound what many people would call un-trumpet-like. The use of air as noise, valve sounds, split-lip multi-phonics, sung multiphonics, throat growls, tongue stops and modulations, changing the resonant shape of the inside of your mouth, playing more with the inside of your lips, playing more with the outside of your lips, playing with less lip pressure, more lip pressure, bouncing the lips of the mouthpiece, opening your teeth more, closing your teeth more, more air, less air, letting air escape, playing very quietly, playing very loudly, very low pitches, very high pitches, circular breathing to make constant harmonic/melodic/rhythmic textures beyond the traditional idiomatic uses of the instruments. Not to mention physically reconfiguring your instrument, taking it apart, adding mutes, tubes and more.

When mixing recordings of myself and others playing trumpet, I noticed while using a FFT visualizer during the EQ stage that, unsurprisingly, when playing notes in a traditional manner, the resonant spikes would fall into a fairly predictable pattern of overtones. And, equally not surprising, when going into extended modes of playing, the patterns of resonant frequencies would become more dynamic and complex. Yet even though it was unsurprising, somehow this was interesting to me, which made me think, is there a way this can be used in my software patch in an improvised music setting. This question was what I explored during my STEIM residency in September 2009.

(more…)

This ‘blog post provides some thoughts following the STEIM orientation of 29 Sept. — 1 Oct. 2009. This is an updated and final post about the Orientation.

The first phase of the STEIM Orientation offered a definition of the aesthetic point of reference, the social mission, and the products developed by STEIM. The aesthetic may be described as a ‘high-touch’ human-performance foundation and motivation for creation of electro-acoustic music. Computers have come to dominate electronic music, yet the best of musical performance, leading to the most ‘human-oriented’ musical composition, requires a much more flexible and ‘human-specific’ interface than is provided by the keyboard/mouse/visual-display standard. Further, the designs of interfaces developed by music industry either emulate acoustic instruments without substantially extending their electro-acoustic potential, or primarily facilitate operation of standard more utilitarian music software, which tends more to support ‘performance of the software’ than intrinsically musical performance. The overriding question behind the interface design may be given as follows:

Does the design of the interface, look back from the design of the software to the human being as an efficient operator, or does it begin with the shape of human performance and then design for its potential expression? (more…)

I have been looking into technologies for expanding a performance system I use with my voice, Conexus, which generates electronic sounds from real time parameter extractions in MaxMSP. I have designed a control interface with sensors (accelerometer, potentiometers) in order to gain better instrumental control of the system live. My goal for the live electronics is to extend the voice and create a unified concept for solo improvisations or with other musicians. I see this project eventually being presented in performances at festivals, experimental music venues and conferences. My stay at STEIM has introduced me to some new software (LiSa and JunXion) that I can involve with my project. I was previously not aware of how powerful these programs are for live performance. I definitely benefited from the range of individual approaches from everyone at STEIM and look forward to reconnecting in the future.

www.vocaldrifts.com

For my project Mixing Cities I asked STEIM if I could place microphones at their building for a registration of the sound environment in front of the building. This way I have in total five sets that do this in different cities. Besides Amsterdam there is Rotterdam, Enschede, Eindhoven and The Hague. The sound is streamed over the internet to my server.
The audio streams of the five locations come together in Mixing Cities. A mixing panel with five faders allow you to listen to one of the streams, for instance Amsterdam, or more at the same time.
Besides audio also the light environment is represented in the installation. Simple LDR sensors receive the light situation of the locations and this information is translated in the strength of the light bulbs on the mixing panel.

The audio streams are transmitted in the ‘ogg’ audio format with help of the open source radiostreaming project Ices2 and Icecast. Arduinos, PD and MaxMSP are used for the sensor data and the mixing panel.
The installation I made two years ago and was premiered at TodaysArt07 in The Hague.

Mixing Cities is presented at RamFoundation in Rotterdam until the 4th of October 2009.

More about Mixing Cities on my website http://a-bort.org.