Anticipation leading up to the gathering
A real sense of anticipation for this Jamboree. I had had a very apprehensive feeling walking from the Metro to STEIM on my last visit back in November, my first time back since Michel’s passing. It was enormously reassuring to see the place well, and just keepin’ on going. Then the news that financial support had been reinstated was a joy of relief. The Jamboree affirms this continued livelihood and liveliness of the place, and the community.
I rushed up on an early train Monday to be able to work w/ Jorgen to finish my instrument, and as I worked away in the workshop, the afternoon leading up to the first session was a wonderful reunion. David Z walked in, first visit to STEIM, went out for lunch w/ Frank and Taku. Then there was Dan Overholt in the software room, tinkering around w/ his new touch-joystick instrument. As if he had never gone away. Then I turhed around and Sukandar had walked in -
Reception and intro’s
7pm and Minouk had catered Mozambique nibbles, mmmm.
8pm and Taku opened the session. The first public words affirming that STEIM was alive and well, thanking everyone for the outpouring of support. Thinking about the criticism that came from the national funding agency, that we were “niche and closed”. But why are these two words associated? Nice doesn’t have to mean closed. Activity doesn’t have to be mainstream to be visible. Those are the issues as STEIM opens a new chapter.
Taku then introduced the new director, Dick Rijken. Dick had been on the board of STEIM for years, and is connected outside the music area to the education, human interaction, and media fields in Utrecht, Den Haag, and elsewhere. He looked back at the double blow of Michel’s diagnosis and the initial rejection from the National council of culture. Looking now to the future, we have 4 more years to figure it out, to reinvent STEIM, to create a future.
Frank Baldé - Release and demo of new features in junXion 4
Frank Baldé, software designer at STEIM showed the latest version of the JunXion software, v. 4
Fragmentary notes on the fly. Should he call it 5.01? Max is 5.06 already so can’t keep up
The software reads input sensors from the outside world and translate to MIDI or OSC. It was a close collaboration w/ Michel designing software.
The origins: Sensorlab programmable hardware box program on host computer, upload code, to have standalone MIDI controller reliable, never crashed. one drawback - the price - in the 1990’s. The only institution to have such a device available - EUR1500, artists got it because it was the only thing. Eventually became obsolete. Commercial companies started creating lower cost sensor input devices. In any case, the model: Sensorlab hardware programmed using programming language - Spider - for artists, simple programming language too complicatedartists simply don’t think that way.
A new effort: try to make a modular, cheaper device, but as they started designing, adding features, the cost kept going up. Meanwhile artists kept coming to STEIM to build systems, instruments, but STEIM had no sensorbox any more. What to do? Joysticks were useful and the HID utility could read data from them. So 5yrs ago he programmed JunXion v. 1.0 as a joystick USB to computer utility.
Dan Overholt came to STEIM on a Fulbright Grant and made the foundation of the JunXion hardware board - 16 input acquisition, registers over USB as a joystick so the JunXion software could read from it, giving a way to read realworld data. Software on the computer side allowed sophisticated mappings
v.2 2005: Seeing the hurdles of end users programming in Spider for the Sensorlab, ideally, JunXion would be a scripting language. So when creating the new version of JunXion, he took that into account.
The challenge: how do you give this programmable power yet make it accessible to artists? The pitfall: it could very well quickly end up creating a graphical programming environment. But no need to create another Max-like thing… so,
v3: Feature: graphical user interface. Left side - inputs - user interface devices (mouse, keyboard, USB multitouch screens), then later MIDI input, then timers, then OSC output, then Wii-mote input. All this was used in the Mobile Touch exhibition, a 7 hour jam session , w/ no glitch, no crash
375 patches per state
Audio events - - amplitude and pitch. Pitch tracking . . . Frank says, “I am not a very good sine wave singer”. But with vocal input and amplitude tracking, Frank played a drum demo - “tick tick boom” triggering off drum samples, with some switches doubling tempo, for play along mode w/ fills and rolls.
Arduino input think of it as a kind of tiny Sensorlab. Similar issue - need to write a sketch in C. With Junxion je provides another solution. Some question: Can you talk back out to Arduino? No. What is the Arduino sampling rate? It opens a virtual serial port, and JunXion polls Arduino at 1ms interval
Videotracking - live camera input. Later over drinks, Frank answered Kazuhiro Jo’s question of whether there was a camera tracking API, that in fact the history came from former STEIM colleague Tom Demeyer’s Big Eye software, originally written for a Dutch choreographer to run on an Acorn RISC based computer, then w/ the arrival of the Motorola PowerPC chip, for Macs in the mid 90’s. This code was then dug up a couple of times by interns at STEIM to be revived finally by Saskia who had also done the Wii interface.
Camera input can use various techniques including a difference algorithm to do object tracking, or color tracking - use of iPhone w/a super duper custom app to…. display a solid color… Video useful as a conducting input, to be augmented by other samples, for example with the LiSa live sampling software.
Frank finished by standing up to play a musical demo, cranking up JunXion, a softsynth in Garage Band, and played w/ iPhone as OSC controller using the accelerometer of the iPhone and sequencing done entirely using the timers in JunXion. Mapping was…
- Timer - step sequencers
- multiple sequencers
- levels mixed by accelerometer
- note tables dynamically changing (from other timers, writing data into tables) feeding variables
- another timer reads value of variable in table
- iPhone accelerometer X tilt - filter
- iPhone accelerometer Y tilt - pan betw sequences
- iPhone accelerometer Z shaking - pumping up volume
- iPhone accelerometer Z big shake - drum volume
David Zicarelli - On Max 5 and cognitive creative processes
David Zicarelli from Cycling 74 made the trip from Santa Cruz, on his way back from Berlin after meetings w/ Ableton. It’s David Z’s first time at STEIM
Frank Baldé introduces him - “met” him w/ Intellingent Music end of 80’s, when David did a couple of
programs which blew him away - different than typical MIDI software, non traditional. . . M, and Jam Factory, inspired by user interface, happy that finally after all these years, here at STEIM
David in his classic hands in pocket, slouched shoulder soft spoken style, says…
“ I’ve been hearing about STEIM for 25 years as a place that shares ideas I have… I’m happy that it’s ‘opened up’ enough to have me. I would have been happy just to be visitor. My talk will be less about what you can do w/ Max but more about what I think about when I work on the program. Imagine that you’re faced w/ my job: To maintain a complex piece of software, Where lots of people have ideas of what I should be doing. How should I decide what to do?
“There are 3 really boring criteria -
- is it currently embarrassing - crashing etc? most of the working day. Everything else is a distortion of reality (that my job is boring)
- is it painfully necessary - Max 5 had to rewrite from scratch - needed to start over - 80’s Mac calls, Windows was emulating that
- can’t ruin people’s lives - would like to get rid of some features, but that creates anger and bad feelings
Let’s try not to give people a hassle
“What I wanted to talk about is more conceptual and visionary
It’s more than just fixing bugs
People teach the program, and people learn the program. From my teaching experience, it’s far harder to get people to use the computer than to program the computer. So ultimately, using the computer is the problem. So let’s take away the stupid excuse of getting started.“
Common trasks like:
- how are you going to read the documentation? tutorials - print - waste of paper. Negotiate screen so that PDF onscreen at same time as screen
- what does it look like when the program starts up? do you know where to start? what do the tutorials require?
In Max 5, all documentation in XML format. Rewrote it w/ Luke Dubois so the tutorials to reflect what people have - just the comoputer, less people have MIDI keyboards. What tools to people have at their disposal? What programs do people use? Web browser has ubiquitous usage and people are used to the UI of a browser. People also use iTunes - a database of content people care about and manipulate database in a meaningful way.
Detail: converting the program to use Unicode was a big effort but important for Asian users. There is a localized Japanese version.
“ Let’s quickly get to the interesting thing - programming - giving yourself a chance for people to find out what youre about. Creative work and how you might do that in software is my primary interest. Sound synthesis, video, and controllers are cool, but I’m not particularly good at those things. I’m better at thinking on an abstract level. I’m interested in the cognitive aspects of being creative - that’s what I’m passionate about.
In the work that we do, we’re all liars. Our main goal is to deceive others, and to do that, we deceive ourselves. Software needs to facilitate cycle of deception.
Mouse draw drum demo - I’ve defined rules of a system or game where I win everytime. When I was learning piano, I had the frustration of not being to play perfectly. Later I got into styles of music where errors are used creatively, well, so people don’t know whether it’s an error or not.
Back to the cognitive process.
- I told you how it worked
- Then I played it
There was a change that happened between the two, and this represents the the cycle that anyone goes through when making an artifact - switching between building something and seeing if it meets that expectation - this is the cycle of deception…
As soon as you stop building and evaluate, you have imagine being the audience. Necessarily there is some sort of magic involved. You ask yourself, ‘have I transcended the mechanics of this system to a result that means more than that?’
A typical patch - where to look first? The organization of how it works is not the same as if you’re evaluating the magic. Need to know what you can control and what’s interesting to manipulate. Don’t need to know what things are connected how. Need to know how you can interact with the system, and be able to construct a mental model that reflects how you think about it.”
In the old Max there was the feature “Hide on Lock”. In Max 5 there is Presentation Mode - you can change the position of some boxes, and only show boxes of the user interface. Use this via the “New view” menu item. There are supporting features (tinging) to show relatioships betwen - by making a selection in one view shows where the object is in the other view.
In the old Max, subpatchers used to be frozen 0 couldn’t listen to the change because there was no way to edit it in context. In Max 5, there is Modify Read Only - you can edit a subpatcher and save and all instances update live in the encompassing patch.
” Fluency - is all about forgetting. Being fluent in something means you can do something w/o being forced to remembering things…”
In Max 5, there is the Clue window.
- The message menu
- No need to open documentation - it’s in the software
- Context sensitive help
- Suggested object list icon on left
- Getting access to your past work and reuse it: Save as Prototype feature
“To be a fluent user - need predictive knowledge. Sometimes the user can help you”
Upcoming features, efforts:
Patching up - patching up from an inlet to use suggested things to plug in to create submenus of possible structures
Relationship betw a visual language and computational performance. Why should a visual language be slower than a compiled language? Three places where computational perforamce is important -
- Open GL
- Arduino - constrained computational environment, small memory, limited processor, can’t debug. Can we create a visual way to program and debug the Arduino?
Translating from visual coding into source code domain and compiling dynamically while Max is running.
“My highscool job at newpaper to arrange pages for editors. Needed to do it and show it, they’d never agree otherwise. Implementation creates its own reality. We need to make something to get people to talk about it.”
So what about this debate on round and squared off objects?
“The roundedness to give an unconscious level cue of what boxes do what”
“It concerns be me that 98% percent of user base is male”
What happened to Pluggo?
“Wait for January 14, 2009, and we’ll make an announcement with Ableton (a project working on for 2 years).”
“A lot of things in Max 5 will make more sense on that day”
I heard that Max uses JUCE
“Yes. This means that there is the possibility eventually to deploy Max on Linux”
“One personal area of interest for me is the universe of ubiquitous computing devices…like the Trash Mirror“