steim | about | contact | events | project applications| products| resources

STEIM Writings

Exploded view: the musical instrument at twilight
Nicolas Collins

The twentieth century has witnessed a radical transformation in the mechanics of music production, and of music's role in society. The first half of the century saw the development of the recording industry, and with it an elaboration of the path from composer to listener, and a redistribution of power amongst new personnel and technologies. The second half of the century has seen a similar redefinition of the musical instrument: the physics of plucked strings and vibrating reeds has been overtaken by electronic manipulation of every link in the sonic chain that stretches between finger and ear. Such dramatic changes have affected the way music is made and the way music is heard. They have altered our very sense of what music is and can be.

Thomas Edison did not invent the phonograph, contrary to the popular American myth. Rather, as he had for so many other of his 'inventions', he recognized the consumer potential of an untried machine. Various technologies for sound recording and reproduction had been under development in several countries for several years -- Edison picked the one that seemed most practical and profitable, tweaked it in his lab, and introduced it to the American public. With a foresight worthy of a modern day computer mogul, he realized that the key to financial success with the phonograph lay in controlling both the hardware (the recorders and players) and the software (the recordings), so he manufactured both. Partially deaf as a result of a punch to the head as a child , Edison claimed to have no ear for, or interest in, music. He saw the phonograph record as a sonic autograph, and the player as a way to hear the speaking voices of famous persons. Musical recordings were initially introduced as a novelty, and were not taken seriously by Edison for years. It would seem to have been a case of right technology, wrong vision.

Edison did, however, have a canny insight into the effect the phonograph would have on the act of listening. By choosing to record that most personal of sounds, the spoken voice, he anticipated a fundamental change in the social role of music that took place in the course of the twentieth century -- the shift from music as a predominately public activity to music as a predominately private activity. If the voice of President William McKinley did not become a valued possession in the home of every American, the voice of Caruso did. Professional music left the concert hall and entered the parlor. Putting on one's own choice of a record, to listen to alone, replaced attending a concert with the masses. And putting on a record in the presence of friends replaced playing in a quartet with them.

The phonograph had a precedent in the 19th century popularity of the piano, which linked the virtuoso on stage and the amateur at home: the only difference was the time you spent practicing. The phonograph was an even greater leveler: a seemingly infinite array of instruments and ghost performers at your fingertips and no need to practice. Perhaps this was too strong an affront to the late Victorian work ethic, because the phonograph was immediately relegated to the status of furniture rather than being treated as a musical instrument -- a situation which remained unchallenged, with a few exceptions such as John Cage's 'Imaginary Landscapes #1' (1939), until rise of the 'turntable artist' in the 1980s. Society was split into two distinct categories: a small group of professionals who made music and the large mass of society that consumed it. The phonograph represented a milestone in the gradual distancing of people from the act of making music -- a process that had been taking place since the rise of art music in Europe. Edison's invention effectively replaced the Victorian amateur with the modern consumer.

Recording music is no more 'natural' a process than taking a photograph or making a film. Much effort and artifice goes into making a recording seem effortless and artless. Each new generation of recording technology is touted for its verisimilitude, its accuracy, its transparency. In the CD era it is hard to believe that anybody took these claims seriously at the time of the Edison cylinder. The ever-savvy Edison , when introducing the phonograph, set up two important support systems: the marketing forces required to convince the public that a seat in front of this small box with a horn was indistinguishable from one in a box at La Scala, and the recording technicians who pushed feverishly at the limits of the technology in pursuit of this ideal.

The technicians included design engineers who sought to improve the existing technology, and recording engineers and producers who made the best of what they had. The latter two rapidly rose in importance and profoundly changed the distribution of power within the chain from composer to consumer. They were to recorded music as the conductor was to orchestral music, and not since the emergence of the conductor as a charismatic figure had the singular authority of the composer been so seriously undermined. The engineer and producer were responsible for making the recording sound like the 'original' music, the composer's auditory vision, but they also influenced the kind of music that made it to the record's surface, and many of its formal and structural details. They became the orchestrators of recorded music -- they knew what sounded good on record -- and for the same reason they became its censors. And because the producer was also 'the man who wrote the checks', he soon became the single most important person in the recording chain.

The film composers of the 1930s and 1940s were the first to learn the technique of 'studio scoring': they wrote music that was only heard over loudspeakers. They were followed by the Tin Pan Alley songwriters, who understood that a two and a half minute song on one side of a 78 was a much more effective use of the medium than a symphony chopped up on a dropping stack of ten. But it was Phil Spector who perfected the art of making music for vinyl. By positioning himself as a producer first and foremost, in the middle of the recording chain, Spector extended his power and influence over the entire production process: he wrote the tunes and lyrics, did the arrangements, picked the musicians and singers, invented new recording techniques, owned interest in the companies that pressed and distributed the disks. Not until a record had proven itself a hit did he bother to assemble, for the sake of live shows, a group to match the name on the label. Total control.

Or was it? The recording age saw the emergence of two other significant new powers in the musical machine: the disk jockey and the consumer. Thanks to radio, the record may have been one of the first products to act as its own advertisement, but records didn't play themselves on air, disk jockeys did. One DJ was worth a thousand listeners, more or less, depending on personality, wattage and demographics, and thus he was a force to be reckoned with, courted, and bought, if necessary. On radio, and later in a booth or on stage, the DJ was acknowledged as the virtuoso of the turntable -- as much for his encyclopedic knowledge of recorded repertoire as for his physical touch. Any vestige of the musical instrument that remained in the phonograph was appropriated by the DJ.

The downside of making records, record players and radios affordable to the masses was that the masses could pick and choose with careless ease amongst a myriad of musical offerings. Flipping off a record midway through a side, or scanning the radio stations, had none of the social stigma or economic recklessness of walking out on a concert. The phonograph and radio may have been no match for the piano in terms of musical expressiveness, but they did give the user an unprecedented degree of control over his or her musical environment. Feedback from listeners to record companies was quickly formalized into the 'charts' that continue to drive record marketing today. Baroque and Classical composers survived by winning the patronage of a wealthy few; now the fickle buying habits of the man and woman on the street held sway over composers and steered musical style.

By 1960 the traditional, pre-recording model of musical production and transmission had been exploded. The locus of power had shifted away from the composer and the germ of the musical idea and was distributed amongst specialist technicians and middle men (arrangers, producers, engineers, disk jockeys, A&R men) and consumers. The effect of the 'British Invasion' of popular music in the early sixties was to kill off the American Tin Pan Alley/Phil Spector tradition of producer-centered authority, and to establish a new class of composers who recognized the importance of recording studio literacy. The Beatles and the Rolling Stones may have started out as bar bands but they came of age in the studio under the guidance of gifted producers such as George Martin. Production vision became as essential to a songwriter as melodies and lyrics; studio technique as critical for a band as instrumental competence. Recording technology evolved quickly, and experimentation extended its application beyond the direct, accurate transference of sound to tape. The recording studio became both a musical instrument in its own right and a compositional tool. After Sgt Pepper the challenge was not how to replicate a live performance on record, but how to replicate the record in live performance.

The notion of the studio as instrument had been pioneered years earlier in European electronic music studios -- rooms full of abducted electronic test equipment and spinoff radio technology. Pop music gave the idea public visibility and commercial viability. The next step was for RCA Laboratories to formalize the concept by transforming the room full of electronic objects into an electronic object the size of a room: the synthesizer. Taking its cue from the increasing influence of electronic technology over the transformation of acoustic sound into recorded product, RCA thought to dispense with the acoustic sources -- and the musician's hourly wage -- and to replace them with electronic ones. This pursuit of the union musician's worst nightmare was ultimately fruitless, but synthesizer design succeeded in altering our understanding of the 'musical instrument' in ways parallel to those by which the phonograph had altered the social uses of music: it exploded the chain of command.

Synthesizers -- from RCA to Buchla, Moog, Arp, and Serge to Oberheim to Yamaha and Roland -- have been based on the premise that musical sound could be analyzed, broken down into modular components, and reconstructed (synthesized) by electronic circuits that emulated those modules. It is a seductive idea: in place of dozens of different instruments and skilled players, a single technician could arrange a handful of electronic oscillators, filters, amplifiers, and control circuits to conjure up any existing instrument or even some novel hybrid combination of instruments. But the greatest implication of the synthesizer lay not in how well it replicated sound -- some 35 years later traditional synthesis methods have yet to mimic acoustic instruments accurately enough to pass close scrutiny -- but in the fact that for the first time in the history of instrument design the causal properties of physical acoustics could be ignored.

Synthesizer modules were based not on modeling the physics of the excitation and transmission of sound , but rather on a description of the resulting sound itself. Without the mechanical causality built into buzzing reeds on resonant tubes and bowed strings on soundboards, synthesizers needed a system of interconnecting and controlling the modules. The closest thing to a standard in the first generation of commercially viable synthesizers was Voltage Control. Briefly, all modules in a Voltage Controlled synthesizer were designed so that any adjustable parameter, such as the pitch of an oscillator, could be externally controlled by a variable DC voltage. In a Voltage Controlled Oscillator, for example, increasing the voltage applied to the frequency control input from one volt to 2two volts would raise the pitch one octave. Thus a keyboard might put out a control voltage that increases by one-twelfth volt with each key that, when applied to the Oscillator, would produce an equal tempered scale; or a second, very slow sine wave oscillator could vary the pitch of an audio-frequency oscillator just enough to produce something akin to vibrato. Since the output of any module was a fluctuating voltage of some kind, Voltage Control permitted any module's output to be connected to any control input.

More interesting than merely reversing the analysis process in order to re-synthesize a known instrument was the possibility of rearranging the modules with seemingly infinite variety, producing permutational richness quite beyond the limitations of lips, strings, reeds, and fingers. 'Patchcord music' was born of throwing away the keyboard and interconnecting modules according to systems that were no longer imitative. The 1960s model of analysis/synthesis failed, but its electronic building blocks acquired a structural meaning independent of that original model.

While Voltage Control technology lay at the heart of most early commercial synthesizers, there were variations in its implementations that made it difficult to interconnect the machines of different manufacturers. Voltage Control was accepted widely not as an 'industry standard' but simply because it made modular design possible within a single instrument or a product line. Initially, company strategists encouraged incompatibility: it could provide patent protection or fend off infringement liability, and it was thought to insure customer loyalty and repeat purchases. But in the early 1980s a handful of manufacturers saw the potential benefits of implementing an industry-wide control standard that would permit total compatibility, and MIDI (Musical Instrument Digital Interface) was born.

The inspiration had come from the booming personal computer industry. As microprocessors and memory became cheaper and more powerful, synthesizer manufacturers started to incorporate them into instruments for control and storage functions. For a synthesizer's microprocessor to control another synthesizer's sound instead of only its own all that was needed was an extra pair of jacks and a little bit of programming -- communication protocols were already well established in the computer world. Furthermore, the growth of the computer industry had shattered long-held assumptions about the importance of proprietary standards, and had demonstrated the profitability of open architecture and third-party developers. Total compatibility meant more sales for everyone, seemingly at the expense of no-one.

MIDI employs a serial communication protocol such as that used between computers and peripherals. It requires a microprocessor in both the controlling device (such as a keyboard) and the controlled device (such as a sound generating module), but the cost of such a microprocessor is negligeable. The MIDI language permits the free exchange of a large amount of information describing the articulation of musical events over time. The language was designed by a committee of representatives from synthesizer manufacturers -- not by musicians and certainly not in consultation with any members of the avant garde -- and it reflects a businessman's assumptions of what information is musically important. Thus it is a bit like Esperanto: not beautiful, not natural, but useful, logical and expandable.

This expandability is important. The microprocessors required to transmit, receive, and interpret MIDI information were initially underutilized (rather like the VW Beetle engine block, which remained externally unchanged between 1955 and 1966, but internally was stroked and bored until the horsepower of that very conservative design had doubled.) MIDI initially clumped the modules of electronic music into two groups: controllers and sound modules. But by placing a user-programmable computer between the controller and the controlled, composers could turbocharge MIDI and introduce a whole new order of musical possibilities.

The first commercial programs to take advantage of MIDI's expandability were utilitarian tools that might best be described as 'organizers': sequencers (with which you record and edit 'scores' for controlling MIDI sound modules -- sort of digital piano rolls), voice editing software (which lets you tweak sounds in the comparative luxury of the computer screen rather than using the synthesizer's knobs and curt displays), and librarian programs (for backing up and restoring a synthesizer's programmable variables.) The second wave of products included what might be called 're-organizers,' programs that interpreted and transformed the MIDI data in various ways: arpeggiators, accompanists, variation generators, improvisors. These represent the most recent stage in the dismantling and reconstruction of the musical instrument.

By placing a decision-making device between the player's controller and his sound output, electronic instruments have progressed still further from their acoustic antecedents. The composer can now re-enter the musical chain post-score, post-player, and immediately preceding actual sound, by programming the computer to filter out unwanted actions by players. Or the computer can change or accompany the player's activity according to algorithms specified by the composer or performer but beyond his or her direct control. Or it can act like the server in a computer network, co-ordinating the actions of several players and controlling the sounds of multiple synthesizers. Inserting a computer in the MIDI chain can infuse a performer's instrument with the characteristics of a score, an orchestrator, an orchestra, an accompanist, an improvisor, a conductor, a teacher, a censor, a producer. In short, it opens up the musical instrument -- previously thought of as the self-contained final link between player and sound -- to incorporate any stage of the industrial chain of music production.

MIDI's transformation of the musical instrument has had a profound effect on the nature of both electronic music and the musical instrument industry. A technology once intimately linked to the avant garde has now become a mainstay of commercial pop. By creating an instrument structure that parallels the production chain of a recording, the locus of power can be shifted easily to suit the musical ideology. Control over the end product can rest in the hand of a virtuoso player touching a key, in the sequences prepared by a producer, in software that improvises, in the studio automation package controlled by the mixing engineer, in a technician's samples or voices patches, or in a score embedded in a program. And the style of the music reflects the center of authority. The charmingly shambolic performances by the electronic music group the Hub result from a collective computer network where no-one has direct control. Composer and trombonist George Lewis shares the stage with a digital alter ego, which improvises well because Lewis has taught it how, and Lewis improvises brilliantly. Ed Tomney is woken up each morning by a computer-generated voice that tells him what kind of music to write that day. In a more commercial realm, the precision of House music is built on power sequencing, while Rap makes extensive use of witty sampler appropriation and the speaker-popping kick drum sound of the venerable Roland TR707 drum machine. After MIDI, electronic instruments are no longer contained instruments, and electronic music is no longer a contained style.

Starting with David Tudor in the 1950s, there has been a long tradition of 'homemade' electronic musical instruments, but before MIDI successful commercial production of home designs was limited to those with the financial backing needed to underwrite the considerable tool-up costs of hardware development. MIDI, however, divided the musical instrument and its manufacturing into three distinct parts: controllers, software, and sound modules. Controllers and sound modules incorporate both hardware and software, while software is just software. Software isn't any easier to write than hardware is to build, but it can be cheaper. It can be produced on a very modest scale, free of the demands of investors who want a product's eccentricities worked out along with its bugs. The 1980s saw several composers market programs imbued with their own distinctive personalities (most notably Joel Chadabe's M and Jam Factory, and Laurie Spiegel's Music Mouse.) Ironically, although MIDI was developed as a convenience by and for large companies at the point when electronic instruments began to acquire serious commercial value, it also permitted the flourishing of artist-run cottage industries.

STEIM has, for 25 years, operated as a kind of subsidized cottage industry. Our status as a government-supported foundation has given us the freedom to extend non-commercial, artist-inspired design beyond the software realm into the more cost-intensive, hardware-dependent projects. Our primary function is serving composers whose needs are not met by existing commercial products, or whose technical or financial limitations place their musical vision beyond their grasp. The emphasis has always been on live performance, and since the advent of MIDI we have stressed the design of new controllers, which are not based on the forms of existing instruments the way most commercial ones are, and the development of software that extends the performer's control over complex musical textures.

The 'SensorLab' is the heart of STEIM's new controller design: a small microcomputer that reads switches, pots, pressure pads and other sensors, and translates that information into MIDI data; the accompanying SPIDER control language lets the user program multiple, dynamically re-configurable instruments based on a single, economical hardware core. 'Deviator' and 'The Lick Machine' are software programs for interrupting and transforming the MIDI data stream between the controller and the sound modules. 'Deviator' is essentially an effect processor for MIDI data, producing delays, echoes, offsets, and other transformations of incoming note and controller information. 'The Lick Machine' maps user defined sequences onto specific notes from the controller for playback and manipulation by the performer. The 'Big Eye' software transforms a video camera into a contact-free controller for translating movement and images into MIDI data.

There is a danger inherent in MIDI's modularity similar to one that besets the recording industry: a lack of integration and feedback between the parts. With the separation of modules it is easy to forget one musical component while concentrating on another, to forget the whole while indulging in the parts. Whereas commercial instrument design usually focuses on sound at the expense of controllers, STEIM has for many years emphasized controllers and interpretive software. Mindful of the pitfall of segregation, we have recently embarked on an ambitious Digital Signal Processing (DSP) project. We are designing a compact DSP module that will serve not only as a MIDI sound generator, sampler, and processor, but -- by incorporating elements of the SensorLab -- will also have direct sensor interfaces allowing it to serve as the core of a fully integrated electronic instrument. There are aspects of acoustic instruments that have yet to be adequately emulated in electronic ones, such as the tactile feedback between a player's finger and a stopped vibrating string. We hope that by combining delicate input sensors into the same hardware/software package as the sound source we can begin to recreate this kind of elusive musical touch. This return to a self-sufficient instrument may seem like a step backward, but it is important to remember that while this new module will serve as an integrated controller, software interpreter, and sound module, it will also function as a hub for communication with external hardware and software. After the MIDI-based explosion of the instrument, STEIM is now creating expandable music 'kits' that have multiple options for re-assembly.

So where do we go from here? After the record's transformation of the music industry, and electronic music's transformation of the identity of the musical instrument, there are still links in the musical chain that have remained relatively unchanged since the turn of the century -- most notably the architecture and audience of the concert hall, the audience at home, and the chamber ensemble. Are there ways to open these as well, such that we might foster new musical forms?

Architecture is the final 'instrument' in the acoustic chain from a performer to our ears, imposing its own acoustic signature on the music played within it, but unlike every other stage in that process it is invariably inflexible. With the exception of the rather underutilized performance space at IRCAM in Paris, there are no public concert halls that permit the real-time transformation of architectural acoustics. There are obvious economic reasons for this, in addition to the fact that little music has been written for such architectural instruments. But such music will not be written until appropriate performance spaces are more readily available. If music institutions were to encourage the construction of malleable concert spaces -- with real-time remote control of shape, reverberation time, frequency response, and other physical characteristics -- it might provoke the creation of music of a truly monumental scale.

The development of architecture for music has closely paralleled the gradual disappearance of participatory musical events within the community. Accordingly, the modern concert hall places a passive audience in fixed seating. Despite the fact that music is a three-dimensional, moving medium (as Alvin Lucier has eloquently demonstrated), 'serious listening' has become a motionless activity. Attempts to change this behavior, through interactive audio installations and non-traditional concert hall design, have been generally unsuccessful. We are faced with a fundamental attitudinal difference between the passive and active consumption of music, with the latter reserved for dance music, a small sector of the experimental fringe, and musics outside the European classical tradition. But just as the availability of halls with variable acoustics might inspire new forms of 'architectural music', alternative listening environments might encourage more activity on the part of audiences. Like the nightclubs of the 1980s, a concert hall could be built not as a single 'optimum' space, but as a sequence of acoustically and electronically linked rooms, each with its own character and social function. One space might cater to focused listening and direct visual contact with the players (as in a traditional concert hall), while another might present the music at an ambient or even subliminal sound level. More active listeners could interact directly with the music in 'remix rooms', by adjusting loudness, mix, and balance to suit individual taste, or wander through a labyrinth of corridors and small rooms that would acoustically transform the music more with every stage.

A similar passivity problem exists with 'home audiences'. There has been little increase in listener activity since the advent of the record and radio gave the consumer the power to select. Interactive media such as CD-I are commercially insignificant compared to the music CD, whose major selling point, I'm sure, is not sound quality but the fact that you don't have to turn it over. Even music CDs boast a degree of interactivity unknown with records or tapes, but what percentage of listeners bother to program their own sequences or listen in 'shuffle' or random mode? Let's face it, once beyond passive listening, we enter the realm of activities where satisfaction is based on short-term competent task fulfillment. What fulfillment is there in an alternative sequence of familiar songs? Interaction with home electronics typically consists of scanning channels with a remote control or playing video games. Video games emphasize the speed of hand-eye co-ordination, usually in competition with the computer itself rather than with other players. Scanning lets the viewer edit broadcast material to the exact length of his attention span, while pursuing a futile desire to miss nothing.

Neither of these two activities seems innately musical, but can we develop a new form of parlor music based on their motivation? A task that sits between the passive appreciation of music and the accessible, if competitive, satisfaction of games? Scanning could be a useful model. Multi-channel broadcast media carry tremendous amounts of information that can be used directly in a musical work as sound material, or can be transformed into structural elements -- for example, translating the 'value' of a given station's programming into the amount of time you stay on it before moving on is not far removed from certain practices of improvisational music.

Video games stress competition for its own sake, and only secondarily (if at all) do they have any aesthetic content. A better game model for music might be bridge. Although competitive, bridge has certain characteristics that are similar to those of chamber music and improvisation: it is a group activity (which sets it apart from that other game commonly linked to music, chess); and it has a social value beyond pure competition, with a tradition of a foursome playing together on a regular basis. There are styles, strategies, and 'classic games' -- all contributing to a tradition of theory and analysis. I could imagine a form of home music evolving like a weekly bridge game. A cable scanner, an interactive CD, or a computer program could provide elements of chance, topicality, score, and sound material. Performances could take place at many different levels of skill. Performances could be played back later for analysis or passive listening. The game-like competitiveness could provide the initial hook for pulling a listener off the couch and activating him or her as a performer, while the social factor would encourage the re-integration of musical performance into everyday life.

Whether this would ever happen depends, of course, not on the will of any one composer or any four bridge players, but on the constellation of technology, economy, social norms, and zeitgeist that governs all cultural developments. Edison's genius, after all, lay not in invention, but in a gift for being in the right place, at the right time, with the right machine.


1. In launching the autograph record Edison also anticipated the value of a recording as a limited edition multiple that would have an intrinsic value as a collectable object quite divorced from its sonic content.

2. In the age of MTV the video that was originally intended as an advertisement for the music often supplants it as the primary product, suggesting that the CD, cassette, or record is merely the affordable memento of a multimedia experience.

3. The relationship between composer and studio remained ambiguous, however. As popular music took over the recording studios and exerted an ever stronger influence on the design of recording equipment and facilities, it became difficult to say whether the production chain was being optimized for the demands of rock music or if rock music was evolving to suit the environment of the recording studio.

4. The origins of Voltage Control are disputed. Its invention has been credited to both Robert Moog and Don Buchla (pioneers in the field of electronic music technology), but as with Edison and the phonograph it is more likely that the technique was adapted and optimized from the omnipresent aerospace and defense electronics industry of the time.

5. I've long noticed that certain records sound better when I am in a room adjoining the room with the stereo rather than sitting directly before the speakers.

6. Witness the success of car CD players: compensation circuits compress and re-equalize the 'perfect' CD sound to adapt it to an imperfect listening environment, thereby reducing sound quality to not much better than cassette, but the 10-CD magazine in the trunk offers the convenience of hours of uninterrupted play.

7. My Devil's Music (1985) consisted entirely of instantaneous sampling and processing of live scanning radio.

8. Music has always had a degree of competitiveness. There was a story circulating Wesleyan when I was a student there, about a graduate student in ethnomusicology who was coaching a reading of the Brandenbergs by virtuoso string players in India. One musician decided that the way to prove that he was the 'best' musician was to play faster than the others, so he kept nudging up the tempo. The challenge was answered by the others, and the reading ended up as a race to see who could finish first.

Originally published in catalog for STEIM festival, October 1993.

© Nicolas Collins 1993