Art Style | Art & Culture International Magazine | #13

After the Turn ACOUSTIC PATHWAYS

Cover image © Jörg U. Lensing, Acoustic Pathways: After the Turn, 2024

Academic Editors

DesignbyArtStyleCommunication& Editions

Editor-in-Chief and Creative Director

Christiane Wagner

Senior Editor

Martina Sauer

Co-Editor

Jörg U. Lensing

Associate Editors

Laurence Larochelle

Katarina Andjelkovic

Natasha Marzliak

Tobias Held

Collaborators

Denise Meyer

Marjorie Lambert

Art Style | Art & Culture International Magazine

editorial@artstyle.international

+1 347 352 8564 New York

+55 11 3230 6423 São Paulo

The magazine is a product of Art Style Communication & Editions. Founded in 1995, the Art Style Company operates worldwide in the fields of design, architecture, communication, arts, aesthetics, and culture.

ISSN 2596-1810 (online)

ISSN 2596-1802 (print)

Theodor Herzi, 49 | 05014 020 São Paulo, SP | CNPJ 00.445.976/0001-78

Christiane Wagner is the lead designer, editor, and registered journalist in charge: MTB 0073952/SP

© 1995 Art Style Communication & Editions

Art Style | Art & Culture International Magazine is an open access, biannual, and peer–reviewed online magazine that aims to bundle cultural diversity. All values of cultures are shown in their varieties of art. Beyond the importance of the medium, form, and context in which art takes its characteristics, we also consider the significance of sociocultural and market influence. Thus, there are different forms of visual expression and perception through the media and environment. The images relate to the cultural changes and their time-space significance the spirit of the time. Hence, it is not only about the image itself and its description but rather its effects on culture, in which reciprocity is involved. For example, a variety of visual narratives like movies, TV shows, videos, performances, media, digital arts, visual technologies and video game as part of the video’s story, communications design, and also, drawing, painting, photography, dance, theater, literature, sculpture, architecture and design are discussed in their visual significance as well as in synchronization with music in daily interactions. Moreover, this magazine handles images and sounds concerning the meaning in culture due to the influence of ideologies, trends, or functions for informational purposes as forms of communication beyond the significance of art and its issues related to the socio-cultural and political context. However, the significance of art and all kinds of aesthetic experiences represent a transformation for our nature as human beings. In general, questions concerning the meaning of art are frequently linked to the process of perception and imagination. This process can be understood as an aesthetic experience in art, media, and fields such as motion pictures, music, and many other creative works and events that contribute to one’s knowledge, opinions, or skills. Accordingly, examining the digital technologies, motion picture, sound recording, broadcasting industries, and its social impact, Art Style Magazine focuses on the myriad meanings of art to become aware of their effects on culture as well as their communication dynamics.

Content Editorial

Christiane Wagner

Editor-in-Chief and Creative Director

Introduction

Christiane Wagner

Editor-in-Chief

Jörg U. Lensing Co-editor for this special edition

Essays and Articles

Foreword Immersive, Emmersive by Michel Chion

Soundscape: Selected Historical and Aesthetic Perspectives by Sabine Breitsameter

Inspired by Nature: How Acoustic Ecology Influences the Work of Sound Scenographers by Ramon De Marco and Jascha Ivan Dormann

Acoustic Architecture Progressing Beyond Sound Branding: Why Imprint When We Can Craft Together? by Lars Ohlendorf

Acoustic Lanes and Auditory Leads: SpatioTemporalities of Social Acoustics and Public Address Systems in Late Weimar Germany by Heiner Stahl

On the History and Aesthetics of Noise Reduction by Jens Schröter

The Critique of Power Dynamics Through Sound by Gregory Blair

Sound Design for Electric Vehicles: Fulfilling Laws and Creating Digital Artwork by Alessandro Fortino

129

145

173

Voice in the Machine: AI Voice Cloning in Film by Ross Adrian Williams

Electronic Music: Utopias and Realities by Thomas Neuhaus

Listen Up! Strategies of Theatre Sound Towards Artistic Identity, Sonic Branding, and Acoustic Ecology by David Roesner

191

203

229

The Audiovisual Chord: Invitation to a Dance Between Sound and Image by Martine Huvenne

Soundesign in German Movies by Jörg U. Lensing

Afterword

The Beginning and the End of the Radio Play by Andreas Ammer

239

Scientific Committee Editorial Board & Information

Editorial

Welcome to the latest release of Art Style Magazine! Our online magazine has come a long way in meeting scholarly journal standards and producing high-quality work. As a result, we have been recognized by several prestigious indexes, including the Directory of Open Access Journals (DOAJ), the Web of Science Core Collection, the Emerging Sources Citation IndexTM (ESCI), the European Reference Index for the Humanities and Social Sciences (ERIH PLUS), and Latindex, which demonstrate the editorial team’s strong commitment to maintaining excellence, and more. Our editions cover contemporary themes and are of interest to academics and individuals from various fields. The success of Art Style Magazine is due not only to its editions and editorial team but mainly to the esteemed professors, academics, and authors who share the magazine’s unwavering commitment to providing exceptional content and knowledge that leads to outstanding achievements.

Therefore, this publication on “Acoustic Pathways: After the Turn” is a culmination of extensive research and expert insights from renowned researchers, professors, and specialists in the field. Jörg U. Lensing, a sound design professor at Dortmund University of Applied Sciences and Arts and a director and composer for both theatre and film, has curated this special edition. The aim is to provide a comprehensive overview of this appealing topic. To that end, Professor Lensing and I have edited this valuable publication that offers a deep understanding of acoustic pathways and their relevance in the contemporary world. The sound world has undergone a tremendous transformation over the past few decades. With the advent of new technologies and the rise of digital music, the way we consume, create, and experience audio has changed forever. However, amidst all the changes, one thing remains constant the power of sound to move us, inspire us, and connect us to something deeper than ourselves. When it comes to the world of audio-visual content, the concept of acoustic pathways is crucial. Essentially, acoustic pathways refer to how sound travels through a given space. That can include everything from the physical properties of the space such as the size and shape of the room to the materials used to construct it. From a creative standpoint, understanding acoustic pathways is essential for anyone in audio-visual production. Thus, in this new era of sound design, acoustic pathways have become more critical than ever. As we move away from traditional recording and performance techniques, the need for authentic sound has become increasingly important. Acoustic pathways are the channels through which sound and timbre music flow, connecting the artist and the listener intimately and profoundly What happened after the “acoustic turn” was announced in 2008? Hence, I am thrilled to announce that Art Style Magazine’s latest special edition on “Acoustic Pathways: After the Turn” is a significant outcome, featuring rich content compiled by renowned expert Professor Lensing.

Enjoy this valuable publication, and happy reading!

Acoustic Pathways: After the Turn

The interdisciplinary anthology called Acoustic Turn by Petra Maria Meyer was published in 2008, and gained attention for a sensory branch that was underestimated until then. Since then, the topics related to sound design have expanded, and it is well-known that wellthought-out sound designs lead to higher quality media and multisensory perception in elaborate radio plays, films, scenography, e-mobility, podcasts, and immersive audio-visual forms of design. Acoustic studies have emerged as a scientific field that combines musicology and media studies. Recent publications such as Handbuch Sound (Morat and Ziemer 2018), Handbuch der Filmmusik (Kloppenburg 2012), and books on sound design have significantly expanded the discourse in the German-speaking world. Additionally, translations of essential works by Michel Chion, such as Audio-Vision (1990 in French, 1993 in English, 2018 in German), have contributed to the growth of this field. Internationally, numerous publications on the topics mentioned below have become standard works.

This special issue on “Acoustic Pathways: After the Turn” comprises original contributions that focus on this subject, along with a foreword by Michel Chion. This French film theorist and composer of experimental music discusses the recent trend in France of using the terms “immersive” and “experience” about certain works of art. He questions the value of these experiences, arguing that everyday life is already immersive and that the term emersive would be more appropriate to describe works of art that present themselves in a structured form, such as music. Chion believes this structure creates the conditions for an “emergence,” or event, that is unique to the listener’s experience.

Further, in “Soundscape: Selected Historical and Aesthetic Perspectives,” Sabine Breitsameter explores the neologism “soundscape” which refers to the sounds we hear around us. This concept has been popular since the late 1970s. It encourages people to focus on the sounds they hear and look at the world from a new perspective. This article explains why the soundscape is important, how it relates to society and the environment, and how it has been studied since the 1970s.

Following that issue, the article titled “Inspired by Nature: How Acoustic Ecology Influences the Work of Sound Scenographers,” by Ramon De Marco and Jascha Ivan Dormann, highlights how soundscapes can affect our emotional response to different spaces The authors have extensive experience in sound scenography and have created audio environments for various settings for Idee und Klang Audio Design. Acoustic ecology plays a vital role in their work and has significantly influenced their approach to sound scenography. In their article, De Marco and Dormann demonstrate the connection between human behavior, species extinction, and acoustic ecology. They also explore the positive impact of acoustic ecology on both non-human species and the environment.

Further, while advanced technologies like AI offer new possibilities for sound branding, in “Acoustic Architecture Progressing Beyond Sound Branding: Why Imprint When We Can Craft Together?” Lars Ohlendorf explores acoustic architecture’s implications in transforming how brands communicate. He explains that exciting possibilities for unique soundscapes beyond physical and virtual realms exist through data and technology, with an emphasis on participation.

Looking back, Heiner Stahl wrote an article called “Acoustic Lanes and Auditory Leads: Spatio-Temporalities of Social Acoustics and Public Address Systems in Late Weimar Germany.” In this article, he discusses a compositional approach to space and time that involves measuring and notating sound in soundscape studies. The article explores the social acoustics of sound and noise in Interwar Germany, and how the management of acoustics was conducted as a political affair. Stahl’s idea of social acoustics is shaped by the interconnectedness of behavior, self-convergence, and technologies.

Jens Schröter’s essay, “On the History and Aesthetics of Noise Reduction,” delves into the significance of noise reduction in modern sound production. Schröter notes that while analog media technologies relied on noise reduction filtering, the advent of digital technologies has made these systems obsolete. He also highlights the aesthetic value of noise reduction, particularly in producing silence in cinema and experimental media aesthetics in electronic music.

Certain musicians and sound artists have employed various methods to critique and challenge power formations, leading to the creation of a sub-genre in the history of sound. “The Critique of Power Dynamics Through Sound” by Gregory Blair delves into the use of sound in music and art as a tool for political critique and disruption of power structures. Blair’s analysis examines how music and sound can be used to challenge power dynamics and societal norms that have become normalized. The paper provides a detailed analysis of three projects that showcase the critique of power dynamics through sound: Pussy Riot’s Punk Prayer, Samson Young’s Canon, and Selma Selman’s You Have No Idea.

Also taking into account “Sound Design for Electric Vehicles: Fulfilling Laws and Creating Digital Artwork,” Alessandro Fortino delves into the techniques of sound design used in the past for vehicles with internal combustion engines. His article highlights toolchains and creative approaches that transform EV sound into digital artwork, shedding light on how the sound design industry is adapting to the new demands of the electric vehicle era.

Additionally, concerning that AI companies are revolutionizing the traditional methods of replacing voices on set or creating foreign language versions of films. Ross Adrian Williams’ article, “Voice in the Machine: AI Voice Cloning in Film” delves into the emerging field of AI voice cloning in film production.

In the article, the author highlights the importance of a character’s voice in cinema, as it is closely linked to their body and movements and can define a character’s personality. William discusses the impact of AI voice cloning on film production, especially in automated dialogue replacement (ADR) and film localization.

Moreover, innovation is a powerful force that can cause significant changes in industrial, social, and cultural contexts. Technological advancements have often been the driving force behind such innovation and have given rise to entirely new art forms in the visual arts. While technology’s impact on aesthetic innovations in music may also be significant, it is worth exploring its potential in shaping the future of music. In that way, the article “Electronic Music: Utopias and Realities” by Thomas Neuhaus discusses the pioneers of technology who have always envisioned a utopian future.

In “Listen Up! Strategies of Theatre Sound Towards Artistic Identity, Sonic Branding, and Acoustic Ecology,” David Roesner explores the relationship between sound and theatre, focusing on three main aspects: sound as a creative tool for the ensemble, sound as a branding element, and sound as a means of ethical exploration.

Roesner underlines sound as an interdisciplinary subject closely linked to our identities, showcasing how contemporary European theatremakers use it to transform their creative processes, develop artistic identities, and investigate the relationship between music, voice, and sound with bodies, texts, and spaces. Additionally, he highlights the use of sound in branding efforts. Theatres can also serve as critiques of commercial sonification and as reflections on our rapidly changing acoustic ecology. Roesner provides two case studies to illustrate his points.

In addition, “The Audiovisual Chord: Invitation to a Dance Between Sound and Image” is an introduction to Martine Huvenne’s book The Audiovisual Chord: Embodied Listening in Film The book takes a phenomenological approach to film sound and is grounded in Merleau-Ponty’s phenomenology. It presents analytical tools such as sound as a dynamic transmodal movement, thinking in movement, auditory filmic space, and the audiovisual chord These tools enable the application of insights gained in filmmaking and film analysis. The book’s main case study is Robert Bresson’s A Man Escaped (1956), presented as a phenomenological film. Bresson’s vision of film as a transmission of experience and presentation of a life-world is explored in detail.

As we near the end of this special edition, we have the pleasure of reading Jörg U. Lensing’s insightful article on the subject of film sound design. However, while almost all articles and books on this special topic almost always deal with Anglo-American films, J.U. Lensing analyses the introductions of six exemplary German films for their specific audio-visual interactions and in relation to a somewhat different understanding and aesthetics of what sound design can achieve for film.

Finally, Andreas Ammer wrote an afterword on “The Beginning and the End of the Radio Play,” which sheds light on the current state of radio plays. Radio plays were once a popular form of entertainment but have largely faded away in the digital age due to a lack of funding and competition with other forms of media. Despite these challenges, Ammer shows that some groups continue to create and promote radio plays to preserve this art form for future generations.

Such an endeavour always involves a great deal of work and requires the cooperation of specialists in their field. The editors Christiane Wagner and Jörg U. Lensing are therefore very grateful to have gained a total of 14 renowned authors on the subject of “acoustic pathways,” who were able to contribute topical articles on their subjects. This special issue of Art Style Magazine is a follow-up publication to Acoustic Turn (2008). It is due to the fact that the initiator of this project, Professor Jörg U. Lensing, primarily teaches and researches on the subject in Germany that the majority of the authors are originally German-speaking. Nevertheless, the subject areas are internationally relevant and this publication is intended as a contribution to the international discourse on the value of the auditory, particularly in the field of art and design.

Immersive Music

Michel Chion

There is a current vogue in France concerning the words “immersive” and “experience” (the latter in the American sense). They are advertising new forms of shows, video games, performances, and concerts, as well as domestic listening and viewing installations, and I am constantly receiving information in my mailbox about artistic events promising “a sensory immersion.”

First of all, I am rather sceptical about this notion since, for the eye, it is only theoretical. Indeed, we cannot look around us without the obligation of moving, thus moving our limited visual field. Even a mirror, reflecting what is behind us, does not show us everything, and above all, it is impossible for us to embrace our entire visual field with just one glance.

On the other hand, when the term immersive is used to promise strong sensations, I would like to point out that already our daily reality is immersive, starting with the most banal experiences like lying in bed, being cold, going shopping, rolling in a car, and so on... So, I don’t see what is considered to be so interesting in experiencing a situation of “sensory immersion.”

In my view, a work of art, or a musical or dramatic experience, has the advantage of being non-immersive. A work of art, a play, a performance, or a show in the childish sense of magic and puppets, they do charm our eyes and our ears exactly because there is something being presented in front of us, and we can only project ourselves in a show (visual, sound, etc.) precisely because we are not being immersed by it.

This is why I am not the only one to demand that the audience be oriented in just one direction, the same for all listeners when one of my works is performed on an orchestra of loudspeakers (acousmonium). In fact, I want the audience to be oriented towards an acoustic stage on which a good part of these loudspeakers is installed.

Sure, a certain pleasure can be found in so-called attractions like bumper cars, ghost trains with phonic sensations, thus a feeling of being moved and shaken but can we consider this as being aesthetic? On the other hand, since words like “immersive” are used as commercial arguments, what desires do they serve or correspond to? Do we want to evoke a fusion with an environment in the same way we perceive a fish in water? Is it about making the entire body vibrate, as music, played at a very high level, allows a kind of feeling of being carried like a child, thus a feeling associated with a regressive form of well-being?

Sound, in musical or non-musical form, has long been a means that seems most conducive to “immersion.” By its nature, it potentially addresses two senses: hearing and feeling. By this, I mean that the same cause, a sound wave, can produce in us two distinct physical effects, but they are wrongly taken for one sensation, and this is for the only reason that they are happening simultaneously.

A sound can indeed address both the ear and the body. In what I call the auditory window (vibrations that the ears pick up and transform into an acoustic sensation), we hear sound objects that are more or less melodic, shaped, continuous, or discontinuous, that have a matter, a pitch, have or do not have a mass, a spatial path, etc. At the same time, if this sound is loud and has low frequencies, we feel disruptions in other parts of our body, vibrations that are missing textural qualities but that are able to make us want to move and dance just naturally when this rhythm is being pulsed. I call this reaction to certain sounds (not all of them) a reaction of the body, things, and objects that I call a co-vibration It is to this covibration that Martin Luther alludes when he writes in one of his hymns:

Heilig ist Gott, der Herre Zebaoth, Sein Ehr die ganze Welt erfüllet hat, Von dem Geschrei zittert Schwell und Balken gar,

Or else Georg Trakl: Sanfte Glocken durchzittern die Brust.

Or Virginia Woolf in her novel Mrs Dalloway: The throb of the motor engines sounded like a pulse irregularly drumming through an entire body.

Some low frequencies at a certain level of intensity cause the listener’s body to resonate in co-vibration while at the same time drawing an acoustic picture in the “auditory window” of our ear However, due to their lower intensity and highfrequency spectrum, other sounds simply fit into the window in question and do not evoke co-vibrations.

Consequently, as I said, we are left to assume that the simultaneous sensations of sound object, acoustic figure (in the auditory window), and bodily co-vibration are identified and referred to by the same word, “sound.” This is in spite of the fact that they are profoundly different. And this is happening only because they are perceived as being the effect of these causes and because they occur simultaneously. In the same way, a specific perception of light that would be systematically associated in synchrony with a specific perception of sound cannot be separated and consciously isolated from each other. They would be perceived as “the same ” thing in two forms.

In short, what we call “the” sound its singular needs to be questioned indeed could then, in certain specific cases, be bi-sensorial (i.e., affecting two senses at the same time).

This would be one of the reasons why the physical, “immersive” investment of the viewer/listener is more immediate and irrepressible via sound than via picture, the latter being monosensorial. Other reasons for this difference in physical investment lie in the impossibility of “turning away from listening” in the same way as one can turn away from looking, as well as in the frequently non-directional character of sound, thus allowing certain sounds to surround us.

For a long time, only certain specific musical forms military music or certain dance music made extensive use of sounds that produced intense co-vibrations, inviting movement. But this required live performers and instruments with a large range.

However, current technology makes it possible to use recorded sounds that do not ‘tire’ and amplify them to high power. Some hi -fi brands have developed a new generation of loudspeakers. Very small in volume, they are able to produce an intense sound with a strong bass. These speakers can be connected via Bluetooth to a mobile phone and, on the latter, to virtually endless playlists, which provoke “immersive” sensations whose vibrations create a feeling of being carried.

In France, immersive art is fashionable, especially for music, and there are already ‘immersive art manifestos.’ Is this the future of music? Is it only possible form from now on? I do not think so.

It turns out that the electro-acoustic means that I have been using for the last fifty years to compose works of concrete music can be used for making “immersive” music, inhabiting all space and time uninterruptedly and uniformly, without gaps, without holes, and striking the body. However, these means are as suitable for making music that, by the opposition, I would call “emersive:” a music that presents itself in the form of a work of art, music that obeys a form. This form creates conditions for an “emergence,” an event.

The most common form of novel and drama, form and narration, can highlight an event that can be spectacular but also discreet, humble, and intimate, and this can even happen in non-narrative music.

For example, let ’ s look at the beginning of my Requiem, composed in 1973. I have chosen to attack the first movement of this 37-minute work with a sharp, brutal sound that stops after about 40 seconds. Even in a concert, the listener cannot anticipate this brutal attack since the sound comes from loudspeakers and not from an instrument. Then, a male voice is heard saying a short introductory text. Only then, with a piano intensity, do electronic chords begin, from which, gradually, two other voices emerge, a whispered female voice and a male voice heard from a distance. At first, they are barely audible, but thanks to the concentration I have created by starting with a violent sound and then giving it up, the audience is able to taste the intimate character of this duo, almost sexual, I was often told, although this was not conscious on my part when I composed it.

Even though I, who composed this music fifty years ago, heard and played it very often, I am still moved by this passage of vocal intimacy: it only lasts a few seconds, but I am happy that I created the conditions for these few seconds. They only exist due to the context of the whole work.

This moment is indeed the result of a general form, and this must be emphasized. At this point in the concert, I only use a few loudspeakers, choosing those in front of the listeners. At other moments in the Requiem (the beginning of the Evangile), the emergence of a moment is created by opposite means: an explosion of sound that occurs from all sides at once.

In a much more abstract work like my Sonate en trois mouvements, my ThreeMovement Sonata, which I composed in 1990, the event is constituted by a ‘false ending’ in the course of the second movement: when it seems to have ended after a brief silence, it resumes for a few seconds from sounds similar to those already heard, but filtered, as if reappearing in a dream.

For a musical work, concrete or not, to be “emersive,” it is necessary that it does not occupy time in an equal, continuous, regular way... and that it does not occupy space and our body in a constant way, by regular co-vibrations. A work like the Requiem and the others that I composed thereafter and in the course of half a century all of them aim, among other things, at creating such moments in which one is invited to listen, in which one creates the conditions of an opening, in which one recreates a new space of attention, and at the same time in which one makes space happen.

In his diary Am Felsfenster morgens (und andere Ortszeiten 1982-1987), Peter Handke writes:

Jetzt weiß ich, was mich so stört an der meisten Musik: sie nimmt mir den Raum; sie verzerrt ihn (Nacht, Garten, Grillen).

With the means given to me by concrete music (which allows me, for example, to create sudden silences, real silences, by making the sound interrupt radically, without anything visible announcing the interruption), I also seek to recreate space, to allow us feeling it again, to make it happen. Immersive forms of music are destroying this space

Michel Chion is a French film theorist , composer of experimental music and associate professor at the University of Paris III (Sorbonne Nouvelle)

Soundscape Selected Historical and Aesthetic Perspectives1

Abstract

The neologism soundscape refers to the acoustic envelope that surrounds the listener. Since the late 1970s it has become continuously popular. In recent years, there has been increasing criticism that the term soundscape is used blurrily and arbitrarily, and thus risks to lose its explanatory power in artistic and scientific discourse. Yet soundscape is more than just a term: it stands for a paradigm. In the following, the origins, prerequisites, implications, consequences, and perspectives of the term soundscape will be explained in order to make its core accessible and its original intensions clear: The soundscape concept leads to an aesthetic perspective that consistently approaches the world and its auditory phenomena from the perspective of hearing, encouraging people to change focus and thereby enabling an alternative world “picture ” At the same time, the holistic acoustic environmental experience that the term implies, in the context of R. Murray Schafer’s theory of Acoustic Ecology, becomes tangible as an expression of the society from which it emerges: a soundscape is a reflection of a specific society’s conditions, tensions, and namely its ecological conflicts. Finally, since the 1970s, the Soundscape Studies have been establishing a vocabulary for the description of the auditory world and its phenomena that makes the important elements of acoustic environmental experience conceptually comprehensible and thus sustainably accessible to scientific as well as aesthetic reflection. By demonstrating why the world sounds the way it does, the soundscape paradigm also opens up starting points for its applied design and the artistic forms that emerge from it. This is especially important with regard to the upcoming immersive technologies and the audiomedial forms of experience that derive from them.

Terminological Considerations

The term ‘soundscape’ emerged at the end of the 1960s in the North American Sprachraum (anglophone region). As one might surmise, it is a combination of the words ‘sound’ and ‘landscape’ An early use of the term was documented in 1966 by Richard Buckminster Fuller, who wrote,

When man invented words and music he altered the soundscape, and the soundscape altered man. The epigenetic evolution interacting progressively between humanity and his soundscape has been profound.2

Buckminster Fuller thus describes a dynamic interrelationship between humans and their acoustic environment, in which both exert a formative force upon each other. In 1967, architect Michael Southworth used the term to describe acoustic manifestations of urban space that he had mapped.3 Subsequently, the term was taken up by the Canadian music educator and composer R. Murray Schafer, who used this term which had clearly made its mark on him in order to understand the auditory phenomena of everyday life in their entirety, to perceive them attentively on this basis, and to be able to subject them to a critical evaluation.4

Since then, the term ‘soundscape’ has encompassed the perceived totality of all acoustic phenomena that occur in specific places, spaces, or landscapes. The environmental sounds are, first and foremost, representatives of a given spatial or local situation and its geographical, cultural, technical, and social peculiarities. A village in the desert has a different soundscape than a village by the sea. A Chinese city in an industrial region sounds different from the industrialized urbanity of Central Europe. Forests, mountains and climate can be reflected in a soundscape just as much as religion, architecture, the degree of technologization of a society and many other contexts.

Generally speaking, in a soundscape not only geographical and physical conditions can manifest themselves auditorily, but also flora and fauna, language, social values, communication behavior, everyday activities, and the many practical aspects of the interaction between humans, nature, and technology. The term ‘soundscape’ is also applicable to closed spaces, such as a concert hall, office, or apartment, as well as to artistically created, media-related, and virtual spaces, such as a composition, a radio program, or a computer game.

When compositions are presented, an entirely new sound environment is created with its own sonic atmosphere and its own timespan of listening. Any music and its use in any context (whether in a performance situation or in a mall) is just as much a sound source of environmental concern as, say, car motors or dog barks.5 What is perceptible in a soundscape goes far beyond representation and signifying. The auditory acquisition of an environment and its surroundings that a person is accustomed to accessing visually acts to reconfigure perception in an essential way. Only rarely does the auditory thereby take on a delineated representationality that can be concretely differentiated from other auditory objects. Rather, the sounds of a place or a landscape merge into a totality that is fundamentally different from visual perception and its ability to grasp something in an instant. To present themselves, sounds require time. A soundscape transposes the visual moment into the auditory duration. If the term ‘soundscape’ is about “seeing the landscape with the ears,” as Murray Schafer described it in a WDR radio interview in 1998,6 then the shift from seeing to hearing refers equally to a perceptual attitude in which it is not so much the ‘thing-ness’ that can be experienced but, primarily, the process.

From this confluence of all sounds in a particular soundscape, a distinctive sonic ‘appearance’ can emerge, an acoustic peculiarity or identity that is similarly characteristic to a specific visible manifestation. If one identifies the sounds by which a specific soundscape is characterized, one can also obtain information about the central values of a society, its priorities, deficits, power structures, and its ecological state. Which auditory events are dominant despite irritating or totally disturbing the listener? Which auditory events do not occur or are inaccessible to the sense of hearing, and for what reason? Which soundscapes are unwholesome, noisy, or repellant, and which contain elements that got out of proportion? Which social priorities contribute to the fact that places, spaces, and landscapes sound the way they do? It is precisely these questions that crystallize the socio-critical impetus inherent in the term ‘soundscape’.

An important precursor of the soundscape concept can be found, in particular, in Luigi Russolo’s manifesto L’Arte dei Rumori (The Art of Noise, 1913), in which he points to the acoustic phenomena of the industrial age as the starting point of what he calls a newly emerging mode of listening and musical aesthetics: “We Futurists have all deeply loved and savored the harmonies of the great masters (...).”7

If we pass through a modern metropolis with more attentive ears than eyes, we will be lucky to distinguish the suction of the water, the air or the gas in the metal tubes, the hum of the engines that undoubtedly breathe and quiver like animals, the knocking of the valves, the rise and fall of the pistons, the screeching of the sawmills, the jumps of the streetcar on the rails, the cracking of the whips and the rustling of curtains and flags. We enjoy distinguishing in our minds the noise of the window blinds, of the stores, of the slammed doors, the din and scrape of the crowd, the various sounds of the stations, the spinning mills, the printing presses, the electric power plants, and the subways.8

In this description, Russolo already anticipates in addition to the aestheticization of everyday sounds a number of characteristics of Schafer’s later soundscape concept, in particular the attention to sounds that are usually ignored, the dynamic experience of the listener by means of a ‘sound walk’, and the aspect of orchestration, meaning the deliberately designed symphony of all sounds (“acoustic design”) in the urban sound concert.

Another hitherto little-known vanguard who designed, described, and also mapped the landscape namely the natural landscape as an acoustic concept was the Finnish geographer Johannes Gabriel Granö (1929).9

While the term ‘soundscape’ began to establish itself in the Sprachraum of North America as early as the 1970s, it reached Europe in the mid-1980s, where the idea first became known to a wider audience under the term Lautsphäre (soundsphere) in the early 1990s.10 For the term ‘soundscape’, which was translated inconsistently, the more specific German translation Klanglandschaft 11 (essentially a one-to-one translation of the two root words) was then found. In the late 1990s the original English-language term began to prevail, and has now also established itself in German. The use of the English term has the advantage that it is no longer so clearly framed by geographical connotations, but it does have the disadvantage that it is often used in a very generic sense, even becoming generalized beyond recognition.

Implications of the Term ‘Soundscape’

R. Murray Schafer conceptualized the soundscape within the framework of his teachings on acoustic ecology, which studies the interrelationship between living beings and their environment. It relates the quality of the acoustic environment directly to the significance of listening in a society, in particular to the degree of physiological ability and mental readiness to listen attentively. This is a concern decisive for Schafer’s teachings. In his main work The Tuning of the World (1977), he unfolds a cultural history of listening, establishing a close connection between the listening experiences of various historical epochs (such as Antiquity, the Industrial Revolution, and the Electrical Revolution) and the significance of listening, as well as the auditory practices in the respective society.

Schafer’s springboard is the noise pollution of the late 1960s. Due to the increase in car and air traffic, the construction noise associated with the expansion of cities, and the proliferation of the music amplifier, acoustic ecology not only addresses hearing damage and auditory overload, but also works out the structural conditions of wanting to and being able to listen. As early as the beginning of the 1960s, Schafer’s experience as a music teacher was that many of his students found it increasingly difficult to listen intently and actively for extended periods of time, whether to music or the spoken word. However, Schafer did not look for a solution to the problem in an adaptation of the content (“Beatles instead of Bach”), but looked at the overall situation. He found that in an everyday life characterized by uncontrolled acoustic expansion one that barely offers any intentionally designed input for the ear, and fails to actively avoid unpleasant, communicatively empty and meaningless sound events the majority of people would not expect anything worth hearing and would thus keep their ears closed. An out-of-control, repugnant acoustical environment has conditioned its inhabitants to dull their hearing and abandon the auditory world to indifference. Experiences shape expectations. What one does not become acquainted with, what is not imaginable, what is no longer listened to, becomes inaudible, and thus non-existent.

Implicitly connected to Ernst Haeckel’s systemic idea of oecology (1866), Schafer’s concept of soundscape combines listener, sounds and environment into an “entirety,” a dynamic system in which the change of one factor influences all other factors and finally the auditory result itself.12 According to Schafer, an acoustically well-designed world based on conducive, livable and aesthetic soundscapes would be a necessary prerequisite for a general social appreciation of the auditory and the promotion of auditory reception and attention in general.

Within this framework, Schafer understands soundscape as a counterpoint to a listening that is selective and completely frontally oriented. Such a concept has developed as a result of (here he borrows from Marshall McLuhan13) a predominantly visual and frontally oriented culture. Its attitudes and practices have been largely adopted by the other senses. In the “system soundscape,” the listener must therefore be reconceptualized namely, no longer as a passive and frontal receiver or positioned at a privileged “sweet spot,” but as a living, agile element that contributes to and influences the soundscape. Thus, perceiving a soundscape always connotes participating in it and being “inside” it.

Elements of the Soundscape

Schafer simultaneously developed terminological and perceptual categories by which soundscapes can be described and analyzed. The three most important are ‘keynote sound’, ‘signal,’ and ‘soundmark ’ With these distinctions, Schafer transfers central principles of visual Gestalt psychology to the auditory, in particular the relationship between figure and ground. According to this categorization, the term ‘keynote sound’ refers to a permanently present sound that is characteristic of a place and is a foundation for all the other sounds there. This can be, for example, the sound of the sea that characterizes a particular beach hotel, or the hum of a highway that one hears from a distance, or the characteristic buzzing of a refrigerator in a particular apartment.

Keynote sounds have an effect over a certain duration. Above all, they establish atmosphere. A soundmark, on the other hand, is something more momentary. The word is rather obviously derived from the term ‘landmark ’ However, an acoustic landmark, or signifier, is usually less easy to identify than a visual one. The ringing of Big Ben’s bells, for example, is considered an acoustic landmark of London. The whistles and horns of trains in Canada can be considered a soundmark of the Canadian landscape. The characteristic door-closing sound of certain makes of car also counts as a soundmark. Any sound event that emerges specifically for a place, space, or object as a single, clearly outlined sound can be called a soundmark. The ‘signal’ also appears in the foreground and therefore figurally. An auditory phenomenon becomes a signal when it communicates a message. Signals are coded, such as the sound of the post horn, the ringing of the telephone, the wailing of the police siren, and so on.

Schafer makes an important distinction between so-called hi-fi and lo-fi soundscapes. While hi-fi, according to Schafer, means the transparent audibility of an acoustic environment in which all sounds can be clearly heard and spatially located and even distant, quieter sounds can be perceived a lo-fi soundscape is understood to be a broadband “wall of noise” that overlays and masks other sounds that occur, thus narrowing the acoustic horizon. Lo-fi soundscapes are usually perceived as more irritating than hi-fi soundscapes. Lo-fi is rather typical for urban soundscapes, although not in every case. The broadband noise, such as that of perpetual car sounds, rattling air conditioners, or vibrating construction machines, often masks fine and distant sounds, thus making them inaccessible to hearing. Subtle sounds such as people breathing, and distant sounds such as the footsteps of passersby across the street, are not readily accessible in lo-fi environments. From the perspective of acoustic ecology, hi-fi environments are more desirable.

Schafer is sometimes criticized for harboring hostility to technology and to the present, since the lo-fi soundscape, which he favors less, occurs mainly in environments characterized by technology and urbanity, and thus has a generally negative connotation. However, this criticism only applies to a quite limited extent, since Schafer’s critical attitude towards lo-fi soundscapes does not refer to the causative mechanisms per se, but rather to the effect of auditory leveling and the associated loss of a diverse, unrestricted listening. This criticism thus fails to recognize that Schafer’s argument here is primarily descriptive and phenomenological, that is, guided by the effect rather than the cause. Moreover, he repeatedly refers to the positive effects of a conscious soundscape design made possible by technological development through a criteria-based acoustic design yet to be developed. 14

The World Soundscape Project and Similar Undertakings

After R. Murray Schafer was appointed to Simon Fraser University in Burnaby near Vancouver in the late 1960s, he founded the World Soundscape Project as part of his research work there.15 His goal was to archive and analyze the acoustic phenomena of places and landscapes worldwide, and to record their changes over decades. Collaborators on the World Soundscape Project included the now renowned soundscape composers Hildegard Westerkamp and Barry Truax.

In the early 1970s, Schafer’s team also began to study their own city of Vancouver on Canada’s west coast. The sounds that were contemporary there at the time were initially compared with descriptions of soundscapes found in books and newspapers from the beginning of the 20th century. These sounds were also examined for future changes that were on the horizon for the urban soundscape of the 1970s, which included airport expansion, increasing road traffic, elimination of the characteristic signal horns in shipping, gentrification of the city, and increasingly multicultural daily life, among others. All of this, Schafer’s research team realized, would very soon and fundamentally change Vancouver’s acoustic identity.

The technically outstanding sound recordings that were made as part of the World Soundscape Project (which is still ongoing) are available in an archive at Simon Fraser University.16 They are not only of considerable historical and documentary value, but also of great aesthetic poignancy. This is one of the reasons why many of these recordings became the origin for various soundscape compositions, including within the Soundscape Vancouver Project (1995), in which the recordings from the 1970s by four composers 17 served as the starting point for their own creations (Figs 1-5) 18

In 1975, a research group led by Schafer traveled to Europe to investigate sites there as part of the Five Village Soundscape research project. The group chose the respective places based on the criteria that the location should still present soundscapes largely uninfluenced by traffic and industrial noise, and that traditional soundmarks should still be noticeable. The sites were Skruv in Sweden, Bissingen in Germany, Cembra in Italy, Lesconil in France, and Dollar in Scotland. In 2009, the same places were visited again by a Finnish research group and studied for their changes, especially with regard to the effects of urbanization on the respective soundscapes. This was done under the direction of Helmi Järviluoma, Heikki Uimonen, and Noora Vikman, with the collaboration of Barry Truax, who was a member of the 1975 Canadian group. 19

Soundscape Composition

Beginning in the 1920s, environmental sounds began to establish themselves as integral components of the artistic material canon. The conventional distinction between music worthy of being listened to on the one hand, and unappealing, unremarkable everyday noise on the other began to dissolve. The composer Kurt Weill, the radio producer Hans Flesch, the film artists Dziga Vertov and Walter Ruttmann, and the media scientist Rudolf Arnheim all stood in the mid to late 1920s for a conceptual and compositional development that gave the noise of the everyday environment a prominent and central position. They postulated this noise as being equal in status to musical sound or the spoken word. According to the radio pioneer Hans Flesch, this gave rise to an innovative form of auditory art that could no longer be captured by the traditional concept of music, and which was compelled forward by the new medium of radio. Groundbreaking in many respects was Walter Ruttmann’s audio production Weekend in 1929, which chronologically arranged the significant sounds of a Berlin weekend and ingeniously captured them aurally and montaged them.

The artistic positions of the 1920s described above were primarily concerned with opening up new compositional material and thereby also recognizably incorporating “reality,” the world of labour and machines, and everyday life. In contrast, the Musique Concrète that emerged in the 1940s, which also took everyday noise as its point of departure, favored an abstracting expansion of the palette of sound material beyond semantic echoes. The simultaneously emerging sound experiments of John Cage (4’33” or Imaginary Landscape No. 2, for example) aimed at questioning traditional concepts of perceiving music. Even if soundscape composition differs from the artistic positioning referenced here, it intersects considerably with them in each case.

Luc Ferrari’s compositional approach of a Musique Anecdotique, which emerged in the early 1960s as a critical reflex of Musique Concrète, already draws nearer to the idea of soundscape composition and is sometimes also actually referred to as an early soundscape composition.20 Ferrari used recordings of everyday situations, the immediate content of which is unabstracted and directly narrative in its aural presence, which is both scenically and semantically concrete, but at the same time also unfolds sound-aesthetic and poetic qualities.21

Soundscape composition as its own genre, which emerged in the mid to late 1970s, was defined in particular by composers Hildegard Westerkamp and Barry Truax.22 It is an electroacoustic genre that procures specific and identifiable sounds from the acoustic environment as its starting point. Westerkamp points out that soundscape compositions themselves are never abstract, but the recorded environmental sounds can very well be subjected to abstracting electroacoustic editing processes.23 In her own compositions, for example, Westerkamp often works with slowing down or speeding up the sound material, or with clever cuts that decontextualize the material, thus detaching it from its semantic context and making it accessible to aesthetic reception apart from a semantic binding. Barry Truax applies granular synthesis, in particular, to his soundscape materials.

The soundscape composition is characterized above all by the fact that it conveys, as a basic compositional intention, its interest in the acoustic environment per se or in a specific soundscape an interest that is shaped by the individual attitude of the respective composer.

Soundscape composition is as much a comment on the environment as it is a revelation of the composer’s sonic visions, experiences, and attitudes towards the soundscape. Audio technology allows us as composers to sort out the many impressions that we encounter in an often chaotic, difficult sound world. 24

Common to the soundscape compositions is an approach that makes one’s own critical listening to the everyday environment the starting point of the respective artistic concept.

... the soundscape composer’s attention to ecological issues of the soundscape ideally extends beyond the compositional process in the studio: it starts with listening as a conscious practice in daily life, continues during the acquisition of sound materials, the work in the studio, right through to the presentation of the final piece.25

An important artistic goal of soundscape composition is thus not to transport “reality” through the composition, but the emergence of aesthetic qualities and special features in what is already taken for granted and known: the breaking of perceptual routines and a new, intensified perception of what is already known.26 The artistic goal is mostly based on an aesthetic-instructive goal, namely to increase the listener’s appreciation of the acoustic environment.27

Outstanding soundscape compositions include A Sound Map of the Hudson River (1989) by Annea Lockwood, Basilica (1992) by Barry Truax, and Kits Beach Soundwalk (1981) and Into India (1995) by Hildegard Westerkamp. However, numerous pieces produced for radio play and artistic radio documentary formats also meet the criteria for a soundscape composition, such as Ritratto di Città (1954) by Luciano Berio/Bruno Maderna, The Solitude Trilogy (1969-76) by Glenn Gould, A Winter Diary (1998) by Murray Schafer and Claude Schryer, and Lezioni di Musica (1994) by Stefano Giannotti. Ros Bandt (Australia), Susan Frykberg (New Zealand), Andra McCartney (Canada), and Francisco Lopez (Portugal) have extensive oeuvres in the field of soundscape composition and have each individually expanded the concept of the genre as defined by Westerkamp and Truax. Among the composers of a younger generation who also deal with questions of acoustic ecology are Darren Copeland, Carmen Braden (both from Canada), Leah Barclay (Australia) and Lasse Marc Riek (Germany).

Criticism at the Soundscape Terminus

Exemplary for the criticism of the term ‘soundscape’ are the remarks of the British anthropologist Timothy Ingold.28 He takes offense at the fact that listening from the perspective of soundscape deprives the perceiver of other sensory experiences, such as visual, tactile, and olfactory experiences, among others, and consequently proclaims an alarming sensory reduction. Murray Schafer, however, has repeatedly emphasized the need for sensory wholeness.29 Ingold also fails to recognize that it makes sense to consider the auditory phenomena of the world separately in order to be able to investigate their manifestations and inherent laws, if necessary, so that they can be transdisciplinarily integrated according to Schafer’s declared intention “into the general study of the environment (...).”30 Ingold’s critique would structurally apply much more to disciplines such as art history and architecture, in which visual phenomena are usually studied in isolation from other modes of perception. A focus on exclusively visual perception is only rarely questioned.

Ingold’s criticism also starts from the fact that the term ‘soundscape’ is often identified with the process of hearing, and implies the world of the auditory as something object-like. Ingold does not provide any evidence or citations for this, which makes it difficult to comprehend his critical approach in the fundamentality he postulates, especially since he presupposes a dynamic interrelationship between the listener and the auditory phenomena, and is thus fundamentally and inextricably tied to considerations of process.31 However, his position becomes understandable with regard to what has often become an unreflected and very generic use of the term, with which the everyday world of sounds per se is boiled down to an undifferentiated, auditory “form of intuition” in the sense of an observation category called soundscape, without taking into account that this term is actually a process-oriented concept.

Soundscape as Auditory Gestalt and Figure of Thought

Most often, soundscape is treated as an empirical phenomenon as something that “exists” and can be pragmatically perceived with the “correct” allencompassing listening attitude. However, this runs the risk of misjudging the highly conceptual character of the term ‘soundscape’. The landscape-oriented perception of sounds simultaneously and holistically-comprehensively constructs an auditory gestalt transcending ingrained listening habits. This is a comprehensive gestalt, down to the quietest, most inconspicuous sound. The term ‘soundscape’ thus directs auditory perception towards a spherical 360° hearing, in which every sound has its position in three-dimensional space and in which no distinction is made between important and unimportant, loud and soft sounds. This “formative listening”32 elicits a model that no longer separates between desirable and ignored sounds, no longer promoting a hierarchy between signal and noise. All present sounds are equally important for the auditory appropriation of the environment. Even that which does not occur plays a major role, as reflection on absent or missing sounds contributes enormously to a deepening understanding of a soundscape.

The term ‘soundscape’ is also a figure of thought that reformulates auditory perception. The spherical listening demanded by soundscape sets itself apart from frontal reception as cultivated in reading, stage situations, stereophonic listening (via radio, television, and music systems) or in traditional school lessons. With the soundscape, Schafer replaces inherited frontal concepts of perception and linear patterns of auditory presentation and communication with an audio-

tactile, lifeworld model. Within the soundscape’s cognitive figure of thought, the sounds to be examined by the ear do not exist “objectified,” but rather contextualized in mutual influence and impression in relation to the listeners. The term ‘soundscape’ thus represents a systemic concept.

Perspectives

In view of the proliferation of immersive media, which provide the recipient with an aesthetically consistent all-round experience, the term ‘soundscape’ occupies a key conceptual position, not only in the design and composition of auditory environments in the field of 360° media and virtual reality, but also in the construction of technologies and programs by means of which immersion and allround experience can be acoustically realized. Soundscape’s elements, as well as the structural considerations underlying the term ‘soundscape’, offer helpful basic constructs for the technical development and realization of design parameters. The cultural and aesthetic aspects of the soundscape concept, especially those that identify the role of the listener as part of the environment, specify the designdramaturgical forms of experience that can be produced with the tools of immersive media technologies.33 To develop consistently such three-dimensional auditory experiences, without remaining bound to principles of linear and frontal perceptual habits, is an immense challenge and has yet to be realized. The concept of soundscape offers various starting points for this and is also suitable as a point of departure for multimodal-environmental modeling in the field of virtual reality.

Author Biography

Prof. Sabine Breitsameter is an award winning author, producer, curator and festival director with four decades of practical experience in the international media industry and art institutions, with focus on media art. Since 2006 she researches and teaches at Darmstadt UAS as a professor for “Sound and Media Culture ” Co-founder of the Master’s program Sound Studies at the University of the Arts/Berlin, guest professor for Experimental Audiomedia from 2004-2008. Jury member of e.g. Prix Ars Electronica, German Audiobook Prize, ARD/ZDF Award Women + Media Technologies; appointed advisory member at German Music Council, and co-president of the Hessian Film- and Media Academy (hFMA).

Notes

1 Translation: John Jones. This text is an extended and updated version of my article “Soundscape,” Hansjakob Ziemer, Daniel Morat (Eds.), Handbuch Sound. Geschichte Begriffe Ansätze. (Stuttgart: J.B. Metzler Verlag, 2018), 89-99.

2. Richard Buckminster Fuller, “The Music of the New Life,” Music Educators Journal, 52(6) (1966): 52.

3 cf. Michael Southworth, The Sonic Environments of Cities, Environment and Behaviour (1969): 49-70. The term soundscape is said to have appeared occasionally in Englishlanguage musicological as well as anthropological literature as early as the 1930s. Concrete research results on this are still pending.

4 R. Murray Schafer, The New Soundscape (Scarborough: Berandol Music Limited/New York: Associated Music Publishers, 1969).

5 Hildegard Westerkamp, “Linking Soundscape Composition and Acoustic Ecology.” Organised Sound, An International Journal of Music and Technology, Volume 7 “Soundscape Composition,” Number 1, Cambridge (2002): 51-56.

6 Murray Schafer in conversation with Klaus Schöning, on the ocassion of the premiere of his soundscape composition A Winter Diary on radio station WDR3 Studio Akustische Kunst, 17 April 1998.

7 Luigi Russolo, Manifesto futurista, Milan, 11 March 1913, cited in: http://www.medienkunstnetz.de/quellentext/39/, (01.06.2023) in the translation by Justin Winkler and Albert Mayr.

8 Ibid.

9. Cf. Johannes Gabriel Granö, Pure Geography, A methodological study, enhanced by examples form Finland and Estonia (1929), edited by Olavi Granö and Anssi Paasi, translated by Malcolm Hicks (Baltimore: Johns Hopkins University Press, 1997).

10 R. Murray Schafer, Klang und Krach. Eine Kulturgeschichte des Hörens (The Tuning of the World in German translation), translated by Kurt Simon und Eberhard Rathgeb and Heiner Boehnke, eds. (Frankfurt/Main: Athenäum, 1991).

11 Justin Winkler, “Landschaft hören. Geographie und Umweltwahrnehmung im Forschungsfeld Klanglandschaft,” Regio Basiliensis. Basler Zeitschrift für Geographie 33, 1992, H. 3, pp. 196-206. Justin Winkler, Klanglandschaften: Untersuchungen zur Konstitution der klanglichen Umwelt in der Wahrnehmungskultur ländlicher Orte in der Schweiz, Universität Basel/Schweiz, 31 January 1995.

12 This was analyzed very effectively by Barry Truax, Acoustic Communication (Westport/London: Ablex Publishing, 1984): 12.

13 See also McLuhan’s concept of audio-tactile all-round perception of the electronic age in: Herbert Marshall McLuhan, The Gutenberg Galaxy, The Making of Typographic Man (Toronto: University of Toronto Press, 1962), 11 et seqq.

14. R. Murray Schafer, The Tuning of the World (Toronto: McClelland and Stewart Limited, 1977), 237 ff.

15 https://www.thecanadianencyclopedia.ca/en/article/world-soundscape-project-emc (02.06.2023).

16. World Soundscape Project Recording Library: http://www.sfu.ca/sonic-studiowebdav/srs/index2.html (02.06.2023).

17 Sabine Breitsameter, Darren Copeland, Claude Schryer and Hans-Ulrich Werner.

18. Soundscape Vancouver 1976-1996. CD. Goethe Institute, Munich 1996.

19. Helmi Järviluoma et al. eds., Acoustic Environments in Change (Tampere: University of Joensuu, Faculty of Humanities, Studies in Literature and Culture 14, 2009).

20 The musicologist Karl Traugott Goldbach attributes to the works of the composer Luc Ferrari the beginnings of an “ecological listening,” cf. ibid., “Acousmatic and Ecological Listening in Luc Ferrari’s Presque rien avec filles” Zeitschrift der Gesellschaft für Musiktheorie 3/1, 2006, 127-137.

21 For example, in his piece Presque Rien no. 1 Lever du jour au bord de la mer (1967). Recorded on the beach of Vela Luka, Croatia, the 21-minute piece consists of a condensation and deliberate arrangement of everyday sound episodes that Ferrari had recorded over the course of a single day.

22. Both were members of Schafers research team at Simon-Fraser-University, Burnaby (Canada).

23 Hildegard Westerkamp, Soundscape Composition: Linking Inner and Outer Worlds (Amsterdam 1999), https://www.hildegardwesterkamp.ca/writings/writingsby/?post_id=19&title=%E2%80% 8Bsoundscape-composition:-linking-inner-and-outer-worlds-

24 Ibid.

25 Hildegard Westerkamp, “Linking Soundscape Composition and Acoustic Ecology,” Organised Sound, An International Journal of Music and Technology, Volume 7, Number 1 (2002), 51–56.

26 Cf. Katherine Norman, “Real-World Music as Composed Listening,” Contemporary Music Review, 1996, Vol. 15, Part 1, 19.

27 See also Barry Truax: “Part of the composer’s intent may also be to enhance the listener’s awareness of environmental sound,” in ibid, Acoustic Communication (Westport/London: Ablex Publishing, 1984), 207.

28 Tim Ingold, “Against soundscape,” in E. Carlyle, ed., Autumn Leaves: Sound and the Environment. Artistic Practice (Paris: Double Entendre, 2007), 10-13. Stefan Helmreich refers to Ingold’s critique in “Listening against Soundscapes” Anthropology News, Boston, December 2010, 10.

29. R. Murray Schafer, Die Ordnung der Klänge. Eine Kulturgeschichte des Hörens, The Tuning of the World in its second German translation), updated, newly edited and translated by Sabine Breitsameter (Mainz-Berlin: Schott-International, 2010), 50.

30 Ibid.

31. This connection had been emphasized by Schafer, and was elaborated by Barry Truax in his dynamic and interdependent model of “acoustic communication,” cf. ibid, Acoustic Communication (Westport/London: Ablex Publishing, 1984), 12.

32 Formative listening cf. Sabine Breitsameter, “Radiokunst innerhalb und ausserhalb der Schule,” in Jutta Wermke, ed., Medien im Deutschunterricht 2007 (Jahrbuch).

Thematic focus: Listening Aesthetics Listening Education (Munich: kopaed, 2010), 6474. See also ibid., “Methoden des Zuhörens. Zur Aneignung audiomedialer Produktionen”, Paragrana. Internationale Zeitschrift für Historische Anthropologie, FU Berlin, Band 16, 2007, Heft 2, 223-236.

33. Cf. Sabine Breitsameter, “Plastische Arbeit mit Klang. Grundlinien künstlerischer Forschungsarbeit am 3D-Audio-Lab der Hochschule Darmstadt,” Positionen. Texte zur aktuellen Musik, Heft 110, Mühlenbeck bei Berlin (2017), 53-62.

Bibliography

Breitsameter, Sabine. “Plastische Arbeit mit Klang. Grundlinien künstlerischer Forschungsarbeit am 3D-Audio-Lab der Hochschule Darmstadt.” Positionen. Texte zur aktuellen Musik, Heft 110, Mühlenbeck bei Berlin (2017), 53-62.

Breitsameter, Sabine. “Radiokunst innerhalb und ausserhalb der Schule.” In Medien im Deutschunterricht 2007 (Jahrbuch). Hörästhetik - Hörerziehung, edited by Jutta Wermke, 64-74. Munich: koepaed 2010.

Breitsameter, Sabine. “Methoden des Zuhörens. Zur Aneignung audiomedialer Produktionen.” Paragrana. Internationale Zeitschrift für Historische Anthropologie, FU Berlin, Band 16, 2007, Heft 2, 223-236.

Fuller, Richard Buckminster. “The Music of the New Life.” Music Educators Journal, 52(6) (1966) 52-60, 62, 64, 66-68.

Goldbach Karl Traugott. “Acousmatic and Ecological Listening in Luc Ferrari’s Presque rien avec filles.” Zeitschrift der Gesellschaft für Musiktheorie 3/1, 2006, 127-137.

Granö, Johannes Gabriel. Pure Geography, A methodological study, enhanced by examples from Finland and Estonia (1929), edited by Olavi Grano and Anssi Paasi, translated by Malcolm Hicks. Baltimore: Johns Hopkins University Press, 1997.

Helmreich, Stefan. “Listening against Soundscapes.” Anthropology News, Boston, December 2010, 10.

Ingold, Tim. “Against soundscape.” In E. Carlyle (ed.), Autumn Leaves. Sound and the Environment. Artistic Practice, 10-13. Paris: Double Entendre, 2007.

Järviluoma, Helmi et al. eds. Acoustic Environments in Change. Tampere: University of Joensuu, Faculty of Humanities, Studies in Literature and Culture 14, 2009.

McLuhan, Herbert Marshall. The Gutenberg Galaxy, The Making of Typographic Man. Toronto: University of Toronto Press, 1962.

Norman Katherine. “Real-World Music as Composed Listening”. Contemporary Music Review, 1996, Vol. 15, Part 1, 1-27.

Russolo, Luigi, “Manifesto futurista, Milan, 11 March 1913.” Cited at: http://www.medienkunstnetz.de/quellentext/39/, (01.06.2023) in the translation by Justin Winkler and Albert Mayr.

Southworth, Michael. “The Sonic Environments of Cities.” In Environment and Behaviour (1969) 49-70.

Schafer, R. Murray. The New Soundscape. Scarborough: Berandol Music Limited/New York: Associated Music Publishers, 1969.

Schafer, R. Murray. The Tuning of the World. Toronto: McClelland and Stewart Limited, 1977.

Schafer, R. Murray. Klang und Krach. Eine Kulturgeschichte des Hörens (The Tuning of the World in its first German translation), edited by Heiner Boehnke and translated by Kurt Simon und Eberhard Rathgeb. Frankfurt/Main: Athenäum, 1991.

Schafer, R. Murray. Die Ordnung der Klänge. Eine Kulturgeschichte des Hörens (The Tuning of the World in its second German translation), updated, newly edited and translated by Sabine Breitsameter. Mainz-Berlin: SchottInternational, 2010.

Soundscape Vancouver 1976-1996. CD. Goethe Institute, Munich, 1996.

Truax, Barry. Acoustic Communication. Westport/London: Ablex Publishing, 1984.

Westerkamp, Hildegard. Soundscape Composition: Linking Inner and Outer Worlds (Amsterdam 1999).

https://www.hildegardwesterkamp.ca/writings/writingsby/?post_id=19&title= %E2%80%8Bsoundscape-composition:-linking-inner-and-outer-worlds(02.06.2023).

Westerkamp, Hildegard. “Linking Soundscape Composition and Acoustic Ecology”. Organised Sound, An International Journal of Music and Technology, Volume 7 “Soundscape Composition”, Number 1, Cambridge (2002), 51-56.

Winkler, Justin. “Landschaft hören. Geographie und Umweltwahrnehmung im Forschungsfeld Klanglandschaft.” Regio Basiliensis. Basler Zeitschrift für Geographie 33, 1992, H. 3, 196-206.

Winkler, Justin, Klanglandschaften: Untersuchungen zur Konstitution der klanglichen Umwelt in der Wahrnehmungskultur ländlicher Orte in der Schweiz. Habilitation Treatise. Universität Basel/Schweiz, 31 January, 1995. World Soundscape Project Recording Library. http://www.sfu.ca/sonic-studiowebdav/srs/index2.html

Inspired by Nature

How Acoustic Ecology Informs the Work of Sound Scenographers

Abstract

Consciously designed soundscapes have the power to impart new meaning to visual content and fundamentally redefine the way we feel within spaces. Sound scenography is the art of staging spaces and settings using sound. As a discipline that developed out of conventional scenography, sound scenography primarily plays a role in exhibition scenarios such as museums or media installations, but is also used in businesses and public spaces As a collective specialized in sound scenography, Idee und Klang Audio Design creates auditive environments for these various settings. Sometimes we create our own artistic pieces, but we mostly create applied work for clients. Nonetheless, in our 18 years of existence, we have found that, more often than not, there is room to align our own views and beliefs with those of our projects to some degree. In this context, acoustic ecology, a discipline that studies the relationship between human beings and their environment, mediated through sound, is of great interest and importance to us. For one thing, it has profoundly impacted our approach to practicing sound scenography: some of the main principles of our work (namely how we deal with orientation, attention, association and spatial depth) are directly inspired by and derived from nature. For another thing, acoustic ecology has also found its way into various projects as a core topic. We live in a time in which no conversation is more pressing than the one about how to save what’s left of our planet’s rich natural environments. Here, acoustic ecology provides some brutally clear indicators, for example regarding the correlation between human behavior and species extinction. It shines a new light on the way our lifestyle negatively impacts both non-human species and our environment

Introduction to Sound Scenography

Sound is incredibly underestimated and underused as a design medium. This is especially true in the field of exhibition design, where the auditory dimension has the potential to be a key element. This is where the relatively new discipline of sound scenography comes into play. Also known as acoustic scenography or audio scenography, this discipline is the science or art of applying sound in the design of rooms and environments. Sound scenography combines knowledge and insights from the fields of architecture, acoustics, communication, sound design and interaction design to convey artistic, historical, scientific, or commercial content while creating atmospheric moods. As a discipline, which is mostly seen as an extension to the field of “conventional” scenography, it is primarily applied in the contexts of exhibition design for museums, media installations and art However, it is also used for retail shops, theme parks, planetariums, spas, receptions, ticket halls, and open-plan office spaces.

The deliberate application of sound scenography can introduce an emotional dimension to rooms, exhibits and even individual interactions. It can create atmospheres and moods whose tonalities range from the realistic to the unreal or even the futuristic. It can evoke memories and associations. Soundscapes and sound accents also have the power to reinforce visual content or lend it entirely new meaning. Of course, content can also be conveyed entirely via sound without any connection to visual media. Furthermore, the purposeful design of the auditory dimension in spaces can also eliminate unwanted sounds or noises (via phenomena called absorption, sound masking and noise cancelling), or encourage visitors to act in a way that affects the aural sphere. For example, sound scenographers can create an auratic space in which visitors are encouraged to move around very mindfully and speak quietly with one another. Sound scenography uses the many strengths of sound and combines various auditory components to create a comprehensive transmedia experience.

Can Architecture be Heard?

When we examine the relationship between sound and the environment it is being heard in, we find architecture at its very core. Sound might not be the first thing that comes to mind in the context of architecture, even though architecture provides the fundamental prerequisites for sound. Our perception of space and its dimensions, its shape, and its material properties is strongly informed by our sense of hearing. Only very few people, such as Swiss architect Peter Zumthor, or Danish architect Steen Eiler Rasmussen, have realized this. They have studied the field and implemented projects following explicitly acoustic principles. Rasmussen asks:

Can architecture be heard? Most people would probably say that as architecture does not produce sound, it cannot be heard. But neither does it radiate light, and yet it can be seen. We see the light it reflects and thereby gain an impression of form and material. In the same way, we hear the sounds it reflects and they, too, give us an impression of form and material. Differently shaped rooms and different materials reverberate differently.

In architecture, acoustics are usually confined to the field of building acoustics, such as exterior sound insulation or impact sound insulation between two apartments. The aim is, for example, to prevent the sound of a razor being knocked out on the edge of the washbasin from disturbing people in the apartment below. Building acoustics are basically all about adhering to standards.

Room acoustics, on the other hand, influence the intensity and nature of sound reflection within a room and, as a consequence, its acoustic quality. Room acoustics thus operate at both levels: design as well as functionality. In practice, however, the focus is usually on the latter, and involves handling challenges like “how can we improve the speech intelligibility in our training rooms?” Only in special cases, such as the construction of a concert hall, do acoustics play a role in aesthetic design. Oftentimes, acousticians are only brought in when acoustic problems have been identified in an existing room. These types of belated acoustic renovations are usually expensive and often visually unsatisfactory. They could easily be avoided with smart design and the deliberate selection of materials at the planning stages.

Relatively little research or specialized knowledge is available about the influence of acoustic quality on a room’s atmosphere. This is reflected in the limited vocabulary that is used to describe acoustic qualities and is predominantly couched in visual terms. Usually, one only speaks of “good” or “bad” acoustics or of rooms that sound anywhere from “reverberant” to “dry.” At most, reverberations are described as “bright” or “dark ” Generally speaking, other terms to describe acoustics are not available. However, taking a building’s acoustic properties into account right from the beginning would unlock enormous potential, for sound is one of the fundamental factors defining a room’s atmosphere like a good lighting concept, which considers the role of daylight and other aspects from the very beginning. Just as lighting reveals and influences the characteristics of an architectural space to our eyes, sound reveals and influences its characteristics to our ears.

How We Work with Sound

But of course sound scenography is not limited to room acoustics. In this section we’ll talk about the other steps that go into the creation of interesting sounding, engaging environments. During the course of our many years of international experience in sound scenography, sound design, audio consulting and exhibition design, we have gradually devised and developed our own process: Holistic Sound Scenography.

Holistic Sound Scenography is an integral approach combining sound scenography, story-telling formats, and audio technologies in spatial contexts, and conceptualizing, planning, and implementing them in an integrated manner. The ambitious design that we developed for the award-winning exhibition Sounds of Silence at the Museum of Communication in Berne, for example, would have been quite impossible without a holistic approach. The same goes for the highly complex technical planning we did for the National Museum of Qatar, which opened in 2019

Since we are aware of the remarkable power of sound in general and in particular of sound in space, our aim is to make the most effective use of this potential in our projects. This means planning and designing an exhibition from the perspective of sound scenography the experience of sound in space from the very first idea to the final implementation. We start with the acoustic features of specific rooms and spaces and a basic sound concept and then progress to technical planning, sound composition and production (more on that subject below), and finally to the implementation in the designated space.

Sound scenography operates as an interface between exhibition design, media planning, architecture, and the museum or venue itself. The Holistic Sound Scenography approach guarantees that the content of an exhibition and the necessary technology are planned simultaneously and combined in optimal fashion This prevents audio technology decisions from being made before the role of sound in an exhibition has been determined. In this way, technology can be used purposefully and tailored precisely to the desired effects and to the role that sound is supposed to have in the space. The specific needs and challenges of respective spaces are hereby identified and addressed at the very outset of the project.

In our book, which we released in 2021, we call sound scenography “the art of designing sound for spaces” The main part of our work is to create the sounds that resonate in these spaces. The most obvious difference between composing for a space instead of, say, a music album, is that we usually use a great number of discrete audio channels compared with just two in stereo the format most music is created and consumed in today. The stereo format assumes that the listener remains stationary, either in front of a stereo hi-fi system or wearing headphones. All sound is therefore created and mixed for this singular auditory perspective. Composing for a space means composing for a great number of different auditory perspectives as listeners are free to walk around as they please. Having many discrete audio channels at our disposal means that we can make sounds come from all sides, from above and below and even, if the architecture allows such flexible speaker placement, from inside objects. This can be compared to the rich sound of an orchestra, with each loudspeaker representing one musician or instrument. It’s easy to see why these types of setups are sometimes referred to as “loudspeaker orchestras.” In contrast with the experience of a conventional orchestra in a concert hall, with the loudspeaker orchestra, listeners can even walk around in and among the “musicians ”

When we compose these kinds of spatial soundscapes, we often operate in the grey zone between music and sound design, tonal material and noise, a musical grid and temporal freedom. In our experience, this is a sweet spot, in which a lot of interesting things happen. We usually compose for tailor-made, multichannel formats that were specifically designed for the room they will be situated in. This is our attempt to seamlessly integrate sound into architecture, or rather, to make sound a part of architecture

Four Principles Inspired by Nature

Our approach in sound scenography is strongly influenced by acoustic ecology: all the core principles of composition and sound design for our spatial soundscapes are borrowed from nature and our interaction with it:

Orientation

In outdoor spaces, humans and other animals constantly use the sounds that surround them for orientation purposes. Acoustic landmarks such as waterfalls help them figure out their location within their environment. This is especially important in low visibility situations or at night Acoustic landmarks unconsciously support us in our daily lives by giving us subliminal information that we wouldn’t be able to perceive visually (or with any other sense). For example, we intuitively know where to look for the beach because we can hear the waves crashing from afar. Likewise, we can make a good guess about the direction we should be headed in to avoid the heart of a thunderstorm. Furthermore, the way these landmark sounds are reflected and filtered reveals a great deal of information about the environment, such as its dimensions, its shape, or the characteristics of the materials which are present in it. In man-made indoor environments such as exhibitions, this kind of orientation can be similarly induced if we consciously integrate acoustic landmarks into our sound scenography. Composing our soundscapes in formats that feature several discrete audio channels and for sound systems with a great number of speakers (as we discussed above), allows us to place different sounds or sound “events” in different, distinct locations within a space.

Attention

Unlike visual information, sounds affect humans and other animals regardless of whether we are actively listening and paying attention or not. Another important difference between our sense of sight and our sense of hearing is that the sense of hearing is a global, omnidirectional sense and can pick up a lot of information at the same time and from all around us at once. The sense of sight, on the other hand, is directional and can only selectively focus on a single point or area in space at any given time the rest of the visual field remains blurred, and anything outside of it is not perceived visually at all. This is also one of the reasons why there is a lot of sonification (the use of audio to convey information or perceptualize data) going on in a helicopter cockpit: our ears are capable of observing many more data points simultaneously than our eyes. In nature, the sense of hearing is crucial to spot potential dangers early on and, conversely, to help predators detect potential prey. In man-made environments such as exhibitions, we harness this by using sound to purposefully direct visitors’ attention. Especially in contemporary exhibition design, which can sometimes be overcrowded with media or feature 360-degree content, sound can play a crucial role in guiding visitors’ focus

Association

In nature, many sounds contain crucial information. The specific characteristics of wind sounds can reveal information about an upcoming bad weather front, for example. Such cues can be learned by humans and other animals and help them navigate their lives more intuitively. In sound scenography, we use this feature of sound to condense complex emotions into specific, short sound cues. Sometimes even a fraction of a sound is enough to trigger associations or memories. One vivid example of this is the film soundtrack of Jaws (Steven Spielberg 1975). If you were to play the first double bass note of this soundtrack to a random group of people from various backgrounds and age groups, there’s a high possibility that at least one person in the room would immediately recognize the sound and draw a connection to the film. In this way, the emotional world of an entire film is reactivated by a single tone. Of course, the associations and memories are specific to each person and heavily influenced by culture. But there are sounds that demonstrably trigger similar things in all humans, such as the sound of a beating heart (the first sound we hear before we are even born), the twittering of birds, or the sound of a jackhammer.

Spatial Depth

A single sound, which originates in a singular location, a so-called point source, gives us limited spatial information. However, when two or more point sources are combined, with a physical distance or offset between them, a three-dimensional listening space opens up. Let us imagine an idyllic scene in a forest glade. We hear different birds singing overhead, a bubbling brook behind us, the quiet rustling of the wind in the leaves all around us, and chirping crickets next to us. Each individual sound comes from a specific direction. Some are static (the bubbling brook), some dynamic (the wind), and others come from several different directions and distances (the crickets). It is through the presence of these many point sources of biophonic and geophonic sounds, that nature spoils us with soundscapes of tremendous spatial depth. This is something we aim to emulate in our work by composing for so-called loudspeaker orchestras, as we explained above. Walking around in a spatial soundscape played back on this type of system creates a constant change in acoustic perspective just like walking around in a natural soundscape in nature.

Introduction to Acoustic Ecology

Acoustic ecology is a discipline that studies the relationship between human beings and their environment, mediated through sound. One of the main objectives of this discipline is to sensitize people to important issues of our time, such as climate change, pollution, and species extinction, through sound art and science. It also aims to foster a deeper understanding of the interaction between humans and nature.

The Covid-19 pandemic, for example, abruptly and temporarily altered our daily sound sphere: no more aircraft humming in the sky, noticeably quieter cities and an acuter perception of animal sounds, were among the striking experiences which some people around the globe now fondly remember. And it is precisely this shift in attention and the heightened awareness of the sound sphere and the sounds of nature that acoustic ecology emphasizes. This new mindfulness of acoustic impressions can be seen as an opportunity and could be the starting point for a more considerate approach to the management of our own resources and those of our planet.

This brings us to a discipline that connects most people in the field of acoustic ecology: field recording or the art of capturing sounds in natural environments to preserve complex acoustic landscapes. In some ways, field recording marked the starting point of the acoustic ecology movement when R. Murray Schafer and his team at Simon Fraser University in Vancouver (CAN) started the World Soundscape Project in the 1960s. The focus of said project was to examine entire landscapes regarding their sonic character in order to draw conclusions about their condition and possible changes they were going through Sound environments are diverse and often very complex. To authentically and faithfully capture and preserve the sound experience of significant places and their origins, one must select not only the right location and time for recording, but also the right recording techniques. Specific 3D recording processes allow us to capture environments in three dimensions by directing several microphones in different directions.

For field recordings, various factors need to be considered in order to achieve the intended result. Does the soundscape sound the same at all times of day? Do different physical vantage points have a different sound, and are different sound perspectives therefore required? What is taking place beyond the range of the human ear (for example, at infrasonic and ultrasonic frequencies or in electromagnetic waves), and what can be heard in hidden places (for example, under water or via impact sound, the dissemination of sound by solid objects)? These and other considerations make it possible to transform the soundscape of a natural environment with all its many nuances and hence its full cultural meaning into a recording, whether its intended use is a soundscape composition, an acoustic exhibit, or a scientific research project For spatial audio experts like us, the focus in field recording is just as much on how to faithfully reproduce the recorded sounds as on how to capture them in the first place. We work with various formats. These include our own system called AROS (acousmatic room orchestration system), which we have used in many projects such as the !Khwa ttu San Heritage Center near Cape Town, South Africa, which we’ll elaborate on in the next section

Sound is also becoming increasingly important for the natural sciences for example, bio-acoustic recordings, which are currently one of the methods used to study biodiversity or the sonification of data, as an alternative to visualizations. There is a growing overlap between art and science in this field. Artist Marcus Maeder’s Sounding Soil, for example, is an interdisciplinary research and art

project in which the sounds of soil ecosystems play the main role. The Institute for Computer Music and Sound Technology ICST (part of the Zurich University of the Arts ZHdK) developed special soil sensors and adapted existing recording devices. This equipment makes it possible to listen to the soil and hear the crawling, scratching, communication sounds and eating noises of the animals underground. Sounding Soil’s acoustic soil recordings and the soil samples obtained in parallel in 2017 and 2018 were evaluated and statistically analyzed. The first results show that differently used soils sound different, too. The analyses also indicate that the greater the diversity of living organisms in a soil, the more complex its sound.

Acoustic Ecology as a Core Topic in our Projects

To come full circle, acoustic ecology has also found its way into our projects as a core topic. Sometimes we create our own artistic pieces, but in most cases, we are hired by a client. This means that our influence on the philosophical and ideological message of a given project is limited. However, throughout Idee und Klang Audio Design’s 18 years of existence, we have found that more often than not, there is room to align our own beliefs with those of our projects to some degree. The next paragraphs describe a few of our experiences with commissioned work that focuses on acoustic ecology.

The !Khwa ttu San Heritage Center’s exhibition near Cape Town, South Africa, documents the daily lives of the San, the hunter-gatherer cultures in the African South-West. Their gentle ways of co-existing with their biosphere taught us a lot about acoustic ecology. Sound plays a crucial role in their everyday lives as the sounds of the weather and the fauna provide valuable information. We joined the San in numerous activities in the Kalahari Desert (Namibia), using our 8-channel 3D audio recording rig to record along the way. We then drew on this rich material to create a 21-channel soundscape composition for the exhibition, which was divided into the three areas of nature, village and culture. The focus of this project was to preserve a version of the current acoustic reality and make it accessible to both today’s population and future generations. Very few San still live as huntergatherers today and their ancestors were among the pre-colonial communities that could once be found all over southern Africa. This kind of echoes efforts like the UNESCO Intangible Cultural Heritage Program but for sound. In another example, we are curating and overseeing the entire audio creation process for the new Klangweg Toggenburg. This extensive public hiking trail in an alpine environment in Switzerland’s Toggenburg region, is currently being completely redesigned. Hikers on the trail will discover many sound art pieces created by several different artists This new edition of the Klangweg Toggenburg, which will

open in June 2024, aims to raise awareness for ecology, climate change, sustainability and mindfulness. In a piece by landscape sound artist Ludwig Berger, for example, the trail will branch into five different paths, each featuring a different kind of gravel rock type and grain size. As they walk on each different gravel path, the hikers’ footsteps will create a sound that resembles the sound spectrum of five different crickets respectively. To mimic the crickets even more accurately, a visual score will help hikers imitate the rhythm of the insects’ chirps with their feet. In another piece, Italian sound artist Marco Barotti raises awareness for the electromagnetic radiations we are exposed to every day. He mounts three solar driven electrical woodpeckers onto three different trees along the trail Barotti has been working towards creating a robotic animal that can detect electromagnetic radiation for quite some time. How would it react to people sending and receiving electromagnetic radiations through their smartphones? Eventually, he came up with the woodpeckers, which are a series of data-driven autonomous devices or custom-made robotic birds that detect electromagnetic radiations (EMR). The woodpeckers are configured to read various ranges of the EMR spectrum and reinterpret these waves with a motion pattern similar to that of a woodpecker. They thus transform the invisible radiation used for mobile communication and wireless technology into audible and visible acoustic drum patterns.

For the temporary exhibition “Earth at its Limits” at the Basel Natural History Museum (Switzerland), we created three spatial soundscapes, which simulated the idea of untouched nature. Each soundscape focused on a different environment the forest, the ocean, and the Alps. The exhibition grappled with the notion that this kind of pristine environment, void of human interference, is quickly becoming a relic of the past, one that could soon only be experienced through an artificially created soundscape. As a counterpoint, we also created a fourth soundscape which featured the sounds of a wide array of natural disasters, whose prevalence and severity is directly impacted by climate change. The same museum is currently planning a comprehensive new building. We are working on multiple concepts for the redesign of its permanent exhibition. One of these features a life-size replica of a whale which will hang from the ceiling of the huge stairwell that connects the various exhibitions. The plan is to connect the replica to the tracking device of a live animal one that is actually roaming the ocean. The live data will reveal the current depth the real whale is swimming at and enable us to derive its heart rate The sound of its heartbeat will then be played back via a massive subwoofer built into the body of the model whale.

Authors Biographies

Ramon De Marco studied audio design at Basel University. The year after he left, he set up the Sound Design Studio of Virus (Swiss National Radio) and went on to become Project Manager in the Music Department of Fabrica (Treviso, I). In 2005, he founded Idee und Klang, where he completed several international scenographic projects including for the BMW Museum in Munich (GER), the Imperial War Museum in London (UK) and the National Museum of Qatar (QAT). As a sound artist, he focuses on acoustic ecology and has carried out projects for various art exhibitions, such as the Outer Ear Festival in Chicago, the Centre Pompidou in Paris and Experimenta Design in Lisbon. Ramon regularly lectures at international conferences and he has also been teaching Sound Scenography at the FHNW in Basel and the FHGR in Chur (CH) since 2010. In 2021, he co-authored a book entitled Sound Scenography - The Art of Designing Sound for Spaces. Ramon De Marco – Owner, Sound Scenographer – Idee und Klang Audio Design –www.ideeundklang.com/audio ramon@ideeundklang.com

Jascha Ivan Dormann is a sound artist and spatial audio specialist. He completed his master's degree in audio design at Elektronisches Studio Basel. In addition to his work as a music producer and sound designer for radio, feature films and documentaries, he explores the interaction of sound and space as a sound designer for performances, exhibitions and installations. For Idee und Klang Audio Design, he managed the sound scenography for Sounds of Silence, an exhibition on the topic of silence at the Museum of Communication Bern (CH), which was based entirely on interactive spatial sound. Most recently, he was the leading sound scenographer for the newly opened Holocaust Galleries at the Imperial War Museum London (UK), which were awarded various international prizes. In 2023, Jascha released the music album A Case of Paranoia under his moniker Ivan Eyes (in collaboration with vocalist Elfrid The Third) and played various live shows. Jascha Ivan Dormann – Sound Scenographer – Idee und Klang Audio Design – www.jaschadormann.com jascha@ideeundklang.com

Note

Sound scenography combines approaches from the fields of architecture, acoustics, sound design and interaction design, offering new concepts and possibilities. Strongly influenced by acoustic ecology, we have incorporated core principles derived from nature into our work as sound scenographers. However, our interest in and commitment to acoustic ecology extends beyond the functional. We believe that now is the time to open our ears to nature so we can learn from it, and learn how to live as a part of it rather than as its predators and exploiters. We are trying to shape our work in a way that sensitizes listeners to the intricate relationship we share with our environment on a sonic level. Given the impact sound can have on us humans, we would even venture to say that this, albeit gentle, activist notion lies at the core of some of our work

Bibliography

De Marco, Ramon and Idee und Klang Audio Design, eds. Sound Scenography – The Art of Designing Sound for Spaces Stuttgart: Avedition, 2021. Dormann, Jascha Ivan. “Acoustic Ecology in Commissioned Work ” Conference World Forum for Acoustic Ecology, 2023.

Acoustic Architecture Progressing Beyond Sound Branding

Why Imprint When We Can Craft Together?

Abstract

Acoustic architecture a novel concept and term is reshaping the landscape of brand communication, shifting the paradigm from a monologue to a dialogue, and paving the way for more immersive and personalized brand experiences. This study explores the transformative implications of this shift, highlighting the increasing significance of sound as a vital component of corporate identity and branding strategies. In the evolving branding landscape, the deployment of acoustic architecture demands a focus on interaction, audience engagement, and resonance with the brand’s identity. The advent of advanced technologies, especially artificial intelligence, plays a substantial role in this transformation, promising the potential to streamline and amplify sound branding initiatives. However, this progression comes with its own set of challenges. The paper discusses the risk of potential misuse of AI, which could lead to generic and indistinct creative outcomes, threatening the uniqueness of brand identities. It also emphasizes the potential impacts on the creative industry, especially the threat to artists reliant on generating basic creative work. Drawing parallels to historical instances like the Reformation-era church’s use of participatory music, the study underlines the importance of active user engagement in brand communication. It suggests that the survival and growth of brands in this brave new world of acoustic architecture hinges on the readiness of individuals to engage and interact. Despite potential ethical concerns, the future of acoustic architecture in branding presents an exciting terrain of possibilities. It offers the prospect of crafting unique soundscapes that transcend physical and virtual realms, revolutionizing the way we perceive and interact with brands, be they corporate entities, public institutions, or artists. As we stand on the cusp of this seismic shift, the challenge lies in harnessing data, deploying technology effectively, and nurturing a participatory ethos to cultivate the acoustic landscapes of the future.

Digital Disruption on Music and Audiences

Require a New Concept of Branded Communication: Acoustic Architecture

The universal adoption of digital technology has instigated profound shifts in the patterns of media consumption, catalysing a notable transformation in the musical listening habits of consumers, especially among younger demographics. This alteration has been momentous, commencing with the introduction of digital audio workstations and sampling techniques altering the process of music production as well as the advent of devices such as the iPod and later platforms like music streaming services which acted as significant milestones altering the ways audiences interact with and appreciate music.

Historically, the appreciation of music was largely dictated by the cultural significance of artists and genres. These served as identity markers and signs of social alignment, acting as cultural and social identifiers that listeners associated with, thereby framing their musical inclinations. However, the advent of the iPod, followed by successive technology, introduced a more individualized and immersive way of consuming music, steering away from these traditional markers. The cultural importance now lies not so much with the artist or genre but with the music recording itself in the form of disconnected ‘tracks’ and playlists and the experiential space it fabricates. “[Younger listeners] are no longer so invested in a single genre, one that promises to mold their being or save the world” (Ross 2010, 19). The focus has shifted to the feelings evoked, memories recalled, connections made, and immersive experiences enabled by a track or a playlist.

Coinciding with, or perhaps instigating these shifts in music consumption, the way music is produced and its artistic goals have undergone substantial changes as well. As Simon Reynolds (2011) noted, pop music became increasingly infused with past artistic styles from the 2000s onwards. Revivals of styles from the 1960s, 70s, 80s, and 90s have appeared with increasing frequency, while truly innovative musical styles have been on the decline begging the question, what is the sound of the early 2020s? For audiences who matured during the respective artistic decades, many current pop music publications can well be a welcome reminder of their own formative years, rendering pop appealing to older demographics. However, younger audiences lack the knowledge of the cultural, social, or political significance of these artists and styles, because they simply have not experienced it. They listen to the music as devoid of these signifiers, instead infusing this blank canvas with their own experiences.

Comprehending this transformation is essential due to the significant implications it carries for industries that employ music as a part of their communication strategies, such as branding. Branding, fundamentally, is about creating and managing experiences through the correct auditory cues. However, what constitutes ‘correct’ when the rules of music consumption are undergoing this paradigm shift? As consumers increasingly seek more immersive and individualized experiences, somewhat liberated from socio-cultural signifiers, brands must evolve beyond the simple employment of musical accompaniment targeted at stereotypical age and social classes. They must create an experiential space engaging multiple media and messages, and this is where the analogy of acoustic architecture comes into the equation.

In the context of acoustic branding, acoustic architecture refers to the design and creation of auditory worlds that augment the brand’s identity and message. Essentially, it views the brand as an experiential space, where each sonic element contributes to crafting a holistic, multi-sensory brand environment. This brand environment can manifest in various forms a digital media campaign, an interactive virtual reality experience, or a physical store or event. Regardless, it must transcend the boundaries of a single platform medium or place linking with another to construct a comprehensive architecture in the sense of this article. Because especially “for younger generations, the distinction between physical and virtual worlds is fading: both are just real life” (Wundermann Thompson 2023, 11) It involves meticulous curation of sounds not only music that resonate with the brand’s identity and values, constructing an auditory environment that enhances and augments the overall brand experience. This could encompass everything from the background music playing in a brand’s physical or virtual spaces, the sound effects used in its digital applications, or the voice and tone of its customer service communications. Each of these elements is strategically selected, or trained, and designed to evoke specific associations, emotions, and responses aligning with the brand’s identity and objectives.

However, it’s vital to acknowledge that the effectiveness of such acoustic architecture does not and cannot solely depend on the sounds chosen as well as the strategy of their use. It also hinges on the interaction of these sounds with the other components of the brand environment: The transition between sounds, the interaction of sounds with visual and other sensory elements, the timing and placement of sounds, and especially their general visual and local context all contribute to the creation of an auditory landscape that enhances, rather than detracts from, the overall brand experience.

As brands endeavour to create more engaging and memorable experiences, the concept of acoustic architecture provides an approach to deepen the connection with consumers. It allows brands to truly communicate with their target groups through mediated sounds. This approach can lead to enhanced brand loyalty, as consumers are more likely to remember and associate positive experiences with the brand. However, it necessitates a shift in mindset from perceiving sound as an ancillary feature to acknowledging it as an integral part of the brand experience space. As brands continue to navigate the increasingly competitive and everevolving digital landscape, the role of acoustic architecture in branding will undoubtedly become more crucial.

In the subsequent sections, we delve into the primary historical advancements in the utilization of music within advertising, aiming to highlight the significant paradigm shift in this domain. This exploration is followed by a detailed exposition of acoustic architecture its conceptual underpinnings and varied operational fields. The discourse culminates with a critical evaluation of the social and ethical ramifications in this area.

The Structural Shift: Music in Advertisement and Branding Evolved from Messaging to Crafting Experiences

Music has played a prominent role in advertising and branding since the post-war boom of the 1950s, evolving significantly to reflect and respond to cultural trends, technological advances, and changes in consumer behaviour. In the early postwar years, jingles were all the rage. These catchy tunes, created specifically for commercials and often featuring the brand’s name and slogan, were designed to embed the brand in the consumer’s memory. As the advertising landscape grew more competitive and consumer expectations evolved, brands began to exploit the potential of popular music as a powerful tool for emotional engagement and identification. A watershed moment was Coca-Cola’s 1971 “Hilltop” commercial, which featured an original song, “I’d Like to Teach the World to Sing.” This song became a global sensation, heralding a new era of commercial music and paving the way for the widespread use of popular music in advertising. This trend further evolved in the 1990s, with brands frequently leveraging popular music to connect on a deeper level with target audiences. Another example of this shift is Aral’s use of Fats Domino’s “I’m Walkin ’” This popular song reinforced the brand message and amplified the campaign’s overall impact. The song did not only link to the music experience of the then 40 to 50 year old target groups but shaped the ad’s mood, and narrative, demonstrating how music can transform an advertisement into a holistic audio-visual experience.

In 1995, Coca-Cola introduced a novel approach to its holiday campaigns, creating a distinct holiday atmosphere associated with the brand through the clever use of music and sound. The tinkling sound of ice cubes dropping into a glass of Coca-Cola, the “ pop ” of the bottle opening, the melodic tune “Holidays Are Coming” all these acoustic elements created a unique, recognizable, and emotive brand experience. This trend of utilizing popular music as a core component of branding continued into the 21st century. A notable instance is the 2003 McDonald’s campaign featuring the now-iconic “I’m Lovin’ It” jingle, performed by Justin Timberlake. The jingle didn’t just serve as a catchy backdrop, but it became a central element of McDonald’s global brand identity in the short form of its well-known sound logo. These historical campaigns showcase how music was used as a signifier of cultural, social, and individual frameworks.

Alongside these examples, another modus operandi evolved: the musical “underscore,” serving as an acoustic bed for the advertisement’s voiceover extolling the benefits of a product or brand. Given the cost-effective and speedy production of these formats, their prevalence is substantial. Music here solely serves to appeal to certain target groups according to the old pattern of age group and social class, linked to supposedly apt musical styles, which are then licensed cheaply from music catalogs or, in rarer cases, commissioned from composers. However, as the socio-cultural meaning of music is dissolving, as previously mentioned, creatives more often face a kind of void when making music-related decisions. If any piece of music within a given genre can suitably match the brand, the choice can feel arbitrary, lacking in meaningful distinction. Still, albeit often arbitrary, the use of music beds for the voice over appears to be a standard design rule for them. For example, younger creatives are often shocked by the idea of possibly not using music for a commercial. Not because the sociocultural signifier would be missing, but rather because the film’s diegetic voice, or the voiceover would be completely exposed. This implicates that they are no longer cognizant of this classic function of music in media and rather assume their own listening to be an experiential space.

These musical approaches were predominantly designed for linear media, such as television and radio, with the internet merely extending the reach of these traditional formats. In these campaigns, the target audience had little to no input in the design of their media world; they were primarily passive consumers of predesigned selections. However, as we advance into an era of digital interactivity and consumer empowerment, these traditional methods of integrating music into branding are being challenged: In the realm of social media, most users browse their timelines in silence, tuning into their own selection of music or content curated by their preferred streaming platforms. They craft TikTok videos or Instagram Reels, underscored by their favorite tracks. Their consumption of music has evolved from listening to complete albums to favoring playlists or individual tracks often truncated to no more than 30 seconds for applications such as TikTok or Instagram Reels thus fashioning their own experiential spaces.

Newer approaches, such as Vodafone’s implementation of music in their campaigns predicated on the selection of artists and music in alignment with specific sound branding guidelines herald the initial steps towards conceptualizing brand music as a space of experience. Here, lyrics and musical aspects like melody, rhythm or arrangement share only little common ground. Instead, alongside rather permissive brand fit interpretation, mostly novelty of

both production and artist are predominant criteria, corroborating the hypothesis of music as space succeeding music as socio-cultural signifier. While the concept of sonic brands as experiential spaces perceiving sound as an integral component of the brand’s identity and experience while engaging consumers in the creation and shaping of this multisensory environment is yet to be fully realized, it undoubtedly holds promise for future brand campaigns.

Harmonizing Spaces: Conception and Extent of Acoustic Architecture

Within the domain of branding, “acoustic architecture” signifies a realm exceeding the mere utilisation of background melodies or lone sonic logos, thus encapsulating a brand’s distinct identity within an all-embracing aural environment. This concept introduces an immersive arena of engagement, enabling the establishment of profound relationships. Crucially, this model recognizes the listeners as instrumental contributors to the brand discourse, interpreting them as co-architects of the sonic realms of these brands, engendered by both the auditory stimuli they encounter and contribute through a variety of media and interaction points. This contrasts the classical model of brand communication, which envisages a uni-directional flow of information from the corporation to its target audience, failing to encapsulate communication in its authentic sense.

The Individually Constructed Voice in Architecture

The intricate symbiosis between the brand and its audience emanates from the concept that architectural spaces ‘speak,’ as postulated by Barry Blesser and Linda-Ruth Salter in their seminal work (Blesser and Salter 2007). According to this proposition, every architectural space possesses an idiosyncratic acoustic signature or ‘voice.’ This ‘voice’ influences our perception and emotional response towards a space, nurturing a sentiment of location, context, and belonging. Within the realm of branding, this acoustic ‘voice’ transforms into the sonic identity or ‘voice’ of a brand. This sonic identity permeates all auditory interaction points, crafting a unique soundscape that echoes the identity of the brand. It is imperative to note, both within the realms of architecture and branding, this ‘voice’ is constructively established within the perceiver’s cognition and may differ substantially from the perceptions of other individuals.

This is particularly applicable when sociocultural signifiers cannot be decoded due to lack of knowledge especially regarding valence attribution, that is, how a given environment is liked or indeed disliked. Respectively, architectural structures, and by extension their sounds, may instill a sense of awe in some while causing intimidation in others. This spectrum of emotional responses is notably evident, for example, in the case of Brutalist architecture. Similarly, beautiful buildings may be as well interpreted as pretentious or featureless by different individuals. Thus, when interacting with a given acoustic architecture, the listener must be perceived as an active participant, or rather a constructor of the narrative, experiencing the brand through its sound.

The Importance of Negative Acoustic Space

The constructivist perspective reveals another fundamental essence of this concept: to curate a space conducive to individual engagement, offering freedom to partake in a narrative, it ought not to be saturated with meaning already. Therefore, the architectural principle of ‘positive’ and ‘negative’ space is equally applicable to acoustic architecture. Positive and negative spaces are pivotal elements of architectural design referring to the occupied and unoccupied regions within a structure or an environment. Positive space is characterised by areas filled with physical structures, objects, or elements, such as walls, furniture, or artwork; the tangible components of a building that lend it form, function, and meaning. Conversely, negative space refers to the empty or open areas around or between these objects. This unoccupied space surrounds, highlights, and contrasts with the positive space, crafting balance and rhythm within a design. A spacious lobby, a courtyard, or the gap between buildings serve as examples.

Both positive and negative spaces significantly influence the overall aesthetic and functionality of a design. A harmonious balance between the two enhances the spatial experience, fostering fluid movement, directing attention, and evoking emotions. A miscalculated equilibrium could result in designs feeling either overcrowded or barren. Translated into the realm of sound, positive space is filled with sounds laden with messages that relay the brand’s narrative, such as music, sonic logos, or brand anthems. Conversely, negative space encompasses the silences, pauses, and ambient noises between these message-loaded sounds. In the minds of the listener, both the positive and negative spaces acquire meaning and significance, analogous to how a viewer perceives an architectural space. Both in architecture and acoustic architecture, it is only the negative space that allows individuals to interact with and inhabit the construct.

On a side note, this conceptual understanding also presents an alternate interpretation of the ongoing shifts in music styles, or its application in advertising: while music has been traditionally utilised as positive space, as a carrier of explicit meaning and socio-cultural code, it has progressively shifted towards negative space in contemporary usage, where music rather forms the emotive backdrop of an experiential space akin to how contemporary music listeners utilise sonic environments for their activities, such as playlists for ‘studying’ or ‘dinner with friends’ or ‘Monday chill-out.’ Similarly, pop music has increasingly gravitated towards negative space since the 2000s scarce on melody and artistic uniqueness, but abundant in rhythm, spectral complexity, and artist-/genretranscending referentiality: an ideal reflection for young urban individuals grappling with overwhelming complexity, thus unable to establish a deeper connection with something that has already lost its socio-cultural relevance before their birth. For acoustic architecture, this does not mean that positive space is no longer utilized Positive space can and should be employed, but it necessitates more cautious examination than in the past: In a commercial world inundated with information and meaning, one should exercise caution while creating more.

Acoustic Architecture Scattered Over Time and Platforms

The most intriguing dynamic between the brand and the listener manifests in the non-linear and participatory domains of virtual and social media platforms. In contrast to the direct, one-way dissemination of auditory brand signifiers on traditional platforms such as television and radio, digital spaces facilitate a more diverse and temporally uneven distribution of these auditory cues. From TikTok videos and Instagram stories to branded VR experiences and interactive sonic interfaces on websites and apps, brands disseminate auditory cues across a multitude of channels and formats, each offering different fragments of the brand’s acoustic puzzle. Listeners interact with these auditory cues not in a linear or chronological sequence, but in a fragmented and personalised manner, based on their media consumption habits and preferences. Hence, the entire edifice of the brand’s acoustic architecture does not present itself simultaneously but gradually emerges over time as the listener encounters these auditory cues in different contexts and at different times. In this manner, each listener’s experience and perception of the brand’s acoustic architecture is unique and personal, moulded by their individual journey of engagement with the brand across various media and touchpoints.

From Passive Consumption to Co-Creation: The Dynamic Dance of Acoustic Architecture

The ubiquity of acoustic architecture, diffused across time and platforms, invites an evolution from one-sided broadcasts to dialogic interactions in today’s brand environment. It engenders an interactive dance between the conceived brand vision and the diverse interpretations from its stakeholders analogous to the participatory dynamic emerging in the pop music domain, where artists engage directly with their “superfans” to collaboratively generate novel realms: “These superfans are encouraged by a new generation of creators who themselves are fans of their fans individuals capable of architecting entire universes to maintain a sustained mutual interest” (Spotify 2022, 4, my translation).

Internally, the dynamic crafting of the brand’s acoustic identity hinges on the adaptive capacity of brand sound managers and designers. Far from doggedly following a preordained brand manual, they must exhibit creativity and agility, accommodating changes in strategic direction, market trends, and cultural shifts. This fluidity, while maintaining the brand’s core identity, ensures the sonic identity’s relevance and engagement. Externally, influencers, ambassadors, and users influence the brand’s acoustic identity. An open-ended approach to brand sound design encourages active participation, possibly through brand sound libraries, interactive sound tools, participatory workshops, or creative competitions. This increased openness invites a degree of unpredictability, particularly in the negative space an area with low predictability potentially unsettling for brands accustomed to controlling their narrative meticulously. Usergenerated content or interpretations may occasionally diverge from, or even blatantly challenge, brand guidelines. Yet, akin to how urban graffiti enlivens cityscapes, these deviations can augment the sonic vibrancy and richness of the brand. These variations reflect the brand’s diverse audience, their unique interpretations, and personal brand connections, contributing to an evolving brand soundscape that remains fresh and relevant.

Thus acoustic architecture is less about control and more about fostering a dynamic conversation around the brand’s sonic identity, striking a balance between guiding the sonic narrative and leaving room for improvisation. Brands can cultivate a sense of ownership and engagement among their audience, transitioning them from passive listeners to active participants in the acoustic narrative, deepening brand connections and fostering loyalty. This diversity of voices can morph acoustic architecture into an influential tool that builds personal relationships, creating a memorable and, above all, shared brand experience. The brand’s sonic identity can become a living, evolving entity, growing in harmony with its audience and consistently reinforcing the brand’s essence in its audience’s minds and hearts.

Crafting Sonic Environments:

The Practical Approach to Acoustic Architecture

Central to acoustic architecture is the strategic sculpting of both positive and negative spaces; this facilitates the weaving of a coherent brand narrative through its sonic identity while simultaneously empowering users to interact meaningfully within these spaces. This aim is realised through a myriad of techniques, each possessing its unique scope and application, yet all converging towards the creation of a singular auditory experience that embodies the quintessence of the brand. In the following discussion, we will delve into the most pivotal design fields within this framework that go beyond or significantly expand the scope of classic sound branding: Soundscape Design, Interaction Design, and Brand Voice.

Negative Space: A Triptych of Soundscape Design, Spatial Design, and Virtual Acoustics

A thorough exploration of negative space in sound design necessitates a focus on soundscape design. Here, the objective is the meticulous orchestration of the acoustic environment within physical or digital realms guided by non-linear, algorithmic compositions. This is achieved through the deployment and careful spatial arrangement of pre-recorded, algorithmically generated or live-streamed sounds, thus constructing a distinct and entrancing auditory ambience.

An effective soundscape may be crafted from a deliberate juxtaposition of sound qualities pulsating background music from one side, the melodic chirping of birds from another, complemented by the continuous burble of a nearby stream. This spatial soundscape exploits the subtleties of sound to stimulate and react to desired emotional reactions, modulate behaviours, and bolster brand associations within both retail and office environments. The embodied qualities of music tempo, rhythm, and spectral complexity form the bedrock of these aural constructs by inducing certain associative and emotional states, and as such are of critical importance to their design and analysis. Intriguingly, these interactive soundscapes are able to provide a sonic bridge between real and virtual spaces. Their dual functionality allows for installation in either realm, and more remarkably, they can be deployed to interact with each other: actions in the virtual realm can then influence the real-world soundscape, and vice versa.

Nested within spatial design is the realm of virtual acoustics, which employs similar principles within digital environments. Techniques such as fast HRTF convolution combined with rudimentary early reflection generation can generate a threedimensional auditory illusion of a specific room’s acoustics, even when experienced through headphones within digital spaces such as websites, apps, or VR and AR experiences. In direct vocal communication the user’s voices then appear to resonate within the simulated environment. In doing so, virtual acoustics allows the forging of sonic experiences that mirror the brand’s physical spaces, thereby bridging the divide between the tangible and the digital. This offers users an uninterrupted and consistent auditory journey across diverse platforms.

Interaction Design: Auditory Icons and Ear-Cons

At the heart of positive space design lies Interaction Design, which markedly enhances user engagement and experience through its dynamic facet. This domain harnesses the potential of auditory icons and ear-cons focusing primarily on the most prominent forms of sonification utilized in branding to captivate the listener and foster a deep-seated interaction with the brand’s sonic identity.

Auditory icons are symbolic sounds that signify their source objects and communicate distinct actions or events in user interfaces, much akin to the role that visual icons play in graphic design. Consider the sound of a shutter clicking to symbolize a photograph being taken, the rustle of paper evoking the action of moving an item to the trash can, or a gentle ‘ding’ denoting the receipt of a message. Such sounds not only amplify user interaction through an implicit informational layer but also proffer opportunities for brands to infuse their unique characteristics, thereby subtly underpinning the brand’s sonic identity through user interactions.

Contrastingly, ear-cons are abstract, musical phrases or motifs employed to symbolize a specific function or nugget of information, much like the aforementioned McDonald’s sound logo. Despite the lack of a direct real-world sonic equivalent, these sounds have been firmly associated with their respective brands via consistent use across myriad media touchpoints. Ear-cons, although not immediately intuitive, accrue significance over time. Their inherent musical movement, acting as a carrier of implicit meaning, combined with repeated exposure in conjunction with visual branding, ultimately serves as potent auditory signifiers for the brand.

Within the framework of temporally non-linear environments, such as interactive media, the generation of auditory icons and ear-cons increasingly leans toward lines of software code as opposed to traditional recordings. This progression presents a fascinating schismatic duality: The advent of digitization signifies the first degree, severing the recording from its physical medium. The second degree manifests as a liberation of the idea from the recording itself, empowering a system to produce sound as required, ensuring both contextual pertinence and timely delivery.

The potency of interaction design within acoustic architecture hinges on the judicious amalgamation of both auditory icons and ear-cons. The authenticity of auditory icons ensures immediate recognition and comprehension, while ear-cons present opportunities to instill abstract ideas or brand values into the listener’s memory. The challenge brands face is designing these auditory cues in a manner that aligns harmoniously with their overarching sound design and distributing them effectively throughout their media landscape.

Brand Voice Design and AI Voice Generation

Arguably, the most salient element of positive space design resides in the voice. Brands have the opportunity to craft a distinctive ‘brand voice,’ an actual voice that communicates in a style and tone emblematic of their identity. This voice, be it human or AI-generated, uses fundamental musical parameters such as register, ambitus, tempo, and rhythm to create a specific tone. The human voice, as we’ve experienced in iconic brand commercials, conveys emotion and nuance with efficacy, adding a personal touch to the brand’s acoustic architecture. In contrast, AI voice generation, mirroring a font in visual design with its distinct prosody, offers scalability and consistency. As AI voice models improve and multiply rapidly, we are likely to encounter more artificial voices in corporate communication, particularly in Chat Bots, hyper-targeted advertisements on social media, and interactive voice response systems on telephone waiting lines, rather than high-level campaigns.

In summary, the disciplines of acoustic architecture are as diverse as they are dynamic, evolving under the influences of technological advancement and shifting user behaviours. The focal tasks of curating immersive soundscapes or designing auditory icons share a common, unwavering objective: the manifestation of an engaging, consistent, and unique acoustic brand experience. Within this stimulating and fluid environment, technology, as always, does not substitute creativity but serves to enhance it.

To realize the full potential of these innovative strategies, established brands must embark on significant transformations in their internal organization, interpretation and application of data, and, at the core, their overarching philosophy of brand management. Such comprehensive evolution might not be immediately forthcoming from traditional entities. However, emerging brands are primed to assimilate and deploy these novel approaches from the outset, making them likely the avant-garde of the new era of brand management. Indeed, though we currently witness multiple examples of audio communication, including AIgenerated ones, the principles of acoustic architecture as delineated here are rarely adhered to in their entirety. Nonetheless, as these organisational metamorphoses occur beyond public purview, these measures will undoubtedly punctuate our collective consciousness once they amass sufficient momentum. Consequently, it becomes of utmost importance to scrutinise the social, ethical, and psychological implications of this paradigm shift with diligent care and a prescient mindset.

The Complexity and Ethical Dimensions of Acoustic Architecture

Navigating the juncture of social, ethical, and psychological implications within an increasingly virtual society where brands employ acoustic architecture to construct immersive environments poses an intricate territory for exploration. In the past, a single message intended for everyone sufficed, enduring over comparatively lengthy periods. However, today’s landscape necessitates personalized outreach to many, partly through messages (positive space) and partly through environments (negative space), all within highly regimented intervals. Thus far, our examination of acoustic branding, refracted through the architectural analogy, has illuminated transformative shifts in brand communication. While this perspective facilitates an understanding of complex developments and the cultivation of novel creative approaches, it calls for a critical investigation of the subsequent societal consequences. Indeed, as communication platforms and experiential spaces proliferate, and music consumption patterns undergo significant shifts, the architectural analogy serves not only to demystify the evolution of corporate communication, but also to render its complexities more digestible and navigable. However, it is essential to remember that individual stakeholders, from brand managers to social media producers, head creatives to brand ambassadors, and even unacquainted individuals, possess only limited agency within the grand sweep of technological and societal transformations.

Presently, many corporations compartmentalise branding and communications, often leading to an inadequate incorporation of sound branding into holistic campaign strategies. This fragmentation mostly results in a nebulous acoustic architecture, shaped by numerous campaign-centric creative decisions and an insufficient deployment of linking sound design. Emerging technologies like artificial intelligence promise to bridge this divide, assisting communication departments in selecting brand-compliant music for campaigns, capitalising on contemporary musical trends in social media, and even automatically generating personalised audio-visual content. In light of this, one might question the extent to which human intervention will be necessary for branding endeavors in the imminent future. Within the sound design industry, it will likely shift away from tasks of relatively low-level generic composition and production functions that can be increasingly delegated to machines. Instead, human contributions will be most crucial in the realm of high-level creative processes, imbuing projects with distinctive attitudes and styles. This shift, however, is not without its complications. While the incorporation of such technological solutions may optimize brand communication, it simultaneously gives rise to a multitude of new strategic and pragmatic challenges that, within companies, add to the already unanswered issues of classic sound branding. Specifically, AI solutions pose a unique risk; when misused, they may result in diminishing distinctiveness and promote generic creative outcomes.

From a designer’s perspective, automating basic creative work, such as creating musical underscores, could initially imperil artists reliant on this income source, thus endangering their livelihood. The recognition of the participatory nature of the acoustic architecture model by brand companies could spur a surge in crowdsourced, underpaid or even unpaid creative work. Conversely, however, it could also foster patronage of chosen artists from the global community. This can already be interpreted in some current music campaigns, such as those by Vodafone, or those featuring the artist Marc Rebillet for Fitbit, EDEKA, and the Super Bowl. Such developments carry social repercussions as well. Immersive sonic environments, though captivating, could overwhelm with acoustic stimuli that may not overtly seem commercially driven. Vulnerable demographics, particularly children and the less educated, could become targets of exploitation. Additionally, the potential sensory bombardment might precipitate stress-related conditions and induce greater social withdrawal. Stress-sensitive individuals, notably those on the autism spectrum, could face extreme difficulties. Conversely, it may assist blind individuals in better navigating virtual spaces.

From a political standpoint, these shifts could undermine healthy scepticism towards large corporations, potentially enhancing consumer susceptibility to branding strategies. This susceptibility could again be particularly problematic within vulnerable demographics, underscoring the necessity for conscious and ethical implementation of such tactics. Moreover, by curating and controlling pseudo-public digital spaces, corporate brands may significantly sway social interaction, moulding it according to their commercial, political, and ultimately, cultural ambitions, as it has already been extensively discussed about the platform activities of the Meta Group.

A noteworthy analogue to the modern abstract brand, utilising the principles of acoustic architecture in curated spaces, engaging consumer participation, and patronising artists, can be traced back to religious institutions, notably the Christian church during the Reformation. Martin Luther introduced a participatory dimension to religious practices, most notably by translating the Bible into vernacular language, thereby facilitating an elevated level of engagement amongst worshippers in their faith. He further pioneered the concept of Gemeindegesang or congregational singing in Christian worship.

Before Luther, church music was predominantly the purview of the clergy and choirs, rendering ordinary worshippers largely as passive spectators. Luther, however, fervently advocated for active congregational participation in worship and identified music as a potent tool in this endeavour. Furthermore, eminent composers such as Georg Philipp Telemann and Johann Sebastian Bach, who were notably employed or commissioned by the church, made significant contributions to both global cultural heritage and the church’s sonic architecture through their music. In this respect, the Reformation-era church demonstrated an early embodiment of the essential facets of acoustic architecture consumer (or, in this case, congregational) involvement, a spatially determined acoustic experience of both positive and negative space, and patronage of artists offering a historical parallel to contemporary practices in the commercial sphere.

From a purely secular perspective, there is no difference between the Christian church and a commercial brand, where the product could be abstract, untouchable or interchangeable. It is pertinent, therefore, to question whether a similar brand phenomenon might emerge from the current commercial landscape. The prospect and potency of acoustic architecture in branding, which extends beyond merely corporate spheres, while exciting, demand judicious and ethical implementation, underpinned by a profound comprehension of the far-reaching

societal implications. In particular, corporations must acknowledge their cultural responsibilities within this novel acoustic landscape. Cultural engagement can no longer solely retreat into so-called philanthropic endeavors of the elites, through sponsorships of Philharmonies and high art events. Instead, such commitment must increasingly manifest within the everyday creative spaces of digital and physical platforms. Brands, more than ever, do not belong exclusively to corporations but to global communities. Supporting these communities and collectively evolving in line with humanistic principles must be the overriding objective to preserve their vital ideological relevance.

Conclusion

From the exploration of acoustic architecture in branding, it becomes apparent that we are on the cusp of a seismic shift in brand communication. Sound, which has historically been a less conspicuous component of corporate identity, is steadily rising to prominence as a potent element capable of enriching and enhancing brand experiences. However, the efficacy of this powerful tool is contingent upon active engagement and interactivity. Hence, outdated, noncommunicative imprinting strategies are increasingly proving ineffectual in the current landscape.

The fields of soundscape design, interaction design, and brand voice have been brought into the spotlight, each playing a pivotal role in the creation of immersive auditory environments that resonate with the brand’s identity. The promise of creating unique soundscapes that can seamlessly transcend physical and virtual realms presents an exciting opportunity for brands to engage their audience in a deeply personal manner. The role of technology, particularly AI, in shaping this future is undeniable. While it holds immense potential to streamline and enhance sound branding, its potential misuse or over-reliance raises a host of ethical and societal concerns. These include potential exploitation of vulnerable demographics, increased stress levels due to sensory overload, and potential impacts on the creative industry, among others. Drawing parallels from historical instances such as the use of acoustic architecture in religious institutions, it is clear that this potent tool can be wielded for both benevolent and malevolent purposes. It underscores the imperative for brands to exercise ethical stewardship in their deployment of acoustic architecture, ensuring it serves to enhance the audience’s experience rather than exploit it.

In conclusion, as brands venture forth into the brave new world of acoustic architecture, they must acknowledge that their survival hinges solely on the readiness of individuals to engage and interact. Therefore, each facet of their operations should be strategically calibrated to enrich individual user journeys and cultivate experiential spaces conducive to participation. The requisite user data is readily accessible, and the tools for automated real-time content production and generation are at our disposal. The task at hand lies in the apt interpretation of data, precise goal-setting, and the zeal to craft worlds in which we all aspire to dwell. Earlier, I cited Reynolds’s (2010) rhetorical question concerning the purported sound of the 2020s through the lens of acoustic architecture, we can discern that this question is predicated on outdated assumptions, specifically regarding whose or which creative expressions shape our era. The spotlight now is on how these sounds and soundscapes are coming into being: through creative participation The future of acoustic architecture in branding is abundant with possibilities, teeming with challenges, and undoubtedly set to transform our sonic landscape in ways we are only beginning to imagine.

Author Biography

Lars Ohlendorf holds an M.A. degree in time-related media and works as a sound designer, composer, and Tonmeister. His academic path at Hochschule für Angewandte Wissenschaften Hamburg (HAW Hamburg) focused on sound engineering, media studies, and music psychology, providing him with a grounding in musicology, sound design, and non-linear/generative music approaches. Currently, Ohlendorf holds the position of Head of Design at the sound branding agency WESOUND. In this capacity, he is tasked with the strategic and conceptual evolution of sonic brands. A focus of his development work is on generative systems in music, soundscape and speech applications with a strong link to interactivity and the and the fusion of digital and physical spaces. His portfolio includes sound strategies and designs for brands such as ALDI Suisse, Cornelsen Verlag, Credit Suisse, Duden, real, Targobank and Team D. Contact: ohlendorf@wesound.de

Website: www.wesound.de

Bibliography

Blesser, Barry, and Linda-Ruth Salter. Spaces Speak, Are You Listening? Experiencing Aural Architecture. Massachusetts: MIT press, 2009.

https://culturenext2022.byspotify.com/de-DE

Reynolds, Simon. Retromania: Pop Culture’s Addiction to Its Own Past. New York: Farrar, Straus and Giroux, 2011.

Ross, Alex. Listen to This. London: Fourth Estate, 2010.

Spotify Advertising Global Trend Report: Culture Next. 2022. Wundermann Thompson. Future of Sound. 2023.

https://www.wundermanthompson.com/insight/future-of-sound-report

Acoustic Lanes and Auditory Leads

Spatio-Temporalities of Social Acoustics and Public Address Systems in Late Weimar Germany

Abstract

Soundscape Studies are rooted in a compositional approach to space and spatiotemporal constellations (Schafer 1973). It is about notation and measurement. In this respect, doing space and sound is linked to practices, knowledge and compounds of acoustic (physical) and auditory (physiological) information. Following this perspective, noise or sound, both socially or culturally constructed, are subfunctions of what frequencies, incident energies, loudness or threshold values. Spatiality and temporality are calculable conditions. This contribution highlights another approach. It refers to two concepts, acoustic lanes (RosenstockHuessy 1956) and auditory leads (Anders 1992), that are derived from practical philosophy and its connections to analysis of media and interpersonal communication and to the shaping of noise and sound. Both concepts are gauging acoustic material and its information value along the lines of being ‘useful’ or ‘applicable’ or ‘common’ or whether it can be labelled as a ‘nuisance ’ Taking these designations on board, I explore the social acoustics of noise and sound in Interwar Germany when decoding public addressing systems and the evaluation of ‘good’ sound in urban spaces. Thereby, I challenge the polyvalent concept of acoustic branding as it has evolved since the time when instruments of signaling and alarming began competing with the polyphonies of urban streets and spaces in 1920 and 1930s Europe especially. In addition, I shed light on the modes of how the politics of acoustic management of spaces, temporal situations, moods, and masses has been conducted and professionalized. Diagnosing those relations and its contents, I do, secondly, follow the echoes that sound and noise disposed in scholarly reflections and how émigrés like Rosenstock-Huessy and Anders emphasized the enticing and misleading capacities of phasing audiences and publics. My notion of social acoustics is therefore shaped by the interconnectedness of behaviour, self-convergence of individuals with groups and technologies.

Social theory deals with the presence of objects and bodies in spatial environments (Lefevbre 1967). Public spaces can be conceived of as stages for performances of any kind (Goffman 1963). The analysis of social relations is mostly bound to what one can observe (empirical data), what one wants to acknowledge and how one extracts sense from the ethnographic material that has cautiously been collected (theory). While data and agency are mostly conceptualized within the paradigm of vision, listening and hearing have been very much sidelined in social theory. This bias is also present in fields like communication and media studies and unsurprisingly in cultural or urban history as well. Following acoustic pathways is an occasion to shed some light on the social, cultural and economical hierarchies that frame notions of noise. Such relations establish themselves as polyvalent and polysensorial constellations (Lefebvre 1967, 30) that shape everyday social acoustics for individuals in spaces but also in terms of class Auditory knowledge is constituted through the experience of and engagement with acoustic material. The first hypothesis of this article is that acoustic lanes and auditory leads provide directions. They mark the asymmetries that give expression to relations of power as well as to economic, political, social and cultural dominance. In this respect and as a second conjecture, I argue that noise signifies space. It imprints a signature that brands the temporal and spatial conditions of places and environments and of individual and class behaviour. Whether their texture be rural or urban, bodies and physiological receptions intertwine in constellations of sound and noise. Understanding such conditions, at least from a historian’s perspective, archival materials serve as a written field recordings that refer and comprise to acoustic layers and auditory markers.

Focusing on the work of Eugen Rosenstock-Huessy (1888-1973) and Günther Stern, who later named himself Anders (1902-1992) two scholars who fled Germany in 1933 this paper outlines their philosophical thinking concerning acoustic materialities, auditory experiences and the spatio-temporalities of noise and how this can by applied to construe social, natural or architecturally designed environments or constellations of traffic and mobility or addressing public audiences Both authors sought to decode noise, sound, speech and music in order to understand various settings of communication but chose different approaches to do so. It is all about perception and notation. Rosenstock-Huessy, who was trained as a sociologist and lawyer, coined the term Hörwege (acoustic lanes) and placed the asymmetries of social relations at the center of the concept.

Anders, who became Hannah Arendt’s husband in 1929, but whose plans at a career in academia were frustrated by Heidegger (Dries 2022), outlines the potential impacts of media technologies on the sensory regimes of everyday life. Coining the term akustische Leine (auditory lead), the philosopher considered this impact as much for the individual as with reference to class, control and consumption. Anders places men and women, in the sense of individuals, within spatial conditions of social power and hierarchies. Such positioning comes along with different modes of past and memory as well as expectations to what might occur in the present and in a contemporary environment. This perspective on spatio-temporalities leads to reckoning with the spatio-materialities of emission, exposure and nuisance.

Hörwege (Acoustic Lanes): Eugen Rosenstock-Huessy’s Concept of Social Acoustics

Shortly after the First World War, Rosenstock-Huessy started working in the Department of Marketing, Advertising and Press Relations (Literarisches Büro) at the headquarter of the German car manufacturer Daimler in Stuttgart. He was responsible for the illustrated magazine ‘Werkstatt’ that addressed all members of the workforce and aimed to inform and entertain. As an editor-in-chief (Wilkens 2007), he conceptualized corporate identities and became a craftsman of corporate communication. He and his team of part-time journalists selected news and provided information for reader’s eyes, ears and minds. The bi-monthly magazine nurtured discussions at shop floor level and challenged information coming from trade unions or radicalized factions on both left and right of the political spectrum. An industrial site, a company with various branches and departments was a social organisation in which opinions and ideas could propagate, along with all the positive and negative side effects this could entail. Coherence and harmony proved to be idealistic constructions that remained out of reach. At least, it was an obligation of the Press Relations Department to obtain and to successfully defend opinion leading capacities. In order to lead the discussion at shop floor level and during breaks, it became a necessary resource to persuade, to convince by modes of speech, published statements and confidence. It became key to reach diverse audiences within the workforce with various approaches. All of this was embedded in a setting of connections and relations of hearing, speaking, listening and understanding. Starting and maintaining conversation with the workers meant negotiating different layers of access, participation and interest

The theoretical framework of Rosenstock-Huessy’s Hörwege is rooted in a philosophical argumentation that centers around constellations of communication, especially lanes on which words, sentences, gestures, voices and contents of speech move to the ears, eyes and minds of employees or university students. The relationship between masters and servants was no longer relevant in modern places of work and education Instead, workers were highly specialized, and a skilled and well-trained workforce was in turn a resource for a company’s success For his employer, Rosenstock-Huessy aimed at organizing a printed, illustrated and ready-to-read format of a continuous Betriebsversammlung, a sort of conference that would involve the company’s entire workforce, and that was elevated and transposed to a mediatized platform. As a practitioner of social communication, he hoped for making company managers and trade union functionaries meet, for presenting products as well as ordinary people which were conducting important tasks within the Daimler work environment. He outlines that all aspects of listening and all features of speaking must match and fall into another. The bonds between the different social groups of workforce and of management and its contentions result in calls of conformity and claims of accordance (Übereinstimmungen) are strong and last even a heavy dispute (unerbittlich geführter Streit). He idealized the identity of speech with the auditory threshold (Rosenstock-Huessy 1956, 141) In the best of cases, the messages that are disseminated by the directorate reach the recipients without any kind of distortion. Such auditory lanes combine “mouth and ear” as well as the direction of what is spoken to whom, with what intention, and how this is perceived, processed, consumed and decoded. The sociologist, remembering his own experiences with forced obedience during school lessons or while being properly drilled on a parading field during military service, as well as keeping in mind, that as a result of such harmonization, he was forced to emigrate and to leave Germany in 1933, hoping, Rosenstock-Huessy argued in favour of the civilizing capacities of the better argument and of a more convenient framing of messages, and suggested a complementary and inclusive approach to processes of communication and social relations. He labelled this field Hörwegswissenschaft (Ibid., 142) and conceptualized this as the territory in which the different pillars of the sciences (physiology, psychology and physics) and the humanities (sociology, history and philosophy) interact. He reflects on the way that acoustic materials are rendered present and performed (their Vergegenwärtigungen) and relates this to the modes, potentials and channels of auditory reception He was searching to enable the (self-) harmonization and to nourish occasions of group belonging and are generated by abusive and manipulative leaders (Rosenstock 1925, 68). He thus applies a spatial as well as a temporal perspective to processes of speech, voices and vocal tones, hearing, listening and understanding without having an established and notorious terminology to name his approach properly For him, sound and noise comprise an inner and an external area of social acoustics which are both linked to directions, either looking backwards to store past experiences or foreseeing future agency (Rosenstock 1924, 57).

The Hörwege are paved by party or trade union conventions, parish congregations, works meetings, protests in the streets and public spaces (Aulke 2015), public addresses (Szendy 2015), aggressive policing (Lindenberger 1995, 173-180) or violent routing and rioting by Freikorps or Communist militia (Stahl 2022, 301-309). In this respect, acoustic and auditory lanes are installed and adapted by force or acclamation and fixed repeatedly by consent or endorsement

After his tenure at Daimler, Rosenstock-Huessy managed a social politics think tank and resumed a career in academia, becoming a professor of law at the University of Breslau (Wilkens 2007). As the Director of the Curriculum, he continued listening to employees, now being participants of advanced training courses at the Academy of Labour (Frankfurt/Main), or to his reading group of law students. Even if the sociologist, as a professor of law and a practitioner of jurisdiction and communication, had face-to-face social interaction in mind, he elaborated a bi-directional theoretical approach which focuses on situations and pasts. His manner of decoding the relationships of sensorial regimes can be extended to the bonds between technology and humans, as well as to the materialities of listening and hearing generated through media or in public spaces. Noise appliances, signal horns, instruments of transmission, chants, vocal offences and flout, affirmative or deviant musical repertoires (Lindenberger 1995, 334-358) and vibrating hubs of sound and urban diversities (Helms/Phelps 2016, 7-10) create materialities of social acoustics within built environments and in terms of city planning, issues of infrastructure and urban mobility.

The techno-human bonds of social acoustics (Sterne 2003, 2012) and media technologies for making sound and noise –for example a transistor or a Bluetooth box or smartphones shape the perception of private and public space (Parzer 2008, 83-87) Even though they are supposed to be carefully designed, architecturally arranged (von Fischer 2017; Stoldt 2019) and attentively kept in order, in line and on the acoustic lane and within the auditory side rails, soundscapes are characterized by contingent, involuntary exposure to mutualinvolvements (Goffman 1963, 214; Schafer 1973; Schestag 2022).

Auditory Leads: Günther Stern’s Critique of Media Technology and Consumer Convenience Music

Günther Anders explains his concept of how individuals are tied to and enlaced by auditory leads in an anecdote (Anders 1992, 241-246) He and some friends had chartered a lodge in a natural resort where they went hiking through the forest and up the hillside. On the front side of a nearby hotel a transistor was playing music on high volume and without interruption. Every time the group caught wind of the melodies transmitted by the device the squad’s mood lightened and the atmosphere tipped towards the positive. The natural sounds of their environment, however, did not have the same uplifting effect on their ears. Consuming authentic and natural acoustic information was to some extent an estranged experience that made members of the group feel unwell and insecure. When they were able to detect bits of melody, the individuals made it cohere with their common media experience and their well trained auditory expectations. They had integrated a broad variety acoustic material in their ways of perceiving the environment and used it to understand themselves as participants of world events (news), as consumers in markets of convenience and consent (advertising) and as users (disposals) who aggregate and transform what has been commercialized and monetarized beforehand. Anders detects the ties that connect people to the acoustic qualities of music, sound and noise and their capacities for (self-) bonding to its auditory values. Auditory markers create tangibilities, and thereby lead people, the common man and women in particular, and help them navigate through environments, either built, designed, imagined, contested or accelerated by technologies and devices. The terms Hörigkeit and Gehorsam describe modes of behaviour that are tethered to hierarchies of class, to modes consumption and to self-identification with the opportunities for participating in the market. These manners connect people’s ears to acoustic materials and auditory markers disseminated by media technologies as well as to hearing capacities and recalls of acoustic events or the impacts of such auditory exposure

Addressing Public Audiences: Acoustic Politics in Streets and Urban Spaces

For centuries, church bells engraved their acoustic signatures in urban spaces (Corbin 1994; Hahn 2015) and formed civic auditory memory. With the invention of sirens (Hilgers 2008; Marti 2020; Burger 2022) this primacy was contested, additionally challenged by radio devices and broadcasting consumers (Squier 2003; Birdsall 2012; Stahl 2022, 345). Since the early 1930s, public address systems (PAS) have become a game changer in terms of game of covering acoustic territories and of marking auditory sensorial regimes in urban areas (Ehlert 2005; Hauptstock/Stahl 2018, 2021). Festivities and parades serve as case studies to examine sound and noise in the public space. Celebrating the constitution of the Weimar Republic on August 11th in the late 1920s, municipal authorities reluctantly began to include free concerts, fireworks and public events in parks and marketplaces. Essen (Ruhr) was one such place that opted for such festivities. In 1927, two years after the occupation by French and Belgian troops came to an end, the mayor ordered a main event at Saalbau, the local concert hall, and free concerts with marching bands of the local police force, military bands and the official orchestra of city’s theatre that took place in parks and places of recreation in various quarters of the city (Essen 1927, 66-70). A year later, the orchestra of the municipal police additionally performed at multiple locations across town. Representatives of the municipal and district authorities held speeches using amplifiers, microphones and loudspeakers. An acoustic lane covered the nodal points of urban space, including the public through their listening activities. Using technology to disseminate auditory experiences to female and male voters (Essen 1928, 9-11) was at the side of consuming politics, governmental efforts and ideological advertising a secondary way of targeting the public with information and involvement. In summer 1930, state and communal representatives in Erfurt finally decided to open up constitutional celebrations to the city’s population and discussed where to position loudspeakers and to place the audience at market square when giving addresses to the public are given (Erfurt 1930, 172). In the following years, an organization of working class radio technology enthusiasts (Campbell 2019, 144; Kreis 2020, 419-423) was repeatedly asked to furnish the festivities with their self-constructed public address system (Krumm 1931; Mann 1932). Telefunken, a Siemens outlet for the emerging telecommunication and broadcasting devices market, intervened and claimed expenses and licensing fees due to copyright infringements (Gontard 1932). While the Weimar Republic was in dissolution, acoustic lanes (Hörwege) and auditory leads (akustische Leinen) were confined, delimited and exploited by economic interests. Well-established companies in the media technology sector strove to exclude other players from the profitable field of sound, audiences, acoustic material and auditory expectations and experiences (Stahl 2022, 322-325).

Long before the national socialist party seized parliamentary, administrative and governmental power in spring of 1933, the mediatization of acoustic spaces, of auditory territories and memories was already accelerating. The transition was accompanied by media technologies and the telecommunication company did endorse the NSDAP, which has been propelled from the political opposition to a place of authority, command and control. May 1st, 1933, a newly introduced Bank Holiday, was the first date for staging the capacities of the nationalsocialist movement to harmonize and abate ongoing class conflict. In Essen, they seized the occasion to give an acoustic performance of their conception of völkisch folkish and racially defined citizenship by generating auditory tokens for applauding participants (roadside) as well as members of the Party, civil society and para-military outlets parading on the street (inside). They let the MotorcycleSturm-Abteilung squadron roar their engines and stood in close ranks at public places in Karnap, Steele or Altenessen to collaboratively eavesdropping broadcasts from Tempelhof field in Berlin where the NSDAP had organized an enormous congregation for the Führer (Reismann-Grone 1933). Many local dealers of radio sets supplied the Party with technological means for these public listening events and of course they claimed their expenses at the municipal authority (Radiozentrale 1933). Similar efforts took place in the city of Erfurt (Pichier 1933) Even the Head of the Catholic diocese revealed enthusiasm for what he imagined as a promising New Germany (Freusberg 1933). Loudspeakers were all over places and spaces in urban Erfurt. Acoustic material and auditory territory were marked. This technology of transmission that helped to form public audiences out of individual listeners remained a corner stone strategy for occupying and staffing communal acoustic space (Krumm 1933) throughout 1930s and 1940s Erfurt and Essen or any other city or town in nationalsocialist Germany Politicians provided citizens (as consumers) with events featuring music, sound and noise (as dependents) as well as advertising ideology (as addicts) leading to acoustic branding of space and to spatial regulations of social acoustics.

Lanes Or/And Leads: Coming To Terms with Social Acoustics

When sound is read as some sort of convenient acoustic information, it reproduces fealty and allegiance among those who hear and listen. Noise nuisances, by contrast, operate differently. Even when conceived as isolated cases, they generate the desire for direct abatement, but such immediacy is rarely provided. When negotiating exposure to noxious and unsanitary conditions, the initial are regularly lost in the detail of verbose discussions. Rosenstock-Huessy and Anders, acting as seismographers of social, cultural and technological displacements, learned in what ways messages, media and biased and fabricated information relate to the sensorial preparedness of those who transfer such material into their

individual strategies for making sense of the world and their class, in the sense of socio-political environments. Married to Hannah Arendt since 1929, Anders frequently moved between different dwellings, tenancies and cities, from Potsdam to Heidelberg to Frankfurt/Main. From these locations, he sensed how the acoustic politics of public space and urban political territory shifted from conservative center-right to populist-right and völkisch, with strategies of racialized exclusion becoming well-established and audible in speeches and other practices for acoustically claiming the streets. He and Rosenstock-Huessy both provided accounts or what they had endured and experienced with their fellow men and women, at congregations and festivities in public and by mediated smearing campaigns or physical chivvies throughout urban spaces. Convivial gatherings and uncordial encounters, accompanied by language, tone, speech, wording, sound, gesture, acoustic threshold values and auditory material of remembrance, resulted in vocal violence and sensorial exposure long before violations of sensuous space escalated to physical assaults and bodily harm. Both authors recorded notions of everyday life, of communication and of the dispersion of acoustic materials and the apportionment of auditory contents in public space, but they leaned towards transposing those accounts into conceptual statements of how people interact with media technologies, with order and obedience. Media ecologies of branding are located in space, time and territory. As Eugen Rosenstock-Huessy and Günther Anders outlined, they are related to sensorial economies, to their space value as much as to their face value and to a floating and mobile remembrance threshold. Public addressing systems, the temporal placement of loudspeakers in urban space, formed anthropo-technologies (Liggeri 2018, 82 and 91), containing techno-presences being double-binded to pasts and potential futures Governing acoustic material bears opportunities to regulate range and coverage and to translate music and information into cultural matter and auditory contents Anders and Rosenstock-Huessy can help us readers to understand that the grammar of noise and sound is spatio-temporal. It goes along lanes or leads and signifies space, participation, use, purpose, products and sensory perception of and sensuous behaviour in technologized environments.

Author Biography

Dr. Heiner Stahl is Senior Lecturer at the History Department of the University of Siegen. He published a book on popculture and media in 1960s Berlin (2010) and soundscapes, social acoustics and hearing knowledge in urban spaces in Erfurt, Birmingham and Essen from the 1880s to the 1960s (2022). He co-edited a volume on spatio-temporal relations of rhythms and timing in the processes of place/space-making (2018). In articles he has focused on noise emissions of airports (2013) and municipal hygiene and health policies in socialism (2014). He is finishing a book on media and public relations in postwar West Germany (2023) and is currently conducting a DFG-financed research project on the sensory history of ice cream, taste and table cultures in Europe from the 1770s to 1850s.

Bibliography

Anders, Günther. Die Antiquiertheit des Menschen. Vol. 1. Über die Seele im Zeitalter der zweiten industriellen Revolution. München: C.H. Beck, 1992.

Aulke, Julian. Räume der Revolution. Kulturelle Verräumlichung in Politisierungsprozessen während der Revolution 1918-1920.

Stuttgart: Metzler, 2015.

Birdsall, Carolyn. Nazi Soundscapes. Sound, Technology and Urban Space in Germany, 1933–1945. Amsterdam: Amsterdam University Press, 2012.

Bracht, Franz. “Besprechung über die Veranstaltung der Verfassungsfeier, Essen, 19.07.1928.” In Haus der Essener Geschichte/Stadtarchiv Essen, Rep. 102 I, Nr. 738: 9-11.

Burger, Bianca. “Von Sirenen und anderen Signalen.“ medien & zeit. Kommunikation in Vergangenheit und Gegenwart 37, no. 2 (April 2022): 54-60.

Campbell, Bruce B. The Radio Hobby, Private Associations, and the Challenge of Modernity in Germany. London/Cham: Palgrave Macmillian, 2019.

Corbin, Alain. Les cloches de la terre. Paysage sonore et culture sensible dans les campagnes au XIXe siècle Paris: Albin Michel, 1994.

Eugen Rosenstock-Huessy-Gesellschaft. “Wer ist Eugen Rosenstock-Huessy?”

Eckart Wilkens. Köln (2007). Accessed May 31st, 2023.

https://www.rosenstock-huessy.com/ew-eugen-rosenstock-huessy/ Firma Westdeutsche Radiozentrale. “Rechnung für die Herstellung der Lautsprecheranlage am 01.05.1933, Essen 29.05.1933.” In HdEG/StA Essen, Oberbürgermeister Reismann-Grone, Rep. 102 I, Nr. 739: 1.

Fischer, Sabine von. Das akustische Argument. Neue Parameter in der Architektur des 20. Jahrhunderts. Zürich: gta, 2016.

Freusberg, Joseph. “Domprobst an Stadtoberbaurat Ludwig Boegl. Gottesdienst aus Anlass des nationalen Feiertages. Erfurt 28.04.1933.”

In Stadtarchiv Erfurt, Magistrat Erfurt, Sig. 1-2/009-65: 233.

Goffman, Erving. Behavior in Public Places. Notes on the social organization of gatherings New York: Free Press, 1963.

Gontard, Adolf. “Telefunken Leipzig, Abteilung Elektro-Akustik [ELA]. Lautsprecheranlage für Stadion und Schlachthof. Leipzig 21.01.1932.”

In Stadtarchiv Erfurt, Elektrizitätswerk, Sig. 1-2/811-10847: 3-4.

Hahn, Philipp. “The Reformation of the Soundscape. Bell-ringing in Early Modern Lutheran Germany.” German History 33, no. 4 (November 2015): 525-545.

Hauptstock, Hans and Heiner Stahl. “Rundfunksäulen und (Volks-) Gemeinschaftsempfang. Zur Beschallung der Öffentlichkeit in rheinischen Kommunen während der NS-Diktatur.” In Geschichte im Westen 33 (2018): 177-199.

Hauptstock, Hans und Heiner Stahl. “Akustische Raumordnungen des A-Falls. Reichslautsprechersäulen im Nationalsozialismus.” In Rundfunk und Geschichte 47, 1-2 (April 2021): 11-22.

Helms, Dietrich and Thomas Phleps, eds Sound and the City Populäre Musik im urbanen Kontext Bielefeld: transcript, 2007.

Hilgers, Philipp von. “Sirenen. Lösungen des Klangs vom Körper.”

In Parasiten und Sirenen. Materielle Kulturen der Produktion von Wissen, edited by Bernhard J. Dotzler and Henning Schmidgen, 195-218. Bielefeld: transcript 2008.

Internationale Günther Anders-Gesellschaft. “Jugend und Studium (1902-1924); Wanderjahre 1924-1933.” Accessed May 31st, 2023 https://www.guentheranders-gesellschaft.org/vita

Kreis, Reinhild. Selbermachen. Eine andere Geschichte des Konsumzeitalters. Frankfurt/M.: Campus, 2020.

Krumm, Heinz. “Entwurf eines Schreibens an Telefunken G.m.b.H., Berlin, Erfurt, 12.09.1931.” In Stadtarchiv Erfurt, Städtisches Elektrizitätswerk, Sig. 1-2/81110847: 1-2.

Krumm, Heinz. “Vermerk zu Lautsprecheranlage, Erfurt, 12.04.1933.”

In Stadtarchiv Erfurt, Städtisches Elektrizitätswerk, Sig. 1-2/811-10847: 34-39.

Lefebvre, Henri. “Le droit à la ville.” L'Homme et la société 6, no.1 (June 1967): 29-35.

Liggieri, Kevin. “Schnittstelle ‘Mensch’. Zum Forschungsinstitut für Anthropotechnik.” In Unterwachen und Schlafen. Anthropophile Medien nach dem Interface, edited by Michael Andreas, Dawid Kasprowicz and Stefan Rieger, 77-103. Lüneburg: meson press, 2018.

Lindenberger, Thomas. Straßenpolitik. Zur Sozialgeschichte der öffentlichen Ordnung in Berlin 1900-1914. Bonn: Dietz, 1995.

Mann, Bruno. “Oberbürgermeister Erfurt an Regierungs- und Baurat Kurt Otto. Verfassungsfeier 1932, Erfurt, 05.08.1932.” In Stadtarchiv Erfurt Magistrat Erfurt, Sig. 1-2/009-65: 221.

Marti, Sibylle. “Alarmieren.” In Strahlen im Kalten Krieg. Nuklearer Alltag und atomarer Notfall in der Schweiz. Paderborn: Brill/Schöningh, 2020. 300-361.

Parzer, Magdalena. Phänomen Handy. Eine qualitativ-empirische Untersuchung der Mobiltelefonnutzung im soziokulturellen Kontext. Diploma Thesis, Universität Wien 2008. https://services.phaidra.univie.ac.at/api/object/o:1250512/get Pichier, Theodor. “Oberbürgermeister Erfurt an sämtliche Magistratsmitglieder.

Feier anlässlich der Eröffnung des Reichstages, Erfurt 20.03.1933.” In Stadtarchiv Erfurt, Magistrat Erfurt, Sig. 1-2/009-65: 232.

Reismann-Grone, Theodor. “Besprechung mit den Stadtverordneten Moritz Kadden und Studienrat Dr. Hermann Grutzamp (NSDAP). Vorläufiges Programm Nationalfeiertag (01.05.1933), Essen, 26.04.1933.” In HdEG/StA Essen, Oberbürgermeister Reismann-Grone, Rep. 102 I, Nr. 739: 1-4. Rosenstock, Eugen. Angewandte Seelenkunde. Eine programmatische Übersetzung. Darmstadt: Rötherverlag, 1924.

Rosenstock, Eugen. Die Kräfte der Gemeinschaft. Berlin: De Gruyter, 1925, Rosenstock-Huessy, Eugen. Die Übermacht der Räume, Soziologie, Bd. 1. Stuttgart: Kohlhammer, 1956.

Schafer, Raymond Murray. “The Music of the Environment.” Cultures 1, no. 1 (January 1973): 15-52.

Schestag, Thomas. “Geräuschkulissen.” In Fakten und Verunsicherung. Ordnungen von Wahrheit, Fiktion und Wirklichkeit, edited by Carina Breidenbach et al., 227-275. Hamburg: Felix Meiner Verlag, 2022.

Squier, Susan Merrill, ed. Communities of the Air. Radio century, radio culture Durham/London: Duke University Press, 2003.

Stadtverwaltung Essen. “Programm der Verfassungsfeier im grossen Saal des städtischen Saalbaus zu Essen, 11.8.1927.” In Haus der Geschichte Essen/Stadtarchiv Essen, Rep. 102 I Nr. 738: 66-70.

Stahl, Heiner. Geräuschkulissen. Soziale Akustik und Hörwissen in Erfurt, Birmingham und Essen (1880-1960) Köln: Böhlau, 2022

Sterne, Jonathan. The Audible Past. Cultural Origins of Sound Reproduction

Durham: Durham University Press, 2003.

Sterne, Jonathan. The Sound Studies Reader. London/New York: Routledge, 2012. Stoldt, Milena. “Die Welt ist Klang. Zur Relevanz von gut geplanten urbanen Klanglandschaften.” Forum 44, no. 397 (July/August 2019): 61-63.

https://www.forum.lu/article/die-welt-ist-klang/ Szendy, Peter. Höre(n). Eine Geschichte unserer Ohren. Paderborn: Schöningh, 2015. Tiedemann, Fritz, Dr. “Schreiben an Oberbürgermeister Dr. Bruno Mann. Verfassungsfeier 11.08.1930, Erfurt, 25.07.1930.” In Stadtarchiv Erfurt, Magistrat Erfurt, Sig. 1-2/009-65: 172.

On the History and Aesthetics of Noise Reduction

Abstract

Modern sound production would and could not exist without different forms of noise reduction: Analog media technologies often used noise reduction filtering in different ways. A prominent example is of course the case of analog audio technologies, in which a variety of noise reduction technologies existed (Dolby A, B, C, SR, S; dbx; Highcom etc.). Firstly, the history of the most important of these systems will be media-archaeologically reconstructed. Noise reduction became necessary with the widespread use of analog audio tape technology, which has certain limitations. The focus will especially on the history of Dolby A, B and C since these were the most widely used systems. Their dominance blocked the way for technically better alternatives, especially Highcom. With the advent of digital technologies the discussed noise reduction systems became obsolete the last commercial noise reduction system by Dolby: Dolby S couldn’t be established on the markets for tape decks anymore. Secondly, there is a genuine aesthetics of noise reduction: On the one hand, analog noise filtering produces artefacts especially when reproducing noise reduced tapes in the wrong way or simply by the (sometimes incorrectly calibrated) noise reduction process (like, e.g. ‘hiss flags’ in dbx or a certain ‘muffled sound’ with Dolby). On the other hand, these effects are a rich source for experimental media aesthetics especially in electronic music, as can especially be seen in the work of Maurizio. A very important usage of noise reduction is the possibility to produce convincing silence e. g., in cinema. This can be shown with outstanding clarity for the case of the movie A Quiet Place, in which as its title already says quietness plays an important role.

In Claude Shannon’s (and Warren Weaver’s) communication and information theory basal elements of a communication system are described (Fig. 1). It is emphasized that the channel is threatened by ‘noise’ at any time:

During transmission, or at the receiving terminal, the signal may be perturbed by noise or distortion. Noise and distortion may be differentiated on the basis that distortion is a fixed operation applied to the signal, while noise involves statistical and unpredictable perturbations. Distortion can, in principle, be corrected by applying the inverse operation, while a perturbation due to noise cannot always be removed, since the signal does not always undergo the same change during transmission.1

Noise”. In Proceeding of the IRE 37, no. 1, 11 1949.

Shannon discusses the need to remove as much of the interference that occurs in the channel distortion and noise as possible if the signal is to be transmitted with the best possible quality. Elsewhere he also discusses a ‘correction system’ (Fig. 2).

He relates this to a ‘discrete channel’, which does not have the characteristics of the analog sound reproduction systems that will be the focus here but we will encounter this principle again in a similar form. The ‘correction system’ works like this: An ‘observer’, which of course can be an “auxiliary device”2 (but sometimes is also a skilled sound engineer), taps the signal M from the transmitter and compares it with the signal M’ received. The difference between the two is due to the interference of the channel. So the observer generates ‘correction data’ and forwards it to a ‘correcting device’, which transforms the signal M’ so that it again corresponds to the original signal M (or at least comes as close to it as possible). Without going further into Shannon’s difficult theory here, this scheme of ‘correction devices’ points to upcoming methods of noise suppression the filtering out of the “thermal noise that all matter and therefore also resistors or transistors radiates when operating (according to another one of Boltzmann’s formulas) …”3 To remain in our example: If it would be possible to successfully add the noise of the channel phase-inverted to the signal M’, it would disappear. Also in this sense, “messages themselves can be generated as … filterings of noise.” 4 But this does not have to be the case: Artfully applied filters, e.g., to alienate acoustic or visual signals, serve exactly the opposite purpose of distancing from an original signal, however given, and make the technicality of the channel visible to the extent that the underlying channel (in its institutional forms) can even give its name to the filters, as on Instagram, for example.

This essay focuses on a particular group of media technologies, their history, associated practices and aesthetics, at least sketchily: Analog sound recording on tapes (tape, compact cassette). As one can infer from Shannon’s general theory of communication, ‘noise’ is a problem for any channel, not just a particular type of channel. The concentration on analog tape recording of audio signals can be explained apart from the inevitable need to focus on something simply by the fact that analog audio technology made the concept of ‘noise reduction’ and the filtering methods associated with it, especially such as ‘Dolby’, known for at least a certain time; the Dolby logo (Fig. 3) is, or rather was, widely known.5

On the basis of the technical operations certain cultures, practices and aesthetics of noise filtering developed, which can be described at least in excerpts. Eventually one could get first hints on the connection of theories, technologies and practices like the aesthetics of filtering.

It is not possible to go into detail here about the history of the tape technology and the subsequent development of the compact cassette as a commercial technology.6 It can be stated that audio tapes, which are to be magnetized, as sound storage media pose special challenges with regard to the channel. Thus, in 1940, it was stated laconically: “The reproduction of magnetic sound recordings on steel wire and steel tape, but also on magnetizable film, is, according to the present state of the art, afflicted with a disturbing background noise.” 7 On the one hand, they cannot be recorded at arbitrarily high levels, because then there is a risk of distortion. On the other hand, this means that the signal is not very ‘loud’ compared to the white noise of the tape. Early developments such as premagnetization later dynamized in ‘Dolby HX Pro’ depending on the proportion of high frequencies in the audio signal8 allowed a significant improvement in level control, but still left a hefty noise.9 An early idea to reduce noise (which was also used in broadcasting) is ‘pre-emphasis.’ The idea: Since noise is especially disturbing in the range of higher frequencies, one raises high frequencies before the recording, with an appropriate circuit, and lowers them again during playback so one also reduces the noise.10 The big problem with this is that the high frequencies become louder during recording and the saturation of the tape is reached more quickly, i.e., distortion is created. This in turn means that you have to lower the recording level as a whole, which then leads to a lower signal-tonoise ratio in other frequency ranges.

In addition, the quality of the signal reproduction depends decisively on the tape speed. The faster the tape runs, the higher the frequency range and signal-tonoise ratio. For this reason, many classic tape machines had comparatively high tape speeds of 38.1 or even 76.2 cm/s.11 So, in order to make the channel as noise-free as possible, it would be advisable to use the highest possible tape speed. But obviously, this introduces a new problem: The faster the tape runs, the less the running time of a given reel. If one wanted to reproduce a longer concert, one would have to use very large reels, which make the device bulky and unwieldy and, moreover, create mechanical problems with the acceleration and deceleration of the large, sluggish reels. Making the tape thinner is also a limited option after all, it must not break. The problem becomes even more acute when trying to establish tape technology as a handy commercial technology in the form of the compact cassette. The small i.e., ‘compact’ cassettes do not fit as much tape, but the vinyl records established at that time had an average running time of about 40–45 minutes. So, cassettes were developed that could hold either 90 minutes (45 minutes per side = a whole record) or 60 minutes (30 minutes per side = one side of a record). However, because so little tape fit into the cassette, the tape speed then had to be greatly reduced to 4.76 cm/s for the compact cassette. This significantly reduced headroom and signal-to-noise ratio even taking into account that the compact cassette was intended more for a market that was concerned with recording the less dynamic, popular music.12 Add to that the fact that the tape was quite narrow compared to tape on tape reels,13 again at the expense of headroom. In short, the quality was weak. In order to establish the compact cassette as a popular medium, improving its sound quality and, above all, lowering the noise level was certainly desirable.14 The trade-off between the economics of tape length (which included its commercial fit into an existing media ensemble) and quality motivated the development of noise filters. Even the much better and often more expensive tape recorders still produced considerable noise at economically reasonable lower speeds: “The continuing demand for improvement in quality delivered to the consumer makes further evolution in noise-reduction systems mandatory.”15 This is also a complicated story that has not been presented anywhere as far as I can see. I can only go into one, but still the most famous, case here: Dolby.16

Ray Dolby founded Dolby Industries in 1965 and soon developed his first noise reduction system, the professional Dolby A. Sound studios in particular needed noise reduction methods, because multitrack recordings are used especially in the production of popular music. However, with each track especially since each track must be narrower to fit on the tape more noise is added, so that either a

further increase in tape speed comes into question, which soon reaches both economic and mechanical limits, or noise reduction. In 1967, a paper appeared in which Dolby outlined his process. The basic principle is that, as with other ‘companders’ (of ‘ compressors ’ and ‘expanders’), only in many ways more cleverly realized, the signal is separated into frequency bands by a system of filters before recording. These are treated separately, with the quiet parts being boosted during recording and lowered again during playback and with them the noise:

Low level signal components are amplified in four independent frequency bands prior to recording/sending, which is accomplished by adding the outputs of four filter and low-level compressor channels to the main signal. During reproduction, the filter and compressor network is connected in a complementary way. Low-level components are subtracted from the incoming signal, and noise acquired in the audio channel is thereby subtracted or reduced as well.17

Dolby’s development had a number of merits, so that already in 1970 it could be stated that “there are no technical arguments or quality compromises that could speak against its general use. It is therefore also to this day the only compander process that has found its way on a larger scale into commercial music production for record and radio throughout the world.”18 Soon a slimmed-down process was introduced: Dolby B. It no longer worked with four different frequency bands and only in the high frequency range, where noise was most distracting. It was to be found in practically every consumer tape deck from the second half of the 1970s at the latest.19 It has been noted about this process that it “keeps circuitry costs down and is ultimately good for the wallet.”20 The technology is a “compromise between engineers and marketing experts.” 21

In any case, Dolby A largely prevailed in the professional sector and Dolby B in the consumer sector, Dolby B also in commercial music cassettes (Fig. 5)22 also because cassettes recorded with Dolby B could be played in still acceptable quality on devices without Dolby expanders (an important backward compatibility). This entry of the music industry, as well as tape deck manufacturers into Dolby, which would require its own historical representation, created a path dependency that made it very difficult for competing systems.

A system that was used at least in some tape decks (for example by Technics, the author of this text was a long-time owner of a Technics tape deck RS-B905 with Dolby B, C and dbx) and increasingly also in the studio area was dbx. It produced since it companded the whole frequency range unlike Dolby B a larger signal-to-noise ratio, however the tapes compressed in such a way could not be played back well without dbx and dbx produced easily artifacts like socalled ‘breathing’, i.e. an audibly louder and softer becoming noise around signal peaks above all in the high tone range. 23 From the mid-1970s, Telefunken developed a very good and advanced compander system, HighCom, which was clearly superior to Dolby B,24 but was no longer able to establish itself due to the path dependency especially since Dolby introduced a better commercial compander, at least compared to Dolby B, with Dolby C as early as 1980.25

In 1986, Dolby Labs introduced Dolby SR, the successor to Dolby A for the professional sector, which is considered the crowning achievement in the history of analog audio companders. 26 From Dolby SR there was similar to Dolby A to Dolby B again a simplified procedure for the consumer sector, Dolby S, which was implemented for the first time in 1990 on commercial tape decks (and where care was taken to ensure that tapes compressed in this way could also be played back to some extent with Dolby B). But at that time the CD had already been established and especially the possibilities to copy CDs with computers had grown. The analog tape deck technology gradually disappeared, so that Dolby S was denied the big breakthrough.

The spread of especially Dolby B and other noise reduction methods was also accompanied by some specific practices and aesthetics, 27 which will be briefly discussed below and, as it were, autoethnographically based on my years of tinkering with tape decks. There was the principal problem that the whole principle of the compander presupposes a symmetry between compression and expansion:

Only when all these control processes are exactly mirror images of each other during recording and playback can the original signal be heard again with an intact frequency and phase response, transient response and correct dynamics. This is not a problem in theory. But in practice. Tape devices always bring their own individual frequency response, head mirror resonances and treble drop make the compander believe something different during playback than the compander entrusted to the tape during recording.28

Tapes often sounded muffled, so it was better to play them back without the Dolby noise filter, which made them more noisy but sounded clearer. Another trick was to cleverly mask the notches on the top of the cassette, so that a CrO2 tape cassette, for example, was played back as a Fe2O3 tape cassette, which also raised the treble and thus made it possible to use the Dolby noise reduction without dullness. In general, all companders require very precise calibration procedures; already during recording it is recommended to measure the individual tape exactly manually or with the help of a measuring computer (if available), which was only possible on higher-quality tape decks.29

In the process, the calibration required for good quality companding is also compromised over time: Over the years, tapes may lose magnetization, affecting proper noise filtering; but it may also be that no or no correctly working devices can be found for playback. This is where the problem of archiving arises in addition to tapes, equipment must also be archived. So, a small connoisseur scene has formed around the question of how to play back and restore old tapes correctly and what problems can arise in the process, a ‘Culture of Noise Reduction’ if you will.30 At the same time, these difficulties also mean that one can hear at least with some experience when music is reproduced incorrectly. The music then sounds dull or sharp, pumping, breathing, reverberating or distorted. In such disturbances the filter system, which should make the channel as inaudible as possible, itself becomes audible and possibly the connoisseur can even hear which noise reduction was used.

Nevertheless, this typically analog sound disturbances can itself become the source of an aesthetics. The early cassette culture, often associated with experimental or ‘underground’ music, was accompanied by a corresponding sound, often associated with involuntarily or unconsciously false companding practices (which could also operate as an opposition to the high-quality ‘high fidelity’ perceived as bourgeois 31). Even under digital conditions and a nostalgic desire for the analog32 associated with them, the dull, the breathing, and the lace itself can become aesthetic forms - for example, in electronic dancefloor music. For example, “M Ø6B” by Maurizio cites overtly misrepresented tapes that are highly noisy, while “M07A” follows the aesthetic of the dull.33 In the track “Don’t” by Actress (on the album “Ghettoville”) you hear a sample at the beginning and in the short pause after that the noise level turns up like a Dynamic Noise limiter gone crazy, or a badly calibrated dbx tape played back without dbx. 34

But perhaps the most important aesthetic contribution of noise reduction as a technology of silence is that silence can be produced. Phases of silence as a dramaturgical tool, die Ruhe vor dem Sturm, are only possible if silence is not obstructed by hiss. A brilliant example for this strategy is the 2018 movie A Quiet Place (John Krasinski). The original plot is that earth is invaded by a hostile alien species that killed most of humanity. The species is blind but they have very good ears. The remaining humans can only survive if they behave very, very quiet if you make one noise then the monstrous aliens can locate you, come and kill you. The film is centered on a family that tries to survive. Most of the film is very quiet the slightest noises, for example like dry leaves crackling under the feet, are thereby amplified in a haunting way. Noises that are normally completely overheard in real life and in the movies are blown up to an existential dimension. Every object that could produce a sound becomes an ominous threat. This is even more radicalized by the figure of the daughter. When scenes are shown from her point-of-audition there is an absolute, oppressive silence. Since we heard through her, non-functional, ears, it is unclear if she or someone else made a dangerous noise. This brilliant and terrorizing aesthetics of silence is only possible if you don’t hear hiss all the time. The silence produced by noise reduction is not just an absence of sound it is an aesthetic element in its own right.

Author Biography

Jens Schröter, Prof. Dr., is chair for media studies at the University of Bonn since 2015. Since 4/2018 director (together with Anja Stöffler, Mainz) of the DFG-research project “Van Gogh TV. Critical Edition, Multimedia-documentation and analysis of their Estate” (3 years). Since 10/2018 speaker of the research project (VW foundation; together with Prof. Dr. Gabriele Gramelsberger; Dr. Stefan Meretz; Dr. Hanno Pahl and Dr. Manuel ScholzWäckerle) “Society after Money – A Simulation” (4 years). Director (together with Prof. Dr. Anna Echterhölter; PD Dr. Sudmann and Prof. Dr. Alexander Waibel) of the VW-Main Grant “How is Artificial Intelligence Changing Science?” (Start: 1.8.2022, 4 Years); April/May 2014: “John von Neumann”-fellowship at the University of Szeged, Hungary. September 2014: Guest Professor, Guangdong University of Foreign Studies, Guangzhou, People’s Republic of China. Winter 2014/15: Senior-fellowship at the research group „Media Cultures of Computer Simulation.” Summer 2017: Senior-fellowship IFK Vienna, Austria. Winter 2018: Senior-fellowship IKKM Weimar. Winter 2021/22: Fellowship, Center of Advanced Internet Studies. Recent publications: Medien und Ökonomie, Wiesbaden: Springer 2019; (together with Christoph Ernst): Media Futures. Theory and Aesthetics, Basingstoke: Palgrave 2021. Visit www.medienkulturwissenschaft-bonn.de / www.theorieder-medien.de / www.fanhsiu-kadesch.de

Notes

1 Claude Shannon, “Communication in the Presence of Noise,” in Proceeding of the IRE 37, no. 1, 11. 1949.

2 Claude Shannon, “A Mathematical Theory of Communication,” The Bell System Technical Journal 27, no. 3 (1948): 408.

3. Friedrich Kittler, “Signal-to-Noise Ratio,” In The Truth of the Technological World: Essays on the Genealogy of Presence, ed. idem, trans. Erik Butler (Palo Alto: Stanford University Press, 2014), 167. Kittler refers in the following with Shannon to a further point, namely that the noise could also be a carefully encoded message and is relevant for the further military use of communication theory (and incidentally also for the question of extraterrestrial communication), but I will skip this here.

4 Kittler, “Signal-to-Noise Ratio,” 169.

5. What is not discussed in detail here is the role of Dolby technologies for cinema, see Gianluca Sergi, The Dolby Era. Film Sound in Contemporary Hollywood (Manchester: Manchester University Press, 2004). I will also not go into the technologies of ‘Active Noise Cancellation’, which currently play a major role in headphones, see Jens Schröter, “Technologies of Silence.” In Techniques of Hearing: History, Theory and Practices, edited by Michael Schillmeier, Robert Stock und Beate Ochsner 21–35 (New York: Routledge 2022)

6 Cf. Pia Fruth, Record. Play. Stop. Die Ära der Kompaktkassette: Eine medienkulturelle Betrachtung (Bielefeld: transkript-Verlag, 2018) and Axel Volmar and Judith Willkomm, “Klangmedien,” in Handbuch Medienwissenschatf, ed. Jens Schröter (Stuttgart: J.B. Metzler, 2014).

7 Hans-Joachim von Braunmühl and Walter Weber, Verfahren zur magnetischen Schallaufzeichnung, Reichspatentamt Patentschrift Nr. 743 411, Klasse 42g, Gruppe 10 02, filed June 28, 1940 and issued November 4, 1943. Translation by the author.

8 Cf. Ian Hardcastle, “Quality Improvements in Pre-Recorded Cassettes,” SAE Transactions 95 (1986): 1622–1629.

9. Cf. Braunmühl, and Weber, “Verfahren zur magnetischen Schallaufzeichnung” esp. 2ff.

10 Interestingly, the first CD standard also included a pre-emphasis for CDs, but this was hardly used in practice.

11. Cf. “Tonbandtechnik,” Genesis-Audioline accessed January 17, 2020, genesisaudioline.de/technik/tonbandtechnik/: “You may wonder how the ‘crooked’ values for tape speeds came about: they were created by halving the next highest speed in each case (the number of tracks is also gained by progressive halving, after all). At the beginning of tape technology there was a venerable 76.2 cm/sec, which corresponds to 30 inches per second. Ultimately, in fact, the gradation is based on the British-American system of measurement, and in order not to complicate international tape exchange, it was left at that and values from the metric system were dispensed with (38.1 cm/sec = 15″/sec; 19.5 cm/sec = 7.5″/sec; 9.53 cm/sec = 3.3/4″/sec, (″ is the abbreviation for inches)”, translation by the author.

12. It cannot be surprising that the formats of the distribution media tend to determine not only dynamics but also, for example, length of popular music pieces, cf. Theodor W. Adorno (signed “Hektor Rottweiler”), “The Form of the Phonograph Record,” October 55 (1990): 57f: “The only thing that can characterize gramophone music is the inevitable brevity dictated by the size of the vinyl plate.”

13 3.81 mm compared to 6.35 mm of the narrowest tape.

14 See the notes on the “Phillips Dynamic Noise Limiter” in Anonymous, “London Audio Fair. Review of a Show Attended by more than 70,000,” Wireless World 77 (December 1971): 585/586.

15 David E. Blackmer, “A Wide Dynamic Range Noise-Reduction System,” DB: The Sound Engineering Magazine 6 (1972): 54. However, the quote is from a text that is already about a critical reaction to the Dolby system.

16. See on Dolby’s method fundamentally: O. Diciol, “Dolby-System. Technik zur Verbesserung des Störspannungsabstands,” Hifi-Stereophonie 11 (1972). The method is very well described in Heinrich Sauer, “Immer mit der Ruhe,” Stereoplay 11 (1982).

17. Ray Dolby, “An Audio Noise Reduction System,” Journal of the Audio Engineering Society 15, no. 4 (1967): 388. See also Ray Dolby, “Audio Noise Reduction – Some Practical Aspects,” Audio 52, no. 6 (1968a) and idem, “Audio Noise Reduction. Part 2 (Conclusion),” Audio 52, no 7 (1968b).

18 K. Bertram, “Dynamikverbesserung mit dem Dolby-stretcher,” Fernseh- und Kinotechnik 4 (1970): 123. Translation by the author.

19 Cf. Dolby, Ray. “A Noise Reduction System for Consumer Tape Recording,” in 2nd Audio Engineering Society Convention. 16–18 March 1971.

20 Sauer, “Immer mit der Ruhe,” 43. Translation by the author.

21 Friedrich Kittler, “Gleichschaltungen. Über Normen und Standards der elektronischen Kommunikation,” in Geschichte der Medien, ed. Manfred Faßler, and Wulf Halbach (Munich: Fink, 1998), 261. Translation by the author.

22 And even with the audio tracks of videotapes.

23 For dbx, see Blackmer, “A Wide Dynamic Range.”

24. Cf. Jürgen Wermuth, “Dynamik-Erweiterung durch neuartigen Studio-Kompander,” Funkschau 47, no. 18 (1975) and Gerhard Dickopp, and Ernst Schröder, “Der Telefunken-Kompander,” Rundfunktechnische Mitteilungen 22, no 2 (1978). One of many other alternative, ultimately failed approaches is, for example, the Burwen Laboratories Noise Eliminator, see Richard S. Burwen, “Design of a Noise Eliminator System,” Journal of the Audio Engineering Society 19, no. 11 (1971). Cf. Michael G. Duncan, Davud Rosenberg, Graham W. Hoffman, “Design Criteria of a Universal Compandor for the Elimination of Audible Noise in Tape, Disc, and Broadcast Systems,” Journal of the Audio Engineering Society 23, no. 8 (1975). This text uses computer simulations to design criteria for an ideal compandor (the spelling ‘compandor’, also common in Dolby’s early texts, did not prevail) and measures Dolby, dbx, and the Burwen Laboratories Noise Eliminator against them. None of the systems meets the criteria, the ideal compander does not seem to have been realized, although it would be worth checking whether Telefunken would do better here with Telcom and then HighCom.

25. On Dolby C see, among others, Hardcastle, “Quality Improvement,” 629ff and Ray Dolby, “A 20 dB Noise Reduction System for Consumer Applications,” Journal of the Audio Engineering Society 31, no. 3 (1983).

26 Cf. Karl M. Slavik, and Stefan Weinzierl, “Wiedergabeverfahren,” in Handbuch der Audiotechnik, ed. Stefan Weinzierl (Berlin: Springer-Verlag, 2008): 621–622.

27 We cannot and should not go into the interesting discussions about an aesthetics of noise in literature, for example, see Rüdiger Campe, “The ‘Rauschen’ of the Waves. On the Margins of Literature,” SubStance 61 (1990).

28. Sauer, “Immer mit der Ruhe,” 43. Translation by the author.

29 The most expensive and best tape decks at the time, the Nakamichi Dragon and the Revox B-215, were in various ways technologies for calibration fetishists. The history of these extraordinary media technologies remains to be written.

30 Cf. “Noise Reduction,” Richard L Hess–Audio Tape Restoration Tips & Notes, accessed January 18, 2020, richardhess.com/notes/formats/magnetic-media/magnetictapes/analog-audio/noise-reduction. Apparently there are no digital emulations of the playback side of companders yet, which would facilitate the playback of old tapes and offer greatly expanded calibration possibilities: “The question of noise reduction companders comes up often on discussion boards. I am unaware of any noise reduction (NR) plugins to decode analog signals, it would be a logical item to create.”

31. Cf. Fruth, Record. Play. Stop.

32 Cf. Dominik Schrey, Analog Nostalgie in der digitalen Medienkultur (Berlin: Kulturverlag Kadmos, 2017). Not only the vinyl record, but even the tape deck are making a comeback today – albeit on a modest scale. Let’s see when there will be a Dolby nostalgia ….

33 Both on: “Maurizio – M-Series,” Discogs, accessed January 18, 2020, www.discogs.com/de/Maurizio-M-Series/release/203360.

34. See generally on glitches in electronic music, though without direct reference to noise reduction, Mark Fisher, “The Metaphysics of Crackle. Afrofuturism and Hauntology,” Dancecult: Journal of Electronic Dance Music Culture 5, no. 2 (2013).

Bibliography

Adorno, Theodor W. (signed “Hektor Rottweiler”), and Thomas Y. Levin. “The Form of the Phonograph Record.” October 55 (1990): 56–61.

Anonymous, “London Audio Fair. Review of a Show Attended by more than 70,000.” Wireless World 77 (December 1971): 585–587.

Bertram, K.: “Dynamikverbesserung mit dem Dolby-stretcher.” Fernseh- und Kinotechnik 4 (1970): 123–126.

Blackmer, David E.. “A Wide Dynamic Range Noise-Reduction System.” DB: The Sound Engineering Magazine 6 (1972): 54–56.

Von Braunmühl, Hans-Joachim, and Walter Weber. Verfahren zur magnetischen Schallaufzeichnung. Reichspatentamt Patentschrift Nr. 743 411, Klasse 42g, Gruppe 10 02, filed June 28, 1940 and issued November 4, 1943.

Burwen, Richard S.. “Design of a Noise Eliminator System.” Journal of the Audio Engineering Society 19, no. 11 (December 1971): 906–911.

Campe, Rüdiger. “The ‘Rauschen’ of the Waves. On the Margins of Literature.” SubStance 61 (1990): 21–38.

Diciol, O.. “Dolby-System. Technik zur Verbesserung des Störspannungsabstands.” Hifi-Stereophonie 11 (1972): 1152–1161.

Dickopp, Gerhard and Ernst Schröder. “Der Telefunken-Kompander.” Rundfunktechnische Mitteilungen 22, no. 2 (1978): 63–74.

Discogs. “Maurizio – M-Series.” Accessed January 18, 2020. www.discogs.com/de/Maurizio-M-Series/release/203360.

Dolby, Ray. “An Audio Noise Reduction System.” Journal of the Audio Engineering Society, 15, no. 4 (October 1967): 383–388.

Dolby, Ray. “Audio Noise Reduction – Some Practical Aspects.” Audio 52, no. 6 (1968a): 19–22.

Dolby, Ray: “Audio Noise Reduction. Part 2 (Conclusion).”Audio 52, no. 7 (1968b): 26–30.

Dolby, Ray. “A Noise Reduction System for Consumer Tape Recording”. In 2nd Audio Engineering Society Convention , (Cologne/Germany, 16–18 March 1971).

Dolby, Ray. “A 20 dB Noise Reduction System for Consumer Applications.” Journal of the Audio Engineering Society 31, no. 3 (1983): 98–113.

Duncan, Michael G., Davud Rosenberg, and Graham W. Hoffman. “Design Criteria of a Universal Compandor for the Elimination of Audible Noise in Tape, Disc, and Broadcast Systems.” Journal of the Audio Engineering Society 23, no. 8 (1975): 610–622.

Fisher, Mark. “The Metaphysics of Crackle. Afrofuturism and Hauntology.”

Dancecult: Journal of Electronic Dance Music Culture 5, no. 2 (2013): 42–55.

Fruth, Pia. Record. Play. Stop. Die Ära der Kompaktkassette: Eine medienkulturelle Betrachtung. Bielefeld: transcript-Verlag, 2018.

Genesis-Audioline. “Tontechnik.” Accessed January 17, 2020. genesisaudioline.de/technik/tonbandtechnik/.

Hardcastle, Ian. “Quality Improvements in Pre-Recorded Cassettes.” SAE Transactions 95 (1986): 622–637.

Hess, Richard L. Audio Tape Restoration Tips & Notes. “Noise Reduction.” Accessed January 18, 2020. richardhess.com/notes/formats/magnetic-media/magnetictapes/analog-audio/noise-reduction.

Kittler, Friedrich. “Gleichschaltungen. Über Normen und Standards der elektronischen Kommunikation.” In Geschichte der Medien, edited by Manfred Faßler, and Wulf Halbach, 255–268. Munich: Fink,1998.

Kittler, Friedrich. “Signal-to-Noise Ratio.” In The Truth of the Technological World: Essays on the Genealogy of Presence, edited by idem and translated by Erik Butler, 165–177. Palo Alto: Stanford University Press, 2014.

Sauer, Heinrich. “Immer mit der Ruhe.” Stereoplay 11 (1982): 42–45.

Schrey, Dominik. Analog Nostalgie in der digitalen Medienkultur. Berlin: Kulturverlag Kadmos, 2017.

Schröter, Jens. “Technologies of Silence.“ In Techniques of Hearing: History, Theory and Practices, edited by Michael Schillmeier, Robert Stock und Beate Ochsner 21–35. New York: Routledge 2022.

Sergi, Gianluca. The Dolby Era. Film Sound in Contemporary Hollywood. Manchester: Manchester University Press, 2004.

Shannon, Claude. “A Mathematical Theory of Communication.” The Bell System Technical Journal 27, no. 3 (1948): 379–423 and 27, no. 4 (1948): 623–665.

Shannon, Claude. “Communication in the Presence of Noise.” Proceeding of the IRE 37, no. 1, (1949), 10–21.

Slavik, Karl M., and Stefan Weinzierl. “Wiedergabeverfahren”. In Handbuch der Audiotechnik, edited by Stefan Weinzierl, 609–685. Berlin: Springer-Verlag, 2008.

Volmar, Axel and Judith Willkomm. “Klangmedien.” In Handbuch Medienwissenschaft, edited by Jens Schröter, Simon Ruschmeyer, and Elisabeth Walke, 279–288. Stuttgart: J.B. Metzler, 2014.

Wayback Machine. “Oval Archive.” Accessed January 18, 2020. web.archive.org/web/20070709154158/http://www.meso.net/oschatz/oval/.

Wermuth, Jürgen. “Dynamik-Erweiterung durch neuartigen Studio-Kompander.” Funkschau 47, no. 18 (1975): 103–196.

The Critique of Power Dynamics Through Sound

Abstract

This paper explores and elucidates a method of making sound in the history of music and art that is explicitly political and disruptive. At different moments in history, certain musicians and sound artists have strategically enlisted different modalities in the production of their music or art to critique and disrupt different forms of power. Some of these particular projects have actively tried to resist or challenge certain oppressive power formations and cultural norms. Their creations form a sub genealogy of sorts in the history of sound, connected not through genres, context, or musical compositions, but by a commonality of procedure and method. The primary focus of this paper is an analysis of how political critique and resistance can be made through musical endeavors or sound art and put into action in the world how theory can become praxis. Three specific case studies will be investigated to demonstrate the various permutations of how music or sound can be melodically quite different and originate from a different context, but also strive for the same overall goal of critiquing power, often in the form of societal norms, spatial politics, or unwritten cultural codes of conduct. Because many of these forms of power become naturalized, they often become unnoticed. However, even in their invisibility they serve to frame individual understanding, meaning, and value. The critiques of power dynamics through sound considered in this paper are intended to expose and question the reach, authenticity, and authority that different power formations may possess. To demonstrate how a critique of existing forms of power may be developed using sound, a detailed analysis of three projects utilizing sound will include Pussy Riot’s political protest Punk Prayer, Samson Young’s artwork Canon, and Selma Selman’s performance You Have No Idea.

When the punk band Pussy Riot created a performance of political protest in Moscow’s Cathedral of Christ the Savior, they generated a gripping and raw insurgency rooted in the critique of power that brought global attention and initiated an expansive discourse about feminism, religion, Russian politics, incarceration, corruption, orthodoxy, and the current state of punk rock music. And although their protest was as a powerful attempt to subvert hegemonic power using a fusion of sound and spatial politics, this does not mark the first time that sound has effectively been utilized for politically transgressive means In many ways, the analysis and experience of music and sound has intrinsically carried the potential for disruption or transgression because of its comprehensive and extant categorization and taxonomy.

There has always been a sort of contrived territorialization in the distribution and consumption of music. “Genres” or “scenes” are presented and constructed as collective identities that encompass styles of clothing and personal appearance, socio-economic status, and political platforms. These “ genres ” or “scenes” sometimes have the feel of fortified encampments distinguishing and protecting themselves from the infiltration of outsiders. However, as Simon Frith has pointed out, “genres” as spatialized compartments are highly problematic. As Frith asserts in his “Genre Rules,” the “use of genre categories [has been] to organize the sales process (Frith’s italics).” 1 The mapping and delineation of genres is also exceedingly fluid depending upon context and intended market. Another problematic of “genres” is that certain music simultaneously fits into more than one existing genre labels. Will Straw makes a similar claim about the perceived hermetic borders of musical “scenes” as spatialized compartments. Straw argues that the “diversity of musical practices unfolding within particular urban centers,” undermines “claims as to the uniformity of local music culture.”2 He also reminds us that a seemingly removed and remote “scene” can still be “shaped by economic or institutional globalization.”3

The reason I want to draw attention to the lack or absence of clear boundaries in musical “genres” or “scenes” is because I also want to question the notion of transgression or disruption that is often attributed to musical “genre-benders.” For as long as musical “ genres ” have existed, there have been those who have been perceived as breaking down the walls separating each “genre” as they head out into new musical territories.

Indeed, genre-benders (or breakers) have been around for as long as “ genres ” themselves stretching back to the beginnings of music history. As Roy Shuker has written however, “no style is totally independent of those that have preceded it, and musicians borrow elements from existing styles and incorporate them into new forms.” 4 Throughout the history of music, genre-bending musicians have often been celebrated as being innovative and revolutionary. And in certain instances this may be true: Bach’s combination of religious music and secular theater was trailblazing, and so was Bob Dylan’s mixture of folk and rock on Bringing It All Back Home (1965). Miles Davis’ fusion of jazz and rock on In A Silent Way (1969), and Beck Hansen’s genre-hopping albums Mellow Gold (1994) and Odelay (1996) have also been considered as important musical innovations. However, from what we know from Frith and Straw, the political trangressiveness of these efforts may not be so radical. They may not even be transgressive in the etymological sense of “stepping over” and breaking through an established boundary.

I have taken the time to draw out this distinction because I want to address another form of critique or transgression that has occurred in the sonic arts that is explicitly more spatial and political in the disruption of dominant powers and the recognized social order. It is here that I want to turn our attention towards a few specific examples of the use of sound to create a spatialized critique of power. I am referring to particular projects that have actively utilized spatial politics to critique and resist dominant power formations and cultural norms. These examples form a genealogy in the history of sound studies connected not through genres, context, or musical composition, but by a commonality of procedure and critical inquiry. This paper is an analysis of what the critique of power dynamics can sound like as a method of resistance and rebellion

I have chosen specific projects by Pussy Riot, Samson Young, and Selma Selman because of how they create a rupture “in the screen of received cultural assumptions ” 5 These projects are political acts executed through the use of sound, hinged upon a certain ungrounding and transgression. There is a radical air to these critiques of power a dissatisfied proclamation, an assertive desire for transformation or what Foucault might call a new “discourse that combines the fervor of knowledge,” with the “determination to change.” 6

Before I dig into a detailed case study analysis, I want to clarify what is meant by “dominant power formations and cultural norms” and how sound is well suited to resist and question their legitimacy. In a general sense, I am pulling from the tradition of Practice theory as begun by Pierre Bourdieu and extended by Foucault and others. The notion of resisting the imposition of general cultural schemas or formations comes from Bourdieu’s concept of “habitus” in which the “permanent internalisation of the social order in the human body” is counteracted by the “human ability to act upon and change the world.” 7 I am also drawing upon Foucault’s critique of how ideology and thought “operate upon the entities of our world, to put them in order, divide them in classes, group by name.” 8 As will be detailed below, these case studies apply sound and spatial politics as a strategy and “refusal to be fixed or pinned down.”9 Within this refusal is a turn towards agency enacted in the world. In Giorgio Agamben’s analysis of Foucauldian apparatuses, he identifies and investigates their effects on the subject, as anything that captures, restricts, or determines the thoughts and actions of living beings. Agamben also advocates finding new ways to dismantle them through individual agency and points out “this problem [of reclaiming the desubjective power of an apparatus] cannot be properly raised as long as those who are concerned with it are unable to intervene in their own process of subjectification.” 10 The sound projects studied here are intended to create critique and affect change calling attention to instances of already existing impositions of power.

Yet, one might ask: why sound? What does the application of sound provide that is not achieved or is not as well performed through other modes of creative expression? How does sound enable and accentuate the spatiality needed for transgression? Sound aids in the production of a critique of spatial politics because there are inherent spatial and mobile qualities to sound beyond the physical facticity of how sound waves transfer energy to move the body on a molecular level, music also creates and constructs a spatial and mobile experience.” 11 The experience of sound is unique because it is a multivalent bombardment of the senses – visual, haptic, aural, and aromatic – that creates an embodied engagement. This embodied immersion isn’t just a “way of expressing ideas, it is a way of living them,”12 and as such, the experience of sound “gives us a way of being in the world.”13 The absorption in the material physicality of sound is often also accompanied by critical reflection, what Jane K. Cowan suggests as a “double sense of engrossment and reflexivity.” 14 In terms of opening a possibility for a spatially political critique, Frith give this account of the inimitable spatiality and mobility of music:

“Music is thus the cultural form best able both to cross borders - sounds carry across fences and walls and oceans, across classes, races and nations - and to define places; in clubs, scenes, and raves, listening on headphones, radio and in the concert hall, we are only where the music takes us.”15 Sound and music have the ability to move us, to take us into new territories, whether through a shift in consciousness, on a narrated journey, in a psychic projection, or by creating a performance in a particular place. Sound provides a powerful vehicle for the resistance of the established social order through the performative invocation of a spatial transgressive.

A Pack of Bitches from the Sexist Regime

Much of the backlash in Russian society against Pussy Riot’s performance was because of the “getting-into-place” methodology through which their critique of power was enacted. The fallout of the Pussy Riot performance amongst the various levels of Russian authorities and some media outlets centered around the rhetoric of the sacred and the profane to incite an indictment of Pussy Riot’s actions as “hatred and enmity of religion and hatred of Orthodox Christians.”16 Indeed the use of sound was crucial to Pussy Riot’s performance because the loud, aggressive, and confrontational nature of punk music helped to further complicate the perceived dichotomy of the sacred and the profane (How could they play that music in that place?), but it also aided in immediately grabbing attention by being boisterous, provocative, and assertive. The attacks based on religious grounds were misguided however, since the intentions of Pussy Riot’s critique of power was not constructed on a platform of religious hatred, but rather from a focus on spatial politics and the gendering of space. The members of Pussy Riot themselves describe their performance as a “political gesture.”17 The title of the music video documenting the performance also has a political reference: Punk Prayer - Mother of God, Chase Putin Away! Their decision to perform on the altar in the Cathedral of Christ the Savior was motivated both by the gendered restrictions placed upon that space and the alliance between church officials and political leaders. “We needed to sing it not on the street in front of the temple,” writes Pussy Riot, “but at the altar that is, in a place where women are strictly forbidden (Fig.1).”18 And as for the questionable coalition of church and state they write that “what troubles us is that the very shrine you consider so defiled is so inseparably linked to Putin, who as you say, returned it to the cathedral” 19 and elsewhere, “the words we spoke and our entire punk performance aimed to express our disapproval of a specific political event: the patriarch’s support of Vladimir Vladimirovich Putin.”20

In an effort to affect change, and to critique the power dynamics within Russian society, Pussy Riot utilized a strategy of spatial transgression as a means to reveal and expose. By “getting-into-place,” a place that was off limits, they utilized a method of “sound as performance” to create a critique of the nature of the exclusions for that space. Some of the significant questions that their performance raised included: Why are they (Pussy Riot) excluded from these spaces?

Who has prescribed these restrictions? What is the purpose of this exclusion?

Pussy Riot’s performance exposed and subverted the hegemonic forces at work within place, especially those that attempt to gender, politicize, ritualize, sequester, or deify space. Even though Pussy Riot’s critique of power may seem distinct from the next two to be discussed because of its creation as a more traditional type of musical performance, each of these projects share the crucial commonality of utilizing sound to call upon a critique of spatial politics to produce a political act

Shots Fired

Samson Young is a contemporary artist that integrates sonic elements into the creation of his artworks and installations. This integration is especially evident in his project titled Canon from 2016 (Fig. 2). Consisting of multiple components including a microphone, 3D printed water basin, custom-designed bench, soundtrack, stamped text on wall, wired fencing, and a LRAD. This final component is an acronym for Long Range Acoustic Device, commonly described as a “sound cannon,” and is a piece of equipment often used for police control of large crowds. It can also be employed as a weapon “capable of causing permanent hearing damage by directing sounds up to 2,000 metres at a volume of 150 decibels.”21

As Young writes of the installation: “Canon is comprised of a sound installation and live performance, which is experienced at two different locations. At the ‘broadcasting’ location, with an LRAD unit the artist whistles the calls of birds, in response to a background track that is consisted of recordings of bird calls.”22 Dressed in the uniform of the Hong Kong colonial police circa 1979, Young stands at a microphone which is connected to the LRAD, into which he creates a stream of continuous imitation bird calls. The LRAD unit is pointed across the installation space toward the viewers in a designated space walled off by a wire fence. The fence restricts the movements of the viewers and does not allow them to approach Young during the performance. The use of the LRAD reminds us that sounds can be used to forcible move people and also to enforce spatial control by deciding who should or should not be allowed in specific spaces which also connects to the historical reference in the artwork.

The installation has complex allusions but small clues, such as the word “Skyluck” on the side of the bench, point the viewer to a historical event from 1979 in which the “cargo ship Skyluck arrived in Victoria Harbour carrying 3,000 Vietnamese refugees, it was not given permission by the Hong Kong colonial authorities to land. For twenty-three weeks, the ship operated as a floating prison, its passengers quarantined offshore, until its anchor chains were severed, and the vessel beached at nearby Lamma Island.”23 The space also contains a small red plastic washbasin spewing water, another historical reference because it is identical to those provided to the Skyluck refugees. Both the restriction of the viewer’s movement by Young and the use of a police uniform, manifest the artist in this project as a symbol for state power and control. Through the spatial politics made real for the viewer in the segregation of the installation space and the reference to the immobilization and exclusion of the Vietnamese refugees, Young emphasizes (and critiques) the absolute spatial control wielded by the state and its ability to detain, direct, and dehumanize in order to retain its own dominance. Young also significantly demonstrates how sound is deployed in the maintenance of state control through means such as the distressing projection of sound through a LRAD, but also in the absence of sound as in the intentional disregard of the voices and cries of the Skyluck refugees. A case in which the refusal of state powers to hear any sound or voice wielded even more power than a LRAD.

The title of the artwork (Canon) also refers to another form of control – the rules and the laws by which something or someone is judged. These rules are often either state mandated or are in the form of cultural convention. Either way, they represent the type of power and control that Young wants his viewer to be conscious of through the experience of this artwork. The overall purpose of Canon is meant as a critique of state power and how that power is exercised in ways that restrict, contain, and exclude.

You Have No Idea

Another performance artwork that uses sound as a critical element is Selma Selman’s You Have No Idea (Election Day) from 2020 (Fig.3). While Selman first performed this work in 2016 and has since re-created the performance in various venues, the Election Day version was presented in Washington, DC on election day 2020 and involved her walking along the Black Lives Matter Boulevard towards the White House, while she repeated the phrase: “You Have No Idea.” She was followed by some photographers and curious bystanders which grew in size as the volume of her voice steadily increased until it reached a scream (Fig. 4). Selman has described the project in these terms: “You do not who I am, nor do you know my happiness or sadness. You do not know about the presence or absence of pain in my life, nor how I feel at the moment that I perform this piece to a live audience. You have no idea. Though this piece refers to the specific circumstances of my life, I believe that it carries a universal message. We have no idea about the struggles that others are facing. We have no idea, but we think that we do.”24

By walking through the streets making her noisy proclamation, Selman used the volume of her voice (sound) to disrupt the expected experience of others. All cultures and contexts have unspoken public codes of conduct and within the space of a contemporary American urban landscape, Selman’s performance was received as being disruptive, unusual, and brash which was exactly her point. At a certain moment in the performance, one of the by-standers exclaims “she is on crack-cocaine.”25

While this response is very much a jump to conclusion, it also illustrates the imposed and established rules of conduct for the public sphere and the cultural stigma which dictates that anyone who breaks these rules must have something wrong with them. Selman created a public display that not only disrupted the accepted behaviors of public space, but also addressed intimate personal issues, such as the frustration of exclusion, discrimination, and the discovery of selfreclamation, all of which we are often told to deal with privately and not in public. Selman’s performance asks the viewer to consider not only the impossibility of knowing her or her struggles, but also how the systems and structures that we can’t see in plain sight that contribute or give origin to the prescriptions of who is, and who is not, welcome in any particular place. As Boshko Boskovic has written, Selman’s unruly performance issued a two-pronged challenge: “validation of our own subjective experiences” and “acknowledgement of oppressive mechanisms” which can be startling and abrupt to witness, but utilizes sonic

The Critique of Power

The implementation of sound in each of these case studies has facilitated the creation of an embodied criticism that includes spatial, mobile, and visceral elements. Through the experience of sound, the viewer is moved not always physically, but certainly made aware of, and asked to question, cultural events and conditions that rely on the spatialized exercise of power. By exploring these three performances, I have demonstrated how varying modalities of using sound remain connected through their mutual critique of power. My aim has been to show that a critique of spatial and body politics using sound can be powerful, disruptive, and effective. If sound has an intrinsic mobility and spatiality, it seems particularly well suited to the “stepping over” of establish boundaries or regulations The inherent spatial quality of sound and “its capacity to reconfigure space” lends itself to this effort as well, since to resist something may indeed sometimes require what Kenneth Frampton calls a “clearly defined domain” to push against or transgress.27 In the form of sounds as critiques of power, these disruptions hold significant potential as a means of rebellion and resistance to dominant cultural power formations.

Author Biography

Gregory Blair is an artist, writer, educator, and activist that resides in Evansville, Indiana with his wife and two children. Blair is an Assistant Professor of Art and Design at the University of Southern Indiana where he teaches digital design, contemporary art history, and gender studies courses. Blair has exhibited his artwork and presented his research both nationally and internationally in locations including Portugal, Italy, NYC, New Orleans, Boston, Alaska, St. Louis, Cincinnati, Minneapolis, and Canada. Blair’s artwork and research incorporates interdisciplinary art practices, cultural geography, spatial politics. His latest book project, The Politics of Spatial Transgressions in the Arts, was published in 2021 by Palgrave Macmillan. Email: gblair1@usi.edu, www.gregoryblair.com

Notes

1. Simon Frith, Performing Rites: On the Value of Popular Music (Cambridge, Mass.: Harvard Univ. Press, 1999), 75.

2 Will Straw, “Systems of Articulation, Logics of Change: Communities and Scenes in Popular Music,” Cultural Studies Vol. 5, Issue 3 (1991), 368.

3 Straw, 369.

4 Roy Shuker, Popular Music: The Key Concepts, 2nd ed (London/New York: Routledge, 2005), 121.

5. Greil Marcus, Lipstick Traces: A Secret History of the Twentieth Century (Cambridge, Mass: Belknap Press of Harvard University Press, 2009), 3.

6 Michel Foucault, The History of Sexuality (New York: Vintage Books, 1990), 7.

7. Birgit Bräuchler and John Postill, eds., Theorising Media and Practice, Anthropology of Media, v. 4 (New York: Berghahn Books, 2010), 7.

8 Michel Foucault, The Order of Things: An Archaeology of the Human Sciences (Vintage, 1994). xvii.

9 Robert Deuchars, Ronald Fischer, and Ben Thirkell-White, “Creating Lines of Flight and Activating Resistance: Deleuze and Guattari’s War Machine,” accessed February 12, 2016, http://www.philosophyandculture.org/seminar1003deuchars.html.

10 Giorgio Agamben, What is an Apparatus? And Other Essays (Stanford University Press, 2009), 24.

11 Simon Frith in Questions of Cultural Identity, Stuart Hall and Paul Du Gay, eds. (London: Thousand Oaks, Calif: Sage, 1996), 109.

12. Simon Frith in Questions of Cultural Identity, Stuart Hall and Paul Du Gay, eds. (London: Thousand Oaks, Calif: Sage, 1996), 111.

13 Frith, 114.

14. Jane K. Cowan in “Systems of Articulation, Logics of Change: Communities and Scenes in Popular Music” by Will Straw, Cultural Studies Vol. 5, Issue 3 (1991), 380.

15 Simon Frith in Questions of Cultural Identity, Stuart Hall and Paul Du Gay, eds. (London: Thousand Oaks, Calif: Sage, 1996), 125.

16. Pussy Riot, Pussy Riot!: A Punk Prayer For Freedom (The Feminist Press at CUNY, 2012), 39.

17 Pussy Riot, 15.

18. Pussy Riot, 15.

19 Pussy Riot, 26.

20 Pussy Riot, 42.

21 Seth Kim-Cohen, “The Sound Canon of Samson Young,” Catalogue Text for Samson Young’s Songs for Disaster Relief, Hong Kong Pavilion, 57th Biennale di Venezia. https://kim-cohen.com/wp-content/uploads/2020/07/2017_Kim-Cohen_The-SoundCanon-of-Samson-Young.pdf. Accessed on June 10, 2023.

22 Samson Young, “thismusicisfalse.com/canon,” thismusicisfalse.com, https://www.thismusicisfalse.com/canon. Accessed on June 10, 2023

23 Kim-Cohen, “The Sound Canon of Samson Young”

24 Selma Selman, “You Have No Idea,” Roma Archive, https://www.romarchive.eu/en/collection/you-have-no-idea/. Accessed on Jan.29, 2022.

25 Selma Selman, “You Have No Idea (Election Day 2020),” YouTube, https://www.youtube.com/watch?v=OP03KJPY-pU. Accessed on June 10, 2023.

26 Boshko Boskovic, “Reclaiming Ourselves through our Labour: Selma Selman’s artistic practice and Modes of Healing, ERIAC, accessed Jan. 28, 2022, https://eriac.org/reclaiming-ourselves-through-our-labour-selma-selmans-artisticpractice-and-modes-of-healing/

27 Kenneth Frampton, “Towards a Critical Regionalism: Six Points for an Architecture of Resistance,” in The Anti-Aesthetic: Essays on Postmodern Culture, Hal Foster, ed., (New York: New Press, 2002), 25.

Bibliography

Agamben, Giorgio What is an Apparatus? And Other Essays Stanford University Press, 2009

Born, Georgina Music, Sound and Space: Transformations of Public and Private Experience, 2015.

Chapman, Owen “Sound Moves: Intersections of Popular Music Studies, Mobility Studies and Soundscape Studies,” Journal of Mobile Media 7, no. 1 (2013). Accessed 10/19/16, http://wi.mobilities.ca/sound-moves-intersections-ofpopular-music-studies-mobility-studies-and-soundscape-studies/ Cresswell, Tim In Place/out of Place: Geography, Ideology, and Transgression Minneapolis: University of Minnesota Press, 1996.

Foucault, Michel The Order of Things: An Archaeology of the Human Sciences

Vintage, 1994

Frith, Simon. Performing Rites: On the Value of Popular Music. Cambridge, Mass.: Harvard Univ. Press, 1999.

Keith, Michael and Steven Pile, eds. Geographies of Resistance. Routledge, 1997. Kim-Cohen, Seth, “The Sound Canon of Samson Young,” Catalogue Text for Samson Young’s Songs for Disaster Relief, Hong Kong Pavilion, 57th Biennale di Venezia. https://kim-cohen.com/wpcontent/uploads/2020/07/2017_Kim-Cohen_The-Sound-Canon-of-SamsonYoung.pdf. Accessed on June 10, 2023.

Marcus, Greil, Lipstick Traces: A Secret History of the Twentieth Century Cambridge, Mass: Belknap Press of Harvard University Press, 2009.

Massey, Doreen. Space, Place, and Gender, 1st ed. Univ. Of Minnesota Press, 1994.

Pussy Riot Pussy Riot!: A Punk Prayer For Freedom The Feminist Press at CUNY, 2012.

Shuker, Roy. Popular Music: The Key Concepts, 2nd ed. London/New York: Routledge, 2005.

Straw, Will. “Systems of Articulation, Logics of Change: Communities and Scenes in Popular Music,” Cultural Studies Vol. 5, Iss. 3 (1991).

Whiteley, Sheila, Andy Bennett, and Stan Hawkins, eds. Music, Space and Place: Popular Music and Cultural Identity, Ashgate Popular and Folk Music Series Aldershot: Ashgate, 2005.

Sound Design for Electric Vehicles

Fulfilling Laws and Creating Digital Artwork

Abstract

Over the last decades, the sound of vehicles with internal combustion engines has been developed to perfection, so that car enthusiasts are not only able to hear the differences between car manufactures but also between different engine configurations and exhaust styles. Even people that do not know all the technical details about different engine types will most likely be more aware of approaching cars with a flat-6 engine or legendary V8 engine. Vehicle sounds are experienced both consciously and unconsciously and have a huge impact on the driving emotions, as well as the overall customer experience. Most car manufactures are aiming for a complete transition to an all-electric fleet in the near future and already started the transformation process. Well known structures and all the know-how of the past decades cannot be used anymore to create the unique sound branding of each car manufacturer. This is why acoustic engineers and sound designers are confronted with an almost complete empty spectrogram and need to create a full story for all frequencies and in each driving condition of the electric vehicle. The design process starts very similar to a music producer sitting in silence who uses his imagination to bring ideas into life. This article reflects first the sound design methods of the last decades to create unique sounds of vehicles with internal combustion engines in order to take into account the heritage of customer experience. From this perspective, the challenges for electric vehicle (EV) sound designers are derived in order to not only comply with regulatory requirements for pedestrian safety and homologation but also to create pleasant, emotional and unforgettable soundscapes For this purpose, technical toolchains are shown that have been combined with creative approaches in order to build a productive environment for turning the sound of electric vehicles into digital artwork.

Introduction

On all markets worldwide an increased demand for electric vehicles can be noticed. For an improved product differentiation and to fulfill the customer demands, the variety of models is increasing massively. By 2025, over 500 new electric models are expected to reach the markets. Vehicle manufacturers are therefore also increasing the effort for the product sound design process. While new manufacturers like Tesla, Rivian, NIO or BYD arise with purely electric fleets, traditional manufacturers are changing their vehicle portfolio more and more to full electric vehicles. Acoustic and sound engineers therefore are facing the challenge of maintaining an auditory brand identity or creating an entirely new brand identity for the electric area during the sound design process. Since the original noise emitted by electric vehicles does not create sufficient sound pressure levels in the lower vehicle speed range for a guaranteed audibility, regulations have been introduced that require additional sound emitted to the exterior with Acoustic Vehicle Alerting Systems (AVAS). Starting from July 2021, every newly registered electric vehicle needs to be equipped with such a warning system in order to be street legal. This leads to different company strategies that use the additional sound requirements to either just fulfill the legal requirements or to use the systems also for product differentiation, creating an auditory brand identity and to increase the driving experience in terms of emotions. Finding the right sound concept for different vehicle types can be a challenging task. Comparing e.g. an electric small car to an electric SUV just by its visual representation leads directly to different expectations regarding the sound concepts of both those vehicles. In order to design the soundscape for a variety of vehicle types, new approaches have to be considered during the sound design process in the vehicle development.

Sound of Combustion Engines: The Art of the 0.5th Order

Before considering the sound design process of electric vehicles, it is important to understand the mechanisms of sound generated by vehicles with internal combustion engines (ICE). For the past decades these sound characteristics have influenced the costumer experience. By using advanced engineering techniques, the sound of ICE can be modified in a wide range from calm and comfortable to loud and aggressive. Depending on the costumer expectation and the brand identity, the air-intake, exhaust und combustion process itself need be developed accordingly to achieve the required goals. The main ingredient in this sound design process is the 0.5th engine order. Since the majority of ICE is operated in a fourstroke mode (intake of new gas – compression – ignition – blow off burned gas), only one ignition with a rapid pressure increase takes place over two revolutions of the crankshaft in a full four-stroke cycle which leads to 0.5 combustions per revolution and therefore to a dominant 0.5th engine order as a dominant sound excitation This periodic pattern resembles a beat pattern with a specific BPM value controlled by the revolution speed of the ICE. Since combustion engines usually have more than one cylinder, the 0.5th order of each cylinder is superimposed in the flow rate of the intake and the exhaust system. This leads to the conclusion, that the dominant engine order and the corresponding sound frequency of a four-stroke ICE is generated by half the number of cylinders. One common approach to analyze the sound composition of a vehicle is to conduct engine run-ups and measure the sound e.g., at the tailpipes. The frequency spectrum of such a measurement can be seen in Figure 1.

The spectrogram shows clearly, that the sound of an ICE does not consist of only one single frequency. Although the dominant engine order has the most energy in terms of Sound Pressure Level (SPL), a wide range of engine orders with a distance of 0.5 engine orders is created by burning fuel in multiple cylinders. This effect is caused by two reasons. Since the combustion process itself is a rapid pressure increase, the waveform cannot be described with a single and pure sine wave. This leads to an excitation of harmonic frequencies based on the dominant engine order. Secondly, asymmetries in both intake and exhaust lead to a small phase shift of the 0.5th order of each cylinder, so that not only the integer multiples but also the orders in between can be observed in the pictured frequency spectrum. The ratio of this order mix is designed during the development of new vehicles and can be used to emphasize psychoacoustic effects, such as modulation or roughness, which lead in general to a sportier perception of vehicle sounds. Especially modulation in the range from 10 Hz to 30 Hz is perceived as an emphasizing roughness and is created by the frequency distance in between the single engine orders.

Sound of Combustion Engines and Electric Motors

While combustion engines produce a significant, tonal auditory spectrum in terms of engine orders, electric motors are in general quieter and produce more high frequent noise. Because of the higher frequency range, the sound of electric motors is usually perceived as more annoying. With the physiological structure of the human ear, the frequency range from 1.000 Hz to 5.000 Hz is very sensitive to sound signals and from the sound design point view it is very hard to achieve a good sound quality in this high note area without sufficient bass notes and mid notes.

With proper mechanical engineering, the noise of the electric powertrain is therefore reduced to a minimum and the total noise of electric vehicles can be summarized in terms of road noise and wind noise, which can be described as random noise signals without a clear tonal structure. The spectrum therefore does not have a clear character and the challenge for sound designers is to create the entire storyline for the sound perception in electric vehicles. For this reason it is helpful to highlight some of the key differences between the sound of combustion engines and electric motors. One of the most important features of a combustion engine is the integration of a gearbox with 5 to 8 gears. Although this aspect may seem to be trivial, a gearbox has a significant impact on the perceived sound.

When the gear is shifted up during acceleration not only the revolution speed is decreased but also the resulting sound frequencies of the corresponding engine orders and the sound spectrum pitches down in the specific ratio of the gear change Compared to this, electric vehicles do not need a variable gearbox in general and in most cases have only one fixed gear ratio between the electric motor and the wheels. For the sound design process therefore a decision has to be made, whether the EV sound concept should be linked only to the continuous engine speed or whether the sound should be added an option for shifting to low frequencies while accelerating. Considering the fact that acoustic signals also carry auditory information, increasing frequencies tend to cause the feeling that the driver wants to shift to the next gear, when the frequency of the sound gets into the higher frequency range. This condition can be observed with a lot of drivers of ICE vehicles that do not look at the visual display of the revolution speed but simply judge on the gear selection by listening to the sound of the combustion engine. In order to recreate this natural feeling for an EV sound design, the sound cannot just be linked directly to the revolution speed of the electric motor. Two of the most common techniques used in current EV are introducing single sound effects with lower frequencies in the higher revolution speed range or even creating a virtual gearbox that shifts the sound spectrum with an entire logic improved for the sound spectrum. In order to emphasize the shifting effect of the virtual gearbox even more, the shifting effect does not only affect the sound concept but also the entire powertrain can be excited with a short impulse to recreate the well-known shifting effect of vehicles with combustion engines. Another important difference that needs to be addressed during the sound design process is the stationary sound during standstill. Since electric powertrains do not have the direct need for a gearbox, a dedicated clutch to open the connection between the electric motor and the tires is also not needed. A vehicle speed of 0 km/h therefore corresponds to a revolution speed of 0 rpm in EV, whereas combustion engines usually have an idle speed in between 600 rpm and 800 rpm during standstill. Considering the engine orders e.g., of a six cylinder engine, the dominant sound frequency of the 3rd engine order lies in an area of 30 Hz to 40 Hz. The EV sound concept therefore also needs to consider a strategy how to compensate the lack of revolution speed during standstill and in which frequency range the stationary sound should be designed. Choosing the sound frequency to high can lead to an unrelaxed impression of the sound experience based on the customer expectation. Translating the natural frequencies of combustion engines to an electric sound concept leads to the conclusion that especially the contra octave and great octave should be used to recreate the natural sound of an idling machine.

Noise Regulations for Electric Vehicles

With the introduction of commercial vehicles with electric powertrains in the year 2010, worldwide associations for the blind and visually impaired pointed out the potential danger of EV in the lower speed range. Due to the lack of road noise underneath 30 km/h and wind noise underneath 100 km/h, the powertrain is the only noise source that can potentially emit noise while accelerating the vehicle from a standstill condition. As mentioned above, electric motors are much quieter in this scenario and worldwide working groups have developed regulations for the minimum sound requirements for EV that have been introduced in recent years:

§ UN R138 for European markets, as well as non-EU markets like Australia, South Korea or South Africa.

§ FMVSS 141 for the US market.

Considering the regulation for the European market, electric vehicles have to meet requirements regarding the minimum sound pressure level, the frequency content and the frequency characteristics in terms of a pitch shift of at least 0.8 % per km/h in the speed range from 5 to 20 km/h. However, the US market deviates from the specifications outlined in UN R138 and follows different criteria under FMVSS 141. Key aspects of the US requirements that differ from UN R138 are the speed range in which a hearable sound is required and the frequency range for additional noise emissions. Not only does the US market require a mandatory sound during standstill when the car is ready to drive but also the maximum test speed is increased to 31 km/h. Another interesting requirement is the fact that the US regulation does not require a mandatory frequency pitch shift but a volume increase of at least 3 dB at an increment of 10 km/h. Since the frequency requirements have a significant impact on the sound development of AVAS, it becomes crucial to compare and analyze the contrasting regulations (Figure 2) to gain insights and understand the specific requirements of each regulation.

Both regulations postulate their frequency requirements in terms of sound pressure level requirements in specific 3rd octave bands. Considering UN R138, two 3rd octave bands in the frequency range from 160 Hz to 5000 Hz have to fulfill the illustrated minimum values of the black lines in the blue box. One or both of these two 3rd octave bands has to be equal or underneath the 1.600 Hz frequency band. The regulation FMVSS 141 has taken another approach, which can be seen in the gray lines in the green box. Sound designers are able to choose between a 2-band method in the frequency range from 315 Hz to 3150 Hz (dashed line) or a 4-band method in the frequency range from 315 Hz to 5000 Hz (solid line). As an important difference it has to be mentioned, that in both 2-band and 4-band method the US regulation demands at least one 3rd octave band to be compliant equal or above 1000 Hz. Compared to this, the UN regulation can be fulfilled with the two lowest possible frequency bands. While not mandated by law, it is still possible to incorporate sound components in the lower frequency range below 160 Hz. However, this often introduces conflicts in terms of the required speaker type and speaker enclosure volume, posing challenges in achieving the desired sound quality. For the sound development therefore an important decision has to be made: Should an EV be equipped with one sound for all worldwide markets, or should the sound be adopted to the corresponding market requirements?

EV Sound Design: Acoustic Vehicle Alerting Systems and

Emotionalization

Now that all requirements are known, the goal of the sound design process needs to be defined. For example, clarifying the sound characteristics with a semantic differential and contrary adjective pairs (such as “cool and warm,” “synthesized and natural” or “understated and overt”) helps in the early sound design stage to identify the direction of the entire concept. With the help of an audio-framework, control data such as engine rpm, vehicle speed or engine torque are converted to sound signals for both exterior sound and interior sound of the vehicle. The complexity of the audio-framework has to be adjusted to the desired methods of the sound designers and the available resources on the sound control unit. In this context, two core strategies for the sound generation itself can be seen in current series vehicles: Building the audio completely based on regeneration/oscillators or using sound recordings of existing objects. Although the measurement of different engines, powertrains or in general sound sources is usually very time consuming, this methodology delivers the best organic sound material. In most cases, even in short recordings there is sufficient fluctuation to create very organic and natural sounding concepts. After defining the core of the audio-framework, additional parameters like mixers, adjustable filters, parametric gains and effects are all combined to create the entire soundscape. The sound design process therefore does not only create a sound concept for this specific electric vehicle, but turns it into an entire instrument, which the driver is playing with his hands and feet.

One common problem in the vehicle development and especially in the vehicle sound development is the availability of prototypes in the early development stage, because the sound is vehicle function that needs to be experienced in a full vehicle environment. Starting the sound design process purely in digital audio workstations often leads to conflicts when the sound concepts need to be transferred to automotive capable hard- and software with reduced computational power. In order to solve this problem, vehicle manufactures are using more and more virtual methods to improve the development process. In terms of acoustics and sound design, having a drivable vehicle model in a simulator (Figure 3) can solve this issue and the sound concepts can be developed early on.

When sound is not only considered as a warning signal for pedestrians but also as a driving sound to create emotions, a full motion simulator enhances the virtual development methods in order to experience the acoustic functionality as close as possible to the final setup in the desired vehicle. As it can be seen in this version of the driving simulator by the University of Applied Science and Arts Dortmund, all vehicle controls such as steering wheel, pedals, buttons and the resulting vehicle parameters are linked directly to the audio-framework in order to control the sound environment. Additional Midi-controller enable a fast tuning of the single sound components without further need of detailed work on a PC, so that the sound design process can focus on the best way possible on driving, experiencing and tuning the sound accordingly. Comparing the sound design process of EV to musical artwork, the necessity for an intuitive development tool becomes even more clear: Whereas in musical production a clear structure of tones and semitones can be used as a baseline or a starting point, the pitch of EV sounds and the corresponding starting point (sound at standstill) can be defined completely free and in-between predefined note steps. With increasing pitch shifts of the base-sound, harmonics and non-harmonic overtones can be combined in endless possibilities, so that the soundscape has to be designed and evaluated in each possible condition before the EV and the sound is handed to the customer in the final step.

Conclusion

In conclusion, the increasing demand for electric vehicles worldwide has led to a surge in the variety of electric models in all vehicle categories from small city cars to large SUV or Pick-up trucks or electric supercars. As traditional manufacturers transition their vehicle portfolios to include more electric vehicles, the sound design process has become a crucial aspect for maintaining or creating brand identity in the electric vehicle sector. Regulations have also been introduced to ensure audibility and safety, requiring electric vehicles to be equipped with Acoustic Vehicle Alerting Systems (AVAS). Designing the soundscape for electric vehicles poses unique challenges compared to combustion engines due to the quieter nature of electric motors. While combustion engines produce tonal auditory spectra with the dominant 0.5th engine order, electric motors produce more high-frequency noise, which can be perceived as annoying. Sound designers must consider these differences and find ways to create a compelling sound experience for electric vehicles. Additionally, regulations such as UN R138 and FMVSS 141 outline minimum sound requirements for electric vehicles, which sound designers must adhere to. One of the key aspect of the sound design process for electric vehicles involves therefore the development of an audioframework that converts control data into sound signals in order to create a comprehensive soundscape. Virtual methods, such as driving simulators, have become increasingly valuable for early-stage sound development, allowing designers to experience the acoustic functionality closely resembling the final setup. Overall, the sound design process for electric vehicles is a crucial factor in creating an auditory brand identity, ensuring safety, and enhancing the driving experience by evoking emotions. After the first models with mandatory exterior sound have been introduced in the last couple of years, it will be interesting to see how the customer expectation towards the sound differentiation will develop and if the sound of an EV will become an important factor in the buying decision in the near future. In this case, the sound design process will be intensified among all vehicle brands even more because of the increased competition and more complex sound concepts will be expected in the next EV models.

Author Biography

Alessandro Fortino was born in Bonn (Germany) in the year 1984. After completing his mechanical engineering studies in automotive engineering at the RWTH Aachen University, he continued his work on active acoustic systems in the field of active sound design and active noise/vibration control to obtain his doctoral degree. Combining his passion for mechanical engineering and music production not only led to his work as a scientific employee in the acoustics department of the Institute of Automotive Engineering, but also to his following employment as a Sound Designer for Porsche. After bringing multiple new vehicle sound concepts into series production, along with the allelectric Porsche Taycan, he started his professorship at the University of Applied Science and Art Dortmund in 2021 in order to establish a new research and education field for vehicle acoustics in the Ruhr area.

Bibliography

Bodden, M., et al. “Active Sound Design for High Performance Electric Vehicles.” Aachen Acoustics Colloquium, 2022.

Fortino, Alessandro, et al. “Acoustic Vehicle Alerting Systems (AVAS)regulations, realization and sound design challenges.” 9th International Styrian Noise, Vibration & Harshness Congress. Graz, 2016.

Fortino, Alessandro, et al. “Sound design and implementation methods for synthetic driving sounds in electric vehicles.” Aachener Acoustics Colloquium, 2014

Genuit, Klaus, et. al. “Sound-Engineering im Automobilbereich ” Wiesbaden: Springer-Verlag, 2010

Moon, Soyoun, et al. “Active Sound Design Development Based on the Harmonics of Main Order From Engine Sound.” Journal of the Audio Engineering Society. Audio Engineering Society, 2020.

Penne, Filip. “Shaping the sound of the next-generation BMW”. International Conference of Noise and Vibration Engineering (ISMA). Leuven, 2004

Suffield, Iain. “What Sound Should an Electric Car Make?” Internect Automotive Acoustics and Vibration. Manchester, 2017.

United Nations. „UN Regulation No. 138 - Uniform provisions concerning the approval of Quiet Road Transport Vehicles with regard to their reduced audibility,” 2017.

U.S. Department of Transportation. “FMVSS 141 - Minimum Sound Requirements for Hybrid and Electric ” Washington DC, 2020.

Voice in the Machine

AI Voice Cloning in Film

Abstract

In 2021 Morgan Neville’s film Roadrunner: A Film About Anthony Bourdain created some controversy when it was revealed that three of the lines we hear Bourdain say in the film were never spoken by him (although he had written them) but were instead AI generated. This shone a light on the rapidly developing field of voice cloning and its use in filmmaking. The voice, and the body that creates it, has an interesting history in cinema. For its first decades the body on screen was mute, the sound of its voice had to be imagined in the mind’s ear and the meaning of the unheard words derived from intertitles. The unique quality of a character’s voice is often one of their defining features and actors often devote considerable time in “finding” the voice of the character that they will create. This voice is linked to the body, to its movements, so replacing this voice can fundamentally change how we perceive the character. Replacing an actors on set voice performance with one they have recorded after the capturing of the image track is extremely common. In some cases, a majority of a film’s production dialog has been replaced and the audience is usually unaware of this. Recording, or dubbing a different actors voice to create a new language version of a film is in many countries the common way to experience foreign films. Both processes require time and money and are ones that AI companies are seeking to disrupt. This paper examines AI voice cloning and its developing impact on the voice in film and its possible future impacts on film production with specific reference to automated dialog replacement (ADR) and film localisation.

Introduction

In 2021 Morgan Neville’s film Roadrunner: A Film About Anthony Bourdain created some controversy when it was revealed that three of the lines we hear Bourdain say in the film were never spoken by him (although he had written them) but were instead AI generated. That this was not revealed to audiences provoked outrage in some, and questions about the ethics of the filmmaker 1 It even garnered discussion on US prime-time news (generating significant publicity for the film) and shone a light on the rapidly developing field of voice cloning and its use in filmmaking. This paper examines AI voice cloning and its developing impact on film sound and its possible future impacts.

The voice, and the body that creates it, has an interesting history in cinema. For its first decades the body on screen was mute, the sound of its voice had to be imagined in the mind’s ear and the meaning of the unheard words derived from intertitles. One notable exception was the live narrator or “benshi” of the Japanese silent film era 2 While an offscreen voice was present, the on-screen body remained voiceless and its true nature unknowable by the audience.

When sound and the moving image finally became reliably synchronised it heralded the age of the “talkies” and finally an audience would hear words when they saw an actor talk. The body had a voice, and the voice immediately claimed its central role in a film’s soundtrack 3 These first synchronised words, uttered by Al Jonson between musical numbers in The Jazz Singer (1927), are the beginning of a fascinating journey and ever-changing relationship between the voice, the body and moving images that continues to evolve today.

For most, it was the first time they heard Jonson speak and they would have rightly assumed that the voice they heard did indeed belong to him and so was “his” voice. The unique characteristics of Jonson’s voice, as mediated as they were by the recording and reproduction process, were now able to be associated with his body, his being. How an actor sounded was almost as important as how they looked and quickly became an identifying feature of the movie star, something amusingly explored in Gene Kelly and Stanley Donen’s classic Singing in the Rain from 1952

The unique sonic characteristics that distinguish one voice from another can be divided into spectral content, the frequencies present in the sound and linguistic prosody, how the sounds are organised into speech and the subtleties in how words are formed into sentences. The use of inflection, emphasis, pauses, tempo, and intonation are also some of the characteristics 4 How a voice sounds is derived from subtle variations in the size and shape of the vocal tract. Qualities such as relative pitch and timbre are products of the vocal folds in the larynx that vibrate to produce a rich mix of frequencies known as the frequency spectrum. Some of these are attenuated or resonated as they pass through the vocal tract. The sound of a voice can be modified by changes to these structures; a blocked nose for instance has a noticeable effect on the timbre of a voice as the spectral content is altered. The voice is the product and an expression of the body that contains it and changes and ages as the body does.

The voice that is heard when we listen to a film has undergone transformations that are product of the film making process. In the production stage the actors’ words and other sounds propagate through air and cause the diaphragm of an adjacent microphone to vibrate and generate an analogue of the soundwave it received. This process of transduction transforms acoustic energy into an electrical signal that is converted into a digital signal for storage and later manipulation. Further transformations occur in the postproduction process, often including the extensive editing together of multiple versions of a performance that were

recorded over many hours or even days of production to create the final dialog track. Commonly words, lines or scenes might have been replaced with new recordings performed by the actor in a studio, perhaps months after the image track was recorded. Known as automatic dialog replacement (ADR) or “looping” this ubiquitous technique has a long history and is used extensively. Finally, the sound of the actor’s voice is modified using equalisation and other processes and mixed with other sounds to create what is then reproduced by the speaker system in the theatre where the audience experiences the film. What began as a voice propagating through air to a microphone, ends with a voice propagating through air from a loudspeaker to the audience. The audience is blissfully unaware of a microphone, editing, or any of the other transformations and are convinced that the voice they hear emanates from body they see on screen and not the loudspeakers in the cinema.5

AI and the Voice

There is a somewhat surprisingly long history of machines designed to synthesise the human voice. In the 18th Century, Hungarian Wolfgang von Kempelen invented a mechanical “Speaking Machine”6 and Joseph Faber demonstrated a “machine that could talk” in 1845 7 The Voder developed by Homer Dudly (also the developer of the Vocoder) at Bell labs created quite an impact at the 1939 World’s fair, and was an early form of electronic speech synthesiser but one that require a good deal of human speech input 8 Computerised synthetic speech developed with advancements in computing and mathematical models but it wasn’t until the development of neural networks and deep learning that the full possibility of voice cloning, as we now know it, became achievable. In 2016 Adobe previewed Project Voco their “Photoshop of speech” a text to speech voice model that was met with some concern and never commercially released.9 Project Voco, along with Deep Mind’s “Wavenet,”10 were forerunners of today’s AI voice synthesis/cloning systems.

In simple terms, a voice is cloned by training a machine-learning algorithm, most often a deep neural network, using existing audio recordings to learn the unique characteristics of a voice. This process, known as feature extraction, captures pitch information, spectral envelope (frequencies present in the voice) and duration of phonetic units and other characteristics. From this a digital voice model is created. The greater the quantity and quality of the audio recording, the better the voice model will typically be, with the ultimate goal of creating a model that

can generate a copy of the voice that is indistinguishable from the original. New speech is generated via text input, or an existing voice recording can be used as the input. In that case the AI voice is cloned on to the existing recording. Numerous companies such as Resemble AI, Respeecher, Sonantic and Descript offer AI voice cloning via online platforms, with new ones entering the market frequently.

The film Roadrunner not only featured the cloning of Anthony Bourdain’s voice but also serves as one example among several in which the voices of deceased individuals have been revived. In the documentary The Andy Warhol Diaries, directed by Andrew Rossi (2022), Warhol’s cloned voice is heard, while in the short film component of the installation In the Event of Moon Disaster, 11 a production of MIT’s Center for Advanced Virtuality, President Richard Nixon’s voice is similarly revived.

In the case of Anthony Bourdain and Andy Warhol, their AI-generated voices were utilized as voice-overs and generated by text input. Consequently, the audience does not witness the physical presence of the individuals they associate with the sound they hear. On the other hand, for Nixon, an existing video recording was modified to synchronize his mouth and facial movements with the AI-generated voice cloned to the performance of a voice actor

A discussion of the ethics regarding the recreation of the voice of someone who has not, or is unable, to give consent is beyond the scope of this essay but is one that is far reaching So called deepfake videos have become ever easier to create and their sophistication and realism make them harder to identify, something that the aforementioned In the Event of Moon Disaster explores 12

AI- Automated Dialog Replacement (ADR)

Replacing an actors on set voice performance with one they have recorded after the capturing of the image track is extremely common. In some cases, a majority of a film’s production dialog has been replaced and the audience is usually unaware of this. While there are many reasons that ADR might be required, one of the most common is the inability to record dialog in production that is noise free enough to be usable (even after traditional noise reduction is applied) in the sound mix of the film. ADR (also called “looping”) is a postproduction process that might take place weeks or months after the production stage has been

completed. Very simply the actor repeats their performance in the controlled environment of a sound studio while being recorded by an ADR recordist using appropriate microphones. The image track is projected in the studio and the actor attempts to perform in sync with it while also being able to hear the original production recording. ADR editors then perform the often-laborious task of editing together multiple takes, making sure the new sound recording synchronises tightly with the image track and the performance matches, or improves on, the desired characteristics of the original performance and matches acoustically the visual world depicted in the image track. This process is time consuming and adds expense to the film production process.

Directors and actors typically dislike ADR, and it has always been seen as a necessary evil. Recreating a performance in a studio, especially dramatic emotional scenes, can be extremely challenging for an actor achieve, since they may be asked to reinhabit a character they had exited months prior.

AI has already begun to disrupt this process by, in some cases, removing the need for the actor to re-record their original performance. The actor’s voice can be cloned from existing recordings and then used to generate a replacement of the original performance, with that original performance as the input for the voice model. The performance characteristics (intonation, inflection, tempo etc) of the input are then attempted to be reproduced in the synthesised output of the voice model. As voice model algorithms improve, the more accurate and natural the synthesised voice should sound The voice model can be adjusted to give differing renderings of the original performance in case performance changes are desired, giving the director a chance to change the character of the original performance (as in traditional ADR).

It’s worth noting that AI voice cloning is offered by vendors as an on-line service and not as a stand-alone product that a post-production facility can purchase. Production companies will upload voice samples to a company like Respeecher who then create the voice model, which is cloned onto an existing dialog performance (from any actor). The final output is returned to be incorporated into the dialog edit of the film. This represents a new stage in the postproduction process, utilizing new technologies and skills whose origins originate outside of the creative arts. As this process becomes more commonplace, it is inevitable that the roles and practices of dialog editors, ADR recordists, and ADR editors will undergo significant changes.

The increasing sophistication of the voice models allow for ever greater amounts of manipulation of their output, and it follows that the resultant rendered performance could be significantly different from the original. The normal process of editing an actor’s performance from multiple takes has always afforded opportunities to adjust tone, pacing and other parameters, However, the final version is still created from the performance(s) of the actor we see on screen. When using voice cloning this doesn’t need to be the case; the input for the voice model could be the performance of a different actor. In this case the output of the model will sound like the original actor, but the performance is not theirs. This invites interesting questions about the authorship of a screen performance. Does the actor truly become Altman’s ventriloquist’s dummy?13

The recently concluded 118 day screen actors strike in the United States was prolonged in part because of the complicated issue of AI as the Screen Actors Guild of America sought contract provisions aligned with their statement from March 2023:

The terms and conditions involving rights to digitally simulate a performer to create new performances must be bargained with the union. In addition, any use or reuse of recorded performances is limited by our collectively bargained contract provisions, including those requiring consent and negotiation of compensation.14

Actors who are not members of a union, especially those in the lower echelons of the industry, will need to be careful that contracts they sign address the use of AI voice cloning and to ensure their voice is only used in the manner they approve. This is a serious issue for voice actors, with some finding their voices had been cloned and sold on AI text to speech websites without compensation or their consent 15 Others report being asked to allow cloning of their voice, sometimes without appropriate contractual protection or compensation 16 The extraordinarily rapid advances in AI require advances in how we negotiate the use of the exciting new capabilities they afford and protect actors’ (and other artists’) rights to determine how their voice is used.

AI – Voice De-aging

Like our bodies, our voices age with the timbre and fundamental frequency changing, as well as other elements of speech.17 The most extreme change happens in male puberty, which presents obvious challenges when casting pubescent actors. AI voice cloning, as already described, can be used to replace an actor’s current voice with one cloned from a model trained on recordings of their younger selves. This process became well known when Disney turned to the company “Respeecher” to bring back the youthful voice of the actor Mark Hamill when he reprised the role of Luke Skywalker in the series 2 season finale of The Mandalorian in 2020. While only a few lines of dialog, the result was quite convincing for what it was, and arguably as convincing the AI face replacement that was also employed. This technique was also used in Black Adam (2022) to de-age the voice of the young actor Bodhi Sabongui who played the role of Amon Tomaz. Sabongui’s voice had changed after principal photography had concluded and a cloned version of his younger voice was used, notably in the introductory narration, and the important rally speech he gives prior to the climax of the film At the film’s premiere Sabongui was surprised to hear his cloned voice and had not heard it previously. These performances were not particularly well received by some audience members18 who were unaware that the performance they are responding to was the product of an AI voice model and not truly representative of the performance of the actor. It’s important to note that Sabongui was not afforded the opportunity to have input on what was created by the AI model even though this “performance” is presented as being authored by him.

AI-Localisation

It can be surprising to watch an English language film in a cinema in Germany for the first time and hear the dialog in German rather than the original. The oftendistinct timbre of a well-known actor has been replaced by that of a local actor. For example, voice actor Tobias Meister can be heard as the German voice of Brad Pitt and Sean Penn (among many others) in multiple films. Known as localisation or dubbing, it is a more expensive and time consuming alternative to subtitling and is traditionally preferred in some countries like France, Germany, Italy and Spain 19 The process begins with the original dialog being translated into the local language and adjusted to ensure it sounds natural and to help with synchronisation with the mouth movements of the original actor. A suitable local actor is cast, and a new dialog performance is recorded in a studio with the actor attempting to match the original performance, in a manner quite similar to ADR. Streaming giant Netflix dubs its content into over 30 languages as it sees audiences preferring this to subtitles 20

The unique quality of a character’s voice is often one of their defining features and actors often devote considerable time in “finding” the voice of the character that they will create. This voice is linked to the body, to its movements, posture, gait, and facial expressions, so replacing this voice can fundamentally change how we perceive the character. AI voice cloning offers a partial solution to this. As described earlier, the input to the voice model can be the performance of another actor, so the performance of the local actor can be used as this input and the resultant output will have timbral qualities of the original cloned actor, but now in the local language. It would be fascinating to hear how this technique could cope with very strong and idiosyncratic accents such as the now-iconic voice that Tom Hanks created for the adult Forrest Gump for the film of the same name. Hank’s Gump has a strong American Southern accent, a southern drawl with a distinct half swallowed and then regurgitated “G”. The slow, stilted and somewhat clumsy voice wonderfully matches the awkward body movements and countenance of the man-child we see on screen. When dubbed in the traditional way, Hank’s unique vocal performance is lost and with it an essential aspect of the character he created. Could current AI voice models create a foreign language Forrest Gump that maintains Hank’s Oscar winning performance?

Naturally, the movements of the mouth and lips used for words in the original language may not align seamlessly with the dubbed language, resulting in noticeable discrepancies between what is seen and heard. However, the advent of AI image generation holds the potential to address this issue by adjusting the actors’ lip movements to correspond with the new language, thus achieving accurate dialogue and image synchronization. Deep Mind demonstrated this capability (Yang et al. 2020) and the AI company Flawless (co-founded by Director Scott Mann) assert their ability to deliver “perfect lipsync in any language ”21 Demos showcasing this technology are quite impressive and it was used to replace f-words in the originally R-rated 2022 thriller Fall, directed by the aforementioned Scott Mann, so that it received a more audience-friendly PG-13 rating 22 This isn’t an example of localisation as there was no change in language and the new sound recordings were created using traditional ADR techniques.

Text to speech conversion doesn’t require a local voice actor since the voice model generates its output from written text. This has the potential to remove a costly step in the dubbing process by taking away the need to make a new recording in the local language. In 2021 it was announced AI dubbing company Deepdub would localise the independent English language feature film Every Time I Die (2019) into Brazilian Portuguese and Latin American Spanish using this technique.23 It is not clear if this was successful or even completed as no mention can be found on the company website and inquires to them and the film’s production company have gone unanswered.

Addressing the significant challenge of the art of language translation, and the complexities of cultural context is beyond the scope of this paper; however, it is important to acknowledge that it encompasses a fundamental human element in the localization process, which would pose a formidable hurdle for an AI system to replicate.

Looking Forward - Disruptions

The implications of AI voice cloning on the voice in film are wide ranging. One can imagine that before long nearly all ADR will employ AI voice cloning to replace the traditional recording process in some form. This assumes that the output from AI voice models is indistinguishable from an actual voice recording and is more economical than traditional methods. The “R” in ADR will disappear, replaced by a synthesised clone from the machine. Air pressure causing a microphone’s diaphragm to vibrate will no longer be the first stage in the journey of the voice to the cinema. The voice becomes truly disembodied.

With localisation the race is very much on to create a solution that would automatically create almost unlimited new language versions of visual media content, with flawlessly cloned voices that convey and translate the original performance faithfully and are precisely and naturally lip-synced with the original image. The ultimate deepfake if you will, except one that is created (hopefully) with the consent of all parties. The rapid progress in this technology suggests that such a solution is not so far away, but is this a good thing?

In the future we might see AI localisation be able to be automatically applied to any content on any device in real time Conceivably, it might be possible to never hear a foreign language unless one happened to travel to a new country or encounter visiting tourists in their own. Instead of requiring subtitles, film festivals could require all films to be dubbed into the local language. Will there need to be a foreign language category in the Oscars if there is a way to perfectly dub any film into English?

Taken to the extreme, AI localisation could create versions of regional accents, something that was foreshadowed when George Miller’s Mad Max (1979) was dubbed from Australian English into American English for its release in the US.24 It is not inconceivable to envision a future where individuals are exclusively exposed to their own dialect within their native language. This homogenization of language, tailored to increasingly specific audience groups, could have profound social ramifications with far-reaching consequences. One would hope this extreme is unlikely, given the importance of the unique cultural identity and linguistic heritage that different accents and dialects represent. Something that is essential in the creation of textured, believable, and interesting characters.

The absence of a language barrier has the potential to inundate local markets with foreign content, potentially displacing locally produced offerings. It is conceivable that large media companies will further expand their reach by creating local language versions for even smaller markets. This raises an important question: Will smaller productions have access to this technology and find it economically viable to employ? If AI dubbing leads to a decline in audience tolerance for subtitles, will it diminish the viewership of films that cannot afford AI dubbing? It is hoped that these new AI tools will be accessible to productions of all sizes and budgets, ensuring that access or the lack thereof does not become a barrier to reaching a wider audience.

The effect of AI voice cloning on production sound recording is interesting to consider. If AI voice cloning can generate a voice output that is an exact virtual recreation of the on-set performance, it is not difficult to imagine a scenario where production dialog recording is replaced with an AI clone as a matter of routine. A single wireless lavalliere will be all that will be needed to capture a suitable guide track for the AI voice to be cloned on to. The services of a boom operator, someone who places a directional microphone in the optimal position to capture natural sounding dialog, might no longer be required.

AI voice cloning is an essential element in the creation of digital doubles, such as the virtual Peter Cushing, who passed away in 1994, that we see as Grand Moff Tarkin in Rogue One: A Star Wars Story (2016). Similarly, the legendary voice of Darth Vader, famously brought to life by James Earl Jones, has been successfully cloned with his consent by Respeecher, opening up possibilities for its implementation in upcoming projects 25 It is a fitting coincidence that the character of Darth Vader, being a fusion of man and machine, will have his future vocalization achieved through the cloning capabilities of AI.

AI voice cloning, like many other AI technologies, presents exciting and disruptive opportunities for the how the voice is rendered in film. Like all technologies it has the potential for misuse, but if used in the service of creativity, offers tools to do what was once impossible to consider. As with the dramatic improvements in the fidelity and believability of visual effects, similar improvements should be expected with voice cloning such that it will eventually become impossible to discern the difference between the original and the clone with the naked ear. It is important to acknowledge the ethical implications of this technology. Safeguards and guidelines must be put in place to ensure responsible use and protect individuals’ rights. Striking a balance between the possibilities of AI voice cloning and ethical considerations is crucial for its continued development and acceptance in the film industry.

Time will tell if future screens will be populated with digital doubles speaking with their cloned voices. Ghostly marionettes, ventriloquists’ dummies that never saw a camera or utter sounds received by a microphone

Author Biography

Composer and sound designer Ross Adrian Williams has written music and designed sound across a range of styles for theatre, film, concert hall, dance, museum installation, 360° video and VR. His work for award winning abstract, documentary and narrative films have been shown in festivals, and installations around the world. As Assistant Professor of Sound Design at the School of Art Design and Media at Nanyang Technological University, his research focuses primarily on sound and music for moving images, particularly the aesthetics of sound and impact of sound design on visual memory in narrative film.

Email: rawilliams@ntu.edu.sg ORCID: 0000-0001-5218-0777

Notes

1 Rosner, “The Ethics of a Deepfake Anthony Bourdain Voice.”

2 Dym, “Benshi and the Introduction of Motion Pictures to Japan.”

3 Chion, The Voice in Cinema

4 Belyk and Brown, “Perception of Affective and Linguistic Prosody.”

5 Altman, “Moving Lips.”

6 Dudley and Tarnoczy, “The Speaking Machine of Wolfgang von Kempelen.”

7 Katz and Assmann, The Routledge Handbook of Phonetics

8 Gold, Morgan, and Ellis, Speech and Audio Signal Processing

9 “Adobe Voco ‘Photoshop-for-Voice’ Causes Concern.”

10 Oord et al., “WaveNet.”

11 Panetta and Burgund, “In Event of Moon Disaster.”

12 Panetta and Burgund.

13 Altman, “Moving Lips.”

14. “SAG-AFTRA Statement on the Use of Artificial Intelligence and Digital Doubles in Media and Entertainment.”

15 Colbert, “Your Favorite Voice Actors Call Out AI Sites Copying Voices Without Consent”; Murphy, “Voice Actors ‘Shocked’ to Find AI Copying Their Speech, Tone.”

16 Cox, “‘Disrespectful to the Craft.”

17 Rojas, Kefalianos, and Vogel, “How Does Our Voice Change as We Age?”

18 whtgnnd, “Why on Earth Did They Decide to Use This Super Bland Narrator in Black Adam?”

19 Esser, Smith, and Bernal-Merino, Media Across Borders

20 Roxborough, “Netflix’s Global Reach Sparks Dubbing Revolution.”

21 Flawless, “TrueSync - Using the Power of Generative AI to Visually Dub Film Content.”

22. Goldsmith, “Director Scott Mann’s AI Startup Helps ‘Fall’ Nab PG-13 Rating, $2.5M Open – Specialty Box Office.”

23. Gamerman, “The Rise of the Robo-Voices.”

24. Whittaker and Wright, Locating the Voice in Film.

25. Edwards, “Darth Vader’s Voice Will Be AI-Generated from Now On.”

Bibliography

Altman, Rick. “Moving Lips: Cinema as Ventriloquism.” Yale French Studies, no. 60 (1980): 67. https://doi.org/10.2307/2930005.

BBC News. “Adobe Voco ‘Photoshop-for-Voice’ Causes Concern.” November 7, 2016, sec. Technology. https://www.bbc.com/news/technology37899902.

Belyk, Michel, and Steven Brown. “Perception of Affective and Linguistic Prosody: An ALE Meta-Analysis of Neuroimaging Studies.” Social Cognitive and Affective Neuroscience 9, no. 9 (September 1, 2014): 1395–1403. https://doi.org/10.1093/scan/nst124.

Chion, Michel. The Voice in Cinema. New York: Columbia University Press, 1999.

Colbert, Isaiah. “Your Favorite Voice Actors Call Out AI Sites Copying Voices Without Consent.” Kotaku, February 13, 2023. https://kotaku.com/aivoice-actors-steve-blum-elevenlabs-contracts-sell-1850108292.

Cox, Joseph. “‘Disrespectful to the Craft:’ Actors Say They’re Being Asked to Sign Away Their Voice to AI.” Vice (blog), February 7, 2023.

https://www.vice.com/en/article/5d37za/voice-actors-sign-away-rights-toartificial-intelligence.

Dudley, Homer, and T. H. Tarnoczy. “The Speaking Machine of Wolfgang von Kempelen.” The Journal of the Acoustical Society of America 22, no. 2 (March 1950): 151–66.

https://doi.org/10.1121/1.1906583.

Dym, Jeffrey A. “Benshi and the Introduction of Motion Pictures to Japan.” Monumenta Nipponica 55, no. 4 (2000): 509.

https://doi.org/10.2307/2668250.

Edwards, Benj. “Darth Vader’s Voice Will Be AI-Generated from Now On.” Ars Technica, September 26, 2022. https://arstechnica.com/informationtechnology/2022/09/james-earl-jones-signed-darth-vader-voice-rights-todisney-for-ai-use/.

Esser, Andrea, Iain Robert Smith, and Miguel Á. Bernal-Merino, eds. Media Across Borders. Routledge, 2016.

https://doi.org/10.4324/9781315749983.

Flawless. “TrueSync - Using the Power of Generative AI to Visually Dub Film Content.” Flawless, 2023.

https://www.flawlessai.com/tools.

Gamerman, Ellen. “The Rise of the Robo-Voices.” The Wall Street Journal, October 7, 2021.

https://www.wsj.com/articles/the-rise-of-the-robovoices-11633615201.

Gold, Bernard, Nelson Morgan, and Dan Ellis. Speech and Audio Signal Processing: Processing and Perception of Speech and Music. 2nd ed. Hoboken, N.J: Wiley, 2011.

Goldsmith, Jill. “Director Scott Mann’s AI Startup Helps ‘Fall’ Nab PG-13 Rating, $2.5M Open – Specialty Box Office.” Deadline (blog), August 14, 2022.

https://deadline.com/2022/08/director-scott-manns-ai-startup-helps-fallnab-pg-13-rating-2-5m-open-specialty-box-office-1235091858/.

Katz, William F., and Peter F. Assmann, eds. The Routledge Handbook of Phonetics. Routledge Handbooks in Linguistics. London/New York: Routledge, Taylor & Fancis Group, 2019.

Murphy, Margi. “Voice Actors ‘Shocked’ to Find AI Copying Their Speech, Tone.” Bloomberg.com, April 19, 2023.

https://www.bloomberg.com/news/newsletters/2023-04-19/voice-actorsshocked-to-find-ai-copying-their-speech-tone.

Oord, Aaron van den, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. “WaveNet: A Generative Model for Raw Audio,” 2016.

https://doi.org/10.48550/ARXIV.1609.03499.

Panetta, Francesca, and Halsey Burgund. “In Event of Moon Disaster.” In Event of Moon Disaster, 2020. https://moondisaster.org/about.

Rojas, Sandra, Elaina Kefalianos, and Adam Vogel. “How Does Our Voice Change as We Age? A Systematic Review and Meta-Analysis of Acoustic and Perceptual Voice Data From Healthy Adults Over 50 Years of Age.”

Journal of Speech, Language, and Hearing Research 63, no. 2 (February 26, 2020): 533–51.

https://doi.org/10.1044/2019_JSLHR-19-00099.

Rosner, Helen. “The Ethics of a Deepfake Anthony Bourdain Voice.” The New Yorker, July 17, 2021. https://www.newyorker.com/culture/annals-ofgastronomy/the-ethics-of-a-deepfake-anthony-bourdain-voice.

Roxborough, Scott. “Netflix’s Global Reach Sparks Dubbing Revolution: ‘The Public Demands It.’” The Hollywood Reporter (blog), August 13, 2019.

https://www.hollywoodreporter.com/tv/tv-news/netflix-s-global-reachsparks-dubbing-revolution-public-demands-it-1229761/.

“SAG-AFTRA Statement on the Use of Artificial Intelligence and Digital Doubles in Media and Entertainment.” Accessed April 25, 2023.

https://www.sagaftra.org/sag-aftra-statement-use-artificial-intelligenceand-digital-doubles-media-and-entertainment.

Whittaker, Tom, and Sarah Wright, eds. Locating the Voice in Film: Critical Approaches and Global Practices. New York: Oxford University Press, 2017. whtgnnd. “Why on Earth Did They Decide to Use This Super Bland Narrator in Black Adam?” Reddit Post. R/DC_Cinematic, November 17, 2022.

www.reddit.com/r/DC_Cinematic/comments/yy50bl/why_on_earth_did_t hey_decide_to_use_this_super/.

Electronic Music

Utopias and Realities1

Abstract

Cultural developments are often not continuous, but interrupted and erratic. They are not necessarily linked to technological developments. Nevertheless, it can be useful to examine the extent to which technological developments have influenced cultural developments (and perhaps vice versa). The discovery of electricity as a source of energy, the development of increasingly complex electronic devices, and not least the virtualisation of formerly physical machines into software, as well as the development of purely virtual machines (also known as programs), have each triggered innovative processes of upheaval in industrial, social and cultural contexts. It is to be expected that these innovations will also find aesthetic resonance. In the visual arts, for example, technology has repeatedly given rise to entirely new art forms. Think of photography, film, video art, light art, various forms of computer-generated art, etc. In the field of music, these technological innovations seem to have less influence on the development of aesthetic innovations, not least in musical composition. After a few decades in which it has been possible to speak of ”electronic music” in any context, we can see that these aesthetic resonances are certainly present, but perhaps not to the extent and perhaps not in the way one would expect. If one compares the use of these technologies with the utopian potential that the pioneers saw in each new stage of technological development, one is struck by the strong discrepancy between the claimed or supposed artistic revolution and the actual aesthetic practice. The article presents a very personal view of the respective technical innovations and their aesthetic potential, and attempts to reconcile them with the musical reality of their history.

In (European or European-influenced) music, significant technological developments, for example in instrument making, are rare even when viewed over centuries. The earliest finds of bone flutes are based on the same soundgenerating principle as our present-day recorders. Reed instruments have been known since antiquity, the musical bow (a hunting bow used as a musical instrument) is considered the oldest stringed instrument, the oldest images of which are thought to be on cave paintings dating back about 13,000 years. Our main instruments in the orchestra today have essentially existed since the 18th century and have not been fundamentally changed since then. In the 19th century, a few newer instruments were added (saxophones, Wagner tubas…).

In the 20th century, the percussion section of the orchestra was expanded, mostly with instruments that had existed for a long time but had not been used in the orchestra before. What influence does technology have then in musical genres whose self-definition is already based on technology, such as electronic music? It seems reasonable to consider the inventions of dynamos (generators), the electron tube (amplification), magnetic tape (sound storage) and the transistor (voltage control) as individual stages of development in each case, as well as the introduction of digital systems as the greatest paradigm shift in music technological development to date. However, there seems to be a certain inertia in thinking in terms of musical structures that in large parts still have an immanent instrumental character and are partly based on centuries-old patterns.

Dynamos

Electric charges and forces have been known since the 17th century. The first instruments to make use of them were, for example, the ”clavecin électrique,” a glockenspiel developed by Jean Baptiste de la Borde that used the attractive and repulsive forces of charged pendulums to strike individual bells and was controlled by a keyboard. Electromechanical pianos were invented by Matthäus Hipp in 1867 and Elisha Gray in 1876 (Ruschkowski 1998)

The instrument, that was first able to generate individual vibrations electromagnetically and combine them into more complex spectra, was the Dynamophone or Telharmonium of the American inventor and lawyer Thaddeus Cahill, who developed this instrument less for its musical qualities than to implement his business idea. This consisted of playing music ” on demand” and transmitting it over the telephone lines to restaurants and other locations. Since there were no amplifiers at the time (the first prototype was made in Washington around 1900 and the electron tube had not yet been invented), the Telharmonium had to generate the currents needed to power the speakers at the other end, and to compensate for the power losses of the telephone network, itself using steamdriven generators. This was over 10,000 watts, and the prototype alone weighed over 200 tonnes. To realise his vision, Cahill managed to attract a number of investors, so that he was later able to build another, even larger instrument in New York, which also went into operation for a while. Cahill also succeeded in building (and financing) a third instrument in 1911. Unfortunately, there were frequent technical difficulties and interruptions, up to and including the collapse of the New York telephone network, so that the whole enterprise was a commercial failure and operations ceased in 1916. There are no recordings, so it is not clear what the instrument sounded like. The sound production principle, voltage induction by means of gears, was later adopted in the Hammond organs (patented by Laurens Hammond in 1934 and introduced in 1935). The music played on the Telharmonium (mostly four-handed) was primarily classical music, from Bach to Grieg, played for the ”dignified” audience.

Cahill was not a musician, so one cannot accuse him of having utopian visions that were more commercial than aesthetic. To his credit he was able to realise them with some skill and verve, albeit with a technology that was not yet really capable of doing so. Considering that all this happened before the invention of radio and far before today’s streaming services, one can only take one’s hat off to the

potential of Cahill’s visions at that time. Aesthetic implications of his invention were, however, outlined by Ferruccio Busoni with regard to the production of microtones (Busoni 1907). However, the use of tonal systems with smaller intervals than the (tempered) semitone and their aesthetic use has been researched and used primarily in instrumental music, although this can sometimes only be implemented with difficulty and inaccuracy on existing instruments, yet it was and is quite easily possible electrically since the Telharmonium.

Tubes

In 1906, Robert von Lieben in Germany and Lee de Forest in the USA each registered slightly different techniques for amplifying electrical signals based on gas-filled tubes, which later developed into the high-vacuum triode as a component for amplification. In 1926, the pentode was developed in the Phillips laboratories in Eindhoven, which then became the standard amplifying element of the tube era.

This eliminated the need to generate stronger currents directly in the soundgenerating or sound-transmitting devices in order to operate loudspeakers. Transmission losses, for example in telephone signals over long distances, could also be compensated for. In the field of music technology, electron tubes and the amplifier circuits that could be realised with them made it possible to build instruments that were much more manageable in size and weight than the Telharmonium. As a result, a number of such musical instruments were invented in the 1920s and 30s. The best known of these are certainly the Theremin by Lew Termen in 1919 (Ruschkowski 1998), the Trautonium by Friedrich Trautwein in 1920 (Donhauser 2007), or the Ondes Martenot by Maurice Martenot in 1923 (Bloch 2004).

The Theremin and Ondes Martenot both work according to the principle of the Schwebungssummer, in which the signals of two high-frequency oscillators with closely spaced frequencies are mixed so that the resulting difference frequency falls into the audible range. One of the two oscillators can be detuned so that different audible frequencies are produced. With the Theremin, this is done by means of a kind of ”capacitive distance sensor ” in which the player’s hand functions as a ground and influences the capacitive part of the inductor-capacitor (LC) oscillating circuit via the ”antenna.” With the Ondes Martenot, something

similar happens via the hand ring, which was later joined by a piano keyboard, initially for orientation, but then also as a playing aid. The Trautonium produces tones by pressing down on a string wound with resistance wire to establish electrical contact. By means of a connected capacitor and a glow lamp, a sawtooth-like oscillation is produced. A downstream formant filter can be used to emphasise individual parts of the spectrum. Even before the outbreak of the Second World War, Oskar Sala further developed the Trautonium into the ”Mixturtrautonium,” which could also generate subharmonic spectra by frequency division.

These (and a few other) instruments did generate curiosity and interest among audiences and musicians alike when they were first introduced, and a number of works were (and still are) written for them. Most of them tried to combine electronic with acoustic instruments. Although the original playing techniques required no reference to the Western tonal system at all, and all were basically continuously playable, keyboards (in the case of the Ondes Martenot) or metal reeds (in the case of the Trautonium) were developed and added as aids. Theremin virtuosos such as Termen himself, Clara Rockmore or Lydia Kavina developed playing techniques that made it possible to play tempered semitones precisely and quickly. The volume, which can also be controlled by hand movements, usually imitates vocal or instrumental phrasing. In addition to original compositions written for these instruments, they are essentially used to interpret existing instrumental music differently (and not necessarily in a novel way), including the classical and romantic repertoire as well as rock and pop music.

The Theremin experienced a certain renaissance as a ”strange” instrument in the 1990s and 2000s (partly until today), not only but especially in (more or less experimental) pop music. In live performances, the unusual way of playing the instrument is often used as a performative element, but a musical style of its own, fed by the peculiarities of the instrument, has not yet developed. Some pure Theremin ensembles have been founded (the largest was the Japanese Matryomin Ensemble with 289 members, which performed Beethoven’s Ode to Joy, among other works). Here, too, most were concerned with making their instrument useful for traditional musical forms. There may be or have been praiseworthy exceptions, such as the ICEM Theremin Ensemble, which was founded in 2010 but disbanded quite quickly. However, these have only become little known.

The Ondes Martenot, for which, according to Jeanne Loriod (1928-2001, one of the most famous ”ondists”), 15 concertos and over 300 chamber pieces were composed, and which was used in many other projects up to Daft Punkt or Radiohead, also did not evoke music based on the special sound production and playing technique whose characteristics would then have led to newer musical structures

Oskar Sala and his Mixturtrautonium is here, at least in part, a great but singular exception. In 1962, for example, he created a score for Hitchcock’s ”The Birds” on the Mixturtrautonium, which is both the sound of the birds, and thus part of the sound design, and at the same time the music of the film (the otherwise usual orchestra does not play a role in this music). Oskar Sala used both here and in several other works a unique playing technique that he developed for his instrument, with which he was able to create virtuoso transitions between sound and noise, tempered and untempered, and between fixed and glissandoing notes. After a brief attempt by the Telefunken company in the 1930s to bring the Trautonium to the commercial market, the instruments built and further developed by Sala himself remained almost exclusively the only ones in use, and Oskar Sala their sole, highly virtuoso performer.

Sound Storage

Probably the greatest and most groundbreaking development in the field of music technology is that of storing sounds on a physical medium. Without this possibility, if you wanted to listen to music, you either had to go to a concert or make the music yourself. Although it has been possible since the Middle Ages in European culture to record musical sequences in written notation, and thus to pass them on and study them by reading them in or outside of real musical time, all these notes and scores are nothing more than instructions for the players as to when which note is to be played and how.2

The first, and at the point of their development, still mechanical devices for sound storage were the phonograph by Thomas Alva Edison (US Patent US200521US200521A1878), in which sound waves were mechanically inscribed in rotating rollers, initially covered with tin foil and later with wax. The deflection of the sound wave corresponded to the penetration depth of the writing stylus (Depth inscription). The phonograph was superseded by Emil Berliner’s gramophone (US patent US372786, 1887), whose discs were cheaper to produce

and took up less space in storage. The lateral inscription used here (the amplitude corresponds to a side-to-side deflection of the inscribing or scanning needle) also ensured better sound quality. However, the recording and production of the record were separate from its playback. The former was reserved for production facilities, while the playback devices were offered on the mass market. The principle of the gramophone has remained the same up to today’s (vinyl) record, despite many technical, mainly electronic, improvements.

The first device for magnetic sound recording was probably the Telegraphon made by the Danish engineer Valdemar Poulsen (US Patent US661619A 1900), in which first a wound steel wire, initially a piano string then later a steel tape, was magnetised on coils. In 1928, Fritz Pfleumer in Dresden developed a tape made of paper on which a magnetisable layer was applied. In 1935/36, the company BASF developed the first audio tape with a plastic carrier layer. Also in 1935, the company AEG presented the first tape recorder (Magnetophon K1) at the Great German Radio Exhibition, whose mode of operation essentially corresponds to tape recorders to this day (wherever they are still used today).

Tape machines and records, amplifiers and increasingly high-quality microphones and loudspeakers finally decoupled the performance of music from the space and time of its creation. If radio transmission had already made it possible to bridge the spatial distances to the place of performance, it was now possible to store music in a small amount of space on inexpensive sound carriers and thus to separate it from the time of its performance, as well as to distribute it commercially on a large scale.

Recording technology itself, especially tape, forms the common basis of what began in Paris in 1948 as ”musique concrête” (a term introduced by Pierre Schaeffer) and in Cologne in 1951 as ”electronic music ” In order to compose, it was no longer necessary to notate sounds symbolically in musical notation as instructions for a performing musician, rather one could compose the sounds directly on the tape. The sounds themselves could be any audible sounds. They could have pitches or not. There was no need to be bound to a tonal system. With the possibilities of the tape machine, the sounds could also be manipulated: They could be cut, recombined, slowed down or sped up (and transposed at the same time), combined with other sounds, layered on top of each other or played backwards. In particular, they could be copied with tolerable loss of quality, and in the process manipulated by further electronic interventions (modulation, filtering, etc.).

While the Parisian Musique Concrête concentrated on using the musical properties of sounds of all kinds in an aesthetic way, in Cologne they tried to create completely new, hitherto unknown sounds with the help of equipment from radio technology, and thus to control the structures of the sound itself compositionally. Herbert Eimert in particular, followed in Paris by Pierre Schaeffer, endeavoured to give their discoveries of the aesthetic possibilities offered by the new technique a corresponding theoretical basis. Both encountered different problems in realising their respective ideals in real works.

Thus Schaeffer primarily emphasised the musical properties of the sounds and wanted to separate them from the connotations of their origin. The kind of reduced listening demanded here requires (at least in my understanding) a certain processing of the sound away from its source, which was hardly possible with the means of the time, and so every listener of the early Etudes des Bruits will probably not be able to avoid hearing railways, pot lids and the like, rather than the desired pure, musical sounds and rhythms.

Cologne’s Elektronische Musik was based on the findings of de Fourier and Helmholz, i.e., that every sound can be analysed in terms of its constituent sinusoidal components, and so it was concluded that, conversely, every conceivable sound can also be composed of sinusoids. If this insight is completely correct in theory, its implementation in the synthesis of sounds of appreciable complexity is an almost impossible undertaking. Therefore, the sounds of the early works based on this idea often consisted of only a few sinusoids, because these then had to be layered on top of each other on tape, cut and mixed to achieve the final result.

Stockhausen’s short (3:20) Studie II , for example, consists of 193 different tone mixtures, each consisting of five sinusoids. In order to minimise the copying processes necessary for this and the accompanying accumulation of unwanted noise, Stockhausen sent tapes with a short sequence of the 5 tones into a reverberation chamber, and then re-recorded the reverberation and cut out from it the part in which the five individual tones were reasonably evenly present. Gottfried Michael Koenig, also a pioneer in the Cologne studio, who supported many composers in the realisation of their works in the most creative way (Stockhausen’s ”Kontakte” is certainly the most famous), developed a compositional aesthetic in his own electronic works that made the technical possibilities of the studio, its available (limited) equipment and wiring possibilities a decisive determinant. This is described, among other things, in his essay on the creation of the score of his work ”Essay” (Koenig 1959).

Yannis Xenakis reacted artistically in his own way to the aesthetic problems that arose from the discrepancy between theoretical basis and technical realisation. He writes about his piece ”Concret PH”, realised in Paris in 1958:

Start with a sound made up of many particles, then see how you can make it change imperceptibly, growing and developing, until an entirely new sound results… This was in defiance of the usual manner of working with concrète sounds. Most of the musique concrète which had been produced up to the time of Concret PH is full of many abrupt changes and juxtaposed sections without transitions. This happened because the original recorded sounds used by the composers consisted of a block of one kind of sound, then a block of another, and did not extend beyond this. I seek extremely rich sound (many high overtones) that have a long duration, yet with much internal change and variety. Also, I explore the realm of extremely faint sounds highly amplified. There is usually no electronic alteration of the original sound, since an operation such as filtering diminishes the richness.3

In ”Concrete PH”, Xenakis recorded the sound of glowing charcoal (until the destruction of the microphone, which was positioned quite close and then died of heat) and cut the tape into small, ca 1 cm pieces which were then reassembled in a different order. In this way, he created sound structures and realised something very early on that only reappeared much later, as (then digital) granular synthesis.4

Despite the sometimes radical approaches in the creation of unusual, or at least in music unusual, sounds, the use of noises not initially meant to be musical, the integration of the technical process into the compositional disposition, as well as the possibility of dissolving the separation between stage and audience through multi-channel techniques, viewed from today many works seem surprisingly ”instrumentally” conceived. Often, one has to deal with the temporal arrangements of individual sounds, which in their rhythmic density and complexity, often do not go beyond those of advanced instrumental music. ”Motifs” and ”phrasings” can be discerned, often with a quasi-melodic character. Most of the structures could certainly be ”played” by human interpreters on appropriate instruments.

Stockhausen’s ”Studie 2”, for example, a piece that is considered paradigmatic for early electronic music (from Cologne), is based on a tuning system whose basic interval is the factor 5!⁄"# The Factor 5!⁄"# corresponds in the tempered tone system to a minor second that is a little too large. The tempered tone system itself is based on the factor 2!⁄!" . The sounds Stockhausen generates with this are 5note ”clusters” whose individual notes are each one, two, three, four or five such basic intervals apart. Although he succeeds in avoiding both the traditional intervals and the lower overtones by altering the intervallic basis (the system only meets the overtone series again after two octaves and at the third–5th partial), the principle of his tuning system itself, in which there is a smallest interval and all others are multiples of it, is very similar to that of the tempered tuning system that has been a constituent of Western music since at least Bach’s time. Having generated the repertoire of possible individual sounds, Stockhausen arranges them in time just as he might in an instrumental score, i.e., with the attendant rhythmic complexity we associate with him. Thus, although he himself has compositional control over the spectral composition of the sounds, no further compositional consequences of the potential of tape music composition seem to have been drawn by Stockhausen at this point.

In the mid-1950s, the first 4-track tape recorders were available (e.g., Telefunken M10 four -track). In addition to the simplified possibility of layering sounds on top of each other, these also offered the possibility of reproducing these four layers on separate loudspeakers.

Without additional technical aids such as ”click-tracks,” it was and still is necessary in instrumental music to position the musicians relatively close to each other so that they can hear each other well and without significant delay and thus synchronise. This leads to the usual separation, which is virtually part of the ritual of the music concert, between a stage arranged ”in front ” for the musicians and an auditorium positioned in front of them. Of course, there are exceptions, such as the orchestral work Terretektorh by Xenakis, 1965. While this situation was still common in the early performances of electronic music, in which one or two loudspeakers were placed on or next to an otherwise empty stage in the concert hall (which occasionally led to a certain irritation on the part of the audience), the arrangement of four loudspeakers, usually in a square or rectangle around the listeners, made it possible to eliminate the rigid allocation of stage space and auditorium.

Many composers of electroacoustic music gratefully accepted these possibilities and composed their works for several (initially four, later considerably more) loudspeakers, which were usually positioned around, occasionally also within the audience. Among these early 4-channel pieces were Koenig’s ”Terminus I+II” and ”Gesang der Jünglinge im Feuerofen” by Karlheinz Stockhausen. 5 It is interesting to note, however, that the use of multiple loudspeakers in a room, and the associated possibilities for immersive sound shaping, have not yet become the norm in concert performance. This is despite the fact that, since the 1950s, composers of electroacoustic music have been using increasingly large loudspeaker configurations, up to and including so-called loudspeaker orchestras (such as the ”Acousmonium” of the French Groupe de Recherches Musicals –GRM, founded by Schaeffer, or the Birmingham Electro Acoustic Sound Theatre – BEAST, of the University of Birmingham).

The normal case in the reproduction of music through loudspeakers is the depiction of a ”stage” in the stereo image, and the placement of the musical events (whether played with acoustic instruments or generated electronically) on this stereo stage. Even advanced composers have long held on to ”stereo” as the normal musical case. For example, Trevor Wishart (*1946), probably one of the most important composers of electroacoustic music and otherwise not suspected of being too attached to instrumental paradigms in his musical aesthetics, composed his first 8-channel piece, ”Encounters in the Republic of Heaven,” only in 20116 (Wishart 2011).

The possibilities offered by multitrack technology were initially used in the production of popular music to make it easier to layer several musical structures on top of each other, via ”overdubbing.” Thus productions were created that could not be performed live, or at least only with great difficulty. As a consequence, the Beatles renounced live concerts from 1966,7 and were able to continue working on their studio productions independently of the restrictions of the live performance situation. In doing so, they were able to use the studio as an instrument and integrative component of the creative process, and not only as the place where something already completed is only ”recorded” (Levinson 1993) Other bands were happy to take up these impulses. Pink Floyd, for example, used recordings of everyday noises (Alan’s Psychedelic Breakfast) or Liverpool FC fans intoning ”You’ll Never Walk Alone” (Fearless). Alan Parson, George Martin’s assistant on some Beatles productions, created the legendary tape loop of cash register noises (in 7/4 time) for ”Money,” among many other examples of applied tape technology. Pink Floyd were also one of the first to produce an album quadraphonically (Atom Heart Mother) and to perform live with a quadraphonic PA system.

There were other artists and bands who creatively used the possibilities of the studio to make themselves musically more independent of instrumental practice in their means of expression. Nevertheless, the live performance and thus the requirements of an instrumental/vocal practice and performance remained an important part of the artistic self-conception of the vast majority of bands. Even Kraftwerk still play large parts of their music ”by hand” on appropriate keyboards during a live performance.

Only the producers and DJ(ane)s of techno and related ”club music ” or in HipHop use playback devices as instruments in their own right, but they usually create very instrumental structures with them, consisting of ”drums,” ”bass,” simple melodies and occasional recordings (samples), such as short vocal interjections. Sometimes, especially in HipHop, the scratching sound of the needle moving quickly on the record (or the record under the needle) is used as a sound effect.

The possibility of storing sound events on physical objects and, by manipulating these objects, manipulating the sound in turn, created in many respects the technical preconditions for breaking away from the instrumental paradigm of music: The sound itself could be produced in a compositionally controlled way and was not dependent on prefabricated sound generators. Any sound could be used, not only those produced by trained musicians on instruments. One was therefore not dependent on a symbolic notation of a desired sound production. Last but not least, it was no longer necessary to position the musicians close to each other on a stage in order to synchronise their sounds. The synchronisation of the sounds already took place in the studio and they could then be placed anywhere by arranging them on an appropriate number of loudspeakers in space.

Composers of electroacoustic music have made and continue to make ample use of these possibilities, and over the course of time many works have been created that in one way or another set them apart from instrumental music. It is all the more astonishing that concerts and festivals of contemporary music (as long as they are not explicitly dedicated to electroacoustic music) are still for the most part festivals of instrumental music, which are only occasionally extended by playbacks, live electronics or electric/electronic instruments.

Transistors and Voltage Control

A further step in technological and subsequently also electronic music development, was the invention of the transistor. Theoretically invented (and patented) by Julius Lilienfeld in 1925 (patented in 1927 as Canadian patent NR CA272437) as a kind of field-effect transistor (FET), it took until 1947 for the first bipolar transistor to be presented at Bell Laboratories. (Bardeen+Brattain 1948). Some improvements to the first designs enabled this bipolar transistor to largely replace the electron tube as an amplifying element in the course of the 1950s. Moreover, its light weight and lower power requirements made the transistor one of the most important electronic components in approximately all electronic devices. For the development of music-electronics, the transistor was decisive because it made more complex circuits possible in a smaller space and with less heat waste. Due to its relatively low noise and loss, it was also easier to control the settings of individual devices or assemblies with a control voltage than it was with electron tubes (triodes).

In 1959, the engineer Harald Bode, who emigrated from Germany to the USA in 1954, developed the first modular synthesizer (Audio System Synthesizer), which was presented at the AES Convention 1960 in New York, and described in detail by Bode in Electronics Magazine in 1961 (Bode 1961). In this unit, various modules, such as tape delays, filters, a ring modulator and others, could be interconnected in different combinations. At the presentation of this device at the AES Convention was a young man named Robert Moog, at that time a manufacturer and seller of Theremin kits. Fascinated by Bode’s synthesiser, it was here that Moog developed his first ideas for what would later become the ”Moog Modular” synthesiser. In fact, Moog later had licensed some of Bode’s circuits for his modular synthesiser, including a vocoder, the ring modulator and the pitch shifter.

In 1963, Moog began working with composer Herbert Deutsch on the concrete design for the ”Moog Modular” synthesiser. In 1964, he was able to present prototypes of the first modules at the AES Convention and deliver the first complete systems shortly thereafter. What was really exciting about Moog’s development was not only an expansion of the number of possible modules (compared to the comparatively few in Bode’s synthesiser) but the possibility of controlling almost every adjustable parameter of the individual modules (frequencies, amplitudes, trigger and gate signals etc.) simply by means of voltages. Since the signals of the system were also realised as electrical voltages, the modules could control each other and thus create novel sound structures through the interconnections of the individual modules.

In an interview, Deutsch told of an important design decision that Moog and Deutsch had made at the time:

At the time I was actually still thinking primarily as a composer and at first we were probably more interested in the potential expansion of the musical aural universe than we were of its effect upon the broader musical community. In fact when Bob questioned me on whether the instrument should have a regular keyboard (Vladimir Ussachevsky had suggested to him that it should not), I told Bob, I think a keyboard is a good idea, after all, having a piano did not stop Schoenberg from developing twelve-tone music and putting a keyboard on the synthesizer would certainly make it a more sale-able product!!!8

Why the piano, with its 12 keys per octave, should have hindered Schönberg’s development of 12-tone music is beyond me. On the contrary, I think it was rather helpful. However, Deutsch was probably absolutely right in his prediction that the Moog synthesizer would sell better with a keyboard than without: the Moog Modular Synthesizer celebrated its greatest successes with recordings such as ”Switched on Bach” by Wendy Carlos (1968, Columbia Records) in which she reinterpreted works by Johann Sebastian Bach on the Moog Modular by recording individual voices one after the other (the keyboard was only monophonic) using overdub procedures. Artists such as Isao Tomita became known for reinterpreting classical works on the synthesizer, imitating classical instruments quite convincingly. The tension between familiar musical structures, electronic imitations of well-known instruments and sometimes unfamiliar new sounds is what might make these interpretations so appealing on the one hand, but also might make them aesthetically vulnerable on the other. The Moog synthesiser also celebrated great successes in the rock and pop music of the 1960s and 70s, probably first used by the Beatles on the Abbey Road album.

In 1970, the Mini -Moog came onto the market, and quickly became a must-have for pop-keyboard virtuosos of the time. This comparatively portable instrument had only a few circuit options and was essentially based on a standard combination of a few ”modules,” along with a few modulation options. It was a pure keyboard instrument that could produce novel sounds that were used for melodic solos or occasional bass lines, but it offered hardly any possibilities for generating novel musical structures.

Also around 1963 and largely independent of Robert Moog, Don Buchla, in constant exchange with the composer Morton Subotnik, also developed a modular, voltage-controlled system at the San Francisco Tape Music Center, which later became known as ”Buchla 100 ” Specific characteristics of the sound synthesis, which was based on non-linear modulation processes rather than Moog’s linear-subtractive filter model, ensured that, to this day, Buchla-like synthesis systems are referred to as ”West-Coast” and Moog-like ones as ”EastCoast ”9

Although the main motivation of Buchla’s (and Subotnik’s) efforts had been to ease the cumbersome production process in the tape studio by creating a system that could generate musical structures much more quickly, as well as being used for live performances, Buchla did not see the point of using a traditional chromatic keyboard as an interface. Instead, he looked for other ways to transform physical gestures of musical expression into control signals for his synthesis machines. Among other things, he developed the ”Thunder,” a touch-sensitive programmable surface, or ”Lightning,” a controller that could convert threedimensional movements of a conductor’s baton-like objects into control signals. His ”Multi-Dimensional Kinesthetic Input Port,” converted signals from touchsensitive surfaces and the position of two wireless finger rings into control voltages. Buchla’s designs were such that the individual modules could be used very flexibly for a wide variety of musical purposes. The interfaces for manually ”playing” the devices were much more adapted to the method of creating electronic sounds and structures than traditional keyboards were and are. Thus Buchla’s developments, and those of others who have been inspired by them, can be seen as a successful attempt to make the means and methods of electronic sound and structure generation, which per se do not require a human player, playable by performers without resorting to traditional playing techniques. In this way, he avoided the danger of manifesting traditional forms of musical virtuosity with his new devices.

All this together certainly had a great influence on the world of electronic composition in contemporary music, but a commercial success like the instruments of Bob Moog and the many others based on similar designs afterwards (ARP, Korg, Roland…) remained elusive for the Buchla devices. In addition to the more or less commercial systems, bespoke developments of voltage-controlled systems were advanced at a few larger academic or public institutions in the 1960s and 1970s. In 1964, Gottfried Michael Koenig became artistic director of the ”Studio vor Elektronische Muziek” at the University of

Utrecht, which was renamed the ”Institut voor Sonologie” in 1967. There they developed their own voltage-controllable modules such as the ”Variable Function Generator (VFG)” with which a wide variety of voltage functions could be generated via a series of sliders, and which could be both a sound source and a control signal for further processing. Here Koenig was able to extend his idea that the interconnection of the individual devices is itself a basis of the composition and not just an instrument on which the composition is performed to the control parameters of the individual devices themselves (which still had to be set ”by hand” to a large extent in the Cologne studio). He composed a whole series of pieces in 1969, the so-called ”Funktionsstücke” each based on a different setting of the VFG (Koenig 1967-69). 10 The Institute for Sonology is now part of the Conservatory in The Hague and is directed by Kees Tazelaars, a student of Koenig. It still operates a studio in which analogue, voltage-controlled sound generation is the focus.

Also in 1964, the Elektronmusikstudion (EMS) was founded in Stockholm. Initially part of Swedish Radio, it was transferred to a state-supported foundation in 1969. By 1970, the large computer studio was operational, a world -leading hybrid technology of digital control of analogue sound synthesis at the time. This innovation attracted many composers of electroacoustic music to Stockholm to realise their pieces here. The hybrid system was dismantled when the EMS moved from downtown Stockholm to the southern part of the city and was lost for a long time. According to some reports on social media, individual components have since reappeared. Currently, the EMS has two Buchla systems (one of them a replica) in addition to some commercial rack-mounted synthesisers.

In 1971, Dirk Reith brought electronic music to the Folkwang Hochschule (now Folkwang University of the Arts) in Essen. Inspired by his studies with G.M. Koenig in Utrecht, he worked with the Berlin company Hofschneider to develop a system consisting of about 80 individual modules: the SYNLAB. One of the design principles of this was that each module should function electrically as precisely as possible. The SYNLAB was completed in 1978 and was for a long time the most important system in the training of electronic composition in Essen. The SYNLAB is still fully functional and is housed in its own analogue studio. It is still used in the training of composers.

After a time when almost all electronic music was based on digital technology, there has been a renaissance of analogue music for some years now. In 1995, the company Doepfer Musikelektronik introduced the Eurorack format for recording various modules in connection with its M100 modular system, which was adopted

by many other companies and thus became the quasi-standard for current modular systems. Since then, there has been an ever-growing number of modules that are integrated into these systems and, even if some of them are internally digital, can at least be controlled in some aspects by means of control voltages. Separate analogue keyboards for the virtuoso performer are usually not provided here, but external control via the MIDI protocol is often used. This means that there is also an interface to keyboards, but even more so to digital MIDI sequencers (software), and here too we are often back in an instrumental context.

Digital Systems

Mechanical aids for carrying out complicated calculations existed quite early. However, it was probably not until the British mathematician, philosopher and inventor Charles Babbage made the leap to a kind of universal machine, comparable in some respects to our computers, with his ”Analytical Engine” (Babbage 1864), which was never actually built. Ada Lovelace was probably the first person who, during the course of translating an article about this Analytical Engine, discovered that it was much more than a mere calculating machine. She also developed the quasi first ” program ” for this machine (for calculating Bernoulli numbers).

In 1937, Alan Turing presented a mathematical concept of a ”universal machine” (later named ”universal Turing machine” after him). If one disregards the necessary limitation of available memory, our computers today are all ”universal Turing machines” (Turing 1937). In 1938, Konrad Zuse presented the first electrically driven mechanical calculator that worked with binary numbers (Zuse 1969). In 1945, John von Neumann presented a first draft of a computer architecture, according to which the vast majority of computer systems are still constructed today (Neumann 1945), further known as ”Von Neumann Architecture ” Towards the end of the 1940s, Alan Turing discovered that it was possible to generate tones through appropriately rapid successive periodic clicks that his computer at the time could produce (Manchester Mark II). As he was less interested in music, he used these as signal tones, which, for example, indicated to him that a calculation had come to an end. Christopher Strachey, one of the programmers, wrote a long programme that played ”God save the Queen” in this way.11

Clearly more serious attempts to use the universal machine, the ”computer”, for the generation of musical sounds and structures were made by the engineer and programmer Max Mathews at the Bell Laboratories. In 1957, the first 17 seconds of computer-generated music were produced with his ”MUSIC” programme. MUSIC was further developed by Mathews (MUSIC II, III, IV, V) and other developers generated derivatives (MUSIC IVb, MUSIC 360, MUSIC 11, CMusic, CSound, CMix…). The Structured Audio Orchestra Language (SAOL), an imperative MUSIC-N Language, is part of the MPEG-4 audio standard. MUSIC can be seen as the ancestor of systems such as Max/MSP (which was even named after Max Mathews), Common Lisp Music, ChucK, and others.

What prompted Mathews, and with him many other engineers, programmers and composers, to devote themselves to digital sound production when it was far more cumbersome with the technology of the time than working with analogue devices? There was seemingly endless computation time only to be flawed by a forgotten semicolon. Then, finally, a sound or a musical structure that often did not correspond to what you had intended and what you wanted to change and hopefully improve, and then you had to start the whole process all over again. Perhaps it is because of the fascination one can feel when one begins to understand what it means to have a universal machine at one’s disposal that can theoretically be transformed into (almost) any other machine one can imagine (as long as it produces a ”computable sequence ” according to Turing)12

Harold Abelson, co-author of the ”Wizard Book” (Abelson et al. 1984), describes it this way in his series of lectures on an introduction to computer science 13 at MIT:

Computer science deals with idealised components. We know as much as we want about these little program and data pieces that we’re fitting things together. We don’t have to worry about tolerance. And that means that, in building a large programme, there’s not all that much difference between what I can build and what I can imagine, because the parts are these abstract entities that I know as much as I want. I know about them as precisely as I’d like. So as opposed to other kinds of engineering, where the constraints on what you can build are the constraints of physical systems, the constraints of physics and noise and approximation, the constraints imposed in building large software systems are the limitations of our own minds.

In terms of generating musical structures, digital systems also offer this kind of liberation from the limitations of the real world. If you need, say, 100 oscillators to realise a musical idea, you are almost lost in the analogue world. The purchase of such a quantity of modules alone would be so expensive that even statesupported and -financed institutions could not afford it. Perhaps one could have helped oneself with a clever production plan with multi-track technology and quite a few intermediate mixes, but then one would have to solve the inevitable problem of unwanted noise.

In software like the Music-N languages, on the other hand this did not present a problem. It may not have worked in real time but it would work. Depending on the performance of the hardware and the complexity of the desired result, you may have needed a very long time to calculate the result and save it in a sound file before you could hear it, but it was possible.

How far then have the ”limitations of our own minds” gone for the developers and users of these systems? Often, indeed, very far. Many composers and sound researchers have tried the experiment with those hundreds of oscillators. At ICEM, for a long time it was a ”compulsory piece” to produce a study using only sinusoids, but their number was not limited. So it often seemed obvious to the students to create a different richness of sound and structure with a lot of oscillators, as Stockhausen was able to do in his Studie II, for example. Composers who were also able to develop their own software, at least in rudimentary form, were able to push the boundaries of the musical further and further. Software such as MaxMSP, Pure Data, SuperCollider and the above-mentioned Music-N derivatives gave even the less programming-savvy composer tools to realise even unusual ideas. Nevertheless, some things are striking here: The ”Music-N” languages have in common that the user can combine individual smaller procedures for signal generation or processing ( ”opcodes” or ”unit generators”) into larger complexes. These combinations were activated by an event list at certain musical times and could be provided with additional start parameters. In the Music-N languages, such an interconnection is called an ”instrument”, the collection of several such ”instruments” here is called an ”orchestra” and the event list that activated the instruments of the orchestra and provided them with values is called a ” score ” .

Jean Claude Risset, one of the pioneers of early computer music who collaborated a lot with Mathews in the 1960s, is often honoured for his research on digital simulations of musical instruments rather than for his (no less important and innovative) genuinely electronic compositions See, for example, the entry about him on the IRCAM site14 or the obituary for him in Musikologie.org.15

In the ”Amsterdam Catalogue of CSound Instruments,”16 an extensive collection of CSound sound synthesis algorithms, these are indeed arranged according to the respective synthesis method, but the even somewhat complex interconnections are almost exclusively, mostly unsuccessful attempts to simulate acoustic instruments.

Until the early 1980s, there was no easy way to remotely control or synchronise electronic musical instruments from different manufacturers. But the polyphonic (keyboard) synthesisers of the time had only a few (4-8) voices; if you wanted more voices, you had to add another synthesiser with its own keyboard. Playing several instruments from the same keyboard was not possible. Ikutaro Kakehashi, president of Roland at that time, thought that this was a hindrance to the further development of the electronic music industry and suggested to develop a common standard. This initiative then gave rise to the MIDI standard. In 1983, the first MIDI connection (between a Roland Jupiter 6 and a Prophet 600 from Sequential Circuits) was introduced at the NAMM show, and the first specification was published in August 1983. 1983 also saw the release of the first MIDI drum machine and the first MIDI sequencer (both from Roland). In 1984 the ”Midi Manufacturers Association” (MMA) was founded with ”all interested companies”, in 1985 the MMA published the ”MIDI 1.0 Detailed Specification” as a standard. Subsequently, other MIDI standards were developed (MIDI File Format, General MIDI, MIDI Machine Control). In 2020, the MIDI 2.0 standard was introduced.

The MIDI standard protocol transmits what a keyboardist can do to a device that converts this information into sound, or records it. In the main, it transmits which key is pressed and released, and with which velocity. In addition, a whole range of playing aids can be transmitted. The ”pitch wheel” introduced by the MiniMoog and common to many synthesisers, or other sensors to influence the pitch (pitch bend), an unspecified modulation wheel, often used to control the amount of vibrato, the pressing on the keys (aftertouch), changing synthesis presets (programme change) and a whole series of additional ”controllers” (pedals, volume sliders, knobs of all kinds), etc.

This made it possible to separate the keyboard from the sound generator, and as a result there was a whole series of MIDI controllable synthesizers and samplers that came onto the market in space-saving 19” rack or even smaller format, all of which were controlled by a common master keyboard, by a sequencer or a computer-based sequencer programme.

This simplified many production processes. Musical information did not need to be recorded as sound, and given that even quite complex arrangements could be realised with MIDI sequencers, musicians were often considered obsolete so much could be realised in the home studio, even with a modest amount of studio equipment. The advantage of this standardisation was also its disadvantage: due to the omnipresence of MIDI, the paradigmatic ”keyboard” became more or less ubiquitous. Anything that could not be represented as a pressed key from a chromatically arranged tone set could only be realised with difficulty and some trickery. Also, the limitation to mostly only 128 different values that a parameter could assume often hindered the realisation of ideas that needed a better resolution.

The space- and material-saving design meant that the operation of the devices itself became increasingly difficult if one wanted to get into sound design oneself. Access to individual parameters of sound synthesis or processing was often only possible after extensive switching through various menu levels, and the display was often limited to 2 LCD lines, so that many musicians limited themselves to the use of presets supplied or purchased. Yamaha even added a special slot to their flagship, the DX7, where additional cartridges could be inserted that contained commercially available additional sound programs for this synthesiser. The MIDI standard allowed manufacturers to receive data via so-called ”system exclusive messages, ” which could contain manufacturer-specific sound programs and thus made it possible to configure the synthesis machines of the respective synthesisers with the support of suitable programs in a laborious way, but at least better than with the provided interfaces on the device itself. For composers of electronic music who wanted to design their own sounds, however, these limitations were more than annoying as they were in fact quite restrictive.

Nowadays, most of the functionalities that required stand-alone equipment in earlier studios are available in software. At the centre is usually a DAW (Digital Audio Workstation), a piece of software that replaces the multitrack tape machine and the mixing console. The individual tracks can contain audio material or MIDI data. Software-based ”effect devices” (reverberators, compressors, equalisers, etc.) can also be integrated in the form of so-called ”plug-ins.” In addition, software simulations of well-known or newly invented synthesisers or samplers can be integrated as plug -ins, which are then controlled by the MIDI data of the corresponding MIDI track.

In many forms of plug-ins, the emphasis is placed on simulating the physical surface of the ”device,” often even when there has never been a physical realisation. This goes as far as simulating modular systems in which the virtual cables are pulled down by virtual gravity. You often even need to turn virtual rotary knobs with the mouse.

Of course, in a digital system, it is completely unnecessary to submit to the limitations of MIDI (7-bit resolution of important parameters, chromatic pitches, virtual keyboard paradigm), and some attempts have been made to establish more advanced standards, the best known being probably the network-based protocol ”OpenSoundControl (OSC) (Wright and Fried 1997), which some manufacturers now use to control their devices and software. However, a replacement of MIDI is still far away. The MIDI Association (successor organisation of the MMA) has meanwhile specified the MIDI standard 2.0,17 which addresses some of these limitations. For example, controllers can be resolved with 32 bits. However, the basic keyboard paradigm is maintained and the new protocol has yet to penetrate the market and its user -base even though MIDI 2.0 is fully backwards compatible with MIDI 1.0.

Artificial Intelligence

In an article dealing with the impact of technology on aesthetic production, one can hardly avoid dealing with the current buzzword ”artificial intelligence ” The term itself was probably first used by John McCarthy and Marvin Minsky to apply for funding for a research project. Many things that were thought to be indicative of artificial intelligence over time, as long as they could not be realised, are now standard applications without being called artificially intelligent any more (playing chess, understanding natural language, expert systems). What is understood by artificial intelligence today are systems whose functioning is based on the (presumed) functioning of our nerve cells, (deep neural networks) and which are very good at recognising patterns in large amounts of noisy data (e.g., images or sounds). Once trained with appropriate data, the system can also be reversed, and in turn generate data and thus e.g., images or sounds. However, since such a system can only operate on already existing material, original, novel results are not to be expected. Where the results do appear original, they are more likely to be due to the shortcomings of the algorithm than to any form of artificial creativity. However, when it comes to producing (even more) music in a certain musical style by imitating it, this form of artificial intelligence is already quite capable today. For a lot of applied music (advertising, gaming, etc.), this can be an economic advantage. Whether this will also bring the world forward artistically and aesthetically remains to be seen.

Conclusion

My impression is that, despite the rapid technological developments of the last 100 years or so, there is a certain insistence on traditional structures and paradigms in aesthetic developments of music, especially in those musical areas that are based on technology. I think the new technologies are used initially, perhaps even mostly to reproduce existing aesthetics rather than inspiring new forms. For developers of music technology, the proof of function of their developments often seems to lie more in the fact that you can make music with their developments that is very similar to music which already exists. Only rarely is a development driven by ideas for a music that does not yet exist, and which can then perhaps be realised by means of this development.

So it’s more about making music in a different way (perhaps more economically), but less about making a different kind of music. Discovering that a different kind of music is possible with a different technology is then more or less left to those among the more advanced composers who can master and use these technologies so confidently that they can free themselves from the inherent implications built in by the developers of this music technology.

Composers who actually ”only” want to make music, and who do not necessarily want to or cannot develop their own technology, are often not in a position to do so. They therefore often adopt the framework that the developers of these technologies suggest to them without being able to question it and thus go on to perpetuate traditional forms and structures more often than they themselves probably would like to admit. By this I do not mean works that consciously engage with current or historical musical models that we all have in our heads as a result of our musical socialisation. It is precisely because they consciously refer to these models and relate them to their own aesthetic ideas that they stand out from them (as long as they do not merely imitate them).

What is meant here are the many individual compositional decisions that composers often make unconsciously because the available technique encourages them to do so, or even suggests that there is no other way. In my opinion, it is necessary to begin training composers on the one hand to sharpen their awareness of the aesthetic implications of particular technologies being used and, on the other hand, to make future composers so competent and knowledgable about the handling of this technology that they are ultimately able to control it aesthetically rather than let themselves be controlled by it.

Author Biography

Thomas Neuhaus (*1961), studied composition at the Folkwang University (today Folkwang University of the Arts) Essen, Germany, with Wolfgang Hufschmidt, and electronic composition with Dirk Reith In the 80s and 90s he was co-developer of the AUDIAC Project for computer aided composition and sound synthesis. As a composer he has worked with Theater der Klänge, Dusseldorf since 1987. Since 1994 he has taught at the Institute for Computer Music and Electronic Media (ICEM) of the Folkwang University of the Arts. From 2000 to 2002 he also taught at the University of the Arts Bremen In October 2004 he was appointed Professor of Musical Informatics at ICEM He is founding member of the Society for New Music Ruhr and was member of the board for many years. Since October 2011 he is the artistic director of the ICEM Thomas has developed several composition languages and interactive realtime environments. His interests include all kinds of connections between electronic music and the performing arts as well as the formalisation and automatic generation of musical structures.

Notes

1 Translated from German by the author (with a little help from deepl.com and some significant improvements by Prof. Dr. Michael Edwards).

2. On the one hand, the notation of music was a prerequisite for the representation and synchronisation of several parallel musical processes, thus making possible the sometimes highly complex polyphonic structures of Western music. On the other hand, it ensured that aspects of the music that could not be notated in the respective notation system (e.g. phrasing or articulation) could not be communicated and thus partially disappeared from the music. (Compare, for example, today's performance practice of classical vocal music with the vocal practices of cultures with oral traditions).

3 Yannis Xenakis in the accompanying text of the release of Concrete PH on Nonsuch Records, 1970

4 However, some theoretical foundations were laid in the 1940s by Dennis Gábor in his ”Theory of Communication” (Gábor 1947).

5 The ”Gesang der Jünglinge” was even originally conceived by Stockhausen as a 5channel work, but due to technical difficulties it was usually performed in 4-channel form, with the material of the 5th track distributed between tracks 3 and 4. According to Rudolf Frisius, the first performances should actually have been 5-channel (with considerable effort) (Frisius 2018).

6 However, there seems to be a 4-channel version of his work ”Vox5” See https://www.emdoku.de/de/work/emdoku/17873

7 Apart from the famous 1969 rooftop concert on the roof of the Apple company (not the one with the computers) for the film ”Let it Be”.

8. Interview with H. A. Deutsch, October 2003, and February 2004: http://moogarchives.com/ivherb01.htm

9. For the history of Buchla, see the obituary by Morton Subotnik in: https://newmusicusa.org/nmbx/the-electric-heat-of-creativity -remembering-donaldbuchla-1937-2016/

10 Interestingly, after the ” functions” he composed only three electronic pieces (Output 1979, Polychromie 2001 and Polytopie 2010).

11 See, among others, the article in the Smithsonian Magazine: https://www.smithsonianmag.com/smart-news/listen-first-recording-alan-turing-playingtune-synthesizer-180960586/

12. This fascination is well described in Steven Levy's book Hackers (Levy 2010).

13. https://ocw.mit.edu/courses/6-001-structure-and-interpretation -of-computerprograms-spring-2005/resources/1a-overview-and-introduction-to-lisp/

14. https://brahms.ircam.fr/en/jean-claude-risset

15 https://www.musicologie.org/Biographies/r/risset.html

16. http://www.codemist.co.uk/AmsterdamCatalog/

17. https://www.midi.org/specifications/midi-2-0-specifications

Bibliography

Abelson, Harold, Gerald Jay Sussmann, and Julie Sussman. Structure and Interpretation of Computer Programs, Boston: MIT Press, 1984.

Babbage, Charles ”Chapter Vii - Of the Analytical Engine ” In Charles Babbage: Passages from the Life of a Philosopher. London: Longman, 1864.

Busoni, Feruccio. Entwurf einer neuen Ästhetik der Tonkunst. Leipzig: Insel, 1907.

. Sketch of A New Aesthetic of Music New York: Schirmer, 1911

Bardeen, J., W. H Brattain. ”The Transistor. A Semi-Conductor Triode.”Physical Review, Band 74, Nr. 2, 1948.

Bloch, Thomas Ondes Martenot Booklet of the CD ”Music for the Ondes Martenot.” Hong Kong: Naxos, 2004.

Bode, Harald ”Sound Synthesizer Creates New Musical Effects ” electronics, McGraw-Hill, December 1st, 1961.

Daniel, Eric D., C. Denis Mee, and Mark H. Clark. Magnetic Recording - The First 100 Years. New York: IEEE Press, 1999.

Donhauser , Peter Elektrische Klangmaschinen. Die Pionierzeit in Deutschland und Österreich Wien: Böhlau, 2007

Eimert, Herbert ”Was ist elektronische Musik?” Melos 1 (1953): 1–5.

Frisius, Rudolf. KLÄNGE - STRUKTUR(EN) - ELEKTROAKUSTISCHE MUSIK

Musik und Sprache / Musik als Sprache: Karlheinz Stockhausens Tonbandkomposition GESANG DER JÜNGLINGE (1955-56)

https://www.frisius.de/rudolf/texte/gesang-juenglinge-quer.pdf)

Gábor, Dennis. ”Theory of Communication.” Journal of the Institution of Electrical Engineers, Volume 94, Issue 73 (January 1947).

Hünermann, Tobias and Christoph von Blumröder, eds Topographien der Kompositionsgeschichte seit 1950. Wien: Verlag Der Apfel, 2011.

Humpert , Hans Ulrich, Herbert Eimert Lexikon der elektronischen Musik Regensburg: Bosse, 1973.

Koenig, Gottfried Michael. ”Essay - zur Genesis einer elektronischen Partitur” (1959). I n Koenig, G.M. Ästhetische Praxis - Texte zur Musik Band 1, Saarbrücken: Pfau, 1991.

Koenig, Gottfried Michael. ”Zu Funktionen” (1967-1969) in: G.M.Koenig: Ästhetische Praxis - Texte zur Musik Band 2 Saarbrücken: Pfau, 1993

Levy , Steven. Hackers. Sebastopol CA: O’Reilly, 2010.

Wright, M., Freed, A. ”Open Sound Control: A New Protocol for Communicating with Sound Synthesizers”, Proceedings of the 1997 International Computer Music Conference, Michigan Publishing, 1997.

Lewison, Mark The Beatles: Recording Sessions. The Official Abbey Road Studio Session Notes 1962–1970 New York: Random House Value Publishing, 1993.

McCarthy J., M. Minsky, N. Rochester, and C.E. Shannon. A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence , 1955.

http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html

Neumann, John von. First Draft of a Report on the EDVAC Philadelphia: University of Pennsylvania, 1945. https://web.mit.edu/STS.035/www/PDFs/edvac.pdf

Ruschkowski, André Elektronische Klänge und musikalische Entdeckungen

Stuttgart: Reclam, 1998

Schaeffer Pierre. Traité des objets musicaux. Paris: Seuil, 1966/2002.

Turing, A.M. “On Computable Numbers, with an Application to the Entscheidungsproblem.” Proceedings of the London Mathematical Society. Nr. 1, 1937

Tewksbury, George E. A complete Manual of the Edison Phonograph 1897, Reprint: Kent: Sevenoaks, 1970

Wishart, Trevor Encounters in the Republic of Heaven (Book&CD), Yorkshire : Orpheus The Pantomime Ltd, 2011

Zuse, Konrad. Der Computer mein Lebenswerk. Berlin: Verlag Moderne Industrie, 1970, 3. Auflage Berlin: Springer,1993.

Listen Up!

Strategies of Theatre Sound Towards Artistic Identity, Sonic Branding, and Acoustic Ecology

Abstract

This article explores a range of interplays between sound and theatre by looking at three relational dimensions of theatre making: sound(ing) and the creative ensemble, sound(ing) as part of branding one’s work, and sound(ing) as an ethical pursuit in relation to a socio-acoustic ecology. Sound is a multifaceted and by nature truly interdisciplinary subject in the middle of nowhere. It is both completely ubiquitous and very elusive. Sound is also deeply connected to our identities. The way individual human beings sound (through speaking, singing, eating, walking, bodily functions, etc.) is as unique as their finger print We are identified by our sound(s), and we develop a sense of identity by how we produce, use and engage with sound(s). The aim of this article is to highlight, how contemporary theatremakers in Europe use theatre Sound as a dispositif of ‘sounding,’ ‘voicing,’ ‘listening,’ and ‘musicking ’ Three aspects will be the focus:

1) Recently, theatremakers have innovated their ways of working by employing Sound as a means to transform their creative processes and create artistic identities. 2) In communicating these, theatremakers have embraced Sound as part of their branding, engaging in a complex confluence of audio marketing, auditory and sonic predispositions,1 on-stage voice styles, composition and sound design in performances, venue architectures, equipment choices etc. 3) Theatres have also become spaces to critique commercial sonification and its manipulative methods of grabbing attention. Theatre thus allows for a more conscious reflection on how music, voice, and sound can relate to bodies, texts and spaces. It reflects our rapidly changing acoustic ecology, and the ethics of the inclusion and exclusion of individual sounds from our sonic environment.2 Case studies will serve to illustrate and test the arguments made.

Sound is a multifaceted and by nature truly interdisciplinary subject. It is both completely ubiquitous and very elusive. Sound is the subject of workplace regulations, design processes from cars to medical devices, it is a source for intense pleasure and intense pain, and it is deeply connected to our identities: the way individual human beings sound (through speaking, singing, eating, walking, bodily functions etc.) is as unique as their finger print. We are identified by our sound(s), and we develop a sense of identity by how we produce, use and engage with sound(s).

After extended periods in which ocularcentrism has dominated Western culture3 and in particular Western scholarship, privileging the written word over oral traditions,4 trusting visual more than acoustic evidence,5 scholars and artists from many disciplines have proclaimed a “sonic turn:” Caleb Kelly writes: “The ‘sonic turn’ in recent art reflects a wider cultural awareness that sight no longer dominates our perception or understanding of contemporary reality.”6 While the impulse to distance one from the other, to move sound into opposition with sight is understandable, it oversimplifies matters. Even the notion of the “turn” (sonic turn, acoustic turn, and auditory turn have all been used, sometimes synonymously, sometimes with minor shifts in accent) suggests a departure from something when, in reality, it would be more productive to understand sound and listening in relation to images and seeing, objects and touch, texts and reading etc. As Adrian Curtin warned: “We should be wary of blindly following the latest academic ‘turn,’ especially if this involves a turning away from other, still potentially productive, areas of enquiry.”7 What we need instead, says HansJoachim Braun, “is a continuing integration of sound studies into the study of the senses as an ‘area of attention’ and, more generally, into the cultural and social studies.”8

Notwithstanding the question of how to label this emerging scholarship, it has proven instructive: on the one hand by focussing on “sound as an isolated object of study, but also sound as a more general principle of selection beyond ‘music’ or ‘speech,’” and on the other hand by leading to a “reorientation to denaturalize hearing and reconceive listening practices as historically contingent, material, and social techniques.”9

This rekindled interest in sound has led to a number of publications in philosophy, theatre studies, sociology, and musicology and has helped to establish a dedicated research field Sound Studies.10 Sound is still, however, as Petra Maria Meyer writes in her edited volume Acoustic Turn “insufficiently reappraised academically.”11 While Sound Studies have spearheaded a significant reevaluation of the role of sound in a great variety of contexts, it is also noteworthy that ‘theatre’ is remarkably absent from its discourses. Its core distinctions and findings about the philosophical, phenomenological, technological, and musical meanings of Sound nonetheless provide a very useful map to navigate the topic.

For the purpose of this article, which is intended as a short provocation on how we might think about theatre Sound (beyond the handbooks for sound editing etc.), I will make a distinction between ‘sound’ (as a singular sonic event, such as the sound of a dog barking) and ‘Sound’, with which I refer to a dispositif rather than an acoustic event or quality. Dispositif here means a complex weave of practices, conditions (acoustic and technological), and concepts of sounding and listening. It is entangled with a wide range of related disciplines and thus also requires various methodologies (see fig. 1). I will come back to the notion of the dispositif, but should first introduce the actual objects of study and provide some concrete examples of what I feel needs discussing.

The Sound of Theatre: Sounding, Voicing, Listening, Musicking

I will start by mentioning three ‘snapshots’ or perhaps: ‘eavesdrops’ from (relatively) recent theatre experiences that highlight different aspects of Sound:

1. In a conversation between Ariane Mnouchkine and Katie Mitchell held online on 12 May 2021, host Wes Williams explicitly mentions the impressive sound system of the Cartoucherie, the venue of the Théâtre du Soleil.

When I came to the Théâtre du Soleil and I was watching Le Dernier Caravansérail the piece that was a version of the Odyssey, set in a refugee camp there was a moment from this piece that stays with me, and I actually dream about it. It’s the moment when they are crossing the river. The theatre student in me goes: ‘Oh, they are doing the Brechtian thing with the river and the cloth’ and I know all about that, and I have seen that many times. But what really transformed it, actually, was the sound. The sound of the helicopters and the incredible sound system that you have in that space that meant that you would just be physically transported through sound as well as all the tricks, all the games, all the play that was going on on stage and all the incredible intensity of all the actors, it was also the sound architecture…12

2. For quite a while, the famous comedic choir sequence “Danke,” originally a ‘new spiritual song’ from 1961, sung with falsettic determination by Jürg Kienberger in Christoph Marthaler’s production Murx (2003), was the holding music on the Berlin Volksbühne’s ticketing telephone line.13 The Sound for which Marthaler was and is still known an acting ensemble coming together in song in an often dead-pan delivery of highly charged musical material from Schubert to Schlager here became something like an acoustic logo of the Volksbühne (even though this came with an ironic twist: the delivery of the song in the show was drawn out to much comedic effect employing a durational aesthetic typical for Marthaler something most people enjoyed in the theatre but not while waiting on the phone).

3. In 2015 the then artistic director of the Münchner Kammerspiele, Matthias Lilienthal, employed the Serbian experimental jazz singer Jelena Kuljić and the queer Australian singer and performer Damien Rebgetz as part of the permanent acting ensemble at this theatre. This was a bold move insofar, as the theatre has traditionally been sees as a stronghold of acting and a culture of vocal delivery.14 Both artists, however, are non-native German speakers with accents, not trained as actors, but have become much-loved performers in the repertoire of the Kammerspiele over the next few years. Egbert Tholl, critic at the Süddeutsche Zeitung, wrote about Kulijć: “a shining bright, very friendly star.”15

All three examples in different ways are located at the intersection of theatre, society, and Sound. I will now suggest three directions in which further research at this intersection could and should travel with a particular attention to ‘sounding,’ ‘voicing,’ ‘listening,’ and ‘musicking.’ This research agenda aims to shed more light on what I have called the dispositif of Sound in theatre earlier allowing us to better understand how Sound shapes and communicates artistic identities and becomes the medium and the message of social engagement. The three phenomena to pay attention to are:

1) Recently, theatremakers have innovated their ways of working by employing Sound as a core means to transform their creative processes and create artistic identities.

2) In communicating these identities, theatremakers have embraced Sound as part of their branding, engaging in a complex confluence of audio marketing, auditory and sonic predispositions,16 on-stage voice styles, composition and sound design in performances, venue architectures and acoustics, equipment choices etc.

3) Theatres have also become spaces in which and through which commercial sonification and its manipulative methods of grabbing attention can be critiqued and questions of sonic representation and inclusivity can be addressed. Theatre thus allows for a more conscious reflection on how music, voice, and sound can relate to bodies, texts, and spaces. It reflects our rapidly changing acoustic ecology, and the ethics of the inclusion and exclusion of individual sounds from our sonic environment. 17

By treating Sound as a dispositif, I seek to delimit our understanding of ‘sound ’ It is no longer seen as mainly the acoustic elements of a theatrical performance, but instead accounts for the complex confluence of practices, discourses, and technologies of Sound. These can be analysed by applying principles of dispositif analysis which have been used successfully and made operational in sociology and linguistics18 but have also already been used in theatre studies19 and sound studies.20 Dispositifs include “actions (non-discursive practices),” “discursive practices” and “physical objects.”21 In relation to Sound in/of theatre, we could translate this into three layers22 (see fig. 2):

• Modalities of Sound: sounding, listening, hearing.

• Concepts of Sound: meta-discourses and preconceptions about sound

• Frames of Sound: architectures, equipment, technologies, conditions

This approach changes the existing narrative(s) on auditory aspects in theatre: from a predominant focus on the experiential and semiotic functions of music and sound in theatre performances, the attention moves towards evaluating of what the Sound of a theatre consists of more widely, and how it helps to determine the profile, community, and identity of theatremakers. I thus respond to a recently reinvigorated interest of theatre practitioners in sonic dimensions of conceptualising, creating, and performing theatre. Ahead of his time, Peter Sellars proclaimed as early as 1992:

Sound is no longer an effect, an extra, a garni supplied from time to time to mask a scene change or ease a transition. …Sound is the holistic process and program that binds our multifarious experience of the world. Sound is our inner continuity track. It is also our primary outward gesture to the world, out first and best chance to communicate with others, to become part of a larger rhythm.23

With the increasing velocity and expanding range of possibilities which digital sound production has afforded in the past few decades, the role of Sound in the theatre has changed fundamentally. For many theatremakers sound is no longer an afterthought, but a fundamental part of their process, their creative persona, and a defining element of their ensemble.24 This is also expressed in the frequently highlighted performativity of sounding and musicking. Often, the making of sound and music, the materiality of its instruments, and the eventfulness of the experience they afford are deliberately showcased, staged, and exposed.

Some of these tendencies have been studied before, often through methods of performance analysis. Here, I seek to redirect the attention to the transformations which theatre ensembles undergo by engaging with sonic-led creation, branding, and engagement. Sound is no longer merely a device amongst many. In many practices the centre has shifted towards a dispositif that becomes central to the inward- and outward-facing formation of identities of theatremakers and defines their sound ethics, their “agitatory acoustic aesthetics,” as Lynne Kendrick has called it.25

Sound is therefore understood here both as a sum of sound effects and events and an overarching quality of the acoustic sphere of a theatre; as both an ensemble of practices of sounding and listening and a set of discourses, concepts, and predispositions towards sound; as both an ephemeral event and a more tangible frame of acoustic and technological conditions. I shall now expand briefly on the three directions I referred to earlier: the role of Sound in shaping of artistic identities, in branding one’s company, and in engaging with a given acoustic ecology.

Artistic Identity through Sonic Practices

There are numerous theatre companies or ensembles, who consider sonic practices fundamental to their artistic identity.26 This is a praxeological aspect of this attention to Sound: how do practices of sounding and musicking feature in the processes of group dynamic, communal training, research and development, and rehearsals? How do they feature in the phenomenologies or narratives of artistic identity of the group?

Examples may include practices of rehearsing to curated music tracks or playlists as we find them in Jürgen Kruse, Jan Klata, or Armin Petras27, practices of communal singing and/or shared voice training as we find them in Gardzienice28 or Marthaler, improvised collective instrumental practices in some of Karin Beier’s productions, in UK’s theatre collective Filter or, historically, in Joe Chaikin’s “jamming” exercises in the 1970s. Finally, this may involve practices of environmental listening or using Foley sounds (e.g., with Katie Mitchell). All of these are examples for how theatre companies make Sound an integral part of how they work and who they are. To examine this, music sociology offers rich models on how individuals or groups use music to modulate mood, enhance performance, stimulate creativity, or create cohesion. 29

Coming back to my early example of the Théâtre du Soleil, it is striking how much has been written and said about Mnouchkine’s work as a principal of a theatre and as its only director, while there is a surprising reluctance in the documentaries and articles to discuss the enormous influence of her musical collaborator since 1979, Jean-Jacques Lemêtre, on the process and the aesthetics of the performance. Judith Miller, in a rare exception, writes about his role: Jean-Jacques Lemêtre is the most involved in the day-to-day rehearsal process. Musician, instrument maker, composer, and teacher Lemêtre improvises alongside the actors, often… finding or making an instrument to match the emotional timbre of the actor’s state. …Lemêtre has also helped develop the vocal characteristics peculiar to Soleil actors: a voice pitched from the chest and the head and emanating from the higher vocal registers. This voice is closer to a singer’s and thus better dovetails with the musical accompaniment. …His participation has been determinate, for example, in creating the exhilarating high-energy entrances and exits for which the Soleil is known.30

Miller’s description underlines, how practices of musicking and sounding are formative to the Soleil’s collective identity, their acting and vocal styles. But together with the aforementioned impressive sound system, music and sound are not just a reference point within the ensemble, but also form part of Théâtre du Soleil’s outward appeal, its branding.

Sound Branding

Branding is a term from marketing and economics and would probably be frowned upon by theatremakers and regarded with a healthy scepticism if not open criticism for its perceived capitalist thinking and underlying neoliberal convictions. I would like to try, however, to strip the relatively new term31 from these undertones and see what it has to offer to the context of entertainment 32 or the arts,33 for which it has only rarely been applied. This allows for asking how theatre companies consciously and unconsciously employ forms of sound branding, how this manifests itself, and what artistic identities and sonic profiles are thus communicated. Sound branding in theatre happens via a number of what branding scholars call “code elements:”34 sound equipment, amplification styles, associate musicians, programming (concerts, song recitals), casting (vocal qualities, dialects, accents), acoustics of the venue, discursive mentions of sound

concepts, PR material and communication such as trailers, CD publications, etc. The branding effect is therefore emergent in two respects: rather than consisting of one concerted effort towards a unified sonic trademark, it is the sum of these distributed practices, decisions, frames, and channels of dissemination. Theatremakers thus employ very different means of sound branding and do so on multiple levels, sometimes more, sometimes less intently.

The aforementioned use of a Marthaler chorus as the holding music; the use of well-known musicians or bands in theatre productions from Tom Waits for Robert Wilson to Duke Special in Deborah Warner’s Mother Courage; or the employment of music dramaturgs in theatres, such as Christoph Gurk or Sebastian Reiter at the Münchner Kammerspiele who curate and collaborate with local music communities, DJs, labels etc. as part of the programming of events, often distinct from the actual theatre repertoire all these may serve as examples of practices of sound branding that increasingly shape how theatres present themselves and are perceived.

Theatre Sound and Acoustic Ecology

The example of the music dramaturg/curator, who in the case of Sebastian Reiter places a strong emphasis on developing and nurturing close relations with musicians and sound artist from post-migrant communities, can also be understood through the framework of acoustic ecology. Theatres are places where societal or political phenomena can be contemplated, critiqued, or boldly envisioned. The question that emerges is, how theatre practitioners position themselves vis-à-vis the sonic environments they find themselves in? Which part of an ‘acoustic ecology’ do they seek to play? In the 1960s, R. Murray Schafer coined this term to refer predominantly to environmental issues of noise pollution, marking the intrusion of men-made sounds into nature. I argue for extending ‘acoustic ecology’ to also address the relationship between dominant and marginalised sounds of humans. Whereas much of recent discussions on representation in the arts focusses on questions of “visibility,”35 I suggest exploring the audibility of marginalised sounds, ‘musics’, and voices in theatre.36 This foregrounds the current themes of audibility and sonic presence of a range of intersectionally marginalised communities and sonic agents, who are not sufficiently heard, rendered audible, or included in the mix.

By exploring casting decisions such as the employment of Kuljic and Rebgetz I mentioned earlier, forms of participation, inclusion, accessibility, and the more varied use of music and sound, one can analyse the sonic diversity of theatre in relation to artists and audiences with respect to differences in culture, gender, race, or ability. One can thus pay attention not just to what these communities have to say, but also how they say it and how they sound.

Possible case studies for such “theatres of engagement,”37 using Andy Lavender’s phrase, range widely: from the intercultural work of Swedish/Vietnamese group

The Six Tones and directors such as Rabih Mroué or Toshiki Okada, the European Network Sign & Sound Theatre Europe, which promotes and supports deaf theatre, theatre company Graeae (UK), which trains and stages actors, writers, and directors with disabilities, to artistic director Milo Rau and the NT Gent and their political theatre, which documents voices e.g. from postcolonial conflicts.

Conclusion

While sound was always an important, sometimes even predominant aspect of theatre in its history,38 it seems that given the multitude of re-appearances of Sound in the present now is a good time to expand its place in theatre research

The approaches I have outlined are only three of many. What they have in common is that they extend the remit of sound research in theatre: beyond the analysis of performances itself to acknowledging praxeological dimensions of theatremaking, and beyond studying the aesthetic, narrative or dramaturgical functionality of sound and music towards enquiring about its social, economic, political, and ethical impetus.

Author Biography

David Roesner is Professor for Theatre and Music-Theatre at the LMU Munich. He previously worked at the Universities of Hildesheim, Exeter, and Kent. In 2003 he published his first monograph on ‘Theatre as Music’ and later won the Thurnau Award for Music-Theatre Studies for his article “The Politics of the Polyphony of Performance” in 2007. Recent publications include Theatre Noise. The Sound of Performance (with Lynne Kendrick, CSP, 2011), Composed Theatre. Aesthetics, Practices, Processes (with Matthias Rebstock, Intellect, 2012), Musicality in Theatre. Music as Model, Method and Metaphor in Theatre-Making (Ashgate 2014) and his monograph Theatermusik. Analysen und Gespräche (Theater der Zeit 2019). From 2018–2022 he was Principal Investigator of a DFG research project on contemporary theatre music (see https://theatermus.hypotheses.org/). His podcast is called “Staging Sound”. David also occasionally works as a theatre musician and sound designer. For a full list of publications and projects see: http://mhn.academia.edu/DavidRoesner or https://orcid.org/00000002-4371-1852.

Notes

1. Ross Brown, Sound Effect. The Theatre We Hear (London, UK; New York, NY: Bloomsbury, 2020), xxi.

2 Susan Bennett, Theory for Theatre Studies: Sound (New York & London: Bloomsbury Methuen Drama, 2019); Lynne Kendrick, Theatre Aurality (Houndmills, Basingstoke: Palgrave, 2017); Dylan Robinson, Hungry Listening: Resonant Theory for Indigenous Sound Studies, Indigenous Americas, (Minneapolis: University of Minnesota Press, 2020).

3. Katarina Rukavina, “‘Ocularcentrism’ or the Privilege of Sight in Western Culture. The Analysis of the Concept in Ancient, Modern, and Postmodern Thought,” Filozofska Istrazivanja, no. 32 (2013).

4 Walter J. Ong, Orality and Literacy (London: Methuen, [1982] 1989).

5 Ong, Orality and Literacy

6 Caleb Kelly, Sound, Documents of contemporary art, (London; Cambridge, Mass.: Whitechapel Gallery; MIT Press, 2011). See also Tom McEnaney, “The Sonic Turn,” Diacritics 47, no. 4 (2019).

7 Adrian Curtin and David Roesner, “Sounding out ‘the scenographic turn’: eight position statements.”

With statements from Ross Brown, Adrian Curtin, George Home-Cook, Lynne Kendrick, David Roesner, Katharina Rost, Nicholas Till and Pieter Verstraete,” Theatre and Performance Design 1, no. 1/2 (2015), https://doi.org/http://dx.doi.org/10.1080/23322551.2015.1027523. Original emphasis.

8. Hans-Joachim Braun, “An Acoustic Turn? Recent Developments and Future Perspectives of Sound Studies,” AVANT VIII, no. 1/2017 (2017): 1, https://doi.org/10.26913/80102017.0101.0005.

9 McEnaney, “The Sonic Turn,” 80.

10 Trevor Pinch and Karin Bijsterveld, eds., The Oxford Handbook of Sound Studies (Oxford: Oxford University Press, 2012); Holger Schulze, ed., Sound Studies: Traditionen - Methoden - Desiderate. Eine Einführung (Bielefeld: transcript, 2008); Jonathan Sterne, The Sound Studies Reader (New York: Routledge, 2012).

11 Petra Maria Meyer, ed., Acoustic turn (Tübingen: Fink, 2008), 12.

12 “Ariane Mnouchkine in Conversation With Katie Mitchell”, Torch. The Oxford Research Centre in the Humanities, accessed February 14, 2023, https://www.torch.ox.ac.uk/event/ariane-mnouchkine-in-conversation-with-katie-mitchell.

13 See Stefan Kirschner, “Christoph Marthaler sagt leise Servus”, Berliner Morgenpost, accessed February 14, 2023,

https://www.morgenpost.de/kultur/article208280973/Christoph-Marthaler-sagt-leiseServus.html.

14 A compilation of press cuttings on the early controversies accompanying Matthias Lilienthal’s directorship in Munich demonstrates this. See theaterkritik.de (accessed 16 Feb 2023),

http://testbereich33.nachtkritik.de/index.php?option=com_content&view=article&id=13 249:presseschau&catid=242&Itemid=62.

15 Egbert Tholl, “Jelena Kuljić und ihre Band KUU!”, Süddeutsche Zeitung, 7 Sept 2018 (accessed 16 Feb 2023), https://www.sueddeutsche.de/kultur/musik-jelena-kuljic-undihre-band-kuu-1.4120622.

16 Brown, Sound Effect. The Theatre We Hear, xxi.

17. Bennett, Theory for Theatre Studies: Sound; Kendrick, Theatre Aurality; Robinson, Hungry Listening: Resonant Theory for Indigenous Sound Studies

18 Andrea D. Bührmann and Werner Schneider, eds., Vom Diskurs zum Dispositiv: Eine Einführung in die Dispositivanalyse (Bielefeld Transcript, 2008); Joannah Caborn, “On the Methodology of Dispositive Analysis,” Critical Approaches to Discourse Analysis Across Disciplines 1, no. 1 (01/01 2007); Siegfried Jäger, “Theoretische und methodische Aspekte einer Kritischen Diskurs- und Dispositivanalyse,” in Handbuch Sozialwissenschaftliche Diskursanalyse, ed. by Reiner Keller, Andreas Hirseland, Werner Schneider, Willy Viehöver Duisburger Institut für Sprach- und Sozialforschung (Opladen: Leske + Budrich, 2001).

19 Lorenz Aggermann, Georg Döcker, and Gerald Siegmund, Theater als Dispositiv. Dysfunktion, Fiktion und Wissen in der Ordnung der Aufführung, Theaomai - Studien zu den performativen Künsten, (Frankfurt: Peter Lang GmbH, 2017); David Roesner, Musicality in Theatre – Music as Model, Method and Metaphor in Theatre-Making (Farnham: Ashgate, 2014).

20 Schulze, Sound Studies: Traditionen - Methoden - Desiderate. Eine Einführung

21 Caborn, “On the Methodology of Dispositive Analysis,” 115.

22 Brown, Sound Effect. The Theatre We Hear, 28.

23 Cit. in: Deena Kaye and James LeBrecht, Sound and Music for the Theatre (New York: Back Stage Books, 1992), vii., original emphasis.

24 Duška Radosavljević, The Contemporary Ensemble. Interviews with Theatre-Makers (London & New York: Routledge, 2013).

25 Lynne Kendrick and David Roesner, eds., Theatre Noise. The Sound of Performance (Newcastle upon Tyne: Cambridge Scholars Publishing, 2011), xv.

26 A great collection of such artists and ensembles can be found and studied on Duška Radolsavljević’s research website https://www.auralia.space (last accessed on 18 Feb 2023).

27 Thanks to Anna Burzyńska for directing my attention to this.

28. See Paul Allain, Gardzienice: Polish Theatre in Transition (Amsterdam: Harwood Academic, 1997); Alison Hodge, “Włodzimierz Staniewski: Gardzienice and the Naturalised Actor,” in Actor Training, ed. Alison Hodge (London: Routledge, 2010); Ben Spatz, “To Open a Person: Song and Encounter at Gardzienice and the Workcenter,” Theatre Topics 18, no. 2, September 2008 (2008); Konstantinos Thomaidis, “Singing from Stones: Physiovocality and Gardzienice’s Theatre of Musicality,” in Gestures of Music Theater: the Performativity of Song and Dance, ed. Dominic Symonds and Millie Taylor (Oxford, UK: Oxford University Press, 2014).

29. See Tia DeNora, Music in Everyday Life (Cambridge: Cambridge University Press, 2000); Sara Horsfall, Jan-Martijn Meij, and Meghan D. Probstfield, Music Sociology: Examining the Role of Music in Social Life (Boulder, CO: Paradigm Publishers, 2013).

30. Judith G. Miller, Ariane Mnouchkine, First edition re-issued ed., Routledge performance practitioners, (2018), 56.

31 John Groves, “A Short History of Sound Branding,” in Audio Branding: Brands, Sound and Communication, ed. Kai Bronner and Rainer Hirt (Baden-Baden: Nomos Verlagsgesellschaft mbH & Co. KG, 2009); Paul Steiner, Sound Branding. Grundlagen akustischer Markenführung (Wiesbaden: Springer Gabler, 2018); Alison C. Wong, “Sound branding: The role of music in consumer perceptions, behaviours, and practitioner beliefs” (PhD University of Sheffield, 2018).

32. Elizabeth Barfoot Christian, ed., Rock Brands: Selling Sound in a Media Saturated Culture (Blue Ridge Summit, US: Lexington Books, 2010).

33 Carsten Baumgarth and Daragh O’Reilly, “Brands in the arts and culture sector,” Arts Marketing: An International Journal: Special Issue: Brands in the arts and culture sector 4, no. 1/2 (2014); Daragh O’Reilly, Ruth Rentschler, and Theresa Kirchner, eds., The Routledge Companion to Arts Marketing (London: Routledge, 2014).

34 Steiner, Sound Branding. Grundlagen akustischer Markenführung, 6.

35 Elizabeth Brendel Horn, ““Do you see me?” Power and Visibility in Applied Theatre With Black Male Youth and the Police,” Youth Theatre Journal 31, no. 2 (2017/07/03 2017), https://doi.org/10.1080/08929092.2017.1370756, https://doi.org/10.1080/08929092.2017.1370756.

36 See Natalie Alvarez et al., “Cultural Diversity as Theatrical Practice,” Theatre Research in Canada / Recherches théa trales au Canada 39, no. 1 (2018), https://doi.org/https://doi.org/10.7202/1055475ar; Ella Finer, “Strange Objects/Strange Properties: Female Audibility and the Acoustic Stage Prop,” in Voice Studies: Critical Approaches to Process, Performance and Experience, ed. Konstantinos Thomaidis and Ben Macpherson (New York, NY: Routledge, 2015); Paula Hildebrandt et al., Performing Citizenship. Bodies, Agencies, Limitations (New York: Palgrave, 2019), https://doi.org/10.1007/978-3-319-97502-3. for research on partial aspects of this topic.

37 Andy Lavender, Performance in the Twenty-first Century: Theatres of Engagement (London; New York: Routledge, 2016).

38 Bennett, Theory for Theatre Studies: Sound

Bibliography

Aggermann, Lorenz, Georg Döcker, and Gerald Siegmund. Theater als Dispositiv. Dysfunktion, Fiktion und Wissen in der Ordnung der Aufführung. Theaomai - Studien zu den performativen Künsten. Frankfurt: Peter Lang GmbH, 2017.

Allain, Paul. Gardzienice: Polish Theatre in Transition. Amsterdam: Harwood Academic, 1997.

Alvarez, Natalie, Ric Knowles, Sue Balint, and Peter Farbridge. “Cultural Diversity as Theatrical Practice.” Theatre Research in Canada / Recherches théâtrales au Canada 39, no. 1 (2018): 96–114.

https://doi.org/https://doi.org/10.7202/1055475ar.

Baumgarth, Carsten, and Daragh O’Reilly. “Brands in the arts and culture sector.” Arts Marketing: An International Journal: Special Issue: Brands in the arts and culture sector 4, no. 1/2 (2014): 2–9.

Bennett, Susan, Theory for Theatre Studies: Sound. New York & London: Bloomsbury Methuen Drama, 2019.

Braun, Hans-Joachim. “An Acoustic Turn? Recent Developments and Future Perspectives of Sound Studies.” AVANT VIII, no. 1/2017 (2017): 1–91.

https://doi.org/10.26913/80102017.0101.0005.

Brown, Ross. Sound Effect. The Theatre We Hear. London, UK; New York, NY: Bloomsbury, 2020.

Bührmann, Andrea D., and Werner Schneider, eds. Vom Diskurs zum Dispositiv: Eine Einführung in die Dispositivanalyse. Bielefeld Transcript, 2008.

Caborn, Joannah. “On the Methodology of Dispositive Analysis.” Critical Approaches to Discourse Analysis Across Disciplines 1, no. 1 (01/01 2007): 115–23.

Christian, Elizabeth Barfoot, ed. Rock Brands: Selling Sound in a Media Saturated Culture. Blue Ridge Summit, US: Lexington Books, 2010.

Curtin, Adrian, and David Roesner. “Sounding out ‘the scenographic turn’: eight position statements. With statements from Ross Brown, Adrian Curtin, George Home-Cook, Lynne Kendrick, David Roesner, Katharina Rost, Nicholas Till and Pieter Verstraete. Theatre and Performance Design 1, no. 1/2 (2015): 107–25.

https://doi.org/http://dx.doi.org/10.1080/23322551.2015.1027523

DeNora, Tia, Music in Everyday Life. Cambridge: Cambridge University Press, 2000.

Finer, Ella. “Strange Objects/Strange Properties: Female Audibility and the Acoustic Stage Prop.” In Voice Studies: Critical Approaches to Process, Performance and Experience, edited by Konstantinos Thomaidis und Ben Macpherson, 177–87. New York, NY: Routledge, 2015.

Groves, John. “A Short History of Sound Branding.” In Audio Branding: Brands, Sound and Communication, edited by Kai Bronner und Rainer Hirt, 62-75.

Baden-Baden: Nomos Verlagsgesellschaft mbH & Co. KG, 2009.

Hildebrandt, Paula, Kerstin Evert, Sibylle Peters, Mirjam Schaub, Kathrin Wildner, and Gesa Ziemer. Performing Citizenship. Bodies, Agencies, Limitations. New York: Palgrave, 2019. http://dx.doi.org/10.1007/978-3-319-97502-3

Hodge, Alison. “Włodzimierz Staniewski: Gardzienice and the Naturalised Actor.” In Actor Training, edited by Alison Hodge, 268-87. London: Routledge, 2010.

Horn, Elizabeth Brendel. ““Do you see me?” Power and Visibility in Applied Theatre With Black Male Youth and the Police.” Youth Theatre Journal 31, no. 2 (2017/07/03 2017): 79-91.

https://doi.org/10.1080/08929092.2017.1370756

Horsfall, Sara, Jan-Martijn Meij, and Meghan D. Probstfield. Music Sociology: Examining the Role of Music in Social Life. Boulder, CO: Paradigm Publishers, 2013.

Jäger, Siegfried. “Theoretische und methodische Aspekte einer Kritischen Diskurs- und Dispositivanalyse.” In Handbuch Sozialwissenschaftliche Diskursanalyse, ed. by Reiner Keller, Andreas Hirseland, Werner Schneider, Willy Viehöver Duisburger Institut für Sprach- und Sozialforschung, 81–112.

Opladen: Leske + Budrich, 2001.

Kaye, Deena, and James LeBrecht. Sound and Music for the Theatre. New York: Back Stage Books, 1992.

Kelly, Caleb, Sound. Documents of contemporary art. London; Cambridge, Mass.: Whitechapel Gallery; MIT Press, 2011.

Kendrick, Lynne. Theatre Aurality. Houndmills, Basingstoke: Palgrave, 2017.

Kendrick, Lynne, and David Roesner, eds. Theatre Noise. The Sound of Performance. Newcastle upon Tyne: Cambridge Scholars Publishing, 2011.

Lavender, Andy. Performance in the Twenty-first Century: Theatres of Engagement. London; New York: Routledge, 2016.

McEnaney, Tom. “The Sonic Turn.” Diacritics 47, no. 4 (2019): 80-109.

Meyer, Petra Maria, ed. Acoustic turn. Tübingen: Fink, 2008.

Miller, Judith G. Ariane Mnouchkine. Routledge performance practitioners. First edition re-issued ed., 2018.

O’Reilly, Daragh, Ruth Rentschler, and Theresa Kirchner, eds. The Routledge Companion to Arts Marketing. London: Routledge, 2014.

Ong, Walter J., Orality and Literacy. London: Methuen, 1989 (1982).

Pinch, Trevor, and Karin Bijsterveld, eds. The Oxford Handbook of Sound Studies. Oxford: Oxford University Press, 2012.

Radosavljević, Duška. The Contemporary Ensemble. Interviews with TheatreMakers. London & New York: Routledge, 2013.

Robinson, Dylan, Hungry Listening: Resonant Theory for Indigenous Sound Studies. Indigenous Americas. Minneapolis: University of Minnesota Press, 2020.

Roesner, David. Musicality in Theatre – Music as Model, Method and Metaphor in Theatre-Making. Farnham: Ashgate, 2014.

Rukavina, Katarina. “‘Ocularcentrism’ or the Privilege of Sight in Western Culture. The Analysis of the Concept in Ancient, Modern, and Postmodern Thought.” Filozofska Istrazivanja, no. 32 (2013): 539–56.

Schulze, Holger, ed. Sound Studies: Traditionen - Methoden - Desiderate. Eine Einführung. Bielefeld: transcript, 2008.

Spatz, Ben. “To Open a Person: Song and Encounter at Gardzienice and the Workcenter.” Theatre Topics 18, no. 2, September 2008 (2008): 205-22.

Steiner, Paul, Sound Branding. Grundlagen akustischer Markenführung.

Wiesbaden: Springer Gabler, 2018.

Sterne, Jonathan. The Sound Studies Reader. New York: Routledge, 2012.

Thomaidis, Konstantinos. “Singing from Stones: Physiovocality and Gardzienice’s Theatre of Musicality.” In Gestures of Music Theater: the Performativity of Song and Dance, edited by Dominic Symonds und Millie Taylor, 242–58. Oxford, UK: Oxford University Press, 2014.

Wong, Alison C. . “Sound branding: The role of music in consumer perceptions, behaviours, and practitioner beliefs.” PhD, University of Sheffield, 2018.

The Audiovisual Chord

Invitation to a Dance Between Sound and Image

Abstract

This article introduces the author’s book The Audiovisual Chord: Embodied Listening in Film. It starts with a quote by Walter Murch about film experience and perception as a dance between sound and image, directing our attention to the interval between sound and image in film. The book presents a phenomenological approach to film sound and film as a whole, with a focus on movement, embodied listening and the body as a sense of movement, that brings all sensory impressions together. This approach includes the audiovisual chord as a dynamic knot of visual and auditory movements. From this perspective, auditory spaces in film can be used as a pivot between an inner and an external world. As this book is primarily addressed to film-makers and film sound professionals, reflection always starts from case studies, inviting the reader to first experience film as a phenomenological art: appearing to the audience without immediately revealing a meaning or a story. With Robert Bresson’s A Man Escaped (1956) as a key case, Bresson is presented as a phenomenologist. Reading Merleau-Ponty’s phenomenology from this perspective, reveals a better understanding of Bresson’s vision on film as a transmission of experience and presentation of a life-world (Le cinématographe). Some analytical tools as sound as a dynamic transmodal movement, thinking in movement, auditory filmic space and the audiovisual chord are presented to enable the application of these insights in filmmaking and film analysis. The concept of the audiovisual chord is both a part of the film composition and a tool useful to analyze the complexity of the interwoven dynamic visual and auditory movements, finding their anchors in concrete lifeworlds. In this way, each member of the audience resonates differently with the presented filmic movements and composition and is invited to a personal dance.

The Audiovisual Chord: Embodied Listening in Film is the result not only of research, but also of a long journey of coaching film students in their creative process. What is needed to think about sound and music in the creative process of making a film? Does this necessarily start from the script, from the narrative or from the image? Are different approaches possible? Which ones are preferable for certain forms of creation? Looking at films by Robert Bresson or Andrei Tarkovsky and reading their texts,1 we realize that film is more than telling a story. They speak about film as a composition, presenting a life-world to an audience. So, what happens if we direct our attention to the audience? How does the audience experience the life-world of a film? Is film sound then still an added value to the image? Walter Murch directs our attention to this perspective:

In a film there’s a dance between the words, the images and the sounds. As rich as films appear, they are limited to two of the five senses hearing and sight and they are limited in time the film lasts only as long as it takes to project it. … But a film can provoke the audience’s participation. …So, the film, although it’s materially the same series of images and sounds, should ideally, provoke slightly different reactions from each person who sees it. (Walter Murch in Ondaatje 2002, 46)

According to Murch, each person can experience and perceive the same film in a different way. He also directs our attention to the interval between image and sound. However, reading this quote leaves a number of questions unanswered.

How it is possible that two senses (hearing and sight) can provoke the other senses? Do we have to think about sound and image in film in a non-hierarchical way? What kind of movement is at the basis of this dance? How important is the rhythm of the film for this dance? What is the relation of the audience with the movements seen on screen and heard in the film? Is the audience dancing with the film? What is the relation between this dance and the meaning it provokes? How is it possible to analyze a gap, something that exits in-between, but is not materialized in se? What are the consequences of this approach for the creative process? What are possible key concepts for this approach?

The Audiovisual Chord: Embodied Listening in Film

A formatted approach to film-making following the efficient workflow of the film industry in the first place focuses on narration (script) and image. Sound then is rather an added value. But thinking about the audience’s dance between sound and image presupposes not only sound and image to be on the same level but also a body, movements and spaces.

As this book not only has an academic purpose, but is also addressed to filmmakers and film sound professionals, reflection always starts in line with a phenomenological attitude from concrete case studies: films and film fragments inviting the reader to first experience the film before analyzing and reflecting on the relation between sound and image. This relation is approached from the auditory perspective, complementary to the multitude of writings on film from the perspective of the image or the narration. Sound as a dynamic transmodal movement and embodied listening are at the basis of the presented phenomenological approach to film and film sound.

Different readings of the book are possible. The reader can start with the introduction in phenomenology or can first dive into the case studies.

The book is set up in three parts:

Part 1 introduces a phenomenological approach to film sound and listening, based on the phenomenologies of Edmund Husserl and Maurice Merleau-Ponty, with Robert Bresson’s writings on le cinématographe and his film A Man Escaped (1956) as a case study. This approach is confronted with the film sound theories of Michel Chion, Gilles Deleuze and Véronique Campan.

Part 2 presents sound as a transmodal dynamic movement, thinking in movement, the filmic auditory spaces and the audiovisual chord as new insights to practice a phenomenological approach to the experience and perception of film sound and film with case studies to enable a better understanding of these insights.

In Part 3, the audiovisual chord with the body as the sense of movement in the audience’s film experience is presented in film conceived as an audiovisual composition. A revisiting of the early sound film, reveals how film as an audiovisual composition was a new challenge for film-makers. The last chapters focus on the consequences of this compositional approach to film sound, film editing and filmmaking. An overview of different possibilities of sound relating to image in an audiovisual chord (from synchresis to complete polychrony and polytopy to immersion) shows how rich these relations might be. With the body of the listener/spectator at the center, it is possible to present inner spaces next to external spaces.

A Phenomenological Approach to Film Sound and Listening?

Exploring the intertwining of sound and image in film theory, Michel Chion, Gilles Deleuze and Veronique Campan contribute in a different but complementary way to a better understanding of film as an audiovisual composition, with a special attention to sound and listening. With the concepts of synchresis and audiovisual contract, Chion opens the field of the audiovisual perception in which looking and listening are always interdependent. However, he describes film sound as an added value to the image, thus filtering the audiovisual perception through sight.

Gilles Deleuze goes a step further. Inspired by Pierre Boulez’ non-hierarchical thinking in composition, he gives sound and image a rather autonomous (heautonomous) status in film. Both elements of the same film experience and perception are able to present different life-worlds at the same time. Deleuze drives our attention to the space in-between, to the interval and to the relations between the elements. For him film is a whole, a composition consisting of the totality of relations without a preconceived causal and/or logical succession of signs.

Véronique Campan radically starts from "filmic listening," discussing the movement of (recorded) sound as a dynamic movement containing "acoustic traces" and the spatiality of the original sound. Stressing the specificity of the experience and perception of sound and sound editing as a time-based capacity, she introduces a phenomenological approach to film sound and listening, based on Husserl’s concept of inner time-consciousness. A phenomenological attitude to film composition and film perception allows to reflect on that part of film and film sound that usually escapes our attention.

Embodied Listening Presupposes Movement and Body

Cinematography is a writing with images in movement and with sounds. (Bresson 1986, 7)

As a phenomenologist ‘ pur sang, ’ Robert Bresson challenges us to create a new theoretical framework to reflect on the role that sound plays in his le cinématographe. Images and sounds must sustain one another without being independent from each other, he states (Bresson 1986, 50). The composition of a film has to convey a character’s inner conflict and engagement with an object. This shapes the filmic movement, with rhythm as the coordinating force. Bresson invites us to rethink the intertwining of sound and image in film and also to focus our attention on the audience’s experience and perception to bring the different elements together into a whole.

Cinematography: a new way of writing, therefore of feeling (Bresson 1986, 21)

Film as a composition of sounds (including music and speech) and (moving) images is indeed always experienced by someone. This brings us towards the embodied first-person perspective with the role of the body in film experience and towards the phenomenology of Maurice Merleau-Ponty and Edmund Husserl. According to them, the body is at the center of its experience, bringing all different sensory impressions together without reflecting on it. In this way, they reveal the importance of the pre-reflective non-thematic awareness of something as a bodily experience. An important aspect of film experience that generally is absent in film theory.

It is important to realize that a phenomenological approach to sound and listening in film cannot start from “objective” sound. It always starts from someone’s experience of sound and listening in film. There is thus always a first-person perspective. But it has to be said: as listener/spectators we constantly experience sounds in film without reflecting on them. Film sound almost always escapes our attention. But not reflecting on something doesn’t mean that we are not aware of it. In doing the exercise of listening carefully (as Michel Chion asks his students in film sound analysis), 2 we realize that we heard the sound, but didn’t direct our attention to it, nor were we thinking about the sound in terms of a source or a meaning. Dwelling on the importance of sound in film perception, we have to realize we often bodily react to a sound. This can result in a shift of our attention, ‘dancing between sound and image.’ Taking into account the experience of sound and listening and the way our body resonates with sound, we are invited not only to approach film sound differently but also to film as a whole.

Film as a Phenomenological Art

In phenomenology a distinction is made between the appearance and the intentional object constituted in the intentional act. Applied to film, this means that a film appears to the audience, without immediately revealing a meaning or a story. Film is not a given. A story for example (intentional object), is not presented on a silver platter, but is constituted in an intentional act, taking different perspectives of space, time, standpoints etc. into account. The audience needs time to experience and to perceive a film. During this process, the audience tries to ‘understand’ what is happening. According to Edmund Husserl, this intentional act is motivated by a kinaesthetic experience, a bodily resonance that starts the process. Resonating with a certain character or a certain aspect of the film, is thus at the basis of a possible interpretation. In fact, we all know this from discussions we had after a film screening where the same film evoked different experiences.

Husserl’s concept of inner time-consciousness also helps us to understand how different events and elements, presented in a film, can be brought together in a knot without necessarily reflecting on them. It is the experiencing body of someone (a first person) at the center, bringing all the impressions together, that plays the crucial role in the experience of the film as a whole and in connecting with the body of the other (for example a character in the film).

This attention for the body in experience and perception, brings us to Maurice Merleau-Ponty’s phenomenology exploring and elaborating on the idea of the body as the sense of movement Merleau-Ponty states that when we are born, we move towards the world, without reflecting on our gestures (body-schema). We bring different impressions and experiences together without an external thought or predicted format (intentional arc). Anchored in the world, the body understands through movement which plays a key-role as much for being-in-the-world as being-towards-the-world. In one’s relation to the other, Merleau-Ponty stresses the importance of movements and gestures through which communication or understanding is achieved. He speaks about the "synchronic modulation" of experience and perception. To feel is then to resonate with the dynamic movements without already having interpreted them or placed them in a certain context. Due to the different resonances (kinaesthetic experiences) and inner timeconsciousness, different intentional acts and intentional objects can emerge and this way different understandings flow from the same complexity.

The listener/spectator’s body at the center of its experience brings together the different sensorial impressions evoked in the film. Moreover, in phenomenology, the body is never abstract, but always phenomenal. This means that the space in which we deploy concrete movements is always a given space. It is the present world. For Merleau-Ponty however, concrete movements can be projected into more abstract spaces, provoking abstract movement. With this difference between abstract and concrete movements, he draws attention to the fact that we all live in superimposed spaces: at once in the physical and in the virtual, human lived space. In this way, someone’s body can also be seen as a knot of living significations, resonating with abstract and concrete movements. All these insights are helpful for understanding Bresson’s thoughts on le cinématographe and Bresson’s invitation to feel before understanding. Le cinématographe presents film as a whole able to transmit an experience and a life-world.

After presenting new insights in Bresson’s le cinématographe through the lenses of Husserl’s and Merleau-Ponty’s phenomenologies, the book presents analytical tools to enable the applications of these insights in film-making and film analysis: sound as a dynamic transmodal movement, thinking in movement, auditory filmic spaces and the audiovisual chord

With film as an audiovisual composition and a bodily experienced whole for the audience, film sound as a felt sound opens the body of the listener/spectator to receive transmodal sensorial impressions. Without reflecting on it, a unity of all these impressions is given as an original composition. Two regions of the experiential field can be distinguished: a field of phenomena that unfold entirely for me (from within), for example, dreams or phantasies; and a field of phenomena that are experienced by other embodied subjects, as well as by myself, as phenomena from the outer world (from without).

After Alan Clarke’s Elephant (1989) and Céline Sciamma’s Portrait of a Lady on Fire (2019) as case studies of this phenomenological approach, Merleau-Ponty’s writings on film with Zero for Conduct (Jean Vigo 1933, music by Maurice Jaubert) as a case study are presented. He proposes that the pre-objective body schema is at the basis of the perception of movement in film and states that it is neither the eye nor the mind that is perceiving the movement. It is the entire body schema that lives the movement, offering its proper animation to the movement (MerleauPonty [1953] 2011, 28).

According to Merleau-Ponty, the meaning of the film is incorporated into its rhythm as the meaning of a gesture is immediately legible in the gesture. Finally, it is through the temporal or spatial arrangement of its elements and through perception that we can understand the meaning of cinema (Merleau-Ponty 1964, 48-59). The music has to make us physically sensitive to the internal rhythm of the image without trying to translate its sentimental, dramatic or poetic content.

The audience moves with the intention of the film-maker to co-constitute the meaning of the film as a whole: the body of the listener/spectator first of all resonates and moves with this multitude of impressions and dynamic movements. With the body schema referred from within, different states of being, spaces, distorted experiences, meanings and so forth can be evoked or constituted. Due to the different resonances (kinaesthetic experiences) and inner timeconsciousnesses, different intentional acts and objects can emerge, and with them different understandings of the same complexity presented in a film as an audiovisual chord

The Audiovisual Chord

The audiovisual chord is the key concept of the book. It brings together the compositional and the experiential perspective of a film as well as sound and image, together with different lived times and spaces. The audiovisual chord enables us to discuss the different layers coming together in and passing through an experienced moment. This way, it can be used as an analytical tool to understand the composition of the film, to understand how the film-maker guides the audience through the film and to reflect on the multiple thoughts and feelings that are delivered via the screening of a film.

As a compositional concept, the audiovisual chord enables a superimposition of spaces and places in image, sound, music and speech in a poly-chronic context.3 It is a dynamic multisensorial knot of movements which can be experienced and perceived from different perspectives, and whose intervals invite the audience to participate in the film’s dance of image and sound. Here it is also important to realize that the coexistence and interconnection of (abstract) dynamic movements in film are rooted in concrete life-worlds. As a complex of various viewing and listening perspectives the audiovisual chord also enables the description of two different temporal dimensions: that of linear successive time and that of the polychronic moment itself.

In film as an audiovisual composition, the audiovisual chord arrives at its decisive form in editing and post-production, at the crossroads between the sound engineering, the construction of the film and the choreography of the film-maker’s intentions. Its active, rational composition is then intertwined with a deeply attuned feeling for what is presented.

To Conclude

With the compositional concept of the audiovisual chord, as a coexistence and interconnection of (abstract) movements in film rooted in concrete life-worlds, film sound can no longer function as an added value in film as a whole. With sound and image as equal intertwined dynamic movements evoking the world presented in the film, narratives are no longer a given. They are constituted in time, not only from the film-maker’s side in the script or in the editing, but also in the audience’s process of perception of the film. With the audience’s body at the center of its film experience and the (pre-objective) body schema offering its proper animation to movement, the audience is invited to dance with and in-between the sounds and the images of the film, in order to give a proper interpretation of the film as a whole.

Author Biography

Dr. Martine Huvenne retired after a career of teaching and researching in the audio-visual field. She was a senior lecturer on Sound and Music for Film at the Kask & Conservatorium (Hogent-Howest), Belgium, where she developed a phenomenological approach to sound and listening. As an independent researcher, she continues her work collaborating with film-makers and as a lecturer in the School of Sound. Email: maarhuvenne@gmail.com

ORCID: 0009-0007-6565-7184

Notes

1. Robert Bresson, Notes on the Cinematograph (Translated by Jonathan Griffin. New York: New York Review Books, 1986; Tarkovsky, Andrei. 1994. Sculpting in Time: Reflections on the Cinema. Translated by Kitty Hunter-Blair. London, Boston: Faber and Faber)

2 Michel Chion, Audio-Vision: Sound on Screen (Translated by Claudia Gorbman. New York: Columbia University Press, 1994), 185-198

3. Edward T. Hall, The Dance of Life: The Other Dimensions of Time (New York: Anchor Books, 1986), 44-58.

Bibliography

Huvenne, Martine. The Audiovisual Chord: Embodied Listening in Film Singapore: Palgrave Macmillan, 2022.

Sound Design in German Cinema Films

Jörg U. Lensing

Abstract

I assume that both the flow of images and the overall soundtrack of a film are largely meaningless when separated from each other, as they are conceived and designed in relation to each other in a sound film. This means that the information, emotional strands and dramaturgical contextualisation provided in the moving image are supplemented, reinforced, questioned or enriched with a narrative level of meaning by the soundtrack, so that the actual message of the film can only be understood by simultaneously reading the image and sound information. This applies to all sound films, but is used very elaborately in German film. The actual text on the topic “Sound Design in German Cinema Films” begins with the sound film icon M - eine Stadt sucht einen Mörder (M) by Fritz Lang. The following audiovisual analyses, primarily of the opening sequences of a total of six German cinema films, move quickly into the filmmaking of German auteur-filmmakers from the 1980s until today. Films by Wim Wenders, Joseph Vilsmaier, Tom Tykwer or Michael Haneke determine this sound film canon just as much as more recent films by younger filmmakers such as Johannes Naber, Felix Randau or Nora Fingscheidt. It turns out that German film sound design is much more spartan in its use of audio-visual interactions than is the case in Anglo-American films and that it is precisely the successful film sound designs for German films that were initiated by directors who either have musical backgrounds themselves or are very sound-sensitive. What they all have in common, however, is the very conscious use of (usually very little) music, the clear setting of equally concise but meaningful dialogue and a sound design, which in various concepts from reduction to abundance significantly enriches the storytelling and usually adds meta-levels to the narrative.

Since most of the literature on the subject of film sound/sound design published in Germany usually cites US models, in this text I will deliberately concentrate mainly on German films, which however also illustrate the above-mentioned topic in a varied and intelligent manner.

But first, let’s have a look at the terminology:

Sound design for film in Germany is largely limited to sound editing which is usually understood as the clarification of what is seen in the picture, vamped up with sound. Sound usually synchronously reproduces what is seen in terms of action, gaits, body movements, etc. by protagonists in the picture. The same is true for effects and, to a large extent, for atmospheres in films. The only layer of sound that is not considered to have a clarifying but rather an ‘underscoring’ or mood-creating effect is usually the film music. However, this is often used in such a way that the viewer is told in advance what is about to happen or to make clear emotionally what the speaking protagonist or protagonists supposedly feel in these moments of dialogue or what the viewers are supposed to feel.

Of course, an approaching car can also be heard while it is still off-screen or footsteps of people moving away from the picture can still be heard off-screen1 for a while even if they are no longer seen on-screen. In exceptional cases, a car or an atmo2 announces itself even before a time/location cut, which is to be considered a stronger device, since it challenges the correspondence between sound and image for at least a few seconds and inevitably leads to the question: Why here and what does this mean?

Unfortunately, there is no binding set of rules for the use of film music in films, only ‘best practice’ examples. Since such a set of rules does not exist for producers, directors, editors and not even for composers, there is an inflationary use of music in films, although what one hears for the most part especially in German TV productions as ‘music’ in TV films does not deserve this designation. Often and stereotypically used, it frequently features indiscriminate guitar strumming or tinkling piano. Harmonic, electronic soundscapes are remarkable, but they are usually used just as indiscriminately under dialogue as guitar or piano sounds are used in other TV films. More sophisticated international film and series productions use music in audio-visual contexts in a more differentiated way, even for TV, than the practice described above.

So when music is used with discretion and no longer implies, interprets or delivers an emotionally charged atmosphere over longer sequences, the film offers space for a semantic sound level that does more than just illustrate with sound what one already sees. Incidentally, the series offer crystal-clear speech intelligibility for the sparingly used dialogue, applying creative means at the height of current discourse, which is exemplary, especially for TV.

M – eine Stadt sucht einen Mörder (M), 1931

The first film to make consistent use of differentiated sound design for the moving image was M – eine Stadt sucht einen Mörder by Fritz Lang (Fig. 1). Film music as non-diegetic music is not used in this film at all, which is remarkable because a few years earlier a complete interpretation of films through music was still common practice in the so-called silent movies, and Fritz Lang also collaborated with Gottfried Huppertz as composer on a completely non-diegetic orchestral score for Metropolis, for example. M makes room for sound design and uses music exclusively as music coming from the screen by having a rhythmic children’s rhyme chanted by children seen in the picture on and off-screen,3 just as the Grieg melody4 is whistled in the film, mostly invisibly but clearly diegetically asserted by the sought-after child murderer. Lang works with the means of acousmatic concealment,5 for example with phantom images by showing the murderer only from behind or as a shadow. This way of presentation is transferred to all shots of this person until he is finally unmasked; from then on his face can be seen, and only then can he be seen speaking.

In the film’s exposition, the first appearance of the murderer is a shadow of the person on an advertising pillar speaking to a little girl who rhythmically throws a ball against this advertising pillar. The ball hits a WANTED poster, which names the murderer in writing as a child killer. This constellation the shadow of a male person on a poster naming a child murderer, the child throwing her ball repetitively at the poster and the acousmatic voice of the murderer shown only as a phantom is a constriction of means that is clear in its message and foreshadows

that this child will be this man’s next victim. While the man, who is shown again as a phantom (from behind), first whistles the Grieg melody, he buys a balloon from an obviously blind street vendor for the girl, who thanks him nicely. The girl and the balloon seller are seen in profile and from the front so that it becomes clear that neither of them is whistling. There are only three people in the picture, so we conclude that the man shown from behind is the one whistling the tune. At the end of the exposition, the purchased balloon, along with the ball rolling out of a bush, is a symbol of the girl’s death although neither the murder nor the killed girl are shown and made audible. The use of silence as a dramaturgical device is remarkable here: The murderer kills quietly; the girl is silenced. The whistled song, by the way, becomes a recognition sign later in the film letting the blind salesman conclude that the person whistling is the murderer they are looking for.

Although the sound technology is poor and these means are tried out for the first time in a film, Fritz Lang, in collaboration with his sound designers Paul Falkenberg and Adolf Jansen, already attempts various image/sound effects in the exposition of this film that create semantic added value. For example, the children in the courtyard of a tenement block continue to chant their counting-off rhyme offscreen, although a neighbour had previously strongly admonished the children: “Just you wait a little while / The nasty man in black will come / with his little chopper / He will chop you up!” The murderer is later shown both as a shadow and in costume as a man in black. Unlike today’s depictions of violence in film, however, we see neither a cleaver nor chopped up flesh. Fritz Lang did not have to recourse to this because the symbolic language in the film has a much stronger effect than violent images.

Food and cutlery, however, are symbolically used in the flat of the mother who is waiting for her daughter. The set plate remains empty, the cutlery laid out next to it remains unused. Instead, a cuckoo clock and acousmatic church bells tell us how it takes unusually long for the child to come home from school. Just before the aforementioned sequences in which the ball rolls out of the bush and the balloon gets caught in telephone wires conclude the exposition, the child’s mother (filmed from behind) is seen calling out ‘Elsie’ several times from the window. For each call, however, images of empty places are shown (attic, staircase, kitchen): Elsie is not in any of these places, she is in the bush at the same time as the concluding shot described above makes clear directly after the mother’s call.

Even in unimportant moments in this exposition, Lang uses acousmatic information in a directorial way that would not have been possible in a silent movie: The mother hears children’s footsteps in the stairwell while she is cooking, so she goes to the door, sees neighbours’ children running one floor up and asks them, whether Elsie isn’t with them, to which the children reply in the negative. On the street in front of the school, you don’t hear any urban atmosphere or traffic, but you do hear honking cars at the moment when Elsie tries to cross the street carelessly and a policeman stops the traffic for her. That these are not original sounds6 can be deduced from the character of the ‘cleanly’ recorded horns and in

the omission of the car engine sounds from the moving cars. So only the horns are deliberately used in this scene and thus have a warning, i.e. symbolic meaning. The scene with the policeman is directly followed by the advertising pillar scene discussed above.

In M, a director already explores the meta-information possibilities of sound in relation to image in the exposition, but continues with new procedures throughout the entire film and uses them as a field of experimentation in a way that for decades was the stock approach of advanced sound design for films internationally, making M one of the standard examples to this day of the intelligent use of sound for moving images.

Der Himmel über Berlin (Wings of Desire), 1987

In German filmmaking, one has to wait with a few exceptions7 for the late 1970s or early 1980s until further innovative explorations meant in this way are ventured in relation to audio-visual interactions in film. In the course of technical innovations and the emancipation of sound design from image montage, especially since the mid-, at the latest late 1970s, a new generation of filmmakers returning from the USA realises a new form of sound design. In Der Himmel über Berlin (Wings of Desire), Wim Wenders, a German director returning from the USA, realises an experimental film using off-voice8 and other audio-visual means, which on the one hand is reminiscent of radio plays, but on the other, by using off-voice as a narrator’s voice, a first-person voice9 and a commentary voice, above all expands the possibilities of using voice over,10 which is otherwise almost always used only one-dimensionally.

Even the spoken, but also partly sung, quotation of a text written at the same time surprisingly uses a contradiction between what is said and shown.11 The first stanza of the poem ‘When I was a child’ is spoken/sung while in the picture, the second stanza is written down with a fountain pen on a white sheet of paper! Only the scratching of the fountain pen on paper is synchronous while the written words can only be partially synchronous with the text that is heard at the same time. What does this contradiction tell us?

The title has non-diegetic music, and while seeing an angel standing on the Kaiser Wilhelm Memorial Church in Berlin, a harp can be heard coming into the previously established music. The accompanying close territory sound12 of the city environment reflects cars, braking noises and footsteps of passers-by. This is no urban atmo per se and thus no extension,13 although the setting of the angel on the church with a large Berlin panorama in the background would actually demand this. Wim Wenders obviously does not care about a claim to realism (Fig. 2)!

Only children can see the angels, which is visually clarified by the fact that children from a passing bus and a child on a pedestrian crossing stop and silently look up to the angel on the church. To the adults walking heedlessly past the child, one hears a walla14 soundtrack with no discernible statements, while one of the two children on the bus is clearly heard saying ‘Look at this’. While the camera is then directed upwards to the church spire, where the viewer remotely perceives the man with wings, the previously established street noises disappear (hollow sound15).

Afterwards, several people are seen walking, cycling or driving in the streets of Berlin. Remarkably, you hear their inner voices as several voice-overs to the extent that you hear a Turkish family driving a car thinking in Turkish or the actor Peter Falk, playing himself, thinking in US English on a plane flying into Berlin.

The aeroplane scene again has a differentiated sound design. Only one child on the plane can be heard physically, i.e., through set body motion and the handling of props. Other people obviously speaking loudly to a flight attendant cannot be heard, but the stewardess can clearly be heard speaking to the child. The second angel stands in the aisle of the plane and watches the passengers. The inner thought-voices of the passengers he looks at can be heard. Technically, however, the engine sounds are heard as an atmo and the stewardess is heard over the onboard system (on-the-air16).

While the angel supposedly flies around the Berlin Radio Tower, various international radio programmes are heard as an On-the-air collage. That it must be the flying angel becomes clear in the next sequence when, after a flight over a motorway, the camera flies into flats in which the at times almost motionless protagonists sitting there can be heard again with their first-person inner voices. An auditory dissonance results from the following scene showing a woman renovating her flat, whose inner thoughts can be heard as a non-iconogenic first-

person voice,17 while the song ‘Some Guys’ by Tuxedomoon can be heard simultaneously as an On-the-air sound from a radio in the room. This simultaneity of information is reinforced as the camera continues to fly out the window, seeing the front gardens to the house from above, hearing children playing in the verbal chiaroscuro18 and simultaneously hearing music from another neighbouring window. The music lures the angel through the window into another flat where three people are each by themselves sitting in different rooms. The teenager’s music can be heard, the older man’s TV and only atmo and indirect music from the teenager and the man’s TV accompany the image of an older woman in the kitchen. All the while one again hears their respective inner voices. This game continues in flight until the angel from above spots an ambulance driving over the city motorway shown earlier and shortly afterwards he finds himself in the ambulance in which a woman is giving birth to a child.

What are Wim Wenders and his sound designers Axel Arft, Hartmut Eichgrün, Lothar Mankewicz and Jean-Paul Mugel and the composer Jürgen Knieper doing here? The first-person voices in the respective heads illustrate that the angels are capable of hearing these inner voices. On-the-air music is clearly always assigned to human characters while ‘heavenly instruments’ are used as non-diegetic music when we assume the mode of perception of the angels. Logos discernible on buses, on house walls, etc. illustrate the city of Berlin just as much as the territorially always narrowly and selectively used city sounds. This narrow sonic landscape, which often corresponds to the image framing, always focuses the viewer’s attention on the angels and a maximum of one or two human figures. Another device not yet mentioned here are anticipations or overlappings of sound before or across image cuts. Where they appear, they create an organic continuity similar to the aeroplane which is seen in the sky before the angel jumps into it or the ambulance, which is seen from a bird’s eye view before the angel flies into it. This form of asynchronous image/sound editing illustrates the angels’ multitasking as well as their ability to listen to people’s thoughts. They can perceive other things, even things that are further away, even though they are still or already directing their gaze at something else as in the opening sequence ‘When I was a child’.

Without this form of sound associations to the image, one would in fact have none of the interpretations posited above, except that the film is set in Berlin, angels are watching people, angels are seen by children and obviously can fly. Otherwise, one would only see many lonely people in flats, walking down streets and in an aeroplane.

One could still go into what the first-person voices say in each case, which is hardly meaningful in detail, but in sum results in a spectrum of themes and music heard by the protagonists, which can certainly be interpreted as a time stamp portrait of Berlin in the mid-1980s as surely intended by Wenders and which is seen and heard today as a document of the time.

Drei (Three), 2009

Tom Tykwer is not only a film director but also a musician. For several years, Tykwer took lessons with the Wuppertal jazz pianist Bernd Köppen. Together with Reinhold Heil19 and Johnny Klimek, he also writes the music for many of his films.20 The film Drei (Three) is Tykwer’s first German-language film after 10 years of international filmmaking in collaboration with US companies, and it is also a ‘Berlin film’ (Fig. 3).

Tykwer is very aware of this fact and places himself directly in the cinematic Berlin tradition with the first image of his film by quoting the iconic image of the train entering Berlin past telephone poles from Walter Ruttmann’s Berlin Symphony of a Metropolis. He quotes it visually but uses the random progression of two, three, one and in the end no telephone wires for the interpretive narration of the male protagonist’s off-screen voice. The auditory narration of ‘randomly’ interpreted lines tells in key words the live history of the couple Hanna and Simon: in the end one wire breaks off to the words ‘You die’, the remaining wire also breaks off shortly afterwards to the words ‘Me too’. The telephone poles that the train passes give a rhythm to the image, but they also structure the flow of words in that a new keyword-like piece of information is given for each telephone pole.

Simon’s off-screen voice then becomes an on-screen voice with the words ‘You on top, me on the bottom’ when we see Simon lying on top of Hanna as a said/shown scansion.21 That this has sexual connotations is made clear in the subsequent dialogue with ‘I thought you’d sleep with me again for a change’. In the sequence with the telephone cables, three wires could be seen momentarily, which on the one hand verbally referred to a child which was apparently lost and on the other suggested a ménage à trois, which then dissolved again. The film will be about just such a relationship when the third character, the bisexual stem cell researcher Adam, appears.

Adam appears in the next sequence, which is set up as a split-screen montage in such a way that in just a few minutes we are shown in an exuberant way the lives of Hanna as a cultural journalist, of Simon as an exhibition designer and also of Adam as a visitor to a public swimming pool simultaneously. The sounds from at least two, sometimes three or more images at the same time can be heard next to each other. The art here is to play with the verbal chiaroscuro of the various scenes in loop groups22 in such a way that key words are quite understandable while the dialogue is lost in the general murmur of the walla. From the comprehensible keywords, one learns a lot about Hanna and Simon’s tired and sometimes quarrelsome relationship, their professional backgrounds and so on. It is significant that in many of the pictures, communications are not direct, but are conducted via microphones (Hanna works for a TV culture magazine) and via various telephones (telepheme23). Since the protagonists cannot see each other during these telephone calls, the picture shows us contexts that are sometimes deliberately not named by the people talking to each other or are even lied about. For example, Simon is watching pornography on his laptop during a phone call with Hanna, but claims in the conversation that he is working on something professional on the computer. Similarly, Hanna claims on the phone that she doesn’t want to go to an open-air swimming pool as arranged because she is on her period, but she is in this very swimming pool while she is watching Simon on the phone, who is observing a younger woman openly changing her clothes and has therefore not noticed that Hanna is already in the swimming pool. The introduction of Adam in this scene is very skilful. He comes to meet Hanna as she is leaving and walks past her without either of them taking any notice of each other at that moment. Only at the end of this vanishing line does the repetitive electronic music that began with the train journey end too; this has given both the jour ney past the telephone poles and the split-screen collage the ‘modern’ drive that locates the film in the ‘modern’ present of Berlin in the late 2000s.

Following this scene of excessive information, there is a very quiet choreographic scene24 in a large white, undefined room that responds to what has been shown so far with a complete break in style: In a short trio, three dancers (two men and one woman) anticipate the course of the entire film in a different form. The music’s character here is completely different and, as an orchestral underscoring, musicalises the dance in an almost classical manner. The setting is ‘dream-like’, the costumes and body language of the dancers are in black and white in a classiccontemporary style, like the music. A dancing couple is joined by another man who first takes over from the woman as a partner for the other dancer, then dances as a new partner with the woman before the first man joins them again. Now they dance a trio at the end of which, however, all three dancers, filmed from above, leave in a star shape in different directions.

The next and final sequence of the four-part exposition of this film takes up the previous voice-over of Simon’s voice, replacing it with a voice-over of Hanna’s voice. But while we did not yet know to whom the voice of the acousmêtre25 belonged when we watched the telephone wires in the first part of the exposition, here we see the portrait of the listening and thus silent Hanna while we hear her voice, i.e. her first-person inner voice. It remains unclear whether her voice is iconogenic or non-iconogenic. Hanna struggles with her perception clearly expressed in her inner monologue when she suddenly sees nude pictures of the artist Jeff Koons with the erotic actress Cicciolina on the screen in the lecture room during Adam’s presentation. In the background, we first hear a lecturer who deacousmatises himself as Adam in the following shot. Shortly after she silently asks herself whether she is fantasising these images or whether the stem cell researcher Adam is actually presenting these images in his lecture, we hear Adam’s voice become clearer and more intelligible: He is talking about stem cells and the fertilisation process. At the very moment his voice, illustrated by an animation in his presentation, is talking about ‘breaking the placental barrier’, music in the form of the pop song ‘Space Oddity’ 26 by David Bowie begins. With this music, Hanna steps out into the foyer of a Berlin Congress Center, which when seen from below is reminiscent of the interior of a spaceship. Before this effect ironically reveals the setting and thus the filmmaking, there is another remarkable situation in the lecture hall: Hanna thinks in her first-person voice at one moment, ‘There’s someone else speaking’, while the camera shows a conference guest sitting opposite her talking to his neighbour while Adam gives his lecture. Hanna hisses a lengthy ‘Shhhh’ as a crescendo to him, which he answers with a wink, then continues chatting blithely to the woman sitting next to him. Hanna does not manage to stop the disruptive acoustics here. But when an acquaintance in the foyer approaches her and verbally complains that she is supposedly always ‘picking on’ a certain theatre director in her TV magazine, she silences him, as well as the off-screen music by David Bowie, with a snap of her fingers.

What Tom Tykwer demonstrates here is the masterly use of audio-visual possibilities without having to rely on a US-budget in the setting or the use of devices. Notwithstanding the numerous figurants in the swimming pool, in the split-screen montage scenes or in the conference building, the film essentially concentrates on its three protagonists and is a chamber play in a Berlin setting from 2008. The music is self-produced studio music or bought in, the sound design is essentially produced by his long-standing sound team Frank Kruse and Matthias Lempert. Although the four sequences of exposition discussed above together last just under 10 minutes, the film has already been told twice in different ways, and there is an abundance of devices (uncut sequence, dance film, split-screen collage and staged scenes), which aims to demonstrate virtuosity in the use of a wide variety of cinematic devices.

It is remarkable how important logo27 information is: acoustic off-screen voiceover to charge images that actually say nothing (valeur ajoutée28) to the sequence of telephone poles/concise scansion between what is said and what is shown in the prevented ‘coitus scene’/fragmentary information in the verbal chiaroscuro process and with telephemes in the split-screen collage/body language statements in the dance sequence/simultaneous first-person voice to the initially dark, then brighter voice of the third protagonist, disturbed by off-screen, then de-acousmatised as in-screen voices, and last but not least, the ironic handling of the sound possibilities, including the lyrics by David Bowie (‘Ground control to Major Tom’) and Hanna’s almost naturalistic behaviour against the chatter of third parties.

In passing, numerous written logos in the moving images refer to further informative contexts. Thus we learn the title of Hanna’s TV programme as well as her full name on a name tag in the conference and in Adam’s presentation, we learn his name, his lecture topic and the names of the artists presented. It takes a lot of attention to catch the densely packed audio-visual wealth of information. Tykwer is probably deliberately going for an overload here and assumes that a few things will be remembered unconsciously in the course of the rest of the film and can then be further classified. This film will deal with the themes of sexuality, the desire for children, stem cells, death and, above all, the budding ménage à trois in an upscale cultural milieu in modern Berlin (in the 21st century noughties), all of which are mentioned (verbally) in the exposition. This film also reads today as is the case with Wim Wenders (see above) as a contemporary document.

Das weiße Band (The White Ribbon), 2009

Michael Haneke stands for silence and, as far as possible, the avoidance of nondiegetic music for his films. When music is non-diegetic, it is always rooted in an assertion on the screen. In The White Ribbon he takes this approach to the extreme in collaboration with his sound designers Guillaume Sciama and JeanPierre Laforce (Fig. 4). An elderly male off narrator’s voice, passive off-screen29 sound such as atmo, birdsong, yapping dogs, grasshoppers or even wind are used as well as individual active off-screen sounds such as children, protagonists’ voices, a fallen neighing horse and screen music from scenic action in which music is played. Dialogue is spare, but despite its brevity, it is always meaningful. For example, almost all the adults have an imperious tone while the children address the adults formally, chat among themselves in a chiaroscuro manner and otherwise barely or only very tersely speak to the adults, always polite to the point of obsequiousness. The only non-diegetic sound element at least in the exposition is the off narrator’s voice of the old man, who obviously remembers and who is later, as a young teacher in the story context, also de-acousmatised as a protagonist and thus as a young dialogue voice.

The concept behind this setting gives rise to an aesthetic of spartan purism, which corresponds to the black-and-white image design with available light as well as to the intimidating action and dialogue of the actors, also reduced to the essentials. Silence, slowness, long takes hardly any music, no SFX30 paint an audio-visual picture of a Protestant village at the beginning of the 20th century in Germany.

The silence, which always seems eerie, especially indoors, gives even the smallest sounds heightened meaning (e.g., the creaking of floorboards, swallowing noises, breathing, the actors’ footsteps, a housefly, small stones thrown against window panes).

The camera hardly moves and rarely shows a long shot: It virtually sticks to detailed views, oppressive beams, low ceilings and narrow chambers and illustrates the actors’ lack of freedom of movement. There is a lot of use of said/shown designations, so that atmos and protagonist sounds often only start after they have been named. In dialogue, however, said/shown contradictions are worked with, e.g., when a baroness says to her slowly moving boy who does not want to disturb, that he is ‘jumping about here’. Another dominant feature is very often the acousmatic introduction of voices and sounds, which are only deacousmatised after a period of purely auditory perception. Music is played with in a similar way when, in the above-mentioned Baroness scene, one first hears music from a piano and flute, assumes for a brief moment that it is non-diegetic music, only to see shortly later the Baroness practicing the piano with an accompanying flautist standing in the room. This music is thus not only de-acousmatised, but the music in this film becomes porous31 in perception without ever really being pit music. Even if the music seems for a moment as if it were the usual non-diegetic film music, it is always later unmasked as screen music or, conversely, later becomes non-diegetic music through subsequent sequences of a music-making action with the meaning of a time-synchronous parallel montage.

The film was released in 2009 simultaneously with Inglourious Basterds (Quentin Tarantino), Avatar (James Cameron) or The Imaginarium of Doctor Parnassus (Terry Gilliam). If one compares Das weiße Band with such films from Anglo-American filmmaking that exploit the possibilities of sound design to the maximum, especially in terms of special effects and music, one has to state that Michael Haneke quite deliberately creates a counter-design here that is reduced to the greatest degree possible and could also be shown behind the screen with a good mono loudspeaker. Just as he does in the interests of image perception with the non-grainy, sharply cut black and white images. In contrast to the ‘rich’ films mentioned above, however, the whole thing is not meant to be nostalgic, but rather purist, reduced, ‘poor’, almost to the point of cinematic Spartanism in order to give the viewer/listener an impression of the emptiness, barrenness, oppression and latent, later open violence of a time and a culture that was the breeding ground for subsequent authoritarian, violent and monstrous expressions that came precisely out of this culture.

Zeit der Kannibalen (Age

of Cannibals), 2014

This film by Johannes Naber (Fig. 5) is remarkable because it is conceived as a very consistent chamber play with a formal stylistic will almost reminiscent of the 1920s and as such completely against the mainstream even of German filmmaking. Naber realises this film in such an artificial way that the polished dialogue makes the plot and the cynical personal relationships of the three protagonists very clear. On the other hand, it is precisely the sound design (Benjamin Hörbe, Markus Krohn, André Zacher) that introduces the outside world in the form of off-screen stories, telephemes, verbal chiaroscuro attained through the supporting characters’ English with strong foreign-language accents and, increasingly, gunshots and explosions as territorially indeterminate extensions, which, however, come increasingly closer until they are literally at the door. The unusual film music by Cornelius Schwehr ranging between repetitive single notes and machine-like sounds reinforces the stylised effect of the auditory level.

The music repeating one note begins to a black screen to continue as a vanishing line as the credits fade in against a black background. To these credits, indefinable noises can be heard under the sound of the repetitively struck prepared piano string, which at best are reminiscent of street or grinding noises. To the appearance of a horizon in the picture, one hears an accentuated beat after previous ‘arhythmic’ repetitive beats in the noises. The abrupt brightening of the horizon makes an urban sea of houses with a higher sun recognisable as an abstract model. In addition, a second beat is heard that triggers a hissing sound as a code-switching accent. Below this, further street/grinding sounds and unintelligible mass-human voices (walla) in a foreign language can be discerned. At the moment when the camera moves back and reveals that the picture is seen through a window (with curtains on the right and left), the tempo of the piano tone, which is constantly repeated on one pitch, doubles.

Another accent with a sibilant now shows the reflection of a man working out on an exercise bike in the window pane. This protagonist then appears in front of it to another accentuating sibilant when the camera moves further back. The sibilants, although precisely integrated into the repetitive rhythm, now seem like the artificial breathing of the protagonist. At the moment when the entire image is reframed, we now see the cyclist actively riding the exercise bike in front of a large window behind which the urban cityscape and the higher sun are clearly visible. The picture is framed by house plants, a sofa and an armchair. The impression is that of a high-class hotel room. Exactly at this moment, when the camera comes to a standstill, the repetitive sound is joined by a bass tone that also repeats uniformly and synchronously.

Change of view: One sees a monitor in front of the exercise bike showing a 3D road in the POV. 32 Change of view now from the oblique to a close-up on the first protagonist (P1) with a window in the background; the hissing sounds in the repetition now have an even stronger effect, like the artificial breathing of the protagonist. An abrupt cut to a second protagonist (P2) cuts off the music and we now only hear the protagonist’s telephone conversation as a telepheme (type 2) in which we only clearly hear the visible speaker and, faintly, a female voice through the mobile phone headphones. In the background, however, a muffled urban atmosphere with car horns can be heard again, just as the artificial cityscape can be seen to some extent through the hotel window, which looks the same. The protagonist is apparently having an unpleasant conversation in German with a woman, which he ends with information about an upcoming meeting, that he is in India and that it is 8 o’clock in the morning. While he is on the phone, he ties a tie, which suggests that he is wearing a suit. With a sibilant crescendo, the montage changes to a marble-tiled hallway where both protagonists, dressed in black business suits and carrying briefcases, march past the camera in sync with the monotonous repetitious music that now begins again. Both act like robots. Cut. The music continues as a vanishing line, but we see the face of an Indian man who looks incredulously into the camera; counter-shot of the two protagonists, now sitting behind a table opposite him, both smiling nastily. At the moment when the music ends with a breathy hiss, P2 begins the dialogue in English, which is continued by P1. The dialogue is about confronting the Indian interlocutor with the fact that a major industrial production will apparently be moved from India to Pakistan. The two also use vulgar language and end their messages with ‘India was yesterday’. Only now does the title appear to a (now plucked) multi-tone descending motif, following an accentuating beat, which however becomes a single-tone repetition again as soon as we jump back into the hotel corridor after the title and the insertion of the names of the three main characters. A sign at the end of the corridor legibly displays the logo Jinshan.

Since we see P1 in the corridor dressed in an overcoat and pulling a wheeled suitcase, we have to assume that this is a new place, even though the hotel corridor looks exactly the same as before.

Jump to one of the hotel rooms where P2 is shown again and a voice can be heard making another telephone call in German while he hangs up his clothes on hangers. Only the cut to P1, who is also in this room, shows that he is obviously speaking very emotionally to an answering machine. The subsequent dialogue between the two protagonists, which is partly vulgar again, makes it clear that the two are colleagues who are critical of their company policy and are also cynical and aggressive with each other. After both talk at the same time at the end of the dialogue, the music starts again to a black image, but now first climbs to a crescendo tone, into which the monotonous repetitive tone, now plucked again at half speed, is reintroduced. Cut. In the back, we see P1 sitting in the hotel room eating, while a pan shot to the left now brings the third protagonist into the picture, a woman who is obviously looking out of the hotel window. From this cut onwards, we hear urban atmosphere again with walla, but also for the first time in the music a repeated flute sound synchronised with the plucked rhythm and bongo beats in the hocket. The music stops as soon as the dialogue between P3 and P1 begins with a question from P3, who wants to know where the famous old town is. In the cut to the inside, the atmo becomes noticeably quieter, suggesting that the shot before was meant to be from the outside through the hotel window to the inside. Then we are back in the chamber play setting of the muffled hotel room. P1 describes the life-threatening reality outside and ends his monologue with the later important statement ‘But here in the hotel we are safe’. In the dialogue that follows, the two first get to know each other; the woman, unnamed until then, is apparently also a consultant and new employee. P2 now also appears talking loudly and aggressively on the phone in English, interrupting the dialogue between the two. In the following dialogue between P3 and P2, both names are mentioned casually. P3 is apparently a new colleague in a team of three, which previously consisted of three men. P2 is not at all pleased about this, especially since the former colleague 3 (mentioned as Henninger several times) has apparently risen to become a partner in the firm. P2 expresses his displeasure by hurling a prop to the floor, but then ironically apologises. He then questions the new colleague as if she were an applicant and asks her to leave the two men alone. In the following dialogue, P2 is annoyed that these company decisions were apparently made over their heads, only to tease P1 again who reveals in the dialogue that he is from the East German city of Erfurt, which provokes P2 to make a cynical joke about East German citizens.

The outside world is visually only dimly recognisable as an urban model world through the hotel windows. The different hotels where the protagonists apparently stay in Asian, Arab or African countries all look the same. Only logos on illuminated signs in the hotel corridors indicate where the protagonists, always dressed and behaving in the same way, are currently staying. Extras who appear later are mostly hotel staff who differ slightly from each other visually (in terms of

ethnicities), speaking with different accents in international conference English and who may partly be distinguished locally from each other by the adornments on the hotel uniforms. The two male protagonists in particular behave robotically and synchronously to the underlying repetitive musical tones when walking down the hotel corridors. Likewise, the training sequences of a protagonist on an exercise bicycle are underlaid with the aforementioned machine-like sounds and a hissing that is reminiscent of breathing in an abstract way.

The dialogues largely stand alone, and this being the case, they are clearly understandable. However, music and sounds punctuate the dialogues repeatedly and increasingly so that by and by the fluidity and sophistication of the dialogues are increasingly disturbed by these interruptions. Despite the economy of means, however, the instrumentation varies a lot: strings for the men, plucked strings for the title, sustained sounds, flute and bongo notes for the female protagonist. Towards the end of the film, the formerly sophisticated dialogues transform into individual commands, cries for help and wails, and the formerly irritating but still seemingly harmless background noises take over as the dominant sound in the film and become life-threatening for the three protagonists. At the same time, the telepheme link to the outside world becomes increasingly problematic, until in the end it is completely impossible. Without a connection to the outside world, without the power of their at times imperious language and without the security of their workouts, marching down corridors and reassurances through telecommunication from their country of origin, Germany, the three management consultants are thrown back on their animal natures and as such are at the mercy of the naked violence of the formerly dominated.

Systemsprenger (System Crasher), 2019

Nora Fingscheidt’s Systemsprenger is a successful example of a film produced by a student group that now continues to work professionally as a team. Both the designers Oscar Stiebitz and Dominik Leube, who were responsible for the original sound and sound design and who had previously completed their diploma in film and television at Fachhochschule Dortmund University of Applied Sciences and Arts, as well as the composer John Gürtler were later students at the Filmakademie Baden-Württemberg. The film is characterised by a very selective use of film music which only occurs in certain acts, almost always alternating with an act entirely without film music. The main protagonist Benni’s psychotic episodes, however, are made explicit through strong audio-visual collages and sound worked out as musique concrète, which ultimately won the German Film Award (Lola) 2020 for best sound design for the entire sound team including Gregor Bonse, Marcel Fink, Markus Limberger and Corinna Zink in addition to the above-mentioned individuals.

The child Benni who lives in a children’s home, has had a traumatic experience, which is explicitly named at one point in the film. When she was a baby and she was crying, a nappy was stuffed into her mouth. Now every time somebody touches the girl’s face, she literally sees red. This red colouration in the film, synchronised with the screaming of the child as a vanishing line, leads in each case to collages in which image fragments and sound fragments mix mostly in crescendi in such a way that nightmarish effects arise for the viewer/listener.

The film begins with a cheerfully whistled piece of film music to which the logos of the sponsors are rhythmically and synchronously ‘stamped’ into the black image as the opening credits roll and are equally faded out again in rhythm with the individual notes of the melody. Already in the first image, in a close-up on a child’s foot (pink sock with comic motif), one sees a discrepancy in the recognisable lower leg of a hematoma and a wired sensor glued to the skin. In addition, one hears the soundscape of office machines. The immediately following sequence shot past the recumbent protagonist from her foot to her head shows not only the halfdressed blonde girl, but also that the entire upper body is wired with glued-on measuring probes. The following cut represents a leap in time, since although the girl is still wired up, we no longer hear printer noises but hear, than see a female doctor first as a phantom in the picture, then de-acousmatised, talking to the child about the examination results and about medication. The EASs (doors, several different steps, slight walla) off-screen reveal that this is not a doctor’s office but rather a hospital.

Cut. We suddenly find ourselves in a sequence filmed with a hand-held camera, close up and restless, in which several children can be seen out of focus and the screaming of many children can be heard. The protagonist is fighting with several other children in a courtyard. Several adults run out of a building into the commotion and scold the children. The group of children goes into the house, the child now addressed as ‘Benni’ by an older man is told to wait outside until she has calmed down. The child tries to hit the man, makes angry sounds and kicks a plastic toy lying on the floor. Finally, with Benni in the picture, the door to the building is heard slamming shut off-screen. Benni then starts to scream like an animal again and continues to throw around small plastic tractors. In counter-cuts we see the other children on the first floor of the building behind a large glass window watching Benni, taunting her and beating their fists against the window panes in an inflammatory way. In the recut, the camera is now behind the courtyard door, which is also glazed, so that Benni, who is raging outside, can be seen through it. Behind the door are the older man and a younger colleague holding the door shut by the handle, both commenting on Benni’s throwing plastic tractors, saying: ‘Don’t worry, it’s safety glass.’ Benni, now filmed close up from the outside, screams and hurls more toy tractors against the door before a tractor flies against the glass pane in such a way that, again looking from the inside out, one sees the title Systemsprenger painted on the pane as if it had been written in children’s finger paint. Behind it, Benni clearly picks up a tractor and pushes it so hard against the pane that the lettering disappears in sync with the impact accent and at the same time cracks in the safety glass become visible.

The set-up of dialogue, off-screen scenography, screaming child is maintained in the following scenes. Benni’s first attempt to escape is accompanied by a driving rhythm and the whistled child’s melody now played on a xylophone.

Benni is seen running, stealing a bag in a boutique, before she meets a group of children on the street who harass her, insulting her, trying to snatch her bag and grab her face. When one of the boys actually touches Benni’s face, who is now lying on the ground, she sees red. The whole picture turns red, for a moment there is only red screen to Benni’s renewed scream. The red lightens and we now see a quickly cut collage of blurred images: Benni’s face, the screaming mouth, a bird, skates sliding on ice, Benni looking, then again fighting with other children, an adult woman’s earrings, a dog, Benni’s eye peering through a keyhole. What is shown here consist for the most part of image fragments that only explain themselves later in the film, i.e., anticipations, but here they seem like an associative image collage. The sound for this, however, does not consist of scenic correspondences, but of a harmonic sustained sound that develops from Benni’s scream, which can be heard at first and which changes into a shredded lowfrequency drone and later into the harmonic sustained sound. In the skate images, this is supplemented by a grinding noise and vocal fragments that are varied when Benni is seen watching. For the fragment of the fight with the other children, one hears only a distorted child’s voice shouting loudly in the harmonic bed of sound, for the earring only the pure harmonic sustained sound, and for the dog and the eye in front of the keyhole, grinding noises played backwards. Although Benni’s screaming open mouth is briefly seen as is her fight with the children, neither Benni’s scream nor the children’s tumult are heard.

Immediately following this short but all the more impressive for it collage, we now see Benni sitting in a dark room, shouting. We are now in the main body of the film. The entire exposition discussed here lasted just under 8 minutes and named all the dominant features of the film: acousmatic extensions and voices, childish screaming as an expression of rage, logo settings linked to actions or music, ‘children’s music’ as the phrasing of a narrative ellipsis and, above all, an audio-visual collage of rapid, barely recognisable image fragments to a dreamlike musique concrète. The two musical materials obviously have nothing to do with each other. The pieces actually used as music suggest a happy children’s film, which is easily called into question by the images shown with them (logos and later the escape and theft sequence). In the scenes of Benni’s treatment, the dialogues with her and people talking to her as well as in the aggression sequences, no music is used, but only conventional sound design for documentary-like images, whereby this is distinguished in these scenes by the fact that many things become recognisable and interpretable acousmatically in extensions, which one would not recognise and interpret in this way in most of the close-ups used. Benni’s subjective freak-outs are depicted in the exposition as described above, but are also made clear again and again over the course of the film with the audio-visual collages in which only individual alienated elements from the sound design are used while the ‘children’s music’ plays no role. It is replaced in these scenes by the changing harmonic sustained sound, which has more of an electronic tonality, enriched with fragments from the sound design processed for effects.

The fact that such sequences are possible in a contemporary feature film and that they form small, highly artificial, experimental islands in an otherwise documentary-like film restores hope in a new generation of filmmakers. On the one hand, this generation dares to be more experimental in German film, on the other hand, it offers sound designers a scope for action.

In the 2020s, several more audio-visually interesting films have been released33 that are worth adding to these analyses in the foreseeable future.

Conclusion

What all the films discussed here have in common is a close interlocking of image and sound as well as a complementary, conceptually well-thought-out interweaving of sound design and music. Even more striking, however, is the fact that all the films largely do without SFX battles, which determine the sound design of most US box-office hits. European films take a different approach, and the use of silence, condensed devices (omissions) or the radio-play character of some sound designs seem to be specific approaches to sound design in Germanlanguage films. All the audio-visual analyses discussed here should have clarified how high the semantic value of the set audio-visual interactions is and how much this is also and especially determined by a knowledge of the meanings of image and sound. This is obviously possible when people with in-depth musical training work together: sound designers and composers as well as filmmakers and editors, the sound crew and the image crew. Several films indicate that this cooperation between the film and sound teams must have begun during the development of the material, i.e. before shooting, since many of the interactions shown here can only be realised if they are planned in advance and laid down in the script before the shooting of the relevant scenes.

The british filmmaker Peter Greenaway recently said in a panel discussion at the British Film Institute that he was never interested in making films for the sake of conventional storytelling. He knows that he has to offer the audience a superficial line they can follow. Underneath, however, he always offers a wealth of further information, a meta-level of formal, aesthetic and sometimes even abstract enjoyment and discourse that does not simply follow a causally understandable story. Or, as he himself formulates this more radically: If you want to tell stories, be a writer not a filmmaker! Film is an audio-visual medium and offers for this meta-discourse both the moving image with all its symbolic-iconographic possibilities for the eyes and the auditory level with an equally rich spectrum of meanings for the ears. Added to this are the valeur ajoutée statements through the vertical settings of image to sound and vice versa. These three semantic possibilities alone form the basis for the richness of the language of sound film and are far from being exhaustively explored, as the journey of the abovedescribed possibilities from 1931 to 2019 alone attempts to demonstrate.

Author Biography

Prof. Jörg U. Lensing: Born in Düsseldorf in 1960. Grew up in Düsseldorf as the son of a craftsman and a cartographer. 1975, discovered creativity with his first music teacher Frank Köllges (percussion/drums). 1980s, studied composition/electronic music at the Folkwang Hochschule Essen (Prof. Wolfgang Hufschmidt & Prof. Dirk Reith), and master student of New Music Theatre (with Mauricio Kagel – Musikhochschule Köln). 1987, foundation of the Düsseldorf THEATER DER KLÄNGE, of which he is still artistic director. 1993, theatre lecturer (first international Bauhaus Stage class) at the Bauhaus Dessau. As of 1996 professor of sound design at Fachhochschule Dortmund – University of Applied Sciences and Arts. Works as a director and composer for both theatre and film. Sometimes also works as a choreographer for dance productions. Works as a sound designer mainly for his own documentaries and films by the German auteur filmmaker Lutz Dammbeck. Specialised author and translator (editor) of two standard works by the French composer and scientist Michel Chion. A recognised specialist on the subject of Bauhaus, in particular the Bauhaus stage and the two masters Walter Gropius and Oskar Schlemmer. Married to the dancer and choreographer Jacqueline Fischer (two children).

Publications (selection): “Filmton,” in Harald Schleicher, Alexander Urban, eds., Filme machen. Zweitausendeins (Frankfurt a. Main, 2005); “Audio-visuelle Montage,” in: Dieter Daniels, Sandra Naumann and Jan Thoben, eds., See this Sound (Köln: Walter König, 2010); Michel Chion, Audio-Vision – Ton und Bild im Kino, edited by Jörg U. Lensing (Schiele & Schön, Berlin 2013); Publication of the German version of the book by Michel Chion: AUDIO–LOGO–VISION im Kinofilm, edited by Jörg U. Lensing, 1st edition (Fachverlag Schiele & Schön, Berlin, 2017); Sound-Design – Sound-Montage –Soundtrack-Komposition, 3rd edition, (Schiele & Schön, Berlin, 2017); “Sounddesign,” in: Daniel Morat, Hansjakob Ziemer, eds., Handbuch Sound: Geschichte – Begriffe – Ansätze (1st edition, Stuttgart: J. B. Metzler Verlag, 2018).

Notes

1 OFF-SCREEN-SOUND – According to Michel Chion’s definition, this is a sound in an audio-visual relationship of whatever nature (voice, noise, etc.), whose source is not visible on the screen at the same time but should exist in the place and time of the situation shown: e.g., the voice of a person speaking off-screen, heard by a visible interlocutor; street noise outside a room, etc. See Michel Chion: Audio-Vision – Ton und Bild im Kino Edited by Jörg U. Lensing. Schiele & Schön, Berlin 2013 (181).

2 Atmo = atmospheres/ambiences

3 IN (SOUND) – In an audio-visual relationship, Chion calls the situation IN (sound) in which there is a sound whose concrete source is visible on the screen at the same time and which is connected to a diegetically present and visible reality. See Chion, 181.

4 ‘In the Hall of the Mountain King’ from Edvard Grieg’s 1875 music for Henrik Ibsen’s Peer Gynt

5. THE ACOUSMATIC SITUATION – Situation of acousmatic listening: hearing a sound without seeing its origin (Pierre Schaeffer 1952)

6 Original sounds (location sounds) = sounds or speech recorded on the set during filming

7 Fritz Lang: Das Testament des Dr. Mabuse (The Testament of Dr. Mabuse 1933), some films by Max Ophüls, Bernhard Wicki: Die Brücke (The Bridge 1959) and, from 1972, some films by Wim Wenders, Rainer Werner Fassbinder, Werner Herzog, Helma SandersBrahms or Volker Schlöndorff’s Blechtrommel (The Tin Drum 1979). Das Boot (1982) by Wolfgang Petersen prominently marks a change to sound design on an international level for German cinema. In TV format, Heimat (1984) by Edgar Reitz is the first German film to attract attention in terms of sound design.

8 OFF-SOUND According to Michel Chion, describes a sound of an audio-visual relationship whose source, firstly, is not visible on the screen at that particular moment, and secondly, is said to belong to a different time and real or imaginary place than the one shown in the scene. See Chion, 182.

9. I-VOICE – According to Chion a type of a voice presence in a film or a video –generally, but not obligatorily, a narrative off-voice when it speaks in maximum proximity to the viewer’s ear (felt proximity through special sound indications) and this in a space that is deliberately kept dull and without reverberation. The I-voice, which resounds within us like our own voice, generally has no materialising sound cues (breathing, mouth noises) to make us feel the body behind it. See Chion, 188.

10 VOICE-OVER – Subsequently recorded off-screen voice (usually commentators)

11 SAID/SHOWN CONTRADICTION – When the narrator or the off-voice, which is supposed to refer to what is actually happening, tells the audience something that is contradicted by what they see.

12 ATMOSPHERIC SOUND OR TERRITORY SOUND – According to Chion, sound of an all-encompassing atmosphere that envelops a scene and is anchored in space without resolving the crucial question of the location and visualisation of the source, e.g., birdsong, collective insect chirping, bells ringing, city traffic. See Chion, 180.

13 See note 8.

14. A sound effect imitating the murmur of a crowd of people, internationally referred to as ‘walla ’

15. Hollow sound in an audio-visual sequence is, according to Chion, sound that is suggested by the image but not heard, whereas other sounds associated with the scene are audible, which contributes to the previous ones being ‘tacitly included’ (even though they do not exist). See Chion, 200.

16 ON-THE-AIR SOUNDS (SOUNDS AND MUSIC ‘from a transmission’) – In an audiovisual fiction, even in a documentary film sequence, those sounds which are present in a sequence but are transmitted electrically by radio, telephone, electrical amplifiers, etc. are called ‘on the air’ which places them outside the mechanical (i.e., natural) laws of sound propagation. Thus, they can pass freely through space and still remain anchored in the real time of the scene. So, when it comes to a music and especially a song, they can travel freely from the ‘screen’ position to the ‘non-diegetic’ position. The game often consists of sending changing hypotheses to the spectator in the course of the scene, playing with the sometimes more, sometimes less strong presence of the song as well as with its more or less strong autonomy in relation to the montage and the diegetic space. See Chion, 43-184.

17 One speaks of a non-iconogenic voice or narration when in a film a certain character tells something and one sees only the narrator and his audience(s) and no other images ‘illustrating’ this narration. See Chion, 184.

18 Verbal chiaroscuro is given when one alternately understands and then again does not understand what the persons are saying. See Chion, 61.

19 Interestingly, both musicians mentioned here played in the Nina Hagen Band, Reinhold Heil also in the band Spliff. Both musicians are responsible for the film music of most Tykwer films together with Tom Tykwer himself.

20 Source: Wikipedia

21 One of five examples of said/shown relationships in which the rhythm of what a person does or does not do while speaking is reflected in an event in the sound scene or the visual scene, emphasising the discourse and helping to convey it to the audience. The scansion between what is said and what is shown is one of the most popular procedures in verbo-centric cinema. See Chion, 192.

22 Loop groups are mass-human sequences in which one can partially understand verbal content.

23 A telepheme is a scene featuring a telephone conversation in a film.

24. Choreography by Sasha Waltz

25 According to Chion, the akousmètre is an invisible person who creates an acousmatic off-screen or on-screen voice for the listener, the source of which remains invisible until this voice has sufficient coherence and continuity to entirely construct a person (even if this person is only known acoustically), provided that the ‘carrier’ of the voice is presented as if he could appear on the screen at any moment. See Chion, 198–199.

26 ‘Space Oddity’, David Bowie, 1969

27 “Logo” is used here as a term for readable written information, as well as for audible text information.

28 The valeur ajoutée is a sensory, informative, semantic, narrative, structural or expressive value that leads us to project onto the image a sound that is more audible in a scene, which creates in it an impression that we in fact ‘audio-visualise’. This commonly used effect is usually absorbed without noticing by viewers who simply accept it. See Chion, 173.

29 Off-screen sounds comprise acousmatic sounds that draw a sonic environment around the subjects (the city, birdsong, the sea) and do not raise any questions about the nature of their origin, nor do they evoke any reaction or expectation from the subjects or the viewer. See Chion, 181.

30 Special effects

31. There is porosity between the real and imaginary spaces of a film when there is communication or circulation between the different levels. See Chion, 195.

32 Point of view, here subjectively the protagonist’s point of view

33 Berlin Alexanderplatz (Burhan Qurbani 2020), Undine (Christian Petzold 2020) or the recently highly decorated Im Westen nichts Neues (All Quiet on the Western Front, Edward Berger 2022)

The Beginning and the End of the Radio Play

Turn it on. The radio play is commonly broadcast on the radio. Without the radio, there is no radio play. But what is the radio? It is a medium; that much is clear. One that, in the year of writing these silent lines, has been babbling and broadcasting from all speakers for a century. The planet has listened, and as is typical of the auditory, only a few things have been preserved. The radio is a medium, one of those contentless entities that, according to an early understanding, are said to be the message itself. On the other hand, this specific medium was reputed to have been more of a massage (of the popular soul, especially the German one). However, the radio play has little to do with that. It is neither a message nor propaganda. It was invented solely as an art form due to the purely acoustic existence of the radio, which is bound to a medium. With the advent of digital distribution, it may disappear. And for that reason alone, sentences could be audibly made in the radio play of the late 20th century that claimed, without consequence: “The medium was the knife, the message zero one ” (Ammer and Einheit, Radio Inferno, BR 1993).

Playing with the radio, playing with listening... can the radio play do that? And if so, who can do it, for how long? And how does one do it? Who started it? Who pays the bill? Necessary for the creation of the radio play was a media technology that is now over a hundred years old. At first glance, nothing more than a device full of electronics that transmits events without a carrier medium, acoustically and in real time, from one place to another. In connection with acoustic storage media such as shellac records, tape recorders, and hard drives, it became a device that not only enabled broadcasting over long distances but also the transmission, speech, singing, and playing of the dead beyond time or of long-past events. “Ich zaubere Ihnen auf Wunsch die Stimme und die Kunst längst Dahingegangener herbei” (On request, I can conjure up for you the voice and the art of longdeceased ones) triumphantly proclaims one of the first shellac records even before the radio was invented. In contrast, the essence of the radio seems to be real-time transmission: Initially, stock market news, water levels, later baseball games, and ultimately, the live conference broadcast of the Bundesliga football.

Live broadcasts of major events, party speeches, or opera performances have always been the essence of the radio. It stands for truth and authenticity more than any other medium. However, this is often an illusion. The report of the airship LZ127 Hindenburg on May 6, 1937, is considered the epitome of adrenalinesaturated radio broadcasts from overwhelming reality. However, it did not actually happen live on the radio. The since then world-famous reporter Herbert Morrison (Oh the humanity!) spoke on-site into four “Presto Direct Disks” of a portable disccutting machine. The sound recording was made by which is why his name should also be mentioned technician Charles Nehlsen. The records were taken from Lakehurst to Chicago after the disaster and only broadcast the next morning... and then they also went “viral” in media history.

Playing with this truth is dangerous and not easily understandable The oftenquoted alleged mass panic during the broadcast of Orson Welles’ radio play War of the Worlds precisely denotes the point where the gesture of the medium transitioned into the simulation of truth and suddenly, unexpectedly, art emerged instead of reality. However, it was already known early on that speaking on the radio has something ghostly and unreal about it: “Das leere Zimmer ist voller Menschen, die man verspürt, aber nicht sieht und daher auch nicht identifizieren kann. Das ist gespenstisch …. Sie sind da und doch nicht – Geister!” (The empty room is full of people you sense but don’t see and therefore can’t identify. This is eerie ....They are there and yet not ghosts!) is how the radio theorist Rudolf Arnheim described his listening experience “in the face” of the plastic cubes speaking to him as early as 1932. This was followed first by the aliens, then Hitler.

Let’s listen again to the shellac records that spoke for themselves before the advent of radio, claiming on the one hand to have no life and yet to rejoice. The record “speaks”: “Verehrte Zeit- und Festgenossen! Sehen sie auch heute hier keinen lebenden Redner vor sich, sondern ich, die Sprechmaschine, habe das Vergnügen, Sie alle auf das Herzlichste zu begrüßen“ (Esteemed contemporaries and festive comrades! Do not see a living speaker before you here today, but I, the speaking machine, have the pleasure of welcoming you all most cordially) More precisely, but just as paradoxically between the alleged “life task” of the dead apparatus and its purpose for the living “present ones” another selfstatement on a shellac record expresses it: “Meine Lebensaufgabe besteht darin, alle Anwesenden stets zu unterhalten” (My life task consists in always entertaining all present ones) When one considers that these sayings of the apparatus were shouted into lifeless funnels of sound by living speakers, much of how much a medium, however authentic it pretends to be, is based on a ghostly game with identities, falsehoods, illusions, in other words, art, becomes clear. This is inscribed in the radio from the very beginning, even if it initially only reproduces stock market prices in real time.

Radio is a medium of the 20th century. According to cultural consensus, it was invented in 1904 by Gugliemo Marconi, who relied on the principle of wireless transmission of radio waves by means of a resonant transformer devised by Nikola Tesla. But already in 1881, fictional events, at that time mainly operas, were transmitted to distant rooms still by means of wires. Ader had set up 40 carbon microphones mounted on lead on the stage of the Paris Opéra Garnier, which transmitted the singspiel over telephone lines two kilometers away and already in stereo to a palais on the Champs Elysée, where listeners could follow the action with two earpieces, including the audience reactions (and allegedly also the prompter). Later, coin-operated machines were installed, making the broadcast commercially viable. The opera, as the blueprint of the radio play, thus stood at the beginning of the possibility of a purely acoustic, dramatic event.

King Ludwig II, always curious about the latest techniques for creating fantasy worlds, was interested in the device. Marcel Proust listened to Debussy with the theatrophone. The technique became obsolete after the first opera was broadcast over the new medium of radio in London in 1923. The radio play followed on its heels. What does all this have to do with the radio play? ... Answer: The radio play is unlike the feature film, for example the rare case of an artistic genre that was created by a medium and exists only in that medium. It lives still in its advanced forms of collage, original sound or sound radio play from the gesture of

immediacy, which is inherent in everything acoustic. On the one hand, this has to do with the fact that the acoustic event is, first, “wireless,” i.e., since the invention of the hearing organ by evolution, it requires no other carrier medium than the air. Secondly, it is see Schiller’s insight “wie der Klang verhallet in dem Ohr / Verrauscht des Augenblicks geschwinde Schöpfung”(How the sound fades in the ear / Noisy of the moment’s swift creation) irretrievably lost with its emergence until the advent of shellacs, and thirdly, there is no more direct path of a sense organ to the human brain than the spoken word and the kang. The ear, unlike the eye, cannot be closed. At every bang, we are frightened immediately and before we have consciously perceived it. The brain also reacts immediately to sentimental music with feelings of longing or involuntary memories of all kinds. This is followed by art.

Before we finally arrive at the game of hearing, a short excursion into the as always wise evolutionary biology Only in the 21st century, two Swedish scientists who examined the fossils of a primitive fish named Panderichthys, which is considered the precursor of the first land animals, discovered the origin of all hearing The oversized breathing hole of this fish resembles the middle ear of the first land animals, which would explain how much breathing, unconscious life, and hearing are connected. Whereby still is to be considered that hearing is to be steered only with difficulty into a direction. Who speaks, speaks always already as Lenin demanded later from the radio ”To all!” Absolute rulers took advantage of this early and cleverly: “Ganz Deutschland hört dem Führer mit dem Volksempfänger” (All Germany listens to the Führer with the Volksempfänger) was the advertising slogan that in 1933 equated listening, dictatorship, the radio set and thus a people that received. The corresponding Volksempfänger device bore the serial number VE 301 after January 30, 1933, the day the National Socialists seized power. By 1939, there were 12 million of these receivers where listening to foreign stations was forbidden under threat of punishment (in fact, the concentration camp). This is the disadvantage of radiophonic media: they do not only broadcast “To All,” but everyone who owns a broadcasting device could theoretically broadcast with it. That is why, to this day democracy or not only state-supporting institutions are allowed to operate a radio station. With National Socialism, radio emerges from the shadow of a permeable medium. It becomes a designer. Peter Weibel summed it up with the programmatic words, “Radio is not only a medium of history, but radio also makes history.” From the very beginning, it has been a medium that also invents stories, which are then called radio plays.

The very first of these was called “The Wolf” and was broadcast on August 3, 1922, by the radio station WGY under the direction of the actor Edward H. Smith from the New York suburb of Schenectady at quarter to eight for 40 minutes on that radio, which otherwise only broadcast the stock market prices, weather reports, some news and now and then a baseball game. The station’s reach of 500 miles was enormous for the time, as was the success of the first radio play, an adapted play. It was the starting point of a legendary career, at the peak of which up to 60 million listeners in the USA sat in front of their receivers and listened to the radio plays at that time mostly theater adaptations or Sherlock Holmes detective stories in the vastness of the huge country.

On October 24, 1924, almost one year after the words, “Achtung, Achtung. Hier ist die Sendestelle Berlin im Vox-Haus, auf Welle 400 Meter. Meine Damen und Herren, wir machen Ihnen davon Mitteilung, dass am heutigen Tage der Unterhaltungsrundfunkdienst mit Verbreitung von Musikvorführungen auf drahtlos telefonischem Wege beginnt Die Benutzung ist genehmigungspflichtig” (Attention, attention. This is the Berlin broadcasting station in the Vox building, on wave 400 meters. Ladies and gentlemen, we are informing you that today the entertainment broadcasting service will begin broadcasting music performances by wireless telephone means. The use is subject to approval), the first veritable radio play conceived especially for the medium was broadcast, which already bore the ghostly illusionism of the process in its title. It was called Zauberei auf dem Sender (Magic on the Radio) and contained dialogs that marveled at one’s own existence: “Sag mal, hältst Du das für möglich, kann das sein (…) dass eine Musik erklingt, ohne dass jemand spielt. Kannst du das verstehen?” (Tell me, do you think it’s possible, can it be (...) that a music sounds without anyone playing. Can you understand that?). It’s part of any kind of technology that you don’t have to understand it to enjoy it.

In the first radio play, wisely conscious unreality of the own existence, positively turned, the “abstractness” of the radio event, could not prevail more peculiarly for a long time. Despite the work of great literary figures, whose financing through radio works was one of the noblest tasks of postwar radio (Andersch, Heißenbüttel, Walser), the genre was, for many decades, in a mysterious way and misunderstanding its laws, addicted to realism. Legendary are the anecdotes of how actors in the early days of the genre declaimed classical dramas in knight’s armor in front of microphones. This misguided thinking resulted in the aesthetically fatal misjudgment of the radio play as a form of fading, as a film without images. When a radio play as Günther Eich’s Träume did gave in to

this illusion, it quickly caused a scandal. This aesthetic twilight sleep was not ended until the wonderful year of 1968, when Ernst Jandl and Friederike Mayröcker’s radio play/text collage Fünf Mann Menschen was awarded the prestigious German Kriegsblindenpreis (Radio Play Prize of the War Blind) and the radio play genre finally emancipated itself from the realism that had never been inherent in the medium of radio. The next innovative thrust came in the mid1990s, when with the participation of the author of these lines the so-called pop radio play exploded from Bayerischer Rundfunk into the world of radio plays, which until then had been either avant-garde (Jandl and Mayröcker) or pseudorealistic. Suddenly, every acoustic event not only the noise that had previously been established but also the rhythmically coded music under which the globe had long since gathered had become part of the radio play. When the live radio play Apocalypse Live (BR 1994) by FM Einheit, Ulrike Haage, and the author of these lines received the Kriegsblindenpreis, many conservative radio producers feared the demise of the genre. Against their objection that a work in which popular music carries meaning is not a radio play, only my battle cry, “Radio play is that which is paid for by a broadcasting company” (speech on receiving the radio play prize in 2002) helped. In my acceptance speech at the presentation of the radio play award seven years earlier, I had still dared to claim:

The radio play is one of the few contemporary art forms that exist at all. Through its links with the other arts and through its purely technical existence, it is suited like only a few arts to create the real work of art of our time, as long as it dares to play to its strengths. ...Irrespective of its far-reaching artlessness perhaps the real cultural work of our epoch takes place in niches of pop culture: There, at the highest technical level, musicians sometimes create gems, which may well contain wild sound collages, hard cuts, cranky lyrics or touching melodies, and whose only drawback is that they may last no more than 3 minutes and 30 seconds and are therefore not suitable for telling great, stories. This is a limitation that radio drama does not have.

Our radio play about the end of the world then also built on the greatest advantage that the radio play has over other art forms. Like only literature or fantastic painting, it can depict events that have no reality.

A film about the end of the world even if Hollywood tries to realize it again and again can only be ridiculous, unlike a radio play. A radio play about the apocalypse or about Dante’s hell, as in our radio play Radio Inferno (Ammer and Einheit BR 1993), does not have this problem. Non-existence is the most grateful theme of the radio play. Acoustically, The Wrath of God can be portrayed relatively easily; in a film, this is absolutely impossible.

Once upon a time, the radio play was, therefore, the freest of all art genres since it was bound to nothing, but sound and sound could be everything. Not everything that has a sound will become one. But many things that can. Radio play can be given such. I once went through this with Wittgenstein’s Tractatus logico philosophicus (BR 2015). When the Wittgenstein fan and non-professional narrator Oswald Wiener getting more and more tired reads the elusive text almost in its entirety and the musician Console (Martin Gretschman or Acid Pauli) crafts a hypnotic track to it, on which the philosopher’s words are made to shine and sing for hours before one is silent about what one cannot say ... and then only the music plays ... then this is one of my favorite radio plays, which could not be performed in this way on any stage in a meaningful way, and certainly could not be filmed or only recited just radio play.

In the meantime, in the 21st century, music sound as a radio play element has become completely accepted as a matter of course. Three decades after Apocalypse Live, the musicalization of the radio play can be considered complete to endemic. The near demise of the genre will as we know for sure today occur in a completely different way: Through the death of radio.

Its purely technical form of existence as broadcast shellac, tape, or broadcast file distinguishes the modern radio play from opera. This form of existence is the only thing that the radio play, as the freest of all art genres, sometimes has to take into account The radio, the apparatus that transmits the sound event, is uninterruptible and unlike a musical scale in the case of opera composers does not have to be understood in its function, even by the makers, in order to make radio plays. We can learn from history the fact that this difficult-to-understand electrical apparatus is regulated politically or capitalistically is the greatest danger threatening radio drama. And that brings us to ARD. Just as it stood at the origin of the modern radio play, it is to be feared that the history of the radio play could come to an end to the extent that the individual German broadcasters will possibly not only have dwindling budgets but will no longer have their own radio play departments and the radio plays will no longer be broadcast, but will be filed away in the intermediate realm of the Internet and become untraceable.

Since the end of National Socialism, the Arbeitsgemeinschaft der Rundfunkanstalten Deutschlands, or ARD for short, has been the proud guardian of the art genre of radio drama, which has been spurred on to unimagined artistic heights by the federal competition between different editorial departments. It has begun to dismantle federal diversity in favor of “centers of competence.”

One can imagine this as if there were only one publishing house as is usual in totalitarian states. The declared goal of the managing directors is to bring the many German stations, each of which used to have its own program, into line. This is all the more absurd because the genre of radio drama is the only one that has caught on with audiences in the course of the digital revolution. Despite the babbling, forced podcast mania, ARD radio plays are responsible for a third of all accesses in the audio libraries (with a share of only 8% of the radio total). Nevertheless, this genre of art is disappearing because it is tied to a statemonopolized medium and, unlike opera, has no architectural ark (opera houses) where it can be nurtured by enthusiasts and continue to live. A quick look at the current ARD Mediathek (as of summer 2023) suggests bad things to come. The offering of what was years ago still a proudly modern genre of radio drama has sunk to a subterranean level in times of global competition for click numbers. According to the Internet editors’ presentation, the working group offers radio plays “for hot ears,” “balmy summer nights,” or “to laugh at.” Nearly nothing remains of a once powerful art form in the digital realm. Switch off.

Author Biography

Andreas Ammer is a German author, director, and television journalist. He studied German literature, philosophy, and the history of natural sciences at Ludwig-Maximilians-Universität München, earned his doctorate in 1990, and worked as a lecturer. Since 1995, Ammer has been creating and producing contributions for various cultural programs on the German TV Broadcast Company ARD. He is also a successful author of Spiegel TV reportages.

Art Style | Art & Culture International Magazine

Art Style Magazine is an open access, biannual, and peer–reviewed magazine

Submission

Art Style Magazine publishes both themed issues and issues with a variety of themes. If you are interested in publishing in the current themed edition, check out the call for papers for details on the upcoming editions. You are also welcome to contribute an extended essay or scholarly article on one of our past issues’ themes or a new subject. You just need to follow our Author Guidelines and submit your essay or scholarly article for evaluation. We will get back to you with acceptance or nonacceptance feedback after the reviewers have evaluated your contribution. If your submission is accepted, we will inform you about the publication schedule.

Peer-Review Process

Papers submitted to Art Style Magazine are subject to strict peer review and published in the online issue when two external reviewers have approved them. Deadlines and publication dates are designated in each respective issue’s planning stages. We will get back to authors with acceptance or nonacceptance feedback after the reviewers have evaluated the contributions, and we will inform them about the publication schedule.

Plagiarism

Papers submitted to Art Style Magazine are automatically checked for plagiarism; if a paper is plagiarized, it will not be accepted. All published articles go through the plagiarism scanner and must meet the ethical standards of academic conduct. If plagiarism is discovered in a published article, the plagiarized piece will be removed, and the author will no longer be able to publish in this magazine.

Research Publishing Ethics

Art Style Magazine outlines the best practice principles for publications. We defend the best practices in our publications and follow top institutions’ directions on teaching, science, and research worldwide. With this practice, we wish to remember the fundamental values of recognition of merit and the originality of researchers and authors. Therefore, we present here the central notions of good conduct and research publishing ethics, based on the Committee on Publication Ethics (COPE), the United States National Science Foundation’s policies and procedures, the European Science Foundation’s code of conduct, and the FAPESP São Paulo Research Foundation’s code of good scientific practice.

License and Publishing Agreement

Authors are required to sign a License and Publishing Agreement, when a paper is accepted for publication, retaining copyright while allowing the Art Style Magazine to publish under the terms of that agreement. Authors of Art Style Magazine are invited to accept and agree to be bound by the terms and conditions of Creative Commons Attribution License (CC BY-NC-SA 4.0).

Author Guidelines

Language: American or British English Everything written should be proofread.

The extended essay should be submitted to editorial@artstyle.international

Required structure

Required structure of the extended essay: title; abstract (approx. 300 words); introduction; body of the essay; conclusion; author biography (at least 150 words), endnotes, bibliography (only authors cited in the text should be referred to in the bibliography). Please, use The Chicago Manual of Style system, in Microsoft Word format (.docx). The extended essay must be of a word count of minimum 3,000 and maximum 5,500 (excluding figures, endnotes and bibliography). Page setup A4 (210 by 297 mm), default settings (based on the normal Microsoft Word template). Title: Times New Roman 16 pt, flush left. Sub-title: Times New Roman 14 pt, flush left. Sub-section Heading: Times New Roman 14 pt, flush left. Text body: Times New Roman 11 pt. Line spacing: 1.15, justified. Quotations: more extended quotations should be single-spaced and separated from the text. Also, we accept scholarly articles with a limit of 8000 words (including endnotes and bibliography). Page setup A4 (210 by 297 mm), default settings (based on the normal Microsoft Word template).

Figures: images are required color or monochrome in RGB color mode. Please, set the resolution of images destined for web pages to 72dpi, and save images in jpg or png format only. Resolution: between 1500 and 2500 pixels wide. Don’t enlarge a small image to avoid pixelated images. The related images to incorporate into your paper should have permission from the image owner or use pictures in the public domain. You can choose royalty-free photos with a Creative Commons or similar license. Otherwise, you can create some images of your own. The editorial team will evaluate the images and choose the most appropriate. The author is responsible for the images used in the article, attending to the recommendations indicated in this guide, besides using the image under Creative Commons Attribution License (CC BY-NC-SA 4.0).

Captions for figures

Works of art can be cited using this format:

Figure 1: Artist’s name, Title, Date, Medium, and support. City, Collection. License information.

But include the publication citation for where the artwork’s image was found unless you have viewed the artwork in person. Any image that is being reproduced publicly should consider adding copyright information, i.e., who owns the right to an image or if it is under a Creative Commons Attribution License.

Before you submit your essay, please ensure your paper is ready for submission. Essays and articles submissions that do not meet these recommendations will be returned.

For more information, please see: The Chicago Manual of Style and Magazine’s Author Guidelines

Terms & Conditions

Publishing in Art Style Magazine is free of charge for anyone, there are no article processing charges or other publication fees. Art Style Magazine is independent and supports the Open Access Movement. Art Style Magazine is following what is recommended in international guidelines of the Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities

This magazine is available under the Creative Commons Attribution License (CC BY-NCSA 4.0) All authors (have) permitted the publication under this license There is no copyright transfer towards Art Style Magazine, the authors hold the copyright and publishing rights without restrictions. Papers submitted to Art Style Magazine are automatically checked for plagiarism; if a paper is plagiarized, it will not be accepted. All published articles go through the plagiarism scanner and must meet the ethical standards of academic conduct. If plagiarism is discovered in a published article, the plagiarized piece will be removed, and the author will no longer be able to publish in this magazine. The editors of Art Style Magazine cannot be held responsible for errors or any consequences arising from the use of information contained in essays published on the Art Style Maganize’s website. Authors agree to the terms and conditions and assure that their submissions are free of third parties’ rights. The views and opinions expressed in the essays are those of the author and do not reflect the views of Art Style Magazine. The authors of Art Style Magazine’s essays are responsible for its content. Images from other sources should be fully acknowledged in the caption. The Art Style Magazine website provides links to third-party websites. However, the magazine is not responsible for the contents of those linked sites, nor for any link contained in the linked site content of external Internet sites.

Editor-in-Chief and Creative Director

Christiane Wagner is the editor-in-chief, founder, and creative director of the Art Style, Art & Culture International Magazine, and a member of the College Art Association of America in New York. She has been a visiting research professor at the University of São Paulo. She was a research visiting professor of aesthetics and science of communication, UNICAMP (Fellowship CAPES 2014–2018, qualification for a professorship, Habilitation, Venia Legendi). Both universities (USP and UNICAMP) are ranked as the best universities in Latin America. She was awarded a doctoral degree (Promotion 2014) in Germany in the science of art and aesthetics, recognized by the Hessian Ministry of Science and Art. She has a Ph.D. in science of art and aesthetics from Université Paris 1 Panthéon-Sorbonne with highest honors (mention très honorable/summa cum laude). Also, she holds a Ph.D. in design and architecture (with highest honors) and a master’s degree in science of communication (with highest honors) from the University of São Paulo. She gave lectures and participated in panel discussions at the Stuttgart State Academy of Art and Design, Germany; L’École des Arts de la Sorbonne – Université Paris 1 Panthéon-Sorbonne; Universität Wien; Contemporary Art Museum of the University of São Paulo (MAC USP); Institut für Kunstgeschichte, FAU – Friedrich Alexander Universität, Erlangen Nürnberg; The City University of New York (CUNY), New York City College of Technology (City Tech); University of Hertfordshire, London, and in many international conferences. Prof. Dr. Christiane Wagner is a scientific advisor of the São Paulo Research Foundation FAPESP in cooperation agreements with national and international research funding agencies including the UK Research Councils, the Agence Nationale de Recherche (ANR) in France, the Deutsche Forschungsgemeinschaft (DFG) in Germany, and the European Commission.

Senior Editor

Martina Sauer studied art history, philosophy and classical archaeology at the University of Heidelberg, Germany (mid-term), Paris, France (Socrates fellowship) and Munich, Germany (graduation) and obtained the PhD in science of art from the University of Basel, Switzerland. After her PhD on affective affects and effects of abstract modern art and their consequences, she followed this idea in research and class at the universities and academies of Witten-Herdecke, Weimar, Bremen, Basel, Zürich and currently at Academy Mode & Design Hamburg and Düsseldorf. Martina is advisory board member and co-leader of the section image of German Society of Semiotics and the Society of Interdisciplinary Image Science and senior editor of Art Style, Art & Culture International Magazine

Co - editor

Jörg U. Lensing has been a professor of Sound Design at the Dortmund University of Applied Sciences and Arts since 1996. He studied composition at the Folkwang University of the Arts in Essen from 1981 to 1987. He founded the series of concerts Neue Töne and the ensemble Kunst-Stoff during this time First music theatre composition Ich will zu Dir – Ach komm doch for two dancers. In 1986 he won the Hochschulpreis at the Folkwang University of the Arts. 1987: exams in composition. From 1987 to 1989, he studied (post-graduate) New Music-Theatre with Mauricio Kagel at the University of Music and Dance in Cologne. In 1987 he founded the THEATER DER KLÄNGE in Düsseldorf. Since 1987 he has continued to work as a director, choreographer, and composer for theatre music until now, including 27 productions of this theatre and several compositions of incidental music for theatres and movies. Since 1990 he has also worked on film compositions and sound design for nearly all the films of German film director Lutz Dammbeck. In 1992 he was a guest lecturer in drama direction at the Bauhaus Dessau.

Associate Editors

Dimitra Laurence Larochelle Dimitra Laurence Larochelle is Associate Professor at the Université Sorbonne Nouvelle (Paris, France). She has a PhD in Cultural Studies (Université Sorbonne Nouvelle – Paris 3) and three master degrees in Sociology (Université Paris Descartes), in Communication (Université Panthéon-Assas) and in Anthropology (Université Paris 8 Vincennes, Saint-Denis). She is the co-author of the book Video mapping: heritage in lights and she coordinates the research project entitled “Online fatphobia. Representations, discourses and practices. The case of Instagram.” Additionally, she is secretary of the Research Committee 14 (Sociology of Communication, Knowledge and Culture) of the International Sociological Association (2023 – 2027) and Youth Representative at the United Nations for the International Sociological Association.

Katarina Andjelkovic, Atelier AG Andjelkovic, with a PhD, M. Arch. Eng., is a theorist, practicing architect, researcher and a painter. Katarina’s research, writing and teaching is transdisciplinary and crosses architecture, visual arts and film. She served as a Visiting Professor at the University of Oklahoma (the US), at the Institute of Form Theory and History in Oslo, Institute of Urbanism and Landscape in Oslo, the University of Belgrade, and guest-lectured at TU Delft, AHO Oslo, FAUP Porto, DIA Anhalt Dessau, SMT New York, ITU Istanbul. She has lectured at conferences in more than twenty-six countries in Europe, the UK, the US and Canada; has published her research widely in international journals (Web of Science); won numerous awards for her architecture design and urban design competitions. Katarina won the Belgrade Chamber of Commerce Award for Best Master Thesis defended at Universities in Serbia in all disciplines. Katarina has published two monographs; a book chapter and several journal articles with Intellect.

Natasha Marzliak, Adjunct Professor of History and Aesthetics of Art at the Federal University of Amazonas (UFAM). Doctorate in Visual Arts - Multimedia and Art at UNICAMP, with a research exchange in Cinema and Audiovisual at Université Paris 1 Panthéon-Sorbonne. Master in Visual Arts - Audiovisual Culture and Media. Bachelor of Fine Arts and Bachelor’s Degree in Art Education. As a complementary training, she holds a Diploma of Advanced Studies in French Language from Université Paul Valéry - Montpellier III and has studied subjects such as French Language, Culture and Civilization in France, Cinema, History of Art and Literature. She works in the areas of Aesthetics, Art History, Visual Culture and Visual Arts, with an emphasis on the intersections of contemporary art with cinema, video and new technologies.

Tobias Held, Ph.D. is a german Designer and Design- and Media researcher. Currently he is working as a research associate at the Westphalian University of Applied Sciences in Gelsenkirchen and as a lecturer at the Mu nster School of Design. He holds a Ph.D. in Art and Design from the Bauhaus University Weimar, a Master of Arts from the University of Applied Sciences Mu nster and a Bachelor’s degree from the University of Applied Sciences in Dessau. He works in the areas of Design- and Mediatheory, Design research and cultural and technical history with a focus on topics about perception, communication and technics. His main topic is the media theory of video telephony.

Art Style Maganize’s website: artstyle.international

Art Style | Art & Culture International Magazine

editorial@artstyle.international

+1 347 352 8564 New York

+55 11 3230 6423 São Paulo

The magazine is a product of Art Style Communication & Editions. Founded in 1995, the Art Style Company operates worldwide in the fields of design, architecture, communication, arts, aesthetics, and culture.

ISSN 2596-1810 (online)

ISSN 2596-1802 (print)

Theodor Herzi, 49 | 05014 020 São Paulo, SP | CNPJ 00.445.976/0001-78

Christiane Wagner is the lead designer, editor, and registered journalist in charge: MTB 0073952/SP

© 1995 Art Style Communication & Editions