Sound for picture – the eighth nerve

Generative sound design at University of West London

At the invitation of UWL Lecturer Charlie Norton, Carla Scaletti presented a lecture/demonstration on Generative Sound Design in Kyma for students, faculty and guests at University of West London on 14 November 2024. As an unanticipated prelude, Pete Townshend (who, along with Joseph Townshend, works extensively with Kyma) welcomed the Symbolic Sound co-founders to his alma mater and invited attendees to tour the Townshend Studio following the lecture.

After the seminar, graduating MA students Vinayak Arora and Sabin Pavel (hat) posed with Kurt Hebel & Carla Scaletti (center) and Charlie Norton (distant upper right background)

UWL Professor of Music Production Justin Paterson and Trombonist/Composer/Kyma Sound Installation artist Robert Jarvis discuss the extensive collection of instruments in the Townshend Studio

It seems that anywhere you look in the Townshend Studio, you see another rock legend. John Paul Jones (whose most recent live Kyma collaborations include Sons of Chipotle, Minibus Pimps, and Supersilent among others) recognized an old friend from across the room: a Yamaha GX-1 (1975), otherwise known as ‘The Dream Machine’ — the same model JPJ played when touring with Led Zeppelin and when recording the 1979 album “In Through The Out Door”. Yamaha’s first foray into synthesizers, only 10 were ever manufactured; it featured a ribbon controller and a keyboard that could also move laterally for vibrato. Other early adopters included ELP, Stevie Wonder and Abba.

JPJ recollects his days of touring with the GX1 and the roadie who took up temporary accommodation in its huge flight case, as Alan, two students, Bruno and Robert look on.

Charlie Norton with Alan Jackson (back), JP Jones (at Yamaha GX-1), Carla Scaletti & Kurt Hebel in the Townshend Studio

Alan Jackson and Pete Johnston pondering the EMS Vocoder

Composer Bruno Liberda (the tall one) with Symbolic Sound co-founder Carla Scaletti

The Light Moves Between

Portland-based multimedia artist William Selman’s 8 November 2024 album, The Light Moves Between, poses the question: “Can scenes from the Pacific Northwest of America connect us to entities in distant universes, or are we listening to William’s unique processing of his own surroundings?”

Various Kyma techniques can be heard on each track, with cross-synthesis emerging as Selman’s personal favorite. Here are a few details you can listen for on the album:

“Outshone the Sun”

Sunspots are caused by intense magnetic fields emerging from the interior. Image created by NASA.

Inspired by sunspots and electromagnetic interference reminiscent of the sounds of popping air bubbles that barnacles make during low tide, Selman used a frequency shifter in Kyma to modulate the Serge New Timbral Oscillator for the sharp droning tones near the beginning to match the character of the shortwave radio sounds. Much of the bed of textures in the second half of the piece is made by cross-filtering field recordings of barnacles against shortwave radio recordings.

“Kept in Banks and Vessels”

For the triangle and singing bowl sounds, Selman designed a Jaap-Vink-inspired feedback Sound in Kyma to create the drones. For the final section, he cross-filtered Serge sounds with frog sounds that he recorded in Hawaii. The warbly animal call sounds are made in Kyma with an Oscillator and a LossyIntegrator.

“Flutter at the False Light”

Here you can hear a Kyma frequency shifter modulating various electromagnetic field recordings made near the composer’s house. The bed of textures for the ambience are cross-filtered sounds from recordings made with contact mics in his studio space and on his windows.

“New Topographics”

The last track was inspired by American landscape photographers and photos of empty, non-places primarily in the American West. Selman recorded sounds such as birds, wind storms, electrical lines, and water wells in the Central Oregon high desert. He used a cross filter patch made by Pete Johnston and Alan Jackson during the Kyma Kata sessions, which he finds particularly useful for creating a stream of sounds that flow seamlessly from one into another. Other instrumental elements include a Serge synthesizer, organ, and bowed (and hit) vibraphone.

The Light Moves Between represents Selman’s return to visual work following a long hiatus — this time around, he is bringing his visuals together with his sound work. The first three tracks on the album were written to accompany three short films. Although the first one (“Outshone the Sun”) has been set aside, the second and third pieces are complete and can be viewed on Vimeo:

“Flutter at the False Light”: vimeo.com/1009701215
“Kept in Banks and Vessels”: vimeo.com/1009700845

Specially created Dolby Atmos versions of all four tracks are available via Apple Music.

Vampires in London, Hyper-organs in Lübeck

In October 2024, Franz Danksagmüller, along with his students from the Royal Academy of Music, performed a live soundtrack for pipe organ and Kyma to accompany the silent film Nosferatu twice in London: The first performance, on 4 October 2024, was at the Royal Academy, and the second performance was on Halloween at the cathedral in St. Albans to a sold-out audience and was, by all accounts, a fantastic success (resulting in some very happy students).

St. Albans organ loft during live soundtrack for “Nosferatu” with Kyma Control on the iPad

Audience for the sold-out performance of Nosferatu at St. Albans (view from behind the projection screen)

Students of Franz Danksagmüller performing the live soundtrack for Nosferatu at St. Albans

At the Musikhochschule Lübeck, Danksagmüller’s students in the new master’s program in Hyper-organ, Improvisation, Composition & New Media are presenting a concert for organ, Kyma and live-video on 8 December 2024 in the main concert hall at the MHK. Composer/performers include:

Sarah Proske: Emerson, Lake and Palmer´s “Fanfare for the common man for organ, percussion, live electronics
Patrycja Olszewska: Liquid Motion for organ, live electronics, video
Lennart Pries: Improvisation for the silent film “The Fall of the House of Usher (USA 1927) for organ and live electronics
Karin Lorenz: (Un-)Known for organ and electronics
Sarah Proske: smoke for organ, live electronics and video
Valentin Manß & Wojciech Buczyński: crowds, live & death, Music for (historical) silent film excerpts for organ, live electronics, video
Fabio Paiano: á la ELP for organ and live electronics

Kyma Sound design studies at CMU

Did you know that you could study for a degree in sound design and work with Kyma at Carnegie Mellon University? Joe Pino, professor of sound design in the School of Drama at Carnegie Mellon University, teaches conceptual sound design, modular synthesis, Kyma, film sound design, ear training and audio technology in the sound design program.

Sound design works in the spaces between reality and abstraction. They are less interesting as a collection of triggers for giving designed worlds reality. They are more effective when they trigger emotional responses and remembered experiences.

Thinking through sound — Ben Burtt and the voice of WALL-E

Ben Burtt was recently awarded the 2024 Vision Award Ticinomoda, which describes him as:

“Ben Burtt is a pioneer and visionary who has fundamentally changed the way we perceive sound in cinema.”

In this interview, Burtt shares some of his experiences as Sound Designer for several iconic films, including his discovery of “a kind of sophisticated music program called the Kyma” which he used in the creation of the voice of WALL-E.

The interviewer asked Ben about the incredible voices he created for WALL-E and EVA:

Well, Andrew Stanton who was the creator of the WALL-E character in the story; apparently he jokingly referred to his movie as the R2-D2 movie. He wanted to develop a very affable robot character that didn’t speak or had very limited speech that was mostly sound effects of its body moving and a few strange kind of vocals, and someone (his producer, I think — Jim Morris) said well why don’t you just talk to Ben Burtt, the guy who did R2-D2, so they got in touch with me.

Pixar is in the Bay Area (San Francisco) so it was nearby, and I went over and looked at about 10 minutes that Andrew Stanton had already put together with just still pictures — storyboards of the beginning of the film where WALL-E’s out on his daily work activities boxing up trash and so on and singing and playing his favorite music, and of course I was inspired by it and I thought well here’s a great challenge and I took it on.

This was a few months before they had to actually greenlight the project. I didn’t find this out until later but there was some doubt at that time about whether you could make a movie in which the main characters don’t really talk in any kind of elaborate way; they don’t use a lot of words. Would it sustain the audience’s interest? The original intention in the film that I started working on was that there was no spoken language in the film that you would understand at all; that was a goal at one point…

So I took a little bit of the R2 idea to come up with a voice where human performance would be part of it but it had to have other elements to it that made it seem electronic and machine-like. But WALL-E wasn’t going to Beep and Boop and Buzz like R2; it had to be different, so I struggled along trying different things for a few months and trying different voices — a few different character actors. And I often ended up experimenting on myself because I’m always available. You know it’s like the scientist in his lab takes the potion because there’s no one else around to test it: Jekyll and Hyde, I think that’s what it is. So I took the potion and and turned into Mr Hyde…

Photo from Comicon by Miguel Discart Bruxelles, Belgique This file is licensed under the Creative Commons Attribution-Share Alike 2.0 Generic license

The idea was to always give the impression of what WALL-E was thinking through sound…

But eventually it ended up that I had a program — it was a kind of sophisticated music program called the Kyma and it had one sound in it — a process where it it would synthesize a voice but it [intentionally] didn’t do very well; the voice had artifacts that had funny distortions in it and extra noises. It didn’t work perfectly as a pure voice but I took advantage of the fact that the artifacts and mistakes in it were useful and interesting and could be used and I worked out a process where you could record sounds, starting with my own voice, and then process them a second time and do a re-performance where, as it plays back, you can stretch or compress or repitch the sounds in real time.

So you can take the word “Wall-E” and then you could make it have a sort of envelope of electronic noise around it; it gave it a texture that made it so it wasn’t human and that’s where it really worked. And of course it was in combination with the little motors in his arms and his head and his treads — everything was part of his expression.

The idea was to always give the impression of what WALL-E was thinking through sound — just as if you were playing a musical instrument and you wanted to make little phrases of music which indicated the feeling for what kind of communication was taking place.

True Crime

In early 2025, listen closely to the soundtrack of Lady of the Dunes, the new true crime series airing on the Oxygen Network. The series looks at a recently solved 50 year old cold case.

The director wanted the music to have a haunting quality to tie into the mystery and brutality of the case. That’s when they called on composer/editor John Balcom.

Balcom started out by recording plaintive, mysterious sounds, like wine glasses and a waterphone. Then he took sections from those recordings and treated them using Kyma’s Multigrid. There was a performative aspect to all of this; he would manipulate the sounds going into Kyma, as well as adjusting sounds in real time in the Multigrid, resulting in some disturbingly haunting sounds. Balcom then incorporated those sounds to build out full compositions.

Here are some audio examples of the textures he created using Kyma, along with a couple of tracks that incorporate those textures (Warning: the following clips contain haunting sounds and may be disturbing for all audiences. Listener discretion is advised! ⚠️ ):

Here’s a screen shot of one of Balcom’s Multigrids:

Balcolm concludes by saying: “The music would not have been the same without Kyma!”

Vision of things unseen

Motion Parallels, a collaboration between composer James Rouvelle and visual artist Lili Maya was developed earlier this year in connection with a New York Arts Program artist’s residency in NYC.

For the live performance Rouvelle compressed and processed the output of a Buchla Easel through a Kyma Timeline; on a second layer, the Easel was sampled, granulated and spectralized and then played via a midi-keyboard. The third layer of Sounds was entirely synthesized in Kyma and played via Kyma Control on the iPad.

For their performative works Rouvelle and Maya develop video and audio structures that they improvise through. Both the imagery and sound incorporate generative/interactive systems, so the generative elements can respond to them, and vice versa.

Rouvelle adds, “I work with Kyma every day, and I love it!”

Soundtrack for a Lost World

Performing with Kyma and a huge (but very soft) pipe organ, Franz Danksagmüller generated a live soundtrack for the silent film The Lost World (think silent-film Jurassic Park from the 1920s) on 25 May 2024 at the National Radio Symphony Concert Hall in Katowice Poland for an audience of 1600 people.

Franz Danksagmuller monitoring the VCS, the film, the Multigrid (on the iPad) and his pencil notes

The Polish National Radio Symphony Orchestra in Katowice seats an audience of 1800 people

In a gravity-defying stunt, Danksagmüller placed two mics within the swell boxes of both Manual V and Manual III, at the apex of the organ pipes, so he could process the pipe organ through Kyma without danger of feedback. He and his sound engineer worked until 2:30 am fine-tuning the setup.

Poster for the film “The Lost World” (1925) * First National Pictures – Source, Public Domain, https://commons.wikimedia.org/w/index.php?curid=12690702

* Note that, according to Wikipedia, “This historical image is not a factually accurate dinosaur restoration. Reason: Pronated hands, real T. rex did not have more than two fingers (unlike in this image), outdated posture, tail dragging, lack of possible feathers”

Kyma consultant Alan Jackson’s studio

Alan Jackson’s studio — designed for switching among a variety of sources, controllers, and ease of portability

Musician and Kyma consultant, Alan Jackson, known for his work with Phaze UK on the Witcher soundtrack, has wired his studio with an eye toward flexibility, making it easy for him to choose from among multiple sources, outputs, and controllers, and to detach a small mobile setup so he can visit clients in person and even continue working in Kyma during the train ride to and from this clients’ studios.

Jackson’s mobile studio sessions extend to the train to / from a gig

In his studio, his Pacamara Ristretto has a USB connection to the laptop and is also wired with quad analog in / out to the mixer. That way, Jackson can choose on a whim whether to route the Ristretto as an aggregate device through the DAW or do everything as analog audio through the mixer. Two additional speakers (not shown) are at the back of the studio and his studio is wired for quad by default.

The Faderfox UC4 is a cute and flexible MIDI controller plugged into the back of the Pacamara, ready at a moment’s notice to control the VCS, and a small Wacom tablet is plugged in and stashed to the left of the screen for controlling Kyma.

Jackson leaves his Pacamara on top so he can easily disconnect 5 cables from the back and run out the door with it… which leads to his “mobile” setup.

The Pacamara and its travel toiletry bag

Jackson’s travel setup is organized as a kit-within-a-kit.

The red, inner kit, is what he grabs if he just needs a minimal battery-powered Kyma setup, e.g., for developing stuff on the train, which includes:

a PD battery (good for about 3 hours when used with a MOTU Ultralite, longer with headphones)
a pair of tiny Sennheiser IE4 headphones
a couple of USB cables, and an Ethernet cable

The outer kit adds:

a 4 in / 4 out bus-powered Zoom interface
mains power for the Pacamara
more USB cables
a PD power adaptor cable, so he can run the MOTU Ultralite off the same battery as the Pacamara
a clip-on mic
the WiFi aerial

If you have any upcoming sound design projects you’d like to discuss, visit Alan’s Speakers on Strings website.

When not solving challenges for game and film sound designers, Alan performs his own music for live electronics.

Alan Jackson’s setup for a recent live improvisation for salad bowl and electronics

Out of the fire, into the Kyma consultancy

It’s not every day that a consultant/teacher gets a closing screen credit on a Netflix hit series, but check out this screenshot from the end of Witcher E7: Out of the Fire, into the Frying Pan:

Kyma consultant, Alan M. Jackson has been working with sound designers Matt Collinge, Rob Prynne and Alyn Sclosa from Phaze UK for a while now, and the students surprised their sensei with an end credit on The Witcher

Kyma Consultancy – Alan M Jackson

Alan and his small band of sound designers meet regularly to share screens and help each other solve sound design challenges during hands-on sessions that last from 1-2 hours. Not only are these sessions an opportunity to deepen Kyma knowledge; they’re also an excuse to bring the sound design team together and share ideas. Alan views his role as a mentor or moderator, insisting that the sound designers do the hands-on Kyma work themselves.

There’s no syllabus; it’s the sound designers who set the agenda for each class, sometimes aiming to solve a specific problem, but more often, they aim to come up with a general solution to a situation that comes up repeatedly. They’ve taken to calling these solutions “little Kyma machines” that will generate an effect for them, not just for this project but for future projects as well. Over time, they’ve created several, reusable, custom “machines”, among them: ‘warping performer’, ‘doppler flanger’, ‘density stomp generator’, and ‘whisper wind’.

Rob Prynne is particularly enamored of a custom patch based on the “CrossFilter” module in Kyma (based on an original design by Pete Johnston), sometimes enhanced by one of Cristian Vogel’s NeverEngine formant-shifting patches. In an interview with A Sound Effect, Rob explains, “It’s like a convolution reverb but with random changes to the length and pitch of the impulse response computed in real-time. It gives a really interesting movement to the sound.” According to the interview, it was the tripping sequence in episode 7 where the CrossFilter really came into play.

When asked if there are any unique challenges posed by sound design, Alan observes, “The big things to note about sound design, particularly for TV / Film, is that it’s usually done quickly, there’s lots of it, you’re doing it at the instruction of a director; the director wants ‘something’ but isn’t necessarily nuanced in being able to talk about sound.”

A further challenge is that the sounds must “simultaneously sound ‘right’ for the audience while also being new or exciting. And the things that sound ‘right’, like the grammar of film editing, are based on a historically constructed sense of rightness, not physics.”

Alan summarizes the main advantages of Kyma for sound designers as: sound quality, workflow, originality, and suitability for working in teams. In what way does Kyma support “teams”?

With other sound design software and plugins you can get into all sorts of versioning and compatibility issues. A setup that’s working fine on my computer might not work on a teammate’s. Kyma is a much more reliable environment. A Sound that runs on my Paca[(rana | mara)] is going to work on yours.

Even more important for professional sound design is “originality”:

Kyma sounds different from “all the same VST plugins everyone else is using”. This is an important factor for film / TV sound designers. They need to produce sounds that meet the brief of the director but also sound fresh, exciting, new and different from everyone else. Using Kyma is their secret advantage. They are not using the same Creature Voice plugins everyone else is using but instead creating a Sound from scratch comes up with results unique to their studio.

For Witcher fans, the sound designers have provided a short cue list of examples for your listening pleasure:

Kyma Sound	Heard	Episode
Warping performer	on every creature	in every episode
Warping performer	on portal sounds and magic	S3E03
Warping performer	on the wild hunt vapour	S3E03
Doppler flanger spinning thing	on the ring of fire	S3E05 / S3E06
Density stomp generator	for group dancing	S3E05
Whisper wind convolved vocal	for wild hunt	S3E03

Visit Alan Jackson’s SpeakersOnStrings to request an invitation to the Kyma Kata — a (free) weekly Zoom-based peer-learning study group. Alan also offers individual and group Kyma coaching, teaching and mentoring.