Sound Design: Jonathan Mitchell

photo of Jonathan Mitchell

Intro from Jay Allison: We have a new offering in our new series on Sound Design, this one from Jonathan Mitchell, the creator and producer of Radiotopia’s podcast, The Truth. Jonathan says, "I think everyone involved in audio production should think of him or herself as a sound designer. Good sound design is not decorative, and it's not afterthought; it exists as an integral element of your story, and is embedded in every decision you make." This feature breaks down differences between literal and abstract sound design. Jonathan takes us through his process in designing a tiny moment in a recent episode of The Truth: a puffer fish expanding and popping a plastic bag underwater. Listen to the five audio steps to assemble this scene in service to the story around it.

This is part of our series, "Sound Design." You can find others here: https://transom.org/tag/sound-design/

“Sound Designer”

I’m always a little surprised when I see the title “sound designer” next to someone’s name in an audio production. It’s presented as though sound design is a different department, a separate activity from the writing, producing, recording, or mixing. In fact, I think everyone involved in audio production should think of him or herself as a sound designer. Good sound design is not decorative, and it’s not an afterthought; it exists as an integral element of your story, and is embedded in every decision you make.

I like to think of sound design as the *non-linguistic* aspects of your story. That includes a lot of what you’d expect, like music and sound effects, and the blurry space between. But it also includes things like editing, mixing, and effects processing. It includes vocal inflection, timbre, range, and regional accent — when you cast an actor, you are making a sound design choice. Most notably, it includes the choice to have no sound at all.

Sound design is the conceptual approach to the audio identity of a story. It’s how all of these elements work together to communicate and embody your idea.

So, when you’re deciding what story you’re going to make, after you have your initial concept but before you think about structure, think about what it will sound like. Some questions you might ask include:

  • What point of view do you want your story to convey, and how can that best be expressed through sound?
  • How do you want your listener to feel, and how can sound support that?
  • Does your story call for an “objective/literal” approach to sound effects? Or would it be better served by using a more “subjective/interpretive” approach?
  • Is location important, and if so how will that be conveyed?
  • What about time — how can sound be used to expand or contract your listener’s perception of time?

You might decide to begin by making a sonic stylebook. Seek out visual art that evokes a sensibility that reflects the tone of your story, or think of films or television programs that have been particularly effective at making you feel a particular way. You could pick out a piece of music that sets the tone for what you’re trying to accomplish — it doesn’t need to be music that you end up using in your story. We made a story for The Truth a few years ago called “The Man in the Barn,” and early on I sent the writer Louis Kornfeld a piece of music by Popol Vuh (a German band that I learned about from listening to the scores of Werner Herzog films). That music wasn’t in the final piece, but it set the tone, and suggested a kind of relationship that music could have with the story we were making.

One of the hidden superpowers of audio is that you don’t see anything. Something only exists in your story if your audience imagines it based on what they hear. That means that you can quickly build any environment you want simply by adding a sound. If you want an ocean, you just add an ocean. And the details say a lot: a door is never just a door, it’s a particular *kind* of door. It’s a wooden screen door with a rusty spring hinge that ricochets off the door frame when it slams shut. The sound reflects information about the story, perspective, and who the characters are.

But crucially, that also means you can easily focus your listener’s attention. You can shape the meaning of your story by pushing certain sounds to the front of the mix, or by editing the audio to highlight a particular moment, or by emphasizing certain characteristics of the sound over others (using EQ and other audio processing tools).

And while sound is great at indicating what something is, I think it’s even better at evoking how something *feels.*

The Feel of Driving

Consider this sound of a car engine:

Download
Listen to “Example 1”

This is a recording of a Ford Escort, from the perspective of the driver’s seat. The type of car and the mic placement are both very important. Cars are like characters, they all sound different. And the mic placement establishes perspective; if we were watching the car from across the street, we’d want a very different sound.

It’s also a very literal representation of a car engine. It’s not processed, it’s exactly what you would get if you placed a certain type of microphone inside a car and pressed record.

Now consider this sound:

Download
Listen to “Example 2”

This is a synthesized representation of a car engine. It suggests the experience of driving a car but no one would mistake it for a real engine. If the sound effect in the previous example is like a photograph, this is more like an impressionist painting.

Both of these sounds could be useful, depending on the circumstance. It all depends on what kind of relationship you want your listener to have to your story. It’s about the Objective vs. the Subjective. If we are objectively observing the driver, then you might choose the first sound (or even better, a sound that you recorded on location yourself). On the other hand, if you want to create a more subjective experience — one that feels like we are actually inside the mind of the person driving the car — you might choose something closer to the second sound.

There’s no right or wrong answer for this by the way — it’s all about context and circumstance, and how you want your listener to relate to the material. Maybe the best choice is actually this:

Download
Listen to “Example 3”

This has the momentum and effect of an engine, while not being strictly imitative. If the first example is a photograph, and the second example is an impressionist painting, this is abstract expressionism. And it creates a different kind of momentum than the synthesizer drone or the sound effect.

It gets even more interesting when you begin to combine these elements. We might start a sequence with the sound effect from the first example to establish the driver’s reality. But as the sequence progresses, the driver becomes lost in thought and removed from her reality, so we gradually crossfade to the synthesizer, moving to a more abstract space. But then her thoughts lead to a realization that she needs to take urgent action, so the rhythmic pulse begins.

NOTE: If you are making a documentary, which sound you choose could have ethical implications. This requires an honest and thorough understanding of what your sound design will make your audience think they are hearing. If you are purporting to provide a factual account of an event, you have a responsibility to ensure that your audience isn’t reading false information into your story. This isn’t to say that you can’t use a stylized sound design for a documentary; it just means that if you do, you need to have a good reason.

Anatomy of a Puffer Fish

Sometimes you need to convey something in audio that you don’t have the option to record, because it doesn’t actually make a sound. For example, something that’s primarily a visual experience, or a conceptual illustration. You want to give the listener an experience they can feel, to make it easier for them to digest the information you are presenting. (Radiolab in particular does this often and well.)

We recently made a story for The Truth called “Fish Girl” (written by Becca Schall), about a girl who befriends a talking puffer fish who has lived her entire life in an aquarium. At the climax of the story, the girl steals the puffer fish, puts her in a plastic bag and takes her to a pond, so that she can meet other fish living in the wild. The girl lowers the bag into the pond, and everything goes well until something unexpected appears: a turtle. The puffer fish panics, and suddenly puffs into a big spiky ball, inadvertently poking holes in the plastic bag.

I wanted to give listeners the subjective experience of being that puffer fish. And I felt that the essence of that might be conveyed by the sound of a small cavity expanding into a large one. There were two sounds that immediately popped into my mind that I could work with: the sound of a long inhale (which is our most relatable subjective experience of filling a cavity inside our own bodies), and the sound of a balloon being inflated (which could reflect the physical transformation of the cavity’s stretching surface). I decided to try getting both, and then see if I could mix them together to create the experience of puffing up.

I asked the actress who played the fish (Alison Cimmet) to give me several takes of long inhales. We tried lots of approaches: fast, slow, breathy, pitched. We wanted to get just the right contour of the transformation.

Download
Listen to “Example 4”

She gave me one that I really liked that was a pitched breath which started breathy, then became pitched and glided higher and higher. An interesting thing about this is that it’s really the opposite of what physically happens when a cavity expands; when something gets larger, the pitch actually gets lower and lower, not higher and higher. I didn’t realize this until I started inflating balloons.

My first instinct was to record a balloon myself — that would give me a sound that no one else has, that’s unique to this production. But, I also looked in my sound effects library to see if I happened to have any balloon sounds just sitting around on my hard drive. I’m not gonna lie: sometimes the perfect sound is a sound effect from a library. And that’s where I found the one I liked the most. It was great because it had these rubber-y stretches in it as the balloon got really big. And those stretches made little “pops” into the cavity of the balloon that activated the space and really made you feel how it was expanding.

Download
Listen to “Example 5”

So, I took the balloon sound and Alison’s inhale, and layered them together in Pro Tools. That’s when I noticed that the pitch of the balloon was going down, while the pitch of Alison’s breath was going up. Sometimes that can be very effective: contrary motion is disorienting, and it’s especially good for suggesting feelings that are queasy or unsettling. But for this, it wasn’t quite producing the effect I wanted.

Along the way, I had an idea that after the fish expands, it might be good to make Alison’s voice lower in pitch, to make her voice sound larger along with her body. I put a pitch-shifting plugin on her track (Z-Plane’s Elastique Pitch, highly recommended). I set it up in Pro Tools so that I could automate the amount of pitch shift using breakpoints on the timeline — that way I could change the pitch gradually over time if I wanted. And that’s when it clicked: maybe I could apply a gradual pitch shift to her pitched inhale. What would that sound like?

I started playing around and discovered that if I gradually shifted her breath down in pitch at the exact same rate that she was making her pitch go up, the effect was that the pitch stayed exactly the same. But (and here’s the fun part) even though the pitch stayed the same, the voice itself felt larger and larger over time, because it was, in essence, a bigger and bigger vocal cavity producing the sound. (I think of this technique as roughly analogous to a reverse zoom in cinematography.)

Download
Listen to “Example 6”

There was one last element of the transformation to add: the spikes popping out of her skin. We needed them to pop a hole in the bag. At first I thought about flicking the bristles of a comb, or wooden percussion sounds, maybe plucked violin strings. I tried a bunch of different things, but none of them were quite gelling because they felt disconnected from the fish’s body. I wanted the sensation that these spikes were popping out organically from her skin. Also, they were clashing with the popping sound of the balloon stretching, which I thought was really nice.

And then I thought, what if I used those stretch pops as the spikes. In the original sound effect, they become slightly more sparse as the balloon expands — I wanted there to be a flourish of pops near the end. I found a second balloon sound with pops that had a sharper, crisper sound. I mixed that in, copied and pasted some of the pops, and arranged the sounds to create the flourish effect I wanted.

Download
Listen to “Example 7”

In the finished story, this moment goes by pretty quick. But it’s a crucial moment — it’s a turning point in the story that raises the stakes in an important way, and I wanted the audience to really feel it. Here is the finished sequence:

Download
Listen to “Example 8”

Let Your Purpose Be Your Guide

There are no *rules* per say about what works and what doesn’t. There’s only purpose. What are you trying to accomplish? Find something that motivates you, that excites and thrills you, that you really need to share with other people.

Sound design is not tricks, and it’s not a band aid that will fix a bad piece. It’s simply letting your ideas express themselves fully through audio. The good news is that you’re probably a sound designer already, whether you realize it or not. It’s impossible to make an audio piece without making sound design choices. But consider a broad vocabulary, and always try to make the choice that will have the greatest emotional impact. Sound is a potent and highly expressive tool, but it’s not about style over substance. Good sound design is about finding the most effective way to express the substance of your story.

Jonathan Mitchell

About
Jonathan Mitchell

Jonathan Mitchell is the creator and producer of The Truth, a podcast distributed by PRX’s Radiotopia that makes short films without pictures. He has contributed a wide range of work — documentaries, fictional stories, non-narrated sound collages, and original music — to all sorts of programs: RadioLab, Studio 360, This American Life, All Things Considered, Planet Money, and PBS’s Nova, to name a few. He studied music composition at University of Illinois and Mills College.

Comments

Your email address will not be published. Required fields are marked *

*
*
Website

  • Tom Lopez

    7.04.18

    Reply

    Very nice work. Excellent puffer sound. Made me smile.

Your email address will not be published. Required fields are marked *

*
*