Intro from Jay Allison: Of course, every sonic choice you make is part of your design, but we're in a new age, abetted by remarkable digital tools, where the whole field of sound design is booming, for better and sometimes for worse. Transom is starting a new series commissioning features from some of the best sound designers in public radio and podcasting.
First up, Mark Henry Phillips (Homecoming, Serial, Radiolab and more) who says, "It's an interesting job: you work hard to layer sound atop sound and if you get it just right, all those layers become invisible. The listener thinks, 'That's just what it sounded like'--if they think about the sound at all. The majority of the work is about being unnoticed."
This is part of our series, "Sound Design." You can find others here: https://transom.org/tag/sound-design/
What Does Sound Design Even Mean?
When people ask me this I try to answer honestly: it’s just a fancy name for a bunch of very different audio-related jobs. I get why people use the term “sound design.” It has a cool ring to it! But “sound design” can mean almost anything depending on who you’re talking to, including (but not limited to):
- Dialogue editing
- Sound FX editing
- Music editing
- Editing (like, plain ‘ol editing from from an editorial standpoint . . .)
- Audio restoration
- Sound FX recording
- Designing new sounds from scratch
- Sonic branding
- Sonic UI
And of course all of these different jobs can be applied to different mediums: film, theater, radio, podcast, games, apps, etc . . . In the film-world each of these jobs were traditionally given to individuals (or whole departments) who could quickly produce amazing results in their one area. As budgets shrink, these jobs are increasingly being given to a single individual. In the radio and podcast world this is especially true. So yes, a “sound designer” often has to do a lot of these jobs.
But it’s still important to realize how different they can all be from each other. They can require different tools, skills, and training. Maybe you want to do them all! It’s still worth thinking about all the different jobs that fall under the “sound design” umbrella and which ones will be required on each job that comes your way.
Because it’s such a large topic and because each project requires its own approach to sound design, I’m going to share some broad ideas that I think hold true for all sound design work.
The Most Important Thing in ALL Sound Design Work is Your Ear
Close listening is the most important skill to bring to all sound-related work. If you don’t have a good ear, it doesn’t matter how much fancy gear you buy or how much you practice certain skills. And if you have a good ear, you can figure out almost any problem. But what does having a “good ear” mean? It’s not as simple as being able to pass a hearing test at the doctor’s office (although that’s a good place to start!).
Having a “good ear” is actually more about your brain than your ear. To me a “good ear” means being able to break apart sounds into their component parts. I think this is truly the key to being a strong sound designer. A good practical example of this skill is how good music producers hear a song they’re working on. They don’t hear it as one glob of music. They hear it as a bunch of different tracks stacked on top of each other (vocals, bass, drums, keys, etc.). And in their heads, they have the ability to move from element to element. They can use their “good ear” to focus on just the guitar. In their head the rest of the tracks drop away and they can hear just the guitar. To them it’s almost as if they hit the solo button in Pro Tools.
A good sound designer needs that skill too. And you can practice! Next time you’re at a loud party stand in the middle of the room and try to eavesdrop on one conversation on your left side. Then, without moving, switch to another conversation behind you. Then another on your right side. Our ability to do that is actually a very complex skill and it’s all happening in our brains (not our ears). Think about it — there are many conversations blending together and hitting our eardrums as a giant cacophony. But our brain is able to filter out all of them except one. It’s pretty amazing! And believe it or not, audio software is WAY behind our brains in being able to do this.
And this exactly what you need to do when working on a big sound design project. This is what a Pro Tools session looks like for a small, independent film:
This is a screenshot of a 10-minute reel and it’s actually missing all the dialogue tracks (and about a dozen or so other FX tracks). As you can see, there are a TON of audio tracks being layered together all at once. None of these scenes were too complicated, but to create a believable world of sound you might need to layer 6 background tracks, 8 FX tracks, 6 Foley tracks, 20 music tracks and 6 dialogue tracks. That means at any given time you might be listening to fifty tracks at once — or fifty layers of sound. If something doesn’t sound right, your brain has to be able to zero in on the one layer — out of fifty — that isn’t working.
And it actually doesn’t stop there. You need to be able to break apart each track into its component parts too! For example, maybe one track of wind (out of 50 total tracks) is sticking out to you. You still need to be able to focus on what’s wrong with the wind track. Maybe it needs an EQ tweak because it has too much build-up around 400 kHz. Maybe it needs a little reverb. Maybe it sounds a little distorted and you need to clean it up. So it’s not just listening to 50-some tracks at once — it’s listening to all the parts of those 50 sounds at once. It’s a lot to listen to and that’s why you need a good ear!
...and good eyes, and a good website!
Help Transom get new work and voices to public radio by donating now.
Sound is One of the Most Effective Ways to Alter Our Sense of Space
I don’t have the science to back this up, but I feel strongly that sound is more intimately connected to people’s unconscious mind (whereas visual input is more connected to people’s conscious mind). Put another way, pictures often inspire thoughts whereas sounds often inspire feelings. Let me give you an example. Here’s a picture of Grand Central Station in New York City:
What did you just experience when you looked at the picture? Maybe you thought about the last time you were at Grand Central or a large train station. Maybe you admired the architecture. Did you feel like you were there? Probably not.
Okay. Now put on good headphones, turn the volume up to a decent level, close your eyes, and listen to this recording from that exact spot:
What did you experience when you listened to the recording? I know you didn’t actually think you had been teleported to Grand Central but I argue that listening to the sound subtly tricked your brain into feeling like it was at Grand Central. That sound was so similar to what it would actually sound like to be there. Your brain hears the crowd and the echoes and starts to feel like it’s in a big, crowded space. That’s something truly unique to sound. (Maybe someday VR technology can become so advanced that it tricks your brain in the same way, but right now the resolution, refresh rates, and motion-tracking make it distinctly different than what it actually looks like to be somewhere. Even if we get to that point, I maintain that sound is a super important piece of your brain’s ability to build a sense of space.)
Sound is One of the Most Effective Ways to Alter Our Emotions
And just as this recording altered your sense of space and location, good sound design can alter the listener’s emotions and subjective experience. Maybe you’re designing a scene in a film where two characters — Jack and Jill — are having a conversation over dinner at a restaurant. Let’s say the news that Jack is giving Jill (our lead character) is earth-shattering. Let’s say the director doesn’t want to use score in this scene but wants the audience to be terrified and also feel like Jill is starting to panic. You can totally do that just by designing the sound the right way.
Right after the earth-shattering news is delivered, we could fade down Jack’s voice as he continues talking and pull off the high end from his voice. This will make us feel that he’s muffled or that something is wrong with our ears. We could also add a bit of chamber reverb to make us feel that he’s really far away (even though he’s sitting right across the table from our lead actress). From there we could also pull up the walla (all the voices of the other restaurant patrons talking). This will help add to the sense of panic since it’ll feel like all these people are closing in on us! And maybe we can hear the dishes being bussed nearby, but significantly raise the volume on that and also put some reverb on it so it almost feels like a dissonant musical instrument. As those sounds are getting louder and louder we can make Jack’s voice disappear completely (even though we see that he’s still talking). Then all of the sudden we can have everything go back to normal just as Jack asks Jill if everything is okay.
If you design the sound this way, you’ll start to see the visual performance in a completely different way. It will actually look like different performances or a different edit of the film. A subtle expression Jill makes will be recast as one of sheer terror. A mundane facial tick on Jack could make him look completely menacing. The people in the background will go from looking like normal people enjoying their dinner to a bunch of terrifying Stepford Wives.
Good Sound Design Often Goes Unnoticed
If you design that last scene correctly, the audience won’t even realize that it was the sound that completely transformed how it played out. If you ask them after the movie about the scene they’ll say “The way they shot that scene was so scary!” And when you transport someone to a different space by layering dozens of sound FX and backgrounds, the same is true. You want all those tracks to blend together perfectly so that the the listener doesn’t think about the production process. You never want the listener to think, “Ah, that must have been a complicated Pro Tools session!”
So it’s an interesting job: you work hard to layer sound atop sound and if you get it just right, all those layers become invisible. The listener thinks, “That’s just what it sounded like” — if they think about the sound at all. They don’t think about the different microphones used to record the dialogue. They don’t think about all the edits that were made or the ambiences you added. They’re too focused on the story to notice. Of course there is fancy sound design that is very noticeable to the listener, but the majority of the work is about being unnoticed.
Rather than thinking of this as a bummer (“Ah, no one appreciates good sound design!”) I think of this as a sound designer’s secret weapon. We can alter the listeners’ experience without them realizing it.
We can transport them to another place. We can alter their emotions. We can manipulate them and they don’t even realize they’re being manipulated! That’s an extremely powerful tool if used effectively (and appropriately — I’m not using the word “manipulate” in a sinister sense).
Always Keep Perspective in Mind
In that last example, we designed the sound from Jill’s perspective. That was how she was experiencing the scene and the sound design was effective because it put the audience inside her experience. Even when you’re taking a more realistic approach to the sound design, choosing the perspective for each sound is super important. If it’s a film, when do you want to match the camera’s perspective and when do you want to deviate from it? From a story standpoint, what’s most important? What should we be focused on? What elements are actually going to distract us? All of these are artistic decisions. This is an underappreciated part of the sound design process as people rarely realize you can approach each scene and each sound in a million different ways. But when you have a clear approach to the perspective, a lot of the sound design decisions sort themselves out.
Once you’ve figured out the perspective from an artistic standpoint, you still have to execute it from a technical standpoint. Changing the perspective of a sound is often a multi-step process using EQ, volume rides, reverb, and all the other tools available for sound design. Using all of them for one little sound can feel tedious. And yes, it might not be that important to nail the perspective on any one sound, but when you add them all up across a project it’s the difference between it sounding real or fake. To give you a basic example, let’s take this car passby:
What’s our perspective? We’re basically in the street, looking straight ahead, and the car is passing a few feet in front of us — coming fast from our left to our right. But what if we want it to sound like it’s far away? Maybe 200 feet away, slightly to our left? One of the first things I would do is sum it to mono because as an object gets further and further away from us it’s going to hit both of our ears roughly the same way. Here’s what a mono summed version sounds like:
Already it feels farther away. A lot of people think “A stereo sound FX is better!” but in this case it’s just the opposite. A mono version makes it fit our perspective. Next I’ll take off some high end and a little bit of low end to mimic how sound waves change over distance:
Again, in general, taking the high end off of a sound makes it less rich. So you could argue it sounds worse. But we’re trying to get it to match the perspective we want, so in this case it makes the sound fit better. After that, I would do a little volume automation so it doesn’t get so loud in the middle of the passby. I’ll also pan it a bit to the left and then have it come to the center to give it a sense of movement:
Finally I’ll add a little bit of a reverb and delay to help give it even more sense of distance:
So it took doing these 4 or 5 steps (in the correct order) to make the sound match the perspective we wanted. It might seem like a subtle difference and a lot of work — if this was just one little car in the background of a movie. But I find this is exactly what I spend most of my time doing and it really makes all the difference. Each time a sound’s perspective is off, it subconsciously takes us out of the story and gives us the feeling that something is off and that this is fake.
That said, there’s a difference between how things actually sound and how a listener expects things to sound. With dialogue especially, it might be detrimental to become too focused on perspective. When I first started working on movies I would pan the dialogue from left to right as the character moved from left to right across the screen. This is generally frowned upon because it really only works for the audience sitting in the dead center of the theater. It’s also distracting! For that reason, dialogue almost always goes up the center channel. And when we watch a movie we’re quite used to hearing dialogue come through microphones. The placement and style of microphone often makes something sound much different than how it would sound if we were there. That’s fine! This way we can hear a whisper just as easily as a yell. So always keep perspective in mind, but let the story dictate how you take it into account. Don’t become too obsessed with it being “right.”
Your Environment vs. the Listeners’ Environment
You need to have good headphones and/or good speakers and a well-treated room. You don’t need to spend tens of thousands of dollars, but if you skimp on this you might not be able to hear the material properly. The goal of your playback system should be accuracy. You don’t want speakers or headphones that add tons of bass or that do other tricks to make the sound better. If you have a neutral playback system, and you make it sound good, chances are it will sound good everywhere.
In addition to making sure your system is up to snuff, it’s super important to take into consideration how the audience is going to be hearing the material. Beatles producer George Martin has said that when he was producing the Beatles’ records back in the 1960’s, he discovered that most stereo speakers were only 3 feet apart. From there he reasoned that to get any stereo effect from the mix, he had to hard-pan the different elements. In other words, he put the drums and bass just in the left speaker and the vocals and guitars just in the right speaker. On a 60’s stereo, these didn’t sound like aggressive mixing techniques but now hardly anyone does hard-pans like that when mixing. That’s because today the majority of music is listened to on earbuds or speakers with significant space between them.
I always try and keep this lesson in mind: it only matters how the audience will be hearing this — it doesn’t matter how great it sounds on your speakers in your room. Depending on the difference between your playback environment and the listener’s, portions of your work — things you spent days on — can completely disappear! If you’re mixing a podcast you need to keep in mind that the listener could be listening in a car, on the subway, or at the gym. So if the mix is too dynamic they could miss the quieter sections or be annoyed that they have to keep riding their volume. Obviously the listeners can be listening in all sorts of environments, on all kinds of speakers and headphones, but try and aim for how most people will hear it.
Also make sure your client and collaborators are listening to your work in a way similar to how you’re hearing it. Otherwise their notes could be completely meaningless. Them telling you to turn down the bass could be a reflection of their speakers having too much bass, or yours not having enough. This sounds basic but it’s really important to consider.
So What Does All of This Mean for Sound Designing Podcasts and Public Radio?
At the very least you should see bad sound as the easiest way for people to think of the show as “low budget” or “amateur,” even if the story-telling, writing and subject matter is amazing. But too much sound design might have the same effect as well. I’ve noticed a trend in nonfiction podcast to include more and more sound FX’s to help illustrate a story being told, and I often think this has the exact opposite effect. Don’t underestimate the listener’s ability to imagine things. If an interviewee is telling a story that starts with “I was walking down a dark alley . . .” we immediately picture it. Maybe we can see the fire escapes and dumpsters. Puddles reflecting the lights from the windows. But if all of a sudden we hear a canned footstep sound FX, all of that can disappear and we can see these two perfect shoes clip-clopping on perfect pavement. Maybe the listener wonders for a second, “Wait, do they have a recording from that night? Oh, no they’re just trying to create a reenactment . . .” The whole time they’re having that thought they’re not listening to the story and they’re not using their own imagination to create the scene in their head. So in nonfiction, I often think less is more.
Where sound FX’s can be super useful is in embellishing real tape. Here’s an example: maybe the piece you’re working on has a clip of a politician making a speech but the clip comes from a cable news show and the applause of the audience gets cut off. Let’s say you decide this clip should end a section of the piece and you want the applause to carry over to the next section. The solution might be to blend it with another recording of applause but to do that you need to find the perfect applause. You may need to treat it so that it matches the crowd size in the tape and also the sound quality of how it was recorded. If it sounds like you just dropped a canned sound FX on top of the real clip (with a bad crossfade) it could just be a distraction. Doing a lot of detailed work like this takes a long time, and often goes unnoticed, but to me this is the sound design that can really elevate a piece.
Narrative-fiction podcasts are a growing trend, and again it’s difficult to give hard and fast advice because the form and aesthetics of each project are so different. In my opinion, sound design needs to be baked in at the conception of the show (and considered throughout the writing) to be truly successful. Sound design cannot be an added-at-the-end afterthought, or else it will most likely sound like an old-timey radio drama with hokey sound FX. Whoever is creating the show needs to hear the finished project while they’re creating it. Not everyone has the ability to do that, which is why sound designers need to be main collaborators throughout every step of the process.
Get a field recorder:
You don’t have to get anything fancy but just get some experience recording your own sounds and building your own library. Even a Zoom is a good way to get started. As you get more into it, I suggest getting a good stereo-mic setup (with a good preamp). Sound Devices recorders all have great preamps in them. I love having an M/S (mid/side) setup: it’s compact, easy to use and gives you so much flexibility later on.
A good general rule is to splurge on the mics. Recording devices change pretty quickly. A few years from now a new device could come out that has better digital-to-audio conversion, and for half the price. But good mics will always be good mics. Also remember, if you’re going to be doing production recording for a film that’s synced to picture, you’ll need a recording device that has timecode. That increases the price significantly.
Get Pro Tools:
I won’t get into all the things that frustrate me about Pro Tools (there are a lot) but like it or not, it’s the industry standard for film and radio/podcast mixing. You can get by with other programs for a while (depending on what you’re working on) but at a certain point you’ll have to invest in Pro Tools. If you’re interested in doing sound design for films there’s really no question — you’ll need to get it.
Pro Tools HD is even more expensive, but it’s essential for film mixing (specifically the Advanced Automation and Surround Mixing features). Paying the extra cost of HD is a tough pill to swallow but I can’t imagine not having the advanced automation that it comes with. It saves you so much time and is essential for film work. So if this has become or is becoming your profession, I think it’s worth it.
Get iZotope’s RX:
This is the other essential tool that I can’t imagine living without. It’s the ultimate sound restoration toolkit. Think of it as PhotoShop for sound. Just as it would be hard to be an effective graphic designer without PhotoShop, RX helps you do things that would otherwise be impossible. That said, like any tool, it takes time to learn how to use it artistically. So if you do get RX, invest time in learning all the different modules and figuring out their tricks and secrets.