The Sound of Software
Repetitive. Disruptive. Annoying.
Sound is an outcast in Software Design. We may embrace the aesthetics of animation and visuals, but sound is different. It’s intrusive. Unlike visuals on a screen, you can’t look away or ignore it. It’s enough to make you rip the batteries out of a toy or frisbee an iPad across the room (speaking from experience).
And yet, play a video game without sound and its powerful punch lands with no force. Without music, once moving moments in a film become dull, even comical (Jurassic Park, Rocky). Sound holds an immense power to elevate any experience—including the most boring of software.
Sound in software isn’t inherently bad. It’s just been really badly designed.
We use sound in every !Boring app, and many have called it out as one of their favorite aspects of our apps. We’ve learned a few things about when to use sound, how to design it, and how to implement it. When done right, sound unlocks a path to much richer software experiences
———
If you haven’t worked with sound before, it may be difficult to know where to start. When should you use it? How do you create it? How do you shape it? How do you even talk about it? Let's get in it. <Headphones on>
--- 1 ---
When To Use Sound
Sound design starts with knowing when to use it and when not to. While I can’t tell you exactly where to add sound in your software, I can share a few guidelines on where sound adds value.
- Communicate something. Sound can be thought of as the voice of your software. It can be a gentle “mm” to acknowledge a common action or a shout to call attention to something important—success or error—particularly when it may happen away from one’s gaze. In games, this is common. Bump into enemy, hear a “hurt” sound. Collect an item, hear a “reward” sound.
- Juice a moment. In a previous essay, I wrote about adding juice (a.k.a “feel”) to enhance software. Just as you might use animation or action lines to emphasize a key moment, so too can you use sound. Need to celebrate an achievement? Add a sound. Warn against a wrong path? Sound. Enhance an otherwise boring action? Sound. !Habits uses the sound of a satisfying burst to celebrate the moment when you check off a task for the day.
- Create tangibility. Software design is something of a magic trick. That trick is to fool you into believing our software isn’t just a grid of pixels lighting up on a screen. It’s a tangible tray, a hovering popup, a blobby button. Sound, like drop shadows or overshoot animations, can add a sense of materiality to an interface. Think of the crumpling paper sound when you trash a file on macOS. The sound makes the action tangible and gives it more weight when in reality the computer is simply changing a few bits. In spatial design, sound is even more critical when other tactile senses like haptics are lost.
- Shape a feeling. Sound can play a key role in queueing us in on how we should feel in a moment—happy, reflective, alert. In the todo app Clear, successively checking things off a list plays a rising set of notes that builds a sense of accomplishment.
- Immerse you in a world. Sound can stretch software beyond the screen and immerse us in a deeper world. !Weather shows a large sun, but it’s ultimately just a tiny 1-inch circle on a screen that can slide into your pocket. It’s the sound of distant birds chirping that opens up the world to something that feels much larger and inhabited. Games like film often do this with rich music and diegetic sound effects.
--- 2 ---
Designing Sound
What makes good sound design? As with visual design, good sound design tends to resolve practical issues and enhance the aesthetic experience.
Break the repetition
What’s the main common reason most software sounds are annoying? You might blame it on repetition. And yet we can find some repetitive sounds like wind chimes or mechanical keyboards tolerable, even pleasing. What’s going on here?
The answer: Variation. On a physical button, minor variations in pressure, timing, & angle create subtly different sounds with each press. If you play the same exact waveform over and over again, it doesn’t matter how good it’s designed, it’ll get annoying really quick.
The trick here is to steal a technique from gaming and create multiple variations of the same sound. Create 8-12 variations by varying pitch, volume, timing, or mix and randomly play one variation for every new key press. Or take it further and dynamically adjust a sound (pitch, filters) based on different inputs like hit location, velocity, and hold duration.
Instead of repetition, you create texture. At !Boring, we do this on all of our buttons and it fundamentally changed how I thought of sound in software. !Calculator went from annoying keypresses to a delightful texture. If you’re a sound skeptic, trust me, try this one thing and it’ll blow your mind.
Layer, layer, layer
Sound designers often create new sounds or add dimensionality by layering multiple sounds. Think of it like bringing together ingredients when cooking. If your sound needs a build up, you might add a reverse whoosh. If it needs a satisfying moment of punctuation, you might add a clear percussive hit. If you want to lift the mood, you might sprinkle in a few musical tones. Combining all of your ingredients not only creates a sound that satisfies your needs, but it tends to create something that sounds entirely new.
Layers can be brought together from any sound source—natural recordings, synthesized, musical instruments. It’s in the combining and layering of sources together that you craft what you want your sound to do.
Build a world
Like a visual or brand system, your soundscape is experienced as a whole where each element speaks to one another. Consider how your sounds live together within a coherent world. Does it sound physical, digital, musical? Experiment with different “worlds” of sound early on to find what aligns with what you want to say. Surprise yourself. Try something unexpected.
Create relationships
As with visual design, sounds for actions should never be considered in isolation but in how they relate to one another.
Similar actions like navigation controls or object actions should have sounds that live within the same auditory family and share characteristics like tone, instrumentation, and effects.
Opposing actions—open vs close, prev vs next, send vs receive—shouldn’t use the same sound. They should sound like similar opposites. A sound might be played in reverse or the emphasis may be moved from the beginning to the end. In this way, the sound intuitively reinforces the directionality of the action you perform.
You can hear these relationships when scrolling up vs down in !Timer, going in and out of detail view in !Weather, and opening and closing the overlay menu in !Vibes.
Listen for inspiration everywhere
Sound within the software world can feel like an echo chamber where most sounds follow what’s been done before—beeps, clicks, pops. If you want to sound like something new, go beyond software. Movies, games, and music all use sound in very sophisticated ways that can be co-opted for software. In our app !Vibes, we combined sounds from ambient music with nature.
Listen to the world around you—the twang of a ruler across a radiator or the thrum of the subway train as it resonates through the tunnel at a specific pitch. Becoming a visual designer begins with learning how to see the world. Sound design begins with learning how to listen to it.
Things don’t have to sound like the thing
It’s common to think something should sound like what it is. A camera shutter like a real shutter and a page flip like a real page flip. But sound doesn’t always need to be this literal. What if it sounded very different? The sound for a page flip could share the characteristics of the sound of a real page flip with a long, slow drag but you could use an entirely different tone or instrument to create it. If meeting expectations enables your design to disappear, these moments where you break expectations are what shape the character and personality of your software.
In !Vibes, animals populate the worlds, and if you tap them, will “speak” to you. But rather than use real world sounds of a hare or deer, we created a musical sound with a unique tone and sequence for each animal. Instantly, you recenter your understanding of this world you’re in as not a realistic world but a highly musical one which aligns with the core purpose of the app.
--- 3 ---
Implementing Sound
50-60% of phones are in some form of silent mode where sound won’t play. Are you doomed? Not quite. You can override the device sound setting so that sound comes through even when a device is muted (as is done with videos). Be careful with this one, but it works if you combine it with offering better controls.
Give back control.
Give people control of the sound in your app or experience. Sound is intrusive, so let people decide how and when it intrudes their space. Give them an option to toggle off all sounds, or just the background music, or only key alerts. Let them set when sounds play or set volume levels. In !Timer, we let you set the alarm volume. Useful if you typically keep your volume low but don’t want to miss a critical alarm.
Sound settings for !Timer
Pair with haptics.
Sound and haptics are close cousins and should almost always be paired together. Haptics shape our perceptions of a sound. Think of a keyboard key stroke or swinging a hammer. Sound can make the same action feel soft or precise, clean or jumbled, heavy or light.
Haptics can also be a substitute for sound in moments where it may be too much or just impractical. Good haptics will level up your app while being less obtrusive and less limited by user settings or situations.
Haptics for the !Habits checkmark
Design for your target device
Legendary music producer, Rick Rubin, mastered early Beastie Boys records on his cheap car stereo rather than studio speakers because he knew that’s where the music would be played. Don’t make the mistake of designing sounds exclusively inside your studio headphones. Make sure you listen to your sounds the way others will—phone speakers, earbuds, in a noisy environment. You don’t even need to code it—just play the sound file over a video.
Spatial audio
If you’re really going for immersion, many platforms now support spatial audio. Spatial audio places sounds in a 3D volume and when combined with head tracking can help sounds feel more anchored within a physical space. This usually requires special earbuds or a VR headset to work and is probably overkill for most software. However, in the right instances it'll add a new dimension of immersion.
We use spatial audio in !Vibes to create ultra-present focus spaces where sounds appear randomly around you. Using an interface element from gaming, we use a minimap to display the location and movement of each generated sound.
Spatial audio and the minimap in !Vibes
--- 4 ---
How to Start
If I was just getting started with sound today, this is how I’d start.
Get sounds
Designing sound begins with getting your hands on those sweet waveforms. You can record your own with a simple mic + recorder or pick up existing samples from sound libraries.
To help, we've packaged up a collection of our own sounds we use in our !Boring apps that you can play around with to get started. !Boring Sound Kit
Edit sounds
There are many audio editing tools and none are meaningfully better for this type of audio work, so don’t sweat it. Just grab Logic, Garageband, Adobe Premiere, Quicktime, or whatever you have and get started.
Combine sound + visuals
Unfortunately, most software design tools don’t support audio on the canvas (Rive is one that does). Regardless, one simple technique you can use when getting started is to record a screen capture of your interface or prototype and compose sounds over the video. We still do this all the time as it’s often the fastest way to sketch out early ideas.
Hire a Sound Designer
Playing with sound on your own will get you far. But if you’re serious about crafting great sound & music for your app, I strongly suggest working with an experienced sound designer or composer. I may be mildly musically-inclined, but I’m no sound designer. I’ve never once regretting working with a sound designer. Their work will instantly elevate whatever it is that you build to a new level. If you’re unsure where to find them, a great place to look is in the credits of your favorite video games.
———
I hope this gives you a better understanding and a few tools to start playing with sound in your own app. Sound has been ostracized for too long because most of us never understood where to use and how to shape it. But just like the typefaces we set, the screens we compose, and the animations we keyframe, sound must be designed. As more of our lives are spent in software, we can ignore sound and live in lifeless software or we can design it with intention and embed soul into more moments of our lives.
Let's make the world sound better.
— Andy & Thomas Williams
———
Resources
!Boring Sound Kit : Sound kit for playing & prototyping
Meta Sound Kit : Sound kit from Meta
Designing Sound : WWDC '17 Video
Freesound.org : Free sounds
Vibes, Weather, Habits, Calculator, Timer : Listen to the sounds in the !Boring apps
———
"Serious Play" is an essay series on how we at !Boring steal from video games to level up software design.
Part I : Serious Play
Part II : The Future of Design Tools
Part III : The World's Most Satisfying Checkbox
Part IV : The Sound of Software