Why we built a custom Rust library for Capture

// By the Dropbox Capture Team • Sep 28, 2021

Dropbox Capture is a new visual communication tool designed to make it easy for teams to asynchronously share their work using screen recordings, video messages, screenshots, or GIFs. There's no formal onboarding required, and you can start sharing your ideas in seconds. In fact, simplicity is key to the Capture experience, and it's a value that also extends down to the development of Capture’s underlying code. 

One of our team’s guiding principles is “be a Margherita pizza.” Just as a Margherita pizza is perfect in its simplicity—what more do you need than tomato sauce, mozzarella, and basil?—we’ve tried to keep Capture’s ingredients as simple and straightforward as possible. We knew early on that Electron and Node would make it easy to build a cross-platform TypeScript app for both macOS and Windows. But finding the right third ingredient that would enable us to quickly, simply, and reliably call native OS-level code took a bit more experimentation.  

Ideally, we wanted streamlined a codebase that could target multiple platforms painlessly, consistently, and that was easy for our developers to build. We also wanted more control over our ability to take screen captures and recordings, better error handling, and faster performance behind the scenes. In fact, we were looking for something that would give us more control at every layer—that didn’t require us to jump through quite so many hoops to call native code—and would better support the new features we wanted to build.

There were a lot of ways we could have solved these problems—perhaps more TypeScript, or C++—but in the end we decided to go with Rust.

In some respects, it was an easy decision. Dropbox has a thriving community of developers building Rust into our products. Rust is at the heart of our desktop client’s recently re-written sync engine, which is what makes the Dropbox folder on your computer work like magic. We also use it for file compression, in our crash reporting infrastructure, and in Magic Pocket—our exabyte scale custom storage infrastructure—to optimize the storage of file data

It turned out that Rust was perfect for our needs, too. Building a custom Rust library helped unlock higher-quality screen recording from 720p through 4K, made screenshots and screen recordings available to share more quickly, and dramatically improved our error handling capabilities, allowing us to offer a more reliable experience for our users. 

We also thought it would be a fun excuse to learn the most-loved programming language of recent years.
 

From Hack Week to here

Capture began as an internal Hack Week project where rapid iteration was key. In early versions, we used a handful of third-party libraries to do things like take screenshots and process GIFs. Cobbling together bits of preexisting code helped us quickly develop a prototype, test out our initial assumptions, and experiment with new features. But when we considered the long-term health of Capture’s codebase—and all the complexity these third party libraries introduced—we knew we would eventually have to pay our early technical debt down.

The third-party libraries we used were usually shell-based applications; Capture would send commands to each shell app and receive stderr and/or stdout responses in return. This meant spinning up an application each time we wanted to complete certain tasks, or in some cases having an application running continuously and awaiting input—not exactly ideal.

More importantly, it also meant there was some inherent brittleness in the way Capture communicated with native code. Each line of output from the shell application had to be parsed. If a line failed to parse, we assumed it was an error, and if it was an error, the issue was likely masked and we wouldn’t know exactly what broke in the native code. As you might expect, this made monitoring and handling errors difficult! 

From a developer standpoint, the libraries posed other challenges. We found APIs could be quite different between macOS and Windows—even within the same cross-platform library—which added complexity when developing for the two platforms. And while some libraries were well maintained but missing features that we needed, other libraries had everything we wanted but were not as well maintained. Each presented tradeoffs we had to work around, some more easily than others. 

For example, if we wanted to make any changes to the individual libraries we’d have to have the institutional knowledge of how to build each one and then build them into Capture. Case in point: It took one of our engineers hours of valuable development time to learn how to build the Windows screen recording library just to fix a single parsing bug!
 

Rust to the rescue

Because Rust was new to the Capture team, our early efforts were extremely incremental. Initially we focused on re-writing simple functions that otherwise required a third-party library. For example, activate-windows was previously a macOS-only library that let us bring a window to the forefront and only record that window. We were quickly able to port the feature to Rust on macOS, and then bring the feature to Windows where it didn’t previously exist. 

These early successes gave us the confidence to try more ambitious things. The more we learned, the more features we moved to our custom Rust library, which benefitted Capture in a handful of ways: 

No overhead. With Neon-bindings we could now easily make calls to native OS code from TypeScript without any overhead (and more reliably, too). In other words, we no longer had to spin up separate shell applications to complete certain tasks. Taking screenshots, for example, which was once asynchronous—requiring us to wait for a response from the shell application—was now immediate and fast.

Better error handling. Rust also dramatically improved our ability to handle errors. Once most of Capture’s code was running within our own library, and with a consistent API across both macOS and Windows, we were able to add more robust logging and monitoring. No more trying to interpret the output from shell apps! Having all of our code in one place gave us more insight into how our app actually was actually behaving.

More control. Ownership of the library meant fixes and enhancements could be made more quickly. Having all our code in one place also made it easier to tackle more nebulous issues—like the instability we kept encountering when taking captures or recordings with more than three screens. It resulted in a simpler build pipeline across platforms, too. 

A smaller footprint. Not having to include third-party libraries also reduced the overall size of our app. Around 17MB of Swift libraries were no longer needed on macOS, for example, after re-writing those capabilities in Rust. And now that we could simply call functions as needed—instead of having shell applications running in the background at all times—we also needed less memory than before. 

New features. As we found early on with activate-windows, moving to Rust also allowed us to do things we just couldn’t do before. We were able to bring functionality to Windows that previously only existed on macOS. We were also able to introduce a new crop tool, new recording controls, and add new recording types like audio or camera only, as well as increase recording quality to 720p/1080p/4K. It’s not so much that we couldn’t have built these things with another language, but rather, Rust allowed us to build them faster and with less effort than before.
 

What’s next for Capture and Rust

Capture is available now in Beta, and if you haven’t already you should try it out.

One of our biggest surprises developing Capture was how easy it was to get started with Rust. In just a few weeks we were pushing Rust code to production, and we benefitted greatly from Dropbox’s supportive community of Rust-savvy engineers who offered help and guidance whenever we got stuck. And as our team grows, there’ll be less of a learning curve for new developers. Instead of wrestling with multiple libraries the way we used to, now we have a really simple document that basically says “here’s how to build everything using this single command.”

With Rust, our developers know exactly what to expect—just like our beloved Margherita pizza.

In time, we expect to move more features to our in-house library. The macOS screen recorder has already been re-written in Rust, and a similar re-write of the Windows recorder is on the way. We’ve been moving features like GIF creation and other OS-level integrations into our Rust library, too. And we’re especially excited for the future of Rust on Windows, which just recently reached v0.21.0.

But it’s also not an all-or-nothing approach. We’ve configured Rust so that we can still call third-party libraries using the old shell process approach if needed. This means we can be intentional about which features we choose to re-write and when. Of course, if we had struggled with Rust it would have been easy to turn back. But our enthusiasm and excitement turned out to be justified given the benefits Rust has unlocked. 
 

We’re hiring!

Do you love Rust? Do you want to grow as an engineer? Dropbox is hiring

Many of our teams are solving problems big and small with Rust and other new technologies—and Capture is no different. We're always on the lookout for clever, curious front-end engineers who want to learn new things, take on bigger challenges, and build products our customers love. If you're a front-end engineer with a talent, passion, and enthusiasm for Rust, we'd love to have you at Dropbox. Visit our careers page to apply.

Special thanks to the Capture team (Youcef Es-skouri, Noga Raviv, Joey Diab, Kyle Shay, Andy Liu, Will Hall, Alex Pelan, Alan Chu, Mike Boht, Karan Khanna, Lien Chung, and Lily Lee) as well as Parker Timmerman and the rest of Dropbox’s internal Rust developer community. 


// Copy link