Migrating Our iOS Build System from Buck to Bazel
How Airbnb achieved a smooth and transparent migration from Buck to Bazel on iOS, with minimal interference to developer workflows
By: Qing Yang, Andy Bartholomew
At Airbnb, we are committed to providing the best experience for our engineers. To offer a cohesive and efficient build experience across all platforms, we’ve decided to adopt Bazel as our build system. Bazel is a robust build system widely utilized in the industry. In alignment with Airbnb’s tech initiatives, both our backend and frontend teams initiated the migration process to Bazel. In the first Bazel post, we start with our iOS development migrating from Buck to Bazel.
We’ll describe the migration approach which involved two main pieces of work: migrating the build configuration and migrating the IDE integration. Such a transition can potentially disrupt engineers’ workflows or hinder the development of new features, but we were able to successfully migrate them without disrupting the day-to-day developer experience. Our aim is to help others who are currently undergoing or planning a similar migration.
Migrating the Build Configuration
When it comes to build configuration, Buck and Bazel exhibit significant similarities. They share a comparable directory structure, employ similar command line invocation, and, importantly, both utilize the Starlark language. These similarities present an opportunity for configuration sharing between the two build systems. This would allow us to reuse our Buck configurations in Bazel, while avoiding slowdowns during the “overlap” phase when we were in the process of migrating and still actively using both build systems.
Unfortunately, there’s a major problem: Buck and Bazel employ distinct rules with different parameters. For instance, Buck offers rules such as apple_library and apple_binary, whereas Bazel, depending on the external rule sets, features rules like swift_library and apple_framework. Even in cases where the two systems have rules with the same name, such as genrule, the syntax for configuring those rules is often unalike. The different design philosophies of these two systems result in various incompatibilities as well. For instance, Bazel doesn’t have the read_config function to read command line options in a macro.
Hiding the Differences with rules_shim
After conducting an in-depth analysis of both Buck and Bazel, we devised a comprehensive architecture for the build configuration to leverage the similarities and address the differences between each system.
At the core of this architecture lies the rules_shim layer, which introduces two sets of rules: one for Buck and another for Bazel. These rule sets act as wrappers around the native and external rules, offering unified interfaces to the layers above.
How does rules_shim work, exactly? By making use of local repositories, we can point the rules_shim repository to different implementations depending on the build system.
This is what the result looks like in Buck’s .buckconfig:
[repositories]
rules_shim = rules_shim/buck
[buildfile]
name = BUILDNote that we’ve also configured Buck to use BUILD as the config file, and renamed the existing BUCK files to BUILD, so the same configuration can be recognized by both Buck and Bazel.
In Bazel’s WORKSPACE, we do the following:
local_repository(
name = "rules_shim",
path = "rules_shim/bazel"
)In a regular BUILD file, we use my_library to wrap around the native rules and provide the same interface for each application:
load("@rules_shim//:defs.bzl", "my_library", …)The app-specific rules layer only needs to know the interface, not the implementation. As a result, whenever we execute Buck or Bazel commands, the build system is able to retrieve the corresponding implementation from the rules_shim layer. A notable advantage of this design is that we can easily remove the rules_shim/buck after the migration.
Unifying the genrule interface
Within our iOS codebase, we heavily rely on generated code to manage boilerplate and reduce the maintenance burden for engineers. Given the different syntax for genrule scripts between the two build systems, we also designed a unified interface for genrule. As a result, the same genrule script can function across both build systems. As you may have guessed, the conversion process is implemented in the rules_shim layer.
Replacing read_config with select
Conditional configuration is unavoidable, because there are always different variants of a built product, such as debug builds and release builds. Buck provides a function called read_config that reads command line options in a macro, while Bazel doesn’t have this due to the system’s strict separation of loading phase. It’s worth noting that Buck does support the select function, although it’s undocumented. We have migrated all instances of read_config to select-based conditions.
deps = select({
"//:DebugBuild": non_production_deps,
"//:ReleaseBuild": [],
# SELECT_DEFAULT is defined in rules_shim to accommodate
# the different default strings used by Buck and Bazel
SELECT_DEFAULT: non_production_deps,
}),Overall, this design achieved the utilization of a single build configuration for both build systems, with minimal changes to our BUILD files themselves. In practice, iOS engineers at Airbnb rarely need to manually modify BUILD files, which are automatically updated from an analysis of the underlying source code. However, in cases where it does occur, they can rely on the unified interface without needing to be aware of the specific underlying build system.
Migrating the IDE Integration
iOS Engineers at Airbnb primarily interact with the build system through Xcode. Since first adopting Buck, we have been utilizing Buck-generated Xcode workspaces for local development. Over the years, we’ve developed various productivity-boosting features on top of this setup, including the Dev App, a small development application focused on a single module; Buck Local, which uses Buck instead of Xcode for building and leverages remote cache; and Focus Xcode workspace, which significantly improves IDE performance by loading only the modules being worked on.
In the Bazel ecosystem, multiple solutions exist for generating Xcode workspaces. However, at the time of our evaluation, none of them fully met our requirements. Additionally, any IDE integration needs to support not only building, but also editing, indexing, testing, and debugging. Given the proven track record and stability of our current workspace setup, we deemed the risk of adopting a completely new one to be exceedingly high. Hence, we decided to develop our own generator to create a workspace close to our existing setup. We chose XcodeGen, a popular tool in this area, because it generates Xcode projects from a YAML configuration, serving as an abstraction layer to separate the build system implementation details.
We implemented this migration process in three phases.
Firstly, we utilized buck query to gather all the necessary information from the codebase and generate an Xcode workspace, replacing the buck project command. This new workspace invoked buck build during the build process. By keeping the build system unchanged, we were able to ensure compatibility and evaluate the performance of the new generator.
Secondly, we performed a parallel implementation in Bazel using bazel query and bazel build, incorporating a simple --bazel option in the generation script that enables switching between the two build systems within Xcode. Apart from the build system, the user interface remained identical, ensuring that all IDE operations continued to function as before.
Lastly, after a sufficient number of users opted for Bazel and all Bazel-powered features underwent extensive testing, we made the --bazel option the default, finishing for a smooth transition to Bazel. Although we didn’t need to, we could easily roll back if issues had occurred. A few weeks later, we removed Buck support from the generated project.
The end result of this migration is impressive. Compared to the Buck-generated project(buck project), the generation time with XcodeGen has been reduced by 60%, and the open time for Xcode has decreased by more than 70%. As a result, this new workspace setup received top rankings in an internal developer experience survey, showcasing the significant improvements achieved through this process.
Completing the Migration and Looking Forward
“All problems in computer science can be solved by another level of indirection.” — David Wheeler
Wherever we relied on Buck, we introduced a common interface abstraction and injected separate implementations to handle the differences between Buck and Bazel. Thanks to the “indirection” principle, we were able to test and update each implementation without dramatically rewriting the code, and we successfully transitioned from Buck to Bazel seamlessly across all use cases, including local development, CI testing, and releases. The migration process was executed without disrupting engineers’ workflows and, in fact, allowed us to deliver multiple new features, including SwiftUI Previews support.
Since Bazel became our iOS build system, we have observed notable improvements in build times, particularly for incremental builds. This shift has enabled us to leverage shared infrastructure, such as remote cache, alongside other build platforms within Airbnb. Consequently, we have fostered increased collaboration across platforms.
Migrating our iOS build system is just the first of a number of Bazel migrations underway or completed at Airbnb. We have repos for JVM-based languages (Java/Kotlin/Scala), for JavaScript, and for Go, which are either using Bazel already, or will be in the future. We believe a single build tool across our entire codebase will allow us to more effectively leverage our investments in build tooling and training. In the future, we’ll be sharing lessons learned from these other Bazel migrations.
Qing Yang, the technical lead for the iOS Bazel migration, designed and implemented the configuration architecture and the new project generator. Andy Bartholomew led the migration of all tests and implemented the script abstraction layer. Xianwen Chen migrated and managed the release builds of Bazel.
We are immensely grateful for the invaluable support received from numerous Bazel experts at Airbnb. Special thanks to Janusz Kudelka for providing valuable advice and guidance on the subject.
We also extend our appreciation to the Bazel iOS community for their various open source projects and the assistance they provided throughout our migration journey.
If you are interested in joining us on our quest to make the best iOS app in the App Store, please see our careers page for open iOS and Developer Platform roles.
All product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.
