Check out this guide for using a Monte Carlo simulation to identify the size and confidence percentage of your sampled success metric.

VCR is a Ruby library that records HTTP interactions and plays them back to your test suite, verifying input and returning predictable output.

In Ruby apps it's most frequently used as a testing tool, but having it in your toolbox provides you with a rich set of organizational and debugging tools, even if you choose not to use its popular “automocking” feature.

In 2020, we announced that React Native is the future of mobile at Shopify and since then we’ve been migrating all our native mobile apps to React Native. Since each app is different, there is no single approach that works for all of them. So, we evaluated all the possible options for each app and chose the ones that best suit their needs.

Shopify Point of Sale, for instance, had come a long way since it was first built during an internal hackathon. It was originally designed and built to support small mom-and-pop stores or weekend warriors. However, it has surged in popularity and is being used by some of our biggest merchants and is processing transactions worth billions of dollars each year. The codebase had accumulated a lot of tech debt and the app’s UX was also not serving the needs of large merchants who have hundreds of locations and tens of thousands of products. After a thorough evaluation, it became clear that we couldn’t fix these issues with incremental changes. Hence, we decided to do a full rewrite, which has been a big hit with our merchants.

Shopify Mobile, our flagship mobile app, on the other hand is quite stable and meets our merchants’ needs. It is also our largest app at 300 screens per platform and took over six years to build. Rebuilding it from scratch would be a massive undertaking. Even if we assume that we’d be twice as productive with RN (which is not necessarily the case always), it would take us at least three years to rebuild. That’s a very long time. We would have to halt all new feature development during this time and in the end have the exact same app as we started with. A rewrite then, was clearly out of question.

Shopify is improving CRuby’s performance in Ruby 3.2 by optimizing the memory layout in the garbage collector through the Variable Width Allocation project.

GitHub personal access tokens (PATs) are like a key: a very, very large key that opens a very, very wide door. Long-lived tokens that have all the access of a developer’s account won’t just cause a leak—it’ll be a flood. GitHub’s built-in token is useful, but has limitations of its own: it can’t access repo-external resources and it won’t trigger downstream actions (by design). Given the limitations with these two blessed authentication paths, what do you do when these methods don’t work for your use case? We encountered this problem in some of our workflows, and solved it by building a system to rotate tokens automatically. Here’s how we did it, and how you can use it too.

Earlier this year, we shared our tips for optimizing large stateful Apache Flink applications. Below we’ll walk you through 3 more best practices.

What do you do with a finite amount of time to deal with an infinite number of things that can go wrong?

We prepare for Black Friday Cyber Monday (BFCM) and other high-traffic events during the year to make sure the Shopify platform stays up so our merchants can sell to their buyers. To do this, we built an infrastructure platform at a large scale that is highly complex, interconnected, globally distributed, requiring thoughtful technology investments from a network of teams. We’re changing how the internet works, where no single person can oversee the full design and detail at our scale.

This year over BFCM, we served 75.98M requests per minute to our commerce platform at peak. That’s 1.27M requests per second. Working at this massive scale in a complex and interdependent system, it would be impossible to identify and mitigate every possible risk. This post breaks down a high-level risk mitigation process into four questions that can be applied to nearly any scenario in order to help you make the best use of your time and resources available.

We walk through how we implemented an SSE server that's scalable and load-balanced to simplify and improve a real-time data visualization application.

Performance is crucial for Shopify. After we started using React Native, we had to find a way to confirm our mobile apps are fast. The solution is an open-source @shopify/react-native-performance library by Shopify, which measures rendering times in React Native apps. In this article, you’ll learn more about how the library works, how to get started, and why measuring performance is so important.

Ashwin explains why and how we implemented server-driven UI in the Shop App’s Store Screen, and his experience working on the project as a Dev Degree intern.

Shopify wanted a cache format that would not blow up when we shipped code changes. Part two of Caching Without Marshal describes the MessagePack format, and how we migrated to MessagePack.

Caching is critical to how Rails applications work. At every layer, whether it be in page rendering, database querying, or external data retrieval, the cache is what ensures that no single bottleneck brings down an entire application.

But caching has a dirty secret, and that secret’s name is Marshal.

Marshal is Ruby’s ultimate sharp knife, able to transform almost any object into a binary blob and back. This makes it a natural match for the diverse needs of a cache, particularly the cache of a complex web framework like Rails. From actions, to pages, to partials, to queries—you name it, if Rails is touching it, Marshal is probably caching it.

Marshal’s magic, however, comes with risks.

Currently Shopify is going through the process of updating from the Apollo GraphQL client 2 to client 3. The Apollo client is a library used to query your GraphQL services from the frontend, it has a feature for caching objects/queries you’ve already made which will be the focus of this post. Through the process of migrating the Apollo client we started to discuss bugs we've run into in the past, for which a common thread was misuse, or more likely a misunderstanding of the cache. This was the catalyst for me diving further into the cache and exploring how it fetches, transforms, and stores data that we previously queried. Having worked with the Apollo client for the vast majority of my career, I still couldn’t say I understood exactly what was happening in the cache internally. So I felt compelled to find out. In this post, I’ll focus on the Apollo client cache and the life cycle of objects that are cached within it.

During the infrastructural exploration of a pipeline my team was building, we discovered a query that could have cost us nearly $1 million USD a month in BigQuery. Below, we’ll detail how we reduced this and share our tips for lowering costs in BigQuery.

There are 3 management poles for developer infrastructure teams that are thus tricky to balance: management support, system and domain expertise, and road maps.

We share the framework that’s worked best for structuring our data teams and that's empowered them to influence business decisions and drive impact.