Campaign Budgets at Pinterest

Kelvin Jiang |Software Engineer; Keshava Subramanya | Engineering Manager; Yeming Shi | Software Engineer, Ads Intelligence

Introduction

Pinterest is a visual discovery engine that helps Pinners find inspirational ideas. Advertisers use Pinterest to connect with Pinners on these journeys to inspiration, and seek to promote products or services efficiently.

The Ads Intelligence team at Pinterest builds products that help advertisers maximize the value they get out of their ad campaigns. As part of that initiative, we have recently launched the Campaign Budget Optimization product for Pinterest Ads.

Campaign Budget Optimization, or CBO, is an automated ads product that benefits advertisers by distributing the advertising budget for each campaign across the underlying ad groups in an automated manner. The goal of Campaign Budget Optimization is to:

• Maximize advertiser value, for example driving clicks or conversions, depending on the campaign
• Improve the budget utilization of the campaign by allowing the budget to be shared across ad groups
• Simplify the advertiser experience and eliminate the need for manual budget adjustments

Problem Statement & Motivations

Advertisers create campaigns with a number of ad groups, and set budget amounts for each ad group separately. Budget allocation is a non-trivial task, because advertisers usually have little prior insight into each ad group’s scale and performance, and tuning ad group budgets can be a manual and repetitive process. CBO seeks to address this problem by moving budgets to the campaign level, and automatically distributing this budget to ad groups in real-time.

The problem that CBO addresses can be formulated as a constrained optimization problem: maximizing advertiser value for a campaign by optimally distributing budgets to ad groups, while using a fixed amount of total budget. Advertiser value is a concept that varies depending on the campaign objective. Clearly, not all actions are equal (e.g. a checkout action is more valuable than an add-to-cart action), and the solution should offer a way to represent these differences.

Campaign objectives vary, and may be focused on creating awareness, encouraging Pinners to consider a purchase, or encouraging Pinners to complete a purchase (or other forms of conversion). Creating awareness or consideration of a purchase is considered an “upper” funnel, while motivating an actual purchase is considered a “lower” funnel. Each of these will have different values to an advertiser.

For the upper funnel Awareness or Consideration objectives, CBO seeks to maximize the volume of resulting actions, such as the total number of impressions or clicks. For the lower funnel Conversion objective, CBO also potentially accounts for the conversion value (if available and provided by the advertiser) associated with each conversion action, and seeks to maximize the overall conversion value for the campaign.

Our Approach

The general idea behind CBO is to incrementally allocate each dollar of the overall campaign budget to the ad group with the highest marginal return. Once every dollar of budget has been allocated, we will have reached an optimal, equilibrium state where the marginal return for every ad group is equivalent.

Automatic bidding is a product for Pinterest Ads that automatically adjusts the bids of each ad group to help get results at the lowest possible cost per result while fully utilizing the budget. Generally speaking, increasing the bid of an ad group increases its ability to spend, but reduces its efficiency and return on ad spend (ROAS) since fewer results (e.g., clicks or conversions) are expected as we spend more due to the effect of diminishing returns. The intuition behind this effect is that the target audience for each ad is finite, and as spend increases the cost of attracting the next customer also increases.

In practice, the CBO algorithm automatically and continuously allocates the campaign budget to ad groups by manipulating the bids of each ad group in order to move toward the aforementioned optimal state in which every ad group is equally efficient, or has the same cost. This can be illustrated by the following example, for a Consideration campaign that seeks to maximize clicks. This example shows two ad groups using automatic bidding:

In the first iteration of the CBO algorithm, the next incremental amount of campaign budget is allocated to the orange ad group (ad group 2) because it has lower cost per click (CPC). This is done by increasing the bid for the orange ad group, which as a result increases its CPC to $2.50. In the second iteration, the next incremental amount of campaign budget is also allocated to the orange ad group, after which an equilibrium is reached where both ad groups have equivalent cost, or CPC of $3.00. At this point, any remaining campaign budget will be allocated equally across both ad groups by increasing the bids in an equivalent amount.

By adjusting the bids of each ad group such that the efficiency and cost are equivalent across all ad groups, CBO is able to optimize overall campaign efficiency, as can be seen in the performance of this real-life Awareness campaign before and after switching onto CBO:

In essence, the CBO algorithm performs real-time budget allocation by using the real-time control systems that dictate when each ad group should spend (pacing), and how much to spend (automatic bidding).

Infrastructure

The following diagram describes the main component of the CBO infrastructure, which is composed of a number of stream processing services using Apache Kafka:

To control the bids across the ad groups of each campaign, new campaign-level state stores were created to store campaign-level pacing and bid values. To productize the system in a scalable way, the pacing and automatic bidding services were repartitioned by campaign so that ad groups for the same campaign will be processed by the same machine.

Experimentation

We built and tested the CBO product for each objective, conducting controlled A/B experiments for multiple months (Alpha phase) followed by a multiple month open Beta phase in which advertisers can choose to opt-in to CBO.

Going into the Alpha and Beta experimentation phases, we formulated a number of hypotheses and used the experiment results for validation. Generally, the hypotheses can be divided into two groups — for budget constrained campaigns in which the budget was (nearly) fully utilized prior to adoption of CBO, and for budget unconstrained campaigns.

We formulated the following hypotheses, and were able to validate the hypotheses for each objective using the results from the experimentation phases:

Product Rollout

Following the Beta phase of each objective, we migrated all campaigns that had yet to adopt campaign budgets onto CBO. This effort involved inferring new budget values, start dates, and end dates for campaigns based on the ad group settings in order to closely approximate advertiser intentions prior to using campaign budgets.

For the migration process, we also conducted in-depth analysis on the impact by comparing campaign performance before and after the migration. We observed changes in campaign performance that were consistent, on average, with the hypothesis matrix validated during the experimentation phases.

For example, after the migration of the Consideration objective, we observed that the distribution of revenue between the ad groups within each campaign shifted towards the ad groups with greater efficiency, or lower cost (CPC):

The migrations for various campaign objectives have been big wins for the Pinterest sales teams and advertisers alike, as there are now significantly fewer (about half) budgets to manage, maintain, and tune, and the focus of these teams can be shifted to allocation of budgets across campaigns or other goals altogether.

Future Work

This roll out of the CBO product is just the first step in improving advertiser value in the budget optimization space. In the future, we plan to continue to improve the CBO algorithm by exploring the use of exchange ratios to differentially control the bids used for each ad group, and differentially applying pacing strategies between ad groups to control spend, and other means of further improving overall campaign efficiency,

If this type of work sounds interesting to you, drop us a line and check out one of our job postings for a Machine Learning Engineer.

To learn more about engineering at Pinterest, check out the rest of our Engineering Blog, and visit our Pinterest Labs site. To view and apply to open opportunities, visit our Careers page.

Acknowledgements

We would like to thank the following cross-functional team (not an exhaustive list) for contributions -

Yeming Shi, Tianyuan Cui, Ruixin Qiang, Kelvin Jiang, Flavio Bossolan, Keshava Subramanya, Sungro Yoon, Yu Hao

Michael Lam, Brittany Herr, Grace Si, Randy Perez, Manuel Lara, Frannie Huang, Allan Rezende, Ravi Dharawat, Jayanth Mettu, Kate Liu, Dani Gnibus

Niket Arora, Roman Chen

Sreshta Vijayaraghavan, Shawn Nguyen, Aniket Ketkar

Zeyu Wang

Sameer Bhide, Edward Zhou, Haoyu He, Shu Zhang, Alonso Garcia

Alex Moon, Malav Shah

Simeon Duong

Huiqing Zhou

Ashim Datta, Daniel Karwoski

Peter Allen, Jay Zuniga, Janani Vijaykumar

Julie Piskin

Hazem Younes, Rita Kamil, Dovik Nissim

Katie Keitges, Patrick Dillon-Hughes

Fidel Calderon, Amy Compeau, Kaitlin O’Donnell, Greg Lindley

Kelly Anneken

Kimmie O’Callaghan

Perrye Ogunwole, Mark Otuteye

Mao Ye, Roelof van Zwol

Dan Xie, Alex Rhee, Andrei Curelea, Yiwei Sun, Jeff Wu