Journey Platform: A low-code tool for creating interactive user workflows

Journey Platform: Low-code notification workflow platform that allows technical and non-technical users to create complex workflows through a simple drag and drop user interface.

By: Arjun Raman, Ken Snyder, Mengting Li

Introduction

Effective communication hinges on delivering the right message, to the right audience, at the right time. At Airbnb, our goal is to engage our users — both guests and hosts — by delivering inspirational and informational notifications through various channels, such as email or in-app messages.

Historically at Airbnb, complex notification workflows have been solely managed by engineering teams, with each workflow requiring the deployment of code. As our platform evolved, we recognized the need for a low-code or no-code solution to streamline the creation of these intricate notification workflows. In response, the Marketing Technology team developed the Journey Platform, a powerful tool that enables non-technical users to build and deliver personalized notifications based on our users’ engagement with Airbnb.

The goals of the Journey Platform are:

1. Empower users to easily create event-driven notification workflows using an intuitive drag and drop interface.
2. Enable real-time execution of these notification workflows for timely and relevant communication.
3. Offer a unified interface for managing transaction notifications, such as upcoming trip reminders and promotional notifications.
4. Guarantee Service Level Agreements (SLAs) for processing various types of notification workflows, including transactional and promotional communications.
5. Reduce the time required to develop complex notification workflows.

Journey Platform allows users to iterate faster by allowing self-serve workflow creation. It has reduced the time taken to support a new use-case from 1–2 months to just 1–2 weeks.

Figure 1: Time saved in Journey Platform

Overview

Figure 2: Journey Platform architecture overview

The key components of the Journey Platform are:

1. Journey Platform UI: WYSIWYG tool allows users to drag and drop components and create a workflow. The workflow definition is then converted to a custom DSL (Domain-specific language) which can be interpreted and executed by the workflow orchestrator.
2. Workflow Orchestrator: Brain of the system, the workflow orchestrator takes in the workflow definition DSL from the UI. Once a workflow is launched, it listens for events from the event store that can start the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and relies on the Action store to perform specific tasks.
3. Platform Store:

• Event Store: Pre-configured catalog of Kafka events which Journey Platform can listen to and trigger new executions of a workflow or pass events to existing workflow execution.
• Action store: Repository of predefined, specific-purpose functions allows users to perform various tasks, such as sending emails, push notifications, or emitting Kafka events. Custom actions can be defined and integrated into the tool, making them accessible to all Journey Platform users.
• Attribute store: Central repository for essential data, such as user metadata (e.g. user’s geolocation, Airbnb search history, etc.) and contextual information. It supports decision-making in workflow branching processes by exposing these data as a parameter to set conditions upon through the parameter manager.
• Custom stores: Ability to create custom action or attribute stores which aren’t already defined in the platform.
• Workflow Orchestrator: Brain of the system, the workflow orchestrator takes in the workflow definition DSL from the UI. Once a workflow is launched, it listens for events from the event store that can start the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and relies on the Action store to perform specific tasks.

User Interface

Figure 3: Manipulating and connecting nodes in a Journey Platform workflow

When crafting the UI for the workflow automation system, we aimed to create a familiar and intuitive experience. Drawing inspiration from flow charts, productivity tools with “inspector panels,” and incorporating drag and drop functionality, we wanted a platform where users could start immediately without consulting the manual.

We also had a goal of using progressive disclosure to incrementally enable the full depth of the platform capabilities, while keeping it simple for users who only need a small subset of the features. By using sensible defaults, and moving more complex features into tabs and sub-screens, our advanced users could create unique solutions, going beyond the pre-planned use cases.

To edit the graph, we leveraged React Flow, an open-source library. This enabled us to display the graph, as well as provide basic operations like zooming, panning, moving, and connecting nodes. On top of this foundation, we added our custom node and edge components, along with drag and drop functionality for adding new nodes and an inspector panel for editing existing ones.

Figure 4: The “node inspector” panel can show a variety of form inputs depending on the type of node.

To create the forms displayed in the inspector panel, we implemented a schema-based form system. This system provides a high level of flexibility, allowing us to declaratively specify the UI for specific node input/output fields as part of their type definitions. The system is built in a type-safe manner, making use of Thrift annotations and Java reflection. Based on the schema information and UI-specific annotations, the interface displays the appropriate form fields, help text, and validation, ensuring our UI is automatically up-to-date with the platform’s capabilities.

Backend Design

DSL provides a high degree of flexibility and customization, allowing us to define the structure and behavior of the workflow. Instead of having to hardcode a workflow in the workflow engine, we instead have a generic workflow defined that can execute any DSL-based workflow. Nodes and edges make up a workflow, with nodes representing individual actions or tasks and edges defining the dependencies and relationships between them.

The nodes and edges include all the necessary information to define a workflow such as inputs, outputs, and parameters passed between nodes. The DSL generated by the UI is passed to the workflow orchestrator, where the DSL parser executes it.

Figure 5: Workflow with the translated DSL

DSL representation of a workflow

Journey Stores

The events, attributes, and actions stores are an integral part of the backend, as they allow listening to events to start workflow executions, filter users, and execute tasks in the journey. All these components work together seamlessly to create a flexible and customizable backend that can be tailored to the specific needs of the platform.

Event Store

Journey Platform supports listening to different Kafka events and using them to trigger new executions of a workflow, or use the event to pass signals to a running execution. For example, start a new execution when a guest books a stay, pass a signal to a running execution when a user receives a push notification, etc. Similar to the action store, once an event is on-boarded, all the teams at Airbnb can use it.

Figure 6: Start node with event trigger

Attribute Store

The attribute store functions as a central repository for fetching all necessary data, such as contextual data, user preferences, and device information, which can be used to enrich the workflow branching process and improve decision-making capabilities. These stores are supported by a data storage system that manages various attributes or characteristics of entities.

Imagine you have a new user who just signed up for Airbnb, and you’re interested in determining whether they’ve conducted any listing searches on the platform. If the answer is yes, you’ll send a personalized message based on their search history, and if it’s no, you’ll send a static message.

This is a concrete example of how the Airbnb Journey Platform leverages attributes, such as “listing search history,” to enhance the user experience. These attributes are extracted and defined as parameters, which can be used for various purposes. Each workflow execution has its own parameter data collection, which can be accessed in the parameter manager. More information about parameters will be discussed in the parameter manager.

Figure 7: Setting filter condition using the attribute store

Action Store

The action store is used to execute various tasks, such as sending an email or updating a database record, when a user reaches a specific point in the journey. It is a common library where each function can be shared and reused by different users in their workflow.

Figure 8: Example Actions supported in Journey platform.

Each action implements a common interface, including its metadata required for the UI schema-based forms mentioned above, and its behavior during the actual workflow execution.

Interface all the Actions must implement

Parameter Manager

Managing a complex workflow that involves multiple steps with varying inputs and outputs can be a challenging task, especially if the input and output parameters change frequently or are different for each user. For instance, you might need conditional branching in your workflow or personalized communication content based on user search. This is where parameterized workflows and parameter managers can prove to be invaluable components.

By specifying inputs and outputs (of attribute node / event node / custom node) as parameters, you can reuse them throughout the entire workflow execution. A parameter manager is a critical component that can store and manage your workflow parameters, streamlining the process of creating, storing, retrieving, and modifying them.

In addition to providing an efficient parameter management system, a parameter manager also provides a range of features such as parameter creation, storage, retrieval, modification, versioning, access control, and auditing. These features ensure that your workflow is executed reliably and consistently while also properly managing and storing your parameters throughout the entire workflow.

Figure 9: Adding a param from the parameter library

Workflow Orchestrator

The Workflow Orchestrator executes workflows by interpreting the meaning of each DSL node and performing the corresponding actions. It manages low-level functions such as storing state, interacting with the action store to perform an action, listening for callbacks through the event store, and allowing developers to concentrate on workflow logic rather than technical details. Journey Platform utilizes Temporal as the underlying workflow engine for state maintenance and orchestration. Temporal helps orchestrate workflows through Temporal Workers.

DSL interpreter

Developers can incorporate custom functionality such as new nodes or edges to broaden platform capabilities, making it simpler to create workflows that fulfill the platform’s and users’ unique requirements. Additionally, it supports advanced features like parallel execution and automatic retries and enhancing platform reliability and performance.

Figure 10: Workflow Orchestrator

Scaling the system

Figure 11: Multi-tenant system with dedicated processing lanes

Ensuring SLA for processing different types of workflows (i.e. transactional and promotional) is critical at scale. Transactional notifications initiated by user action (e.g. booking confirmation, guest/Host messaging, etc.) have a strict SLA and require higher priority when compared to promotional notifications. To achieve this, we have implemented the following at different parts of the system:

Event pre-processing:

• Pre-filter: Instead of passing all the events directly to the Workflow Handler, the event processor filters out events that don’t match the criteria. e.g. only pass if the event type is reservation_complete and filter out for all other reservation events. This greatly reduces the QPS feeding into the system.
• Aggregate high QPS events: Events like searches have a high QPS. Instead of directly processing, we batch and aggregate them over a time window. This reduces the QPS by at least a few orders of magnitude.

Dedicated lanes:

• We have dedicated lanes for different categories of workflows through the system. The event listener has different consumer groups with built-in throttling. The workflow handler has dedicated Temporal namespaces for each category and strict limits on the processing QPS, max QPS to the database, etc.

Conclusion

The Journey Platform empowers non-technical and technical users to create complex stateful workflows through a simple drag and drop interface. By leveraging a generic workflow definition DSL, along with action store, event store, and attribute store, the platform facilitates the creation of workflows that respond to real-time events, streamlining communication, and enhancing user experiences.

Interested in working at Airbnb? Check out these open roles.

Acknowledgments

Thanks to Balaji Kalaimani, Davis Wamola, Iris Feng, Jesse Garrison, John Bernardo, Kumar Arjunan, Michael Endelman, Priyank Singhal, Steve Krulewitz, Tej Sudha, Victoria Gryn, Xin Tu, and Zhentao Sun for their contributions in building Journey Platform.

Thanks to Sagar Naik and Michael Kinoti for their leadership and supporting us in this Journey.

****************

All product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.