Streamlining Financial Precision: Uber’s Advanced Settlement Accounting System

In the realm of fintech, settlements, refunds, and chargebacks related to PSPs (payment service providers) are crucial transactions that involve the transfer of funds between different financial entities to complete a payment process. PSPs act as intermediaries between merchants and consumers, facilitating the smooth and secure transfer of funds. Settlements in this context refer to the end-of-day process where funds collected from consumers are disbursed to merchants, minus any fees or charges**.**

Settlement accounting is a critical financial process that involves reconciling and recording all transactions related to settlements, refunds, and other financial activities conducted through PSPs. At Uber, settlement accounting is indispensable due to the sheer volume of transactions and the need for precise financial tracking.

Figure 1: Summary of card payment processing and the key players in the market.

We process about 1.2 billion settlements each month, handling around $130 billion in cash in transit annually from over 50 different PSPs. This process ensures that ‌funds received from various payment methods are accurately accounted for and matched with the corresponding bank statements. Settlement accounting at Uber involves offsetting receivables booked during revenue accounting, verifying the cash deposited in the bank, and analyzing contracts with PSPs to determine fees, taxes, and other charges associated with transactions.

This blog describes Uber’s advanced settlement accounting system. Settlement accounting is crucial for ensuring financial accuracy, preventing fraud, and maintaining regulatory compliance. By automating reconciliation and offering near-real-time reporting, it boosts operational efficiency and supports strategic business decisions. Additionally, it helps manage transaction costs by analyzing PSP fees and taxes. Reliable settlement accounting fosters customer trust through accurate processing and timely resolution of payment issues.

PSP files include various events from payment service providers, each initiated by different parties and crucial for maintaining accurate financial operations. We built the settlement accounting system to manage these events.

When considering the payment ecosystem, Uber-initiated events are actions initiated directly by Uber. They include:

• Collections: Uber initiates the collection of funds from riders for services rendered.
• Refunds: When a rider is owed money back, Uber initiates the refund process. This could be due to ride cancellations, service issues, or other reasons that warrant returning funds to the rider.

Rider-initiated events are triggered by riders, typically in response to issues or disputes. These include:

• Chargebacks: Riders can dispute transactions, leading to a chargeback. This process involves reversing a transaction after a rider questions its legitimacy, often resulting in funds being withdrawn from Uber’s account and returned to the rider.

PSP-initiated events are managed by the PSP. These include several critical financial processes:

• Trip Fees: These are fees directly related to the trips taken by riders, such as a gateway fee, card network fee, etc.
• Non-Trip Fees: These are fees that are applied on an aggregated level rather than per trip, such as a gateway fee or card network fee.
• Taxes: PSPs often handle the collection and remittance of applicable taxes on transactions, per tax regulations.
• Payment Verification: This involves verifying the authenticity and validity of payments. PSPs perform checks to prevent fraud, ensure that payment details are correct, and that funds are available.

Each of these events plays a vital role in the comprehensive financial management system, ensuring that transactions are processed accurately, disputes are handled properly, and regulatory compliance is maintained. Understanding and managing these events effectively is essential for maintaining financial integrity and operational efficiency in the payment processing ecosystem.

Efficient financial transaction management is critical in today’s digital landscape. Payment service providers facilitate these transactions, but reconciling PSP settlement files can be challenging. Here are the common challenges with PSP settlement files:

• Complexity: Each PSP has its own file format, and there are diverse file delivery mechanisms available. 
• Duplicates: Duplicates occur when the same transaction or entry is recorded more than once in the settlement files. This leads to discrepancies in financial records and causes issues in reconciliation. For instance, a transaction for a payment of $100 might be recorded twice, resulting in an inflated total of $200 instead of the actual $100.
• Missing or Wrong ‌References: This challenge involves the absence of necessary reference information or the inclusion of incorrect references in the settlement files, making it difficult to match transactions accurately. A transaction record might lack a unique transaction ID or have an incorrect ID that doesn’t match the records in the payment platform, leading to reconciliation issues.
• Format Issues: These occur when the data in the settlement files don’t conform to the expected format, leading to difficulties in processing and reconciliation. For example, a settlement file might include dates in DD/MM/YYYY format instead of the expected MM/DD/YYYY format, or contain extra columns that aren’t anticipated by the processing system.
• Balancing Equation Isn’t Satisfied: The balancing equation Settlements – Deductions = Payout ensures that the total settlements minus deductions should equal the payout amount. Any discrepancies indicate potential errors in the settlement data. For example, if the settlements amount to $10,000 and deductions are $2,000, the payout should be $8,000. If the file shows a payout of $7,500, there’s a discrepancy that needs investigation.
• Large One-Off Transactions Monitoring: When a PSP conducts a large one-off transaction, it requires special monitoring to ensure it behaves as expected and doesn’t cause disruptions. For example, a PSP might conduct a one-off transaction withholding $10 million USD. Systems must monitor this transaction closely to ensure it’s processed correctly and reflects accurately in all financial records.

The settlement accounting system manages challenges with PSP files and streamlines financial precision. Settlement accounting at Uber has 3 primary components: the Feed Ingestion Service, the Feed Processor Service, and the Reconciliation & Accounting Service. Each service has specific contractual responsibilities that are crucial for understanding their roles in the overall settlement process.

Figure 2: Parts of the Settlement Accounting infrastructure.

The File Ingestion Service is integral to managing settlement files obtained from third-party PSPs. Its primary function includes fetching these files from third-party sources and transferring them to a designated file storage location. This approach ensures that we avoid excessive retries or reprocessing requests from ‌third-party sources by storing the files internally.

Figure 3: The functioning of the file ingestion service, including downloading, parsing, and normalizing.

Downloading files from PSPs involves accessing settlement files published by PSPs through various data sources, such as SFTP, APIs, or blob stores. The File Ingestion Service retrieves these files directly from the source and uploads them to a designated blob store. This store serves as a centralized repository from which the files can be accessed for subsequent reading and processing.

Parsing presents a significant challenge due to the lack of standardized formats among settlement files from different PSPs. This necessitates dynamic parsing tailored to each format and PSP. The File Ingestion Service relies heavily on configuration for parsing the files sent by the PSP. For every PSP onboarding into the system, parser rules are written to parse the files for processing. Before applying parsing rules, we stage the file data into necessary tokens for further processing. This is called tokenization of data. Tokenizing involves splitting using delimiters and tagging the data to configured tokens.

Since PSPs send data in their unique format, it complicates the process of data processing. To address this challenge, the File Ingestion Service performs a crucial function known as normalization. During parsing, the service converts the varied data formats received from PSPs into a standardized, consumable format. This normalization process ensures that the data is structured consistently, facilitating easier and more efficient processing downstream.

To effectively process the PSP feed, essential fields like amount, currency, and transaction date are identified as critical. These key business data elements are extracted and structured into a standardized, normalized format known as PSPEvent. This standardized format ensures consistency and clarity of the extracted information. Later, the PSPEvent is transmitted to the feed-processing component for further stages of processing and analysis. 

Figure 4: Sample normalization for a PSPEvent.

As summarized previously, the Feed Parser divides the incoming PSP file data into distinct buckets called splits. This targeted segmentation ensures that each payment event is processed in a contextually relevant manner, setting the foundation for a more efficient and precise reconciliation process.

Figure 5: The functioning of the batching and aggregating of the normalized events.

Following the parsing phase, the Feed Processor steps in to handle each of these buckets in parallel. This parallel processing approach significantly accelerates the handling of large volumes of PSP events, reducing overall processing time and enhancing throughput. For each PSP event within a split, the Feed Processor calls the Reconciliation Service. This service meticulously matches these PSP events with Uber’s Payment Platform records, ensuring that every transaction is accurately reconciled.

However, not all reconciliation attempts are successful. For events where reconciliation fails, the system marks them as unhealthy and submits them to the DLQ (Dead Letter Queue). The DLQ acts as a safety net, ensuring that these events are retried and resolved at a later time. This mechanism maintains the integrity and accuracy of our payment processing pipeline by ensuring no event is lost or forgotten.

Once the financial events are enriched, they’re sent back to the Feed Processor. After completing the processing of a split, these enriched financial events are forwarded to our accounting engine. At this stage, the financial events are meticulously processed, and the corresponding accounting transactions are persistently stored in our data repository. This ensures that every financial event is accurately accounted for, maintaining the fidelity of Uber’s financial records.

Payment reconciliation, in essence, is the process of cross-referencing and verifying financial transactions between the payment service providers and the bank records. The primary goal is to ensure that all transactions are accounted for accurately and that the settlements by the payment service providers match the expected revenue.

Besides protecting against fraud, effective payment reconciliation enables data-driven decision-making, features the trends among various payment service providers, and manages compliance with legal and financial regulations.

The Reconciliation Service plays a crucial role in reconciling PSPEvents with records on the Uber Payment Platform, ensuring that Uber receives payment for every trip where PSPs were involved in processing transactions. The reconciliation process hinges on matching precise records from both the PSP and Uber Payment Platform, facilitated by a deterministic field called transactionReference shared between them. This acts as a unique reference, established and stored during the payment transaction process between Uber and the PSP. Uber retains these references to validate incoming data from the PSP during reconciliation.

Figure 6: The reconciliation and accounting architecture for the PSP events.

Reconciliation operations aren’t immediate upon trip completion but can occur with a delay due to the PSP sending data to Uber on a scheduled basis—typically daily or weekly, about T+7 days after the transaction. Given Uber processes around 1 billion trips per month via PSPs, the system must accommodate high scalability and robust storage capabilities.

To streamline operations and segregate responsibilities between trip ingestion and PSPEvent processing, we designed the system with a dedicated Trip Ingestion Service. Trip information is ingested into the system via an Apache Kafka® consumer triggered on an event basis, immediately upon trip completion. The Kafka consumer hydrates the cache with essential details such as trip ID and the transactionReference, enabling swift retrieval and correlation when processing PSPEvents.

When reconciliation is successful, the events undergo an enrichment process. They’re supplemented with data from 15 different upstream sources, transforming them into comprehensive financial events. These financial events are self-sufficient data models, containing all the information necessary for precise and accurate accounting.

These financial events are sent to the accounting service to generate the financial accounting transactions.

Upon creation of a financial event, it’s promptly routed to the accounting engine, where meticulously configured accounting rules come into play. Each event type is associated with specific accounting rules that guide the evaluation process. These financial events carry all necessary parameters to facilitate the application of relevant accounting rules.

Every financial event has pre-configured rulesets designed to calculate various segments of the accounting treatment. These events are tagged with attributes such as location, line of business, account type, etc. The account types are broadly categorized into assets, liabilities, and profit and loss. Each of these categories is further subdivided into revenue, contra-revenue, receivables, and payables and expenses.

After ‌accounting rules are evaluated for a given event, a transaction model is generated based on the outcomes. This model represents the accounting treatment applied to the event. The transaction model undergoes a series of validations to ensure accuracy and compliance with accounting standards. Once validated, the transaction is saved in the data store and then published to a Kafka topic for downstream systems to consume.

The validated transactions are aggregated into a ledger, which provides a comprehensive view of all financial activities. This ledger is essential for financial reporting, audit trails, and strategic decision-making.

Our comprehensive settlement accounting system at Uber is designed to efficiently handle the immense volume of transactions processed each month. By using a structured approach that includes parsing, processing, reconciliation, enrichment, and accounting, we ensure that each financial event is accurately captured and recorded.

This robust system not only accelerates processing time but also enhances the accuracy and integrity of our financial operations. Automated reconciliation and real-time reporting improve operational efficiency, support strategic business decisions, and maintain regulatory compliance. Additionally, it helps in managing transaction costs, preventing fraud, and fostering customer trust through precise and timely resolution of payment issues.

Ultimately, this settlement accounting framework underscores Uber’s commitment to operational excellence and reliability, ensuring that our financial processes are secure.

Apache®, Apache Kafka®, Kafka®, Apache Cassandra®, Cassandra®, Apache Spark ™, Spark ™, Apache Hive ™, and Hive ™ are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.