No Code Workflow Orchestrator for Building Batch & Streaming Pipelines at Scale

No Code Workflow Orchestrator for Building Batch & Streaming Pipelines at Scale

摘要

At Uber, several petabytes of data move across and within various platforms every day. We power this data movement by a strong backbone of data pipelines. Whether it’s ingesting the data from millions of Uber trips or transforming the ingested data for analytical and machine learning models, it all runs through these pipelines. To put it in perspective, Uber’s data platform runs upwards of 15,000 data pipelines!