Improving Efficiency Of Goku Time Series Database at Pinterest (Part 2)

Improving Efficiency Of Goku Time Series Database at Pinterest (Part 2)

摘要

Goku is a time series database used by Pinterest for monitoring and alerting services. It offers pre-aggregation as an optimization technique for reducing query latency and cardinality. Users can enable pre-aggregation for metrics experiencing high latency or hitting cardinality limits. The Goku team provides users with tag combination distribution for the metric, allowing them to choose the tags they want to preserve in the pre-aggregated time series. After consuming data points from Kafka, the Goku Short Term host checks if the time series qualifies for pre-aggregation. If it does, the data is entered into an in-memory data structure that records various aggregations. Additionally, 5 aggregated data points are emitted for the time series. Goku Root handles query requests and modifies the metric name to query the right time series. The success story mentions a metric with high cardinality that achieved lower latencies after enabling pre-aggregation. Goku has onboarded over 50 use cases for pre-aggregation.