结构化数据存储(SDS):使用统一的服务堆栈进行多模型数据管理

Authors: Alberto Ordonez Pereira; Senior Staff Software Engineer | Lianghong Xu; Senior Manager, Engineering |

作者:Alberto Ordonez Pereira;高级软件工程师 | Lianghong Xu;高级经理,工程 |

Part 1: HBase Deprecation at Pinterest & Part 2: TiDB Adoption at Pinterest

第1部分:Pinterest上的HBase弃用 & 第2部分:Pinterest上的TiDB采用

In this blog, we will show how the team transitioned from supporting multiple query serving stacks to provide different data models to a brand new data serving platform with a unified multi model query serving stack called Structured DataStore (SDS).

在这篇博客中,我们将展示团队是如何从支持多个查询服务堆栈转变为一个统一的多模型查询服务堆栈的全新数据服务平台,名为Structured DataStore(SDS)。

As a data serving platform, SDS is responsible for various aspects of the data lifecycle management, including:

作为一个数据服务平台,SDS负责数据生命周期管理的各个方面,包括:

  • Online Query Serving, which offers multiple data models such as graph, table, document, etc. at 1–2 digit ms p99 latency and with high availability (99.99+) to access and store data
  • 在线查询服务,以1-2位数毫秒的p99延迟和高可用性(99.99+)提供多种数据模型,如图形、表格、文档等,用于访问和存储数据
  • Offline Query Serving, which is responsible for offering means to run analytical queries on the online tables
  • 离线查询服务,负责在在线表上运行分析查询的手段
  • Streaming (Change Data Capture), which is responsible for streaming changes to the online tables and making them available to external consumers through PubSub
  • 流式传输(变更数据捕获),负责将在线表的更改流式传输并通过PubSub向外部消费者提供
  • Cost Attribution and Analysis, which provides visibility and predictability on the cost of the different use cases served by the system. It also provides insights on potential cost improvement opportunities, query analysis, etc.
  • 成本归因和分析,提供对系统服务的不同用例成本的可见性和可预测性。它还提供有关潜在成本改进机会、查询分析等方面的见解。
  • Catalog/Metadata Management, which is responsible for storing and serving table metadata, including ownership, schemas, SLAs, etc., and also providing runtime metadata about the platform
  • 目录/元数据管理,负责存储和提供表元数据,包括所有权、模式、SLA等,并提供有关平台的运行时元数据
  • Security and Auditability, which ensures GDPR compliance, advanced levels of auditability, security, AAA, etc.
  • 安全性和可审计性,确保符合GDPR的合规性,具有高级审计、安全性、AAA等功能
  • Tool...
开通本站会员,查看完整译文。

Accueil - Wiki
Copyright © 2011-2024 iteam. Current version is 2.139.0. UTC+08:00, 2024-12-26 01:30
浙ICP备14020137号-1 $Carte des visiteurs$