基于Serverless产品，轻松实现低成本的延迟队列。

当今大数据时代，高吞吐、高可靠成为了分布式系统中重要的指标。而Apache Kafka作为一个高性能、分布式、可扩展的消息队列系统，被越来越多的企业和开发者所关注和使用。

本文将介绍Kafka的基本概念，包括Kafka的架构、消息的存储和处理方式、Kafka的应用场景等，帮助读者快速了解Kafka的特点和优势。同时探讨Kafka的一些高级特性，如Kafka的配置、文件存储机制、分区 等，帮助读者更好地使用Kafka构建分布式系统和应用。

Kafka中的实时数据以Topic的概念进行分类存储，而Topic的数据有一定的时效性。在定位一些实时数据的Case时，如果没有对实时数据进行历史归档，在排查问题时，没有日志追述，会很难定位是哪个环节的问题。

本篇文章首先介绍MetaQ消息队列，然后介绍作者对MetaQ和Kafka这两个消息队列的理解。

目前我们在构建一个多租户多产品类网站，为了让用户更好的找到他们所需要的产品，我们需要构建站内搜索功能，并且它应该是实时更新的。本文将会讨论构建这一功能的核心基础设施，以及支持此搜索能力的技术栈。

本文介绍了kafka使用SASL安全认证的配置方式。

Grab’s real-time data platform team, also known as Coban, has been operating large-scale Kafka clusters for all Grab verticals, with a strong focus on ensuring a best-in-class-performance and 99.99% availability.

Security has always been one of Grab’s top priorities and as fraudsters continue to evolve, there is an increased need to continue strengthening the security of our data streaming platform. One of the ways of doing this is to move from a pure network-based access control to state-of-the-art security and zero trust by default.

本文旨在分享阿里云Prometheus在阿里云Kafka和自建Kafka的监控实践。

本文主要从Kafka消费、堆积、稳定性、预案、成本控制等角度等最佳实践。

新浪现有 Kafka 集群主要处理来自新浪新闻、微博等的数据，数据类型包括特征日志、订单数据、广告曝光、埋点 / 监控 / 服务日志等。这些数据经过 Kafka 在线集群、广告专用集群、日志集群、离线集群和机器学习训练等集群的处理后，会用于推荐训练、HDFS 落地、离线数仓、实时监控、数据报表和实时分析等生产目的。

Cruise Control作为Kafka的运维工具，它包含了Kafka服务上下线、集群内负载均衡、副本扩缩容、副本缺失修复以及节点降级等功能。

本文是Kafak万亿消息实践中一次典型的故障进行详细分析和说明。深入到Kafka架构原理层分析故障出现的根因及对应的解决方案。

Uber’s goal is to ignite opportunity by setting the world in motion, and big data is a very important part of that. Presto® and Apache Kafka® play critical roles in Uber’s big data stack. Presto is the de facto standard for query federation that has been used for interactive queries, near-real-time data analysis, and large-scale data analysis. Kafka is the backbone for data streaming that supports many use cases such as pub/sub, streaming processing, etc. In the following article we will discuss how we have connected these two important services together to enable a lightweight, interactive SQL query directly over Kafka via Presto at Uber scale.

Uber has one of the largest deployments of Apache Kafka® in the world. It empowers a large number of real-time workflows at Uber, including pub-sub message buses for passing event data from the rider and driver apps, as well as financial transaction events between the backend services. As Kafka forms a critical component of Uber’s core workflows, it is important to secure the data being published and subscribed from the topics to maintain the integrity of the data and to provide an access control mechanism for who can publish/subscribe to a given topic.

Grab’s real-time data platform team (Coban) covers the importance of moving data in and out of Kafka easily and how Kafka Connect helps with that.

Apache Kafka 作为一个热门消息队列中间件，具备高效可靠的消息处理能力，且拥有非常广泛的应用领域。文章介绍基于 Kafka 的实时数仓在搜索的实践应用。