知鸦日报2024-02-06

Retrieval-Augmented Generation (RAG) is a technique that combines retrieval and generation capabilities. It effectively addresses some issues of large language models (LLMs), such as hallucinations and knowledge limitations. With the evolution of RAG, vector technology involved in RAG has gained attention, and vector databases have become more widely recognized. Established database providers now support vector retrieval, including Elasticsearch, which recently added support for vector retrieval in its latest version. This article introduces the deployment of Elasticsearch and embedding models in RAG, as well as how to use Elasticsearch for document indexing and retrieval within the LLM framework LlamaIndex.

如果系统的控制权、代码完全被掌控，很容易添加中间层；现实情况我们往往无法控制系统的所有细节，所以需要使用一些 “非常规”（拦截） 手段来增加中间层。

介绍容器部署的三个阶段的优缺点，以及Docker部署的案例。

Fluid是一个开源可扩展的分布式数据编排和加速系统，以Kubernetes标准和对用户透明的方式为AI和大数据等数据密集型应用提供数据访问能力，其目标为构建云原生环境下数据密集型应用的高效支撑平台。

本文记录两次报错系统监控现象以及作者针对性的排查过程和分析，最终解决了问题的全过程。

Myntra has two categories of merchandising stores: category stores and brand stores.

The store operations team actively manages category and brand store root pages to run different campaigns in coordination with business and category teams. Some of the brands also manage their brand stores.

They change layouts, add new widgets in a layout, and new content in widgets and also change the positions of widgets.

In this digital age, companies collect multitudes of data that enable the tracking of business metrics and performance. Over the years, data analytics tools for data storage and processing have evolved from the days of Excel sheets and macros to more advanced Map Reduce model tools like Spark, Hadoop, and Hive. This evolution has allowed companies, including Grab, to perform modern analytics on the data ingested into the Data Lake, empowering them to make better data-driven business decisions. This form of data will be referenced within this document as “Offline Data”.

With innovations in stream processing technology like Spark and Flink, there is now more interest in unlocking value from streaming data. This form of continuously-generated data in high volume will be referenced within this document as “Online Data”. In the context of Grab, the streaming data is usually materialised as Kafka topics (“Kafka Stream”) as the result of stream processing in its framework. This data is largely unexplored until they are eventually sunk into the Data Lake as Offline Data, part of the data journey (see Figure 1 below). This induces some data latency before the data can be used by data analysts to inform decisions.

Several years ago, Pinterest had a short incident due to oversights in the policy delivery engine. This engine is the technology that ensures a policy document written by a developer and checked into source control is fully delivered to the production system evaluating that policy, similar to OPAL. This incident began a multi-year journey for our team to rethink policy delivery and migrate hundreds of policies to a new distribution model. We shared details about our former policy delivery system in a conference talk from Kubecon 2019.

At a high level, there are three important architectural decisions we’d like to bring attention to for this story.

How Airbnb achieved a smooth and transparent migration from Buck to Bazel on iOS, with minimal interference to developer workflows.

GC 全称 Garbage Collection，垃圾收集，是一种自动管理堆内存的机制，负责管理堆内存上对象的释放。GC 让程序员把更多的精力集中在实际问题上。Dart 作为 Flutter 的主要编程语言，在内存管理上也使用了 GC。

调度、内存、文件、网络。

本文我们介绍了在 V8 中是如何实现闭包这一特性的，V8 在处理函数的时候采用的延迟解析来提高启动速度，但是延迟解析和闭包存在天然的矛盾，所以当一个函数中存在闭包并且执行时。