优化Apache Flink应用程序的7个提示
by: Yaroslav Tkachenko, Kevin Lam, and Rafael Aguiar
by:Yaroslav Tkachenko, Kevin Lam, and Rafael Aguiar
At Shopify, we’ve adopted Apache Flink as a standard stateful streaming engine that powers a variety of use cases like our BFCM Live Map. Our Flink applications are deployed in a Kubernetes environment leveraging Google Kubernetes Engine. Our clusters are configured to use High Availability mode to avoid the Job Manager being the single point of failure. We also use RocksDB state backend and write our checkpoints and savepoints to Google Cloud Storage (GCS).
在Shopify，我们已经采用了Apache Flink作为标准的有状态流引擎，为我们的BFCM Live Map等各种用例提供动力。我们的Flink应用被部署在Kubernetes环境中，利用谷歌Kubernetes引擎。我们的集群被配置为使用高可用性模式，以避免作业管理器成为单一故障点。我们还使用RocksDB状态后端，并将我们的检查点和保存点写入谷歌云存储（GCS）。
Making sure our Flink applications stay performant and resilient is one of our top priorities. It’s also one of our biggest challenges. Keeping large stateful applications resilient is difficult. Some data models require storing immense state (for example, 13 TB for the sales data, as we shared in our Storing State Forever: Why It Can Be Good For Your Analytics talk) and we’ve spent a lot of time on performance tuning, learning many lessons along the way.
确保我们的Flink应用程序保持高性能和高弹性是我们的首要任务之一。这也是我们最大的挑战之一。保持大型有状态应用程序的弹性是困难的。一些数据模型需要存储巨大的状态（例如，13TB的销售数据，正如我们在《永远存储状态》中所分享的。为什么它对你的分析有好处），我们在性能调整上花了很多时间，在这一过程中吸取了很多教训。
Below we’ll walk you through key lessons for optimizing large stateful Apache Flink applications. We’ll start off by covering recommended tooling, then focus on performance and resiliency aspects.
下面我们将引导你了解优化大型有状态Apache Flink应用程序的关键经验。我们将从推荐的工具开始，然后重点讨论性能和弹性方面。
1. Find the Right Profiling Tools
1.找到正确的分析工具
First things first. Having the right profiling tools on hand are key to getting insights into how to solve a performance problem. While deploying our first applications, we found that using the below set of tools were useful when debugging Flink:
第一件事。拥有正确的分析工具是深入了解如何解决性能问题的关键。在部署我们的第一个应用程序时，我们发现在调试Flink时，使用下面这套工具很有用。
- Async-profiler: a profiling tool built for ...