使用 FastAPI 和 Celery 的异步任务
PDF version is available!
PDF 版本已上线!
Download the PDF version of this post for free:
免费下载本帖的 PDF 版本:
When you have a long running Python function that you want to expose via an API endpoint, you might want to run the function asynchronously to avoid timeouts. Running a function asynchronously means it won’t block the rest of your code. Instead of waiting the function to finish the task, your program can continue executing other tasks, making it more responsive.
当你有一个运行时间较长的 Python 函数,并希望通过 API 端点暴露它时,你可能希望异步运行该函数以避免超时。异步运行函数意味着它不会阻塞其余代码。程序无需等待该函数完成任务,而是可以继续执行其他任务,从而提高响应性。
In this post, I will show you how to use Celery to execute tasks asynchronously in your FastAPI application. We will use the function below to simulate long-running code.
在本文中,我将向你展示如何在 FastAPI 应用中使用 Celery 异步执行任务。我们将用下面的函数来模拟耗时较长的代码。
Python
Python
import math
import time def square_root(num: float): time.sleep(10) return math.sqrt(num)
In the real world applications, this might be sending emails to users, processing long duration videos, or training ML models.
在实际应用中,这可能包括向用户发送邮件、处理长时间视频或训练 ML 模型。
A quick intro to Celery
Celery 简介
Celery is a task queuing in Python. You use Celery to manage and distribute tasks across worker processes or machines. You define tasks in your Python code, and Celery takes care of running them asynchronously. Its a perfect tool for handling things such as time-consuming operations or external API calls.
Celery 是 Python 中的任务队列工具。你可以用 Celery 在多个 worker 进程或机器上管理和分发任务。在 Python 代码中定义任务后,Celery 会负责异步执行它们。它是处理耗时操作或外部 API 调用的理想工具。
Celery requires a message broker to transmit messages between the client (where tasks are initiated) and the workers (where tasks are executed). There are many options for messages brokers to be used with Celery. In this post, we will use Redis.
Celery 需要一个消息代理在客户端(任务发起的地方)和 worker(任务执行的地方)之间传递消息。与 Celery 配合使用的消息代理有很多选择。本文将使用 Redis。
Write the API endpoints
编写 API 端点
We will use FastAPI to build the API endpoints. We will write two API endpoints:
我们将使用 FastAPI 来构建 API 端点。我们将编写两个 API 端点:
-
/square_root
to ...