从OpenAI API流式传输JSON

This blog posts explains how streaming from the OpenAI API improves user experience (UX). More importantly, we look how you can stream JSON data. We provide examples using Next.js, Vercel AI SDK and my very own http-streaming-request library.

这篇博客文章解释了如何通过OpenAI API进行流式传输来提高用户体验（UX）。更重要的是，我们将看到如何流式传输JSON数据。我们使用Next.js、Vercel AI SDK和我自己的http-streaming-request库提供示例。

Why do you need streaming?

为什么需要流式传输？

Have you ever used an Open AI API in your code? If so, you may have noticed that it can take up to 10 seconds for the API to respond. So you may have to display a spinning wheel (of doom) to users. That's not great user experience - people may think your app is broken and leave.

您是否曾经在您的代码中使用过Open AI API？如果是这样，您可能已经注意到API的响应时间可能长达10秒。因此，您可能需要向用户显示一个旋转的轮子（末日之轮）。这不是很好的用户体验-人们可能会认为您的应用程序出错并离开。

spinning wheel

On the other hand, ChatGPT doesn't make you wait - it starts printing out a response almost immediatelly - character by character (well, token by token).

另一方面，ChatGPT不会让你等待 - 它几乎立即开始逐字符（或者说逐令牌）打印出响应。

ChatGPT streaming

So what ChatGPT does is not just a gimmick, which makes it look like a retro sci-fi film, but a very useful UX technique. It lets users get partial results almost straight away.

所以ChatGPT所做的不仅仅是一种花招，让它看起来像一部复古科幻电影，而是一种非常有用的用户体验技术。它让用户几乎立即获得部分结果。

Yes, it still takes the same amount of time to get a full response but:

是的，获取完整响应仍然需要相同的时间，但是：

You can start reading an answer almost as soon as you've submmited a question
您可以在提交问题后几乎立即开始阅读答案
And you don't get an impression that it is broken or slow
您不会觉得它是损坏或者很慢

How can you get started with streaming?

如何开始使用流式传输？

Okay, streaming is useful! But how can you get started with streaming?

好的，流式传输很有用！但是您如何开始使用流式传输呢？

Actually, it's not that difficult and there are already a number of tutorials on the Internet. For the purpose of this article we use Next.js with the Vercel AI SDK.

实际上，这并不难，而且互联网上已经有很多教程。在本文中，我们使用Next.js和Vercel AI SDK。

Streaming text response

流式传输文本响应

Let's start with a simple example - streaming text. We have a simple prompt that returns text data.

让我们从一个简单的示例开始 - 流式文本。我们有一个简单的提示，返回文本数据。

...