How to get GPT3 to Output in JSON

This past Friday I attended an AI Hackathon in Amsterdam where me and my team built an application that scans difficult letters from the government and helps simplify them for low-literate people (it won a prize by the way :)). You can check out the video pitch in this tweet.

The app we made interfaced with the OpenAI API to send our scanned letter with our engineered prompt to in turn receive the output generated by the text-DaVinci-03 model. Our prompt was engineered in such a way that the text the model outputted was in JSON format and could be easily parsed. This allowed us to easily put the data in the right places in the application.

In this post, I will give some tips and findings from building this application and how you can prompt a GPT3 model such as text-DaVinci-03 to return a JSON formatted text. We got prompt engineering help from Jordi Bruin, so make sure to check out his projects on Twitter.

What are a GPT3 and a JSON?

GPT3 is a powerful language model that can generate text from a given prompt. It is capable of understanding natural language and can generate text that is coherent and relevant to the given prompt. GPT3 can be used to generate text for a variety of applications, such as summarizing conversations, generating stories, and generating code.

JSON (JavaScript Object Notation) is a lightweight data interchange format. It is a text-based format that is used to store and exchange data. JSON is easy to read and write and is often used to exchange data between a server and a web application.

An application can interface with an API by making an HTTP request to the API. The application can then parse the response from the API and use the data in the application. In this case, the application can make an HTTP request to the OpenAI API to generate text from a given prompt that is returned from the request.

Getting GPT3 to generate JSON

When writing a prompt for GPT3, it is important to provide enough context for the model to generate a valid response. The prompt should first include the text that you want to process by the model. Next, you want to give any instructions about the processing of the input text, for instance summarizing or rewriting the contents as a poem. Finally, add the requirement that the output should be structured in JSON formatted text.
GPT3 is insanely good at inferring what information needs to be filled in the required fields.

There are two ways to write the JSON output requirement: give an exemplary structure or describe a structure by mentioning the keys of fields and subfields.

Exemplary structure

We have a letter from Tarzan from the jungle to Jane. We want to process this letter and put the relevant information in the JSON. Below you can see the full prompt where we first have that letter, then the processing instructions, and an example JSON structure.

Consider the following message:-------Dear Jane, I am thinking about you when I am swinging through the jungle.Greetings, Tarzan--------Put this message in the following JSON structure{ "sender": "..", "message": "..", "list_of_words": [ "..", ".." ] "addressed": "..", "amount_of_words": ..

}

The output of this prompt is:

{
"sender": "Tarzan",
"message": "Dear Jane, I am thinking about you when I am swinging through the jungle. Greetings, Tarzan",
"list_of_words": [
"Dear",
"Jane",
"I",
"am",
"thinking",
"about",
"you",
"when",
"I",
"am",
"swinging",
"through",
"the",
"jungle",
"Greetings",
"Tarzan",
],
"addressed": "Jane",
"amount_of_words": 16
}

Described structure

With a described structure there is even less information given, consider the following prompt, now with a described JSON structure:

Consider the following message:-------Dear Jane, I am thinking about you when I am swinging through the jungle.Greetings, Tarzan--------

Put this message into a JSON with keys "sender", "message", "list_of_words", "addressed" and "amount_of_words":

The output of this prompt is:

{
"sender": "Tarzan",
"message": "Dear Jane, I am thinking about you when I am swinging through the jungle. Greetings, Tarzan",
"list_of_words": ["Dear", "Jane", "I", "am", "thinking", "about", "you", "when", "I", "am", "swinging", "through", "the", "jungle", "Greetings", "Tarzan"],
"addressed": "Jane",
"amount_of_words": 16
}

Even though in both cases we haven’t described what list_of_words and amount_of_words should contain, the key is descriptive enough that GPT3 infers that it should create an array of the words from the letter and count these words before putting them in the JSON, such magic! Another interesting thing is when we had an exemplary structure, we had the elements of the list_of_words on a new line as per our example. And with the described structure, the list_of_words was contained in a single line!

Another Example: Summarizing a Chat Conversation.

Let’s say we want to summarize a conversation on Slack or another group chat. We can write a prompt for GPT3 that includes the desired output format (JSON) and the context of the conversation. For example, the prompt could be:

Consider the following chat log:--------14:09 PM - John: "Let's add text-to-speech to our reading app! I think it will benefit our customers greatly."14:09 PM - Jane: "Let's include both a male and a female voice."14:10 PM - Ahmed: "I will talk to the dev team and see who picks it up."14:10 PM - John: "Thanks guys!"14:15 PM - Ahmed: "Bob will pick it up, he has previous experience with text-to-speech implementations"14:17 PM - Jane: "Exciting! send me a demo when the first version is implemented."--------Summarize for each person what they have said in the chat log.

Put this data into a JSON list with keys "name", "summary" and "amount_of_messages":

The output of this prompt is:

[
{
"name": "John",
"summary": "Suggested adding text-to-speech to the reading app and thanked the group.",
"amount_of_messages": 2
},
{
"name": "Jane",
"summary": "Suggested including both a male and a female voice.",
"amount_of_messages": 2
},
{
"name": "Ahmed",
"summary": "Offered to talk to the dev team and mentioned Bob would pick up the task.",
"amount_of_messages": 2
}
]

As you can see, it is not perfect. In the summary of Jane, there is no mention of her requesting a demo when the first version is implemented. Some extra prompt engineering and testing are required to generate a more stable response.

Omitting fields

Unfortunately, one thing that the model sometimes does not do well is asking it to leave a field empty when there is no data. For instance, when a piece of text might or might not contain a location. For the following prompt, the model correctly left the field empty, but just in case we added a boolean flag that shows if the field “City” is present. This is a nice fallback if the model sometimes fails to leave the field empty.

Consider the following sentences:------"I love Paris, it has the best Croissants.""Tokyo is beautiful by night.""Mountains are where I come to peace."------Put the sentences in a JSON list with the keys "sentence", "has_city" and "city".

If the sentence does not contain a city, leave the field empty:

Output:

[
{
"sentence": "I love Paris, it has the best Croissants.",
"has_city": true,
"city": "Paris"
},
{
"sentence": "Tokyo is beautiful by night.",
"has_city": true,
"city": "Tokyo"
},
{
"sentence": "Mountains are where I come to peace.",
"has_city": false,
"city": ""
}
]

Conclusion

Writing GPT3 prompts to output to JSON is a powerful way to receive formatted and structured data from a given prompt. By providing enough context in the prompt, GPT3 can generate a valid JSON object that can be easily used in an application. Please note, you must always check if a valid JSON is returned and test the prompt very thoroughly as an invalid JSON can result in parsing errors.

Happy prompting!