How to use ChatGPT for multi-turn conversation

GPT stands for Generative Pre-trained Transformer, and is based on the Transformer model introduced by Vaswani et al. in 2017. The ChatGPT application is a chatbot that uses the GPT-3.5/GPT-4 model, and is known for its ability to understand and generate natural language. GPT-3.5 has 175 billion parameters, while its successor, GPT-4, has 400 trillion parameters. These parameters were trained on a huge corpus of text containing 496 billion tokens, and produce remarkable results in question answering across domains. Both GPT-3.5 and GPT-4 are available via APIs. In this post, we’ll explore how the openai API can be used for multi-turn conversations using the gpt-3.5-trubo model, which is the one used in the ChatGPT application.

Tokens

The GPT models consume and generate language as tokens. Each token is 3–4 letters of text. We can use the tiktoken library to count the number of tokens in a given text. To do so, first encode the text using the get_encoding() method in the tiktoken library. This method will accept the name of the encoding as a parameter. In the case of gpt-3.5-turbo, the encoding used is cl100k_base. The number of tokens is then the length of the encoding generated.

Each GPT model has a limit on how many tokens it can process. This is the number of tokens in the prompt plus the reply. In multi-turn conversations, this number needs to be kept in check as the prompt length continues to grow as the conversation progresses. GPT models can’t process an infinite context, and so the context needs to be truncated when the conversation length increases beyond a certain point.

GPT Models Specs and Pricing

Model

Max Tokens

Pricing $/1K Input Tokens

Pricing $/1K Output Tokens

gpt-3.5-turbo

4,096

0.0015

0.002

gpt-3.5-turbo-16k

16,384

0.003

0.004

gpt-4

8,192

0.03

0.06

gpt-4-32k

32,768

0.06

0.12

Code Example

Let's explore how the GPT model can be used via the API. To begin, replace the text <add-API-key-here> with the OpenAI API key.

Note: We can get an OpenAI API key by following the instructions given in the Educative answer, How to get API Key for GPT-3. Please keep in mind that if your trial with OpenAI API has expired, it will throw up an error unless you purchase credits.

from pprint import pprint
import glob
import openai
import tiktoken
def num_tokens_from_string(string, encoding_name):
    encoding = tiktoken.get_encoding(encoding_name)
    num_tokens = len(encoding.encode(string))
    return num_tokens
openai.api_key = "<add-API-key-here>"
max_tok = 200
chat = [{"role": "system", "content": "You are a tenured Data Scientist."}]
queries = ["Give me directions for research in Automatic Speech Recognition.", 
          "What is the best way to train a model for a low resource language?",
          "I'm getting a high WER as a result of the fact that in Arabic script languages, there are various ways in which a word can be written. How can I resolve this issue?"
]
for idx, query in enumerate(queries):
  chat.append({"role":"user", "content":query})
  reply = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=chat,
    max_tokens=max_tok
  )
  print("Stage",idx+1 ,": number of generated tokens", num_tokens_from_string(reply['choices'][0]['message']['content'], "cl100k_base"))
  chat.append({"role":"assistant", "content":reply['choices'][0]['message']['content']})
  pprint(chat[2*idx+1])
  pprint(chat[2*idx+2])

Code Explanation

  • Lines 1–4: Import the relevant libraries.

  • Lines 6–9: Define the function for finding the number of tokens in a query/response.

  • Line 11: Set OpenAI API key.

  • Line 12: Set the maximum number of tokens that will be generated.

  • Line 14: The system identifier is used to personalize the assistant’s responses by assigning it a role.

  • Lines 15–18: Define the queries for a chat session.

  • Line 20: Iterate through the queries, performing the following at each step.

  • Line 21: Append the query with the identifier user to the chat conversation list.

  • Lines 23–27: Use the create() method of the ChatCompletion module to generate a response.

  • Line 28: Print the number of tokens in the response.

  • Line 29: Append the response with the identifier assistant to the chat conversation list.

  • Lines 30–31: Print the user query and the system response.

Unlock your potential: ChatGPT for everyday tasks, all in one place!

If you've missed any part of the series, you can always go back and check out the previous Answers:

Free Resources

Copyright ©2025 Educative, Inc. All rights reserved