Tutorial

How to stream ChatGPT API responses?

This tutorial introduces a simple method to stream ChatGPT & GPT-4 responses. Streamed responses are served sooner to user side, which improve user experience.

Photo by Mollie Sivaram on Unsplash

Introduction

Streaming enables faster response to user query.

The tutorial builds the necessary code to stream answers with ChatGPT API responses. The entire code is available in my Github with an option to deploy the code in Streamlit.

GitHub - tmgthb/stream_chatgpt_api: How to stream ChatGPT & Davinci model responses (gpt-3.5-turbo…

Let’s get started!

Requirements

This approach requires only standard libraries for the OpenAI API’s own library and time, which is built into Python. So, you do not need any unfamiliar libraries:

!pip install --upgrade openai 
import openai
import time
openai.api_key = os.getenv("OPENAI_API_KEY")
startime = time.time()

Now, we can start using the scripts to stream answers. I will use a code, which first takes user input, then it passes the input to the API and immediately measures events in time and…