How ChatGPT Works, Explained Simply
This page explains how ChatGPT works, from the moment you press Send to the answer on your screen. It uses simple English and no maths. It covers 23 common questions, and says clearly where OpenAI has not published the details.
The short version
ChatGPT cuts your text into tokens and sends them, with the earlier chat, to a large language model. The model predicts the reply one token at a time. It was trained on huge amounts of text, then tuned with human feedback to be helpful.
1. The big picture
Start here if you are new. These answers explain ChatGPT without any maths.
What is ChatGPT, in simple words?
ChatGPT is a chat app made by OpenAI. Behind it is a large language model (LLM). That is a program trained on a huge amount of text to predict what comes next.
When you ask a question, the model writes the most likely answer, one small piece at a time.
What does GPT stand for?
GPT means generative pre-trained transformer. Generative: it creates text. Pre-trained: it first learned from a huge amount of text. Transformer: the kind of neural network it uses. A neural network is a program made of many numbers, called weights, that are adjusted during training.
Does ChatGPT look up answers in a database?
Usually not. The model writes answers from patterns it learned in training. It does not copy them from a stored list of answers.
For recent or exact facts, ChatGPT may search the web on its own, and you can also ask it to. Then it can show you its sources.
How much do we really know about how ChatGPT is built?
Less than many articles suggest. OpenAI has not published the size or exact design of GPT-4 and its later models. It has described the general training method and the product features.
So this page explains what OpenAI has said in public, plus how large language models work in general.
2. Tokens: how ChatGPT reads text
Before the model sees your message, the text is cut into tokens. This step explains some of ChatGPT's strange habits.
What is a token?
A token is a small piece of text: a word, part of a word, or a symbol. The model reads and writes tokens, not letters or whole words. The tool that cuts text into tokens is called a tokenizer.
In English, one token is often about three quarters of a word. Each token has a number. During training, the model learns which numbers tend to appear together, and that is how it captures meaning.
What we measured: OpenAI's GPT-4o tokenizer cuts "How does ChatGPT work?" into 6 tokens. They are "How", " does", " Chat", "GPT", " work" and "?".
Does ChatGPT use more tokens for Hindi and other languages?
Often, yes, especially with older models. Tokenizers learn their pieces from their training text, which is mostly English. Other languages get split into smaller pieces.
More tokens means slower replies and less room in the context window. That is the most text the model can read at once. For developers using the API, it also means a higher bill.
What we measured: "मुझे चाय पसंद है" ("I like tea") takes 17 tokens with GPT-4's tokenizer and 6 with GPT-4o's. The GPT-4o tokenizer has about 200,000 pieces. GPT-4's has about 100,000.
Why does ChatGPT sometimes fail at counting letters?
Because it does not see letters. It sees tokens, and a word may be one token. Counting the letters inside a token is like counting letters in a word you only heard, never saw written.
Newer models do better. Asking a model to spell the word out first often helps.
3. How ChatGPT was trained
ChatGPT was not trained in one go. For the first ChatGPT in 2022, OpenAI described three main stages. Newer models add more training that OpenAI has not fully published.
What is pre-training?
In pre-training, the model reads a huge amount of text and learns to predict the next token. This is where it learns grammar, facts and patterns of reasoning.
The result can continue any text, but it is not yet a helpful assistant.
What is RLHF, and why does ChatGPT need it?
RLHF means reinforcement learning from human feedback. People compare several answers to the same question and rank them. A second model, the reward model, learns to predict those rankings. Then ChatGPT's model is tuned to get high scores from the reward model.
OpenAI said ChatGPT was trained with RLHF, using the same method as its earlier InstructGPT model. This step makes answers more helpful and more polite.
Why does ChatGPT not know about recent events?
The model only knows what was in its training data, which stops at a date called the knowledge cutoff. Anything after that, it does not know unless you tell it.
When it searches the web, it can find newer information.
4. What happens when you press Send
Now follow one message from your screen to the answer.
What happens, step by step, when I send a message?
Your message, the earlier chat and some hidden instructions are sent to OpenAI's servers. The text is cut into tokens. The model runs on powerful graphics chips (GPUs) and predicts the reply one token at a time.
Each new token is sent to your screen as soon as it is ready. That is why the answer appears word by word.
Why do I get a different answer to the same question?
At each step, the model gives a probability to many possible next tokens. The next token is picked with some randomness. In OpenAI's API, a setting called temperature controls how much.
One different word early on changes everything after it. So two answers to the same question can look quite different.
Why are long answers slower?
Each token needs another pass through the model. A 500-token answer needs about 500 passes.
Reasoning models are slower still, because they first write hidden thinking tokens before the answer you see.
Why does ChatGPT need so many GPUs?
The model's weights, the numbers it learned, take a lot of memory. Every token needs a large amount of maths. Millions of people use ChatGPT at the same time.
So the model runs on many GPUs. Servers also group many users' requests together. A GPU does many calculations at once, and would otherwise sit partly idle.
5. Conversations and memory
ChatGPT seems to remember you. Here is how that actually works.
Does the model remember what I said earlier in the chat?
Not by itself. The model has no memory between requests. Each time you send a message, the earlier chat is sent again with it. OpenAI's API works the same way. Even when OpenAI stores the chat for you, the model reads it all again.
The limit is the context window, the most text the model can read at once. Very long chats may lose their oldest parts.
Then how does ChatGPT remember me across different chats?
ChatGPT has a memory feature. It can save facts you share, such as your job, and it can also use your past chats. You can see, turn off and delete memories in the settings.
The model is not retrained on you. OpenAI says saved memories work much like custom instructions, which are added to what the model reads.
What are the hidden instructions?
Before your message, the app adds instructions you do not see, often called the system prompt. They tell the model how to behave and which tools it can use.
Your own custom instructions, if you set them in the settings, are added in a similar way.
6. Tools and features
ChatGPT can use tools to do things the model alone cannot, such as searching or running code.
How does ChatGPT search the web or run code?
The model can ask for a tool, such as web search or running Python code, by writing a special request. The system runs the tool and gives the result back to the model as more text.
The model then continues its answer using that result. This is the same idea behind AI agents.
How does ChatGPT understand and create images?
Newer models are multimodal, so they can read images. In many open models, an image is cut into small squares that become tokens next to the text. OpenAI has not published its exact design.
ChatGPT can also create images, using OpenAI's image models.
What is different about reasoning models?
Reasoning models, first released by OpenAI in 2024, are trained to think step by step before they answer. They write thinking tokens first, then the final answer.
They are often better at maths, code and planning. They are also slower and cost more per question.
7. Mistakes, safety and privacy
Last, what ChatGPT gets wrong, and how to use it safely.
Why does ChatGPT sometimes make things up?
The model is trained to write likely text, not to check facts. When it does not know, a confident guess can still be the most likely text. This is called a hallucination.
Ask for sources, check important facts, and ask it to search the web for recent or exact information.
How does ChatGPT decide what it will not answer?
The model is trained to refuse some harmful requests. OpenAI also runs separate safety checks on messages and replies. These rules are set by OpenAI's usage policies.
Like any filter, they sometimes block harmless requests and sometimes miss harmful ones.
Is it safe to put private data into ChatGPT?
Be careful. What you type is sent to OpenAI's servers. In the data controls you can choose whether your chats may be used to improve the models.
For work data, follow your company's rules. Many companies use business plans or their own systems for this reason.
Learn it properly, not just the answers
Every answer on this page comes from our AI Engineering course: 112 lessons on RAG, evals, agents, serving, security and MLOps. Many of them are built around a real experiment. You learn why the answer is right, which is what an interviewer checks with the second question. 10 lessons are free to read, with no card needed.