What Is a Large Language Model (LLM)? A Plain-English Guide
✦ Key takeaways
- A large language model is an AI system that predicts the next word (or token) based on patterns learned from vast amounts of text.
- Training happens in two main stages: pretraining on huge text corpora, then fine-tuning to make the model more useful and safe.
- The context window sets how much text a model can consider at once, and it is measured in tokens.
- Models can state false information with full confidence (hallucination), so their output must be verified for important tasks.
What Is a Large Language Model?
A large language model (LLM) is a type of artificial intelligence system designed to understand and generate human language. At its core, an LLM is a giant next-word prediction machine: you give it a piece of text, and it estimates which token is most likely to come next, then repeats that step over and over to build full sentences and paragraphs. Despite how simple this idea sounds, training a model on enormous amounts of text makes it capable of translating, summarizing, answering questions, and writing software.
These models are built on a neural-network architecture called the Transformer, introduced in 2017. It transformed natural language processing thanks to its attention mechanism, which lets the model weigh how important each word is relative to every other word in the text.
Invoice & Quotation Maker
Professional invoices that auto-calc & print/PDF in a minute.
Tokens and Parameters: The Building Blocks
A model does not read letters the way we do. Instead, it splits text into small units called tokens. A token can be a whole word, part of a word, or even a punctuation mark. As a rough rule of thumb in English, one token equals about four characters, or roughly three-quarters of a word.
Parameters are the internal numerical values a model adjusts during training, and they are what store everything the model has "learned." The more parameters a model has, the more complex the patterns it can capture. Modern models range from a few billion parameters to hundreds of billions. OpenAI's GPT-3, for example, had about 175 billion parameters, while smaller open models with 7 or 8 billion parameters can run on far less powerful hardware.
How Is a Model Trained?
Training usually happens in two stages. The first is pretraining, where the model is exposed to vast amounts of text from books, websites, and articles, and repeatedly learns to predict the next token. This stage is expensive and can take weeks running on thousands of specialized graphics processors (GPUs).
The second stage is fine-tuning, where the model is refined on more targeted data to make it more useful and safe. One of the best-known techniques is reinforcement learning from human feedback (RLHF), in which people rate the quality of the model's answers to steer it toward responses that are more accurate and appropriate.
Context Window and Hallucination
The context window is the amount of text a model can take into account at once, measured in tokens. If the context window is 128,000 tokens, the model can effectively "remember" the equivalent of hundreds of pages within a single conversation. Any text beyond that limit falls outside the model's attention.
Hallucination is when a model produces information that looks plausible but is actually incorrect or entirely made up, and presents it with complete confidence. This happens because the model generates statistically likely text rather than verified facts. That is why you should always review the output for sensitive tasks such as medicine, law, or anything involving numbers.
| Term | Plain meaning |
|---|---|
| Token | The smallest unit of text the model processes; may be a word or part of one. |
| Parameter | An internal numerical value tuned during training that stores what the model learned. |
| Context window | How much text the model handles at once, measured in tokens. |
| Fine-tuning | Refining a pretrained model on specific data to improve its performance. |
| Hallucination | The model producing false or invented information stated with confidence. |
Use Cases and Their Limits
Large language models are used across many fields: drafting text and summarizing long documents, translating between languages, helping programmers write and debug code, powering customer-service chatbots, and searching knowledge bases. But they have clear limits. They do not "understand" the world the way a person does, they can carry biases present in their training data, their knowledge is frozen at their last training date unless connected to external search tools, and running them at scale can be costly. The golden rule is to treat an LLM as a powerful assistant that needs human oversight, not as a final source of truth.