Text generation is the task of producing written language with a computer, whether a short reply, a summary, a story, a translation or a piece of software code. In modern systems it is done by a language model that predicts the next small piece of text, called a token, given everything written so far, and repeats this step until the passage is complete. It is the core capability behind chatbots, writing assistants, machine translation and code assistants, and it is the best-known form of generative AI. In the field of natural language processing, it is often called natural language generation.
History of text generation
The earliest systems used templates, in which a program fills blanks in a fixed sentence pattern, as in weather reports and form letters. In 1966, Joseph Weizenbaum’s ELIZA generated replies by rewriting the user’s own words. Statistical language models, such as n-grams, arrived in the 1980s and 1990s and predicted the next word from the previous few words. In the 2010s, recurrent neural networks and LSTMs produced more fluent text, and sequence-to-sequence models improved translation and summarization. The transformer architecture of 2017 changed everything. Models such as GPT-2 in 2019 and GPT-3 in 2020 showed that very large transformers trained on vast text collections could write coherent essays, answer questions and follow instructions. ChatGPT, released in November 2022, brought text generation to a mass audience.
How text generation works
The process has four main steps.
- Tokenization: the input text is split into tokens, which may be words or pieces of words, and each is mapped to a number. See tokenization.
- Prediction: the model, usually a transformer using self-attention, reads all the tokens so far and outputs a probability for every token in its vocabulary as the possible next one.
- Selection: one token is chosen from this distribution, by a rule described below.
- Repetition: the chosen token is appended to the text and the process repeats, one token at a time, until the model produces a stop signal or reaches a length limit.
The quality of the output depends on how well the model was trained, on the prompt and on the selection rule.
Decoding and sampling methods
How the next token is chosen strongly affects the style of the text.
- Greedy decoding: always take the most probable token. It is predictable but can become repetitive and dull.
- Beam search: keeps several promising sequences at once and picks the best overall. It works well for translation and summarization, where one right answer is expected, but can produce bland text for open-ended writing.
- Random sampling with temperature: draws a token at random in proportion to its probability. A low temperature makes the choice more conservative, and a high temperature makes it more varied and risky.
- Top-k sampling: samples only from the k most probable tokens. See top-k sampling.
- Top-p, or nucleus, sampling: samples from the smallest set of tokens whose probabilities add up to p. See top-p sampling.
Most chat products use sampling with moderate settings, which gives natural and varied responses.
Training and adaptation
A text generator is first pretrained on a very large body of text to predict the next token, which teaches grammar, facts and patterns of reasoning. It is then adapted. Instruction tuning trains it on examples of prompts paired with good responses so that it follows directions. Reinforcement learning from human feedback uses human preferences to make answers more helpful and safer. Fine-tuning on specialized data, such as legal documents or medical notes, adapts the model to a domain. Retrieval methods let it consult documents at the time of writing, which improves accuracy.
Types of text generation tasks
Tasks include open-ended writing, such as stories and articles, summarization, translation, question answering, dialogue, paraphrasing and style transfer, data-to-text generation, in which tables or database records are turned into reports, and code generation. Some tasks have a fairly well-defined correct output, such as translation, while others are creative and have many acceptable answers.
Applications
Text generation is used in customer support chatbots, virtual assistants, email and document drafting, marketing copy, news summaries and automated reports in finance and sports, educational tutoring, accessibility tools such as automatic captions and alt text, software development, legal and medical documentation and creative writing support. Search engines now combine generated summaries with retrieved sources.
Evaluating generated text
Evaluation is difficult. Automatic metrics compare generated text with reference texts, such as BLEU for translation and ROUGE for summaries, though they correlate only roughly with human judgment. Newer methods use models to judge quality. Human evaluation remains the standard for fluency, relevance, factual accuracy and safety. Benchmarks test knowledge, reasoning and honesty.
Limitations and risks
- Hallucination: models can state false information confidently, invent citations or misreport details, because they predict plausible text and do not look up facts by default.
- Bias: models reflect biases in their training data, which can show up in stereotypes or unequal treatment.
- Misuse: the technology can create spam, phishing and misinformation at scale.
- Plagiarism and copyright: questions remain about training data and about close reproduction of existing work.
- Privacy: models may reveal memorized personal data, and users may share sensitive information with services.
- Inconsistency: answers can vary between runs and can be sensitive to small changes in the prompt.
Tips for better results
Give the model a clear role, audience, goal and format. Supply relevant facts and examples. Break complex tasks into steps. Ask for sources and then check them. Lower the temperature for factual tasks and raise it for brainstorming. Treat the output as a draft, edit it for accuracy and voice and disclose AI assistance where expected.
A worked example
A user types the prompt write a two-sentence welcome message for a library. The model tokenizes it and computes probabilities for the first token, perhaps Welcome with high probability, then adds it and predicts the next, and so on. With a moderate temperature, it may produce a warm message about books and community. If the user asks again, sampling gives a different wording. The user then adds the library’s opening hours to the prompt so that the message includes accurate details, and edits the final text before posting.
FAQs
Is text generation the same as a chatbot? A chatbot is an application that uses text generation to hold conversations. Text generation is the underlying capability.
Why do generated texts sometimes contain false facts? The model predicts likely wording and does not verify facts, so it can produce confident errors known as hallucinations.
What controls how creative the text is? Sampling settings such as temperature, top-k and top-p.
More in Generative AI
Key facts
- Modern text generation uses language models that predict the next token given the tokens so far, and repeat the step to build a passage.1
- Sampling settings such as temperature, top-k and top-p control how random or conservative the output is.2
- Generated text can contain factual errors called hallucinations, so important claims must be checked.3
Related concepts
Quick checkHow does a language model generate text?Show answer
It repeatedly predicts a probability for each possible next token given the text so far, picks one, adds it to the text and repeats until the passage is finished.
Sources
- Jurafsky and Martin, Speech and Language Processing, chapter on large language models, https://web.stanford.edu/~jurafsky/slp3/
- Holtzman et al., The Curious Case of Neural Text Degeneration, https://arxiv.org/abs/1904.09751
- Ji et al., Survey of Hallucination in Natural Language Generation, https://arxiv.org/abs/2202.03629
Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue
