A dataset (also written data set) is an organized collection of data. In artificial intelligence and statistics, datasets are the raw material from which models are built and judged. A dataset may be a spreadsheet of customer records, a folder of photographs, millions of web pages, hours of recorded speech or readings from sensors. For machine learning, a dataset is usually divided into training data, validation data and testing data, each with its own role. The characteristics of a dataset, including its size, quality, balance and origin, set a ceiling on what any model trained on it can achieve.
Structure and formats
Datasets are organized so that software can read them. Common forms include:
- Tabular data. Rows are records, and columns are variables, such as age, income and purchase history. Stored in files such as CSV or in databases.
- Images. Collections of photographs or scans, often with labels, boxes or masks; see computer vision.
- Text. Documents, conversations, code or web pages; see natural language processing.
- Audio and video. Speech recordings, music, film and surveillance clips.
- Time series. Measurements ordered in time, such as stock prices or sensor signals.
- Graphs. Nodes and links, such as social networks or molecules.
- Multimodal data. Paired data from several types, such as images with captions.
Each item in a dataset is often called an example, instance, record or sample. The measurable properties of each are features, and the thing to be predicted, in supervised problems, is the label or target. See feature engineering.
Well-known datasets
Some datasets have shaped the history of the field.
- Iris. A small table of 150 flower measurements introduced by the statistician Ronald Fisher in 1936, still used in teaching.
- MNIST. Seventy thousand small images of handwritten digits, a long-running benchmark for early neural networks [1].
- ImageNet. More than 14 million images organized into categories from the WordNet hierarchy. Its annual challenge, which ran from 2010, saw a sharp drop in error rates in 2012 when a deep convolutional neural network won, helping to launch the deep learning era [2].
- CIFAR-10 and CIFAR-100. Small color images in 10 and 100 classes.
- COCO. Images with object outlines and captions, used for detection and segmentation; see object detection.
- Common Crawl. A huge, openly available archive of web pages that is a major source for training large language models.
- Wikipedia dumps. Free text widely used in language research.
How datasets are built
Datasets come from several sources: existing records collected for other purposes, sensors and instruments, web scraping, surveys and experiments, simulation, and purchases or licensing from data providers. Building one typically involves collection, cleaning, removing duplicates and errors, labeling, splitting and documentation.
Labeling can be done by experts, crowd workers or automated systems. Quality depends on clear instructions, agreement between annotators and review. Some labeling work is done by low-paid workers, which raises questions about labor conditions that responsible developers address. Other approaches reduce the need for manual labels, including self-supervised learning, active learning and synthetic data.
Dataset quality
A dataset should be judged on several dimensions.
- Accuracy. Are values and labels correct?
- Completeness. Are there missing values or gaps in coverage?
- Consistency. Are formats and definitions uniform?
- Representativeness. Does it reflect the people, places and conditions where the model will be used?
- Balance. Are classes or groups sufficiently represented?
- Timeliness. Does it reflect the current world?
- Size. Is there enough data for the task?
Poor datasets cause well-known failures: models that perform worse for underrepresented groups, that learn shortcuts such as background cues instead of the object of interest, or that repeat historical discrimination. See AI bias.
Documentation and governance
Good practice is to document each dataset. The idea of datasheets for datasets, proposed by Gebru and colleagues, suggests recording why the dataset was created, what it contains, how it was collected and labeled, what uses are appropriate and what limitations exist [3]. Related practices include data versioning, tracking provenance, access controls and audit trails.
Legal and ethical issues
- Privacy. Personal data is regulated by laws such as the General Data Protection Regulation in Europe. Techniques include anonymization, consent management and differential privacy.
- Copyright and licensing. Datasets may contain protected works, and each has a license that governs use. Open licenses such as Creative Commons and the Open Data Commons family clarify permissions.
- Consent and harm. Some datasets, including large collections of faces scraped from the web, have been withdrawn or criticized because the people shown did not agree to their inclusion.
- Security. Poisoning, where an attacker inserts bad examples, can corrupt models trained on open data.
Where to find datasets
Public repositories include the UCI Machine Learning Repository, Kaggle, Hugging Face Datasets, government open data portals such as data.gov, and academic archives. Researchers also publish datasets with their papers. Always check the license and documentation before use.
Datasets in the era of large models
The scale of datasets has changed dramatically. Early benchmarks held thousands of examples, while modern large language models are trained on collections containing trillions of words drawn from web pages, books, code and other sources. At this scale, manual inspection is impossible, so developers rely on automated filtering for quality and safety, deduplication, language identification and removal of personal information. Decisions made in this filtering, such as which sites count as high quality, shape what the model knows and which voices it reflects.
Two further trends stand out. First, data mixtures matter: the proportions of code, scientific text, conversation and other types affect model skills, and teams tune them with the same care as other design choices. Second, curated, smaller datasets are increasingly used for instruction tuning and alignment, where a few thousand high-quality examples can change a model’s behavior noticeably.
Dataset shift and maintenance
A dataset is a snapshot. The world changes, and so does the data that a deployed system sees, a problem known as dataset shift or drift. A fraud detector trained on last year’s transactions may miss new tactics, and a language model’s knowledge goes out of date. Organizations therefore treat datasets as living assets: they monitor incoming data against the training distribution, collect fresh examples, update labels and retrain on a schedule. Versioning each release, much like software, makes it possible to reproduce earlier results and to trace a problem to the data that caused it.
FAQs
What is a dataset? It is an organized collection of data, such as a table, set of images or body of text, used for analysis or to train and test machine learning models.
What is the difference between a dataset and a database? A dataset is a collection of data prepared for a particular analysis or model. A database is a system for storing, managing and querying data, which may hold many datasets.
What are examples of AI datasets? MNIST for handwritten digits, ImageNet for images, COCO for object detection and Common Crawl for web text.
How large should a dataset be? It depends on the task, from hundreds of rows for simple problems to billions of items for large models.
Where can I find free datasets? Sources include Kaggle, the UCI repository, Hugging Face and government open data sites.
More in AI Data and Infrastructure
Key facts
- The MNIST dataset contains 70,000 images of handwritten digits and was a standard benchmark for early neural networks1.
- ImageNet contains over 14 million images organized by the WordNet hierarchy, and its annual challenge helped launch the deep learning boom in 20122.
- Datasheets for datasets, proposed by Gebru and colleagues, recommend documenting a dataset's motivation, composition, collection process and intended uses3.
Related concepts
Quick checkWhat is a dataset?Show answer
A dataset is an organized collection of data, such as a table of records, a set of images or a body of text, used to train, validate and test machine learning models and to support analysis and research.
Sources
- Yann LeCun, Corinna Cortes and Christopher Burges, The MNIST Database of Handwritten Digits, http://yann.lecun.com/exdb/mnist/
- Jia Deng et al., ImageNet: A Large-Scale Hierarchical Image Database, IEEE Conference on Computer Vision and Pattern Recognition, https://doi.org/10.1109/CVPR.2009.5206848
- Timnit Gebru et al., Datasheets for Datasets, Communications of the ACM, https://arxiv.org/abs/1803.09010
Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue
