Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Data Leakage

nounCybersecurityalso target leakage, training-test contamination, information leakage

In one line

Data leakage in machine learning has two senses: information from outside the training set, such as test data or future values, that improperly influences model building and evaluation, and the unintended exposure of sensitive data through a model or its outputs.

Data leakage is a term used in two related ways in artificial intelligence. In the first sense, common in data science, it means that information from outside the proper training data, such as the answer itself, test examples or facts from the future, finds its way into model building, so that performance looks better in testing than it will be in real use. In the second sense, common in security and privacy, it means the unintended exposure of sensitive data through a model, its outputs, its logs or its infrastructure. Both are serious, and both are covered here. The topic connects to machine learning, AI privacy, AI cybersecurity and model evaluation.

Leakage that inflates performance

Kaufman and colleagues defined leakage as the introduction of information about the target that should not legitimately be available to a model [1]. The model appears excellent in development, but fails in production because the leaked information is no longer there.

### Target leakage

A feature contains information that would only be known after the outcome. A model to predict whether a patient has pneumonia that uses a feature recording antibiotic prescriptions has leaked the answer, since the prescription follows the diagnosis.

### Train-test contamination

Records from the testing data appear in the training data. This happens when duplicates exist, when several records from the same person or patient are split across sets, or when data are preprocessed using statistics from the whole dataset before splitting.

### Temporal leakage

Using future information to predict the past, for instance training a stock or demand model on data that includes later periods than the test period. Time series should be split by time, not at random.

### Benchmark contamination

For large language models, test questions may be present in the huge web text used for pretraining, so high scores may reflect memory rather than ability; see AI benchmark and AI evaluation.

### Hyperparameter and selection leakage

Repeatedly tuning a model against the test set makes the test results optimistic; a separate validation data set avoids this.

How common is it?

A 2023 survey of machine-learning-based science found leakage problems in papers across many fields, with some results failing to replicate once the leakage was fixed [2]. The problem is not limited to beginners, because it can be subtle and hidden in pipelines.

Detecting and preventing performance leakage

  • Split first. Divide data into training, validation and test sets before any preprocessing, feature selection or imputation, and fit transformations only on the training portion.
  • Group and time-aware splitting. Keep all records of one person together, and split time series chronologically; see cross-validation.
  • Remove duplicates. Check for near-identical items across sets.
  • Review features. Ask for each whether it would be available at prediction time.
  • Suspicion of perfect scores. Unusually high accuracy often signals leakage.
  • Feature importance checks. Look for a single feature dominating the model.
  • Holdout from real deployment. Test on fresh data collected after the model was built.
  • Documentation. Use checklists and model cards that state how data were split.

Leakage as exposure of sensitive data

In the security sense, data leakage means that sensitive information leaves its intended boundary. In AI systems this can happen in several ways.

  • Memorization. Large models may reproduce parts of their training data, including personal details, when prompted in certain ways [3].
  • User input exposure. Employees paste confidential documents into public chatbots, where they may be stored or used to improve a service.
  • Logs and caches. Prompts and outputs saved without protection.
  • Retrieval systems. A search-augmented assistant may show documents a user should not see if access controls are weak; see retrieval-augmented generation.
  • Prompt injection. Malicious text makes an assistant reveal hidden instructions or private data.
  • Model attacks. Membership inference and inversion; see membership inference attack.
  • Third-party and supply chain risks. Vendors or plug-ins handling data.

### Prevention

Use data minimization and redaction, enforce access controls inside retrieval systems, set clear policies on what staff may paste into tools, choose enterprise services with no-training and retention terms, filter outputs for personal data, test models for memorization before release, protect logs and train employees; see differential privacy.

A worked example

A data scientist builds a model to predict customer churn and gets 98 percent accuracy. A reviewer notices that one feature, account closure date, is filled in only for customers who have already left. The feature leaks the outcome. After removing it and rebuilding the pipeline so that features come only from before the prediction date, accuracy falls to 78 percent, which is realistic, and the model becomes useful for planning retention offers. Separately, the company adopts a rule that staff may not paste customer lists into public chatbots and sets up an approved tool with retention controls, closing the second kind of leakage.

Leakage in pipelines and tools

Many leaks come from convenient tooling rather than deliberate choices. Automated feature engineering may compute averages over the full dataset, cross-validation code may apply scaling before folds are separated, and notebooks may hold copies of test data that get reused. Data stores that merge several tables can bring in columns created after the event of interest. Teams can reduce these risks by building pipelines in which every step is fitted inside the training fold, keeping a locked test set that is used once, versioning datasets and recording how each was created. Code review by a colleague who was not involved in building the model catches many problems, as does a pre-registration style plan that fixes the evaluation design before looking at results.

Consequences

Performance leakage leads to wasted investment, unsafe decisions when a medical or financial model is trusted too much, and loss of credibility when published results do not replicate. Exposure leakage can lead to regulatory penalties, lawsuits, lost trade secrets and harm to the individuals whose data are revealed. Because both forms often go unseen until a failure occurs, organizations include leakage checks in AI risk management and model release reviews, and they keep records that show how data were handled.

FAQs

What is data leakage in machine learning? It is when information that should not be available during training or prediction affects a model, or when sensitive data are exposed through a model or system.

What is target leakage? A feature that contains information about the outcome that would not be known at prediction time.

How do you prevent train-test contamination? Split data before preprocessing, remove duplicates and keep related records in the same split.

Can chatbots leak data? Yes, through memorized training text, stored prompts, weak access controls in connected tools or prompt injection.

Why do models with leakage fail in production? Because the leaked information is not available, or not the same, when the model is used on real new cases.

Key facts

  • Kaufman, Rosset and Perlich's 2012 paper Leakage in Data Mining defined leakage as the introduction of information about the target that should not legitimately be available1.
  • A 2023 review by Kapoor and Narayanan found data leakage problems in papers across 17 scientific fields using machine learning2.
  • Carlini and colleagues showed in 2021 that large language models can reproduce memorized training sequences when prompted3.

Related concepts

Quick checkWhat is data leakage in machine learning?Show answer

Data leakage in machine learning has two senses: information from outside the training set, such as test data or future values, that improperly influences model building and evaluation, and the unintended exposure of sensitive data through a model or its outputs.

Sources

  1. Shachar Kaufman, Saharon Rosset and Claudia Perlich, Leakage in Data Mining: Formulation, Detection, and Avoidance, ACM Transactions on Knowledge Discovery from Data, https://dl.acm.org/doi/10.1145/2382577.2382579
  2. Sayash Kapoor and Arvind Narayanan, Leakage and the reproducibility crisis in machine-learning-based science, Patterns, https://doi.org/10.1016/j.patter.2023.100804
  3. Nicholas Carlini et al., Extracting Training Data from Large Language Models, https://arxiv.org/abs/2012.07805

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue