Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Confusion Matrix

nounAI Evaluation and Reliabilityalso error matrix, contingency table, classification matrix

In one line

A confusion matrix is a table that compares a classification model's predicted labels with the actual labels, showing counts of true positives, false positives, true negatives and false negatives, from which most evaluation metrics are calculated.

A confusion matrix is a table that shows how a classification model’s predictions compare with the true labels. Each row (or column, depending on convention) represents the actual class, and each column (or row) represents the predicted class. The cells count how many examples fell into each combination. It is called a confusion matrix because it shows where the model gets confused, for instance mistaking cats for dogs. It is one of the most useful tools in model evaluation, because nearly every common metric, including accuracy, precision, recall and the F1 score, can be calculated from it [1].

The binary case

For a two-class problem, with a “positive” class of interest and a “negative” class, the matrix has four cells [1].

  • True positive (TP). The item is positive, and the model predicted positive. Correct.
  • False positive (FP). The item is negative, but the model predicted positive. Also called a false alarm or Type I error.
  • False negative (FN). The item is positive, but the model predicted negative. A miss, or Type II error.
  • True negative (TN). The item is negative, and the model predicted negative. Correct.

A mnemonic: the second word is what the model predicted, and the first word says whether that prediction was right.

A worked example

A model screens 1,000 emails for spam. There are 200 spam emails and 800 legitimate ones. The model catches 170 spam emails (TP = 170), misses 30 (FN = 30), wrongly flags 40 legitimate emails (FP = 40) and correctly passes 760 legitimate emails (TN = 760).

| | Predicted spam | Predicted not spam | |—|—|—| | Actual spam | 170 | 30 | | Actual not spam | 40 | 760 |

From these four numbers:

  • Accuracy = (170 + 760) / 1,000 = 93 percent.
  • Precision = 170 / (170 + 40), about 81 percent.
  • Recall = 170 / (170 + 30) = 85 percent.
  • Specificity = 760 / (760 + 40) = 95 percent.
  • F1 = 2 x 0.81 x 0.85 / (0.81 + 0.85), about 0.83.

The matrix shows at once that most errors are misses of spam and some legitimate emails are blocked, information a single accuracy figure would not reveal.

Metrics from the matrix

  • Accuracy: (TP + TN) / all.
  • Precision (positive predictive value): TP / (TP + FP).
  • Recall (sensitivity, true positive rate): TP / (TP + FN).
  • Specificity (true negative rate): TN / (TN + FP).
  • False positive rate: FP / (FP + TN), the x-axis of the ROC curve.
  • Negative predictive value: TN / (TN + FN).
  • F1 score: 2TP / (2TP + FP + FN).
  • Matthews correlation coefficient: a balanced single score using all four cells.

Multiclass confusion matrices

With more than two classes, the matrix has one row and one column for each class. The diagonal holds correct predictions, and each off-diagonal cell shows how often one class was mistaken for another [2]. For a digit recognizer, the cell where actual 4 meets predicted 9 might be large, revealing that these digits look alike. For each class, precision and recall are computed by treating it as positive and all others as negative, and then averaged by macro, micro or weighted methods.

Normalization and visualization

Raw counts can be hard to compare when classes differ in size, so matrices are often normalized. Dividing each row by its total shows, for every actual class, the share predicted as each class, so the diagonal equals per-class recall. Dividing each column by its total shows precision. Matrices are usually drawn as heatmaps, with darker cells for larger values, which makes patterns of confusion easy to spot. Libraries such as scikit-learn and many plotting tools produce them in a line of code.

Reading a confusion matrix

  1. Check the diagonal for overall success.
  2. Look for large off-diagonal cells, which show systematic confusions and often point to similar classes, label errors or missing training examples.
  3. Compare rows to see which actual classes are hard to recognize, which relates to recall.
  4. Compare columns to see which predictions are unreliable, which relates to precision.
  5. Consider the costs: a false negative and a false positive rarely cost the same.
  6. Examine examples from the biggest error cells to understand causes.

Thresholds and the matrix

For a model that outputs probabilities, the confusion matrix depends on the threshold. Lowering it turns false negatives into true positives and true negatives into false positives. A different matrix therefore exists for each threshold, and ROC and precision-recall curves summarize all of them.

Uses beyond classification

Limitations

A confusion matrix needs labeled data and a fixed threshold, can be unwieldy with hundreds of classes, and gives counts that depend on the test set’s class mix. For ranking and probability quality, other tools are needed.

Cost-sensitive use of the matrix

The four cells can be given prices. Suppose a bank estimates that a missed fraud (false negative) costs 500 dollars on average, a false alarm (false positive) costs 5 dollars in analyst time and a correct decision costs nothing. Multiplying each cell count by its cost gives the total expected cost of the model at a given threshold. The team can then pick the threshold with the lowest total cost instead of the highest accuracy. This shift, from counting errors to pricing them, turns the confusion matrix into a business tool and often leads to a very different operating point than the default of 0.5.

Common mistakes in reading it

One frequent mistake is mixing up the orientation. Some libraries put actual classes in rows and predicted classes in columns, and others do the opposite, so the labels on the axes must always be checked before interpreting the cells. Another is comparing matrices of different test sets using raw counts, when normalized values are needed. A third is forgetting that the matrix describes one threshold and one data set: a model that looks good on a balanced test set may produce a very different matrix on real traffic where positives are rare. Finally, a matrix computed on training data hides overfitting, so it should be built from testing data the model has never seen.

FAQs

What is a confusion matrix? It is a table comparing a classifier’s predictions with actual labels, showing true and false positives and negatives.

What are false positives and false negatives? A false positive is a negative item wrongly predicted positive. A false negative is a positive item the model missed.

How do you calculate accuracy from a confusion matrix? Add the correct predictions, TP and TN, and divide by the total number of examples.

What does the diagonal of a confusion matrix show? The correct predictions for each class.

Can a confusion matrix be used for more than two classes? Yes, it has one row and column per class, and shows which classes are confused.

More in AI Evaluation and Reliability

Key facts

  • A confusion matrix for binary classification has four cells: true positives, false positives, false negatives and true negatives1.
  • Accuracy, precision, recall, specificity and F1 can all be computed from the four counts of a confusion matrix1.
  • For multiclass problems, the matrix has one row and one column per class, and off-diagonal cells show which classes are being confused with each other2.

Related concepts

Quick checkWhat is a confusion matrix?Show answer

A confusion matrix is a table that compares a classification model's predicted labels with the actual labels, showing the counts of true positives, false positives, true negatives and false negatives, from which most evaluation metrics are calculated.

Sources

  1. Tom Fawcett, An Introduction to ROC Analysis, Pattern Recognition Letters, https://doi.org/10.1016/j.patrec.2005.10.010
  2. Marina Sokolova and Guy Lapalme, A Systematic Analysis of Performance Measures for Classification Tasks, Information Processing and Management, https://doi.org/10.1016/j.ipm.2009.03.002

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue