Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Classification (Machine Learning)

nounMachine Learningalso classification in machine learning, classifier, categorization

In one line

Classification is a supervised machine learning task in which a model assigns each input to one of several predefined categories, such as spam or not spam.

Classification is a task in supervised learning in which a model learns to assign each input to one of several predefined categories, called classes. Deciding whether an email is spam, whether a tumor is benign or malignant, which digit a handwritten character represents or whether a customer will cancel a subscription are all classification problems. The model learns from labeled examples and then predicts the class of new items. Classification is one of the two main kinds of supervised prediction, along with regression, which predicts numbers instead of categories.

How classification works

A classification system begins with a dataset in which each example has features and a class label. The learning algorithm fits a model that maps features to classes. Many models do not simply output a class. They produce a score or probability for each class, and the class with the highest probability is chosen. Where the line is drawn between classes is called the decision boundary. A good model places it so that most examples fall on the correct side, while still working on new data. The final model is tested on examples held out from training to estimate its real-world accuracy.

Types of classification problems

  • Binary classification: two classes, such as fraud or not fraud, positive or negative.
  • Multiclass classification: more than two classes, where each item belongs to exactly one, such as recognizing which of ten digits is shown.
  • Multilabel classification: each item can have several labels at once, as when a news article is tagged with politics, economy and Europe.
  • Imbalanced classification: one class is far rarer than the others, as with fraud or rare diseases, and needs special handling.
  • Hierarchical classification: classes are organized in a tree, such as a product taxonomy.

Common classification algorithms

Many algorithms can classify, and each has strengths.

  • Logistic regression: a simple, fast and interpretable model that estimates the probability of a class. Despite its name, it is a classifier.
  • Decision trees and random forests: split data by a series of questions, handle mixed data types and give good accuracy, especially in ensembles.
  • Support vector machines: find the boundary with the widest margin between classes.
  • k-nearest neighbors: label an item by the majority class among the most similar training examples.
  • Naive Bayes: applies Bayes theorem with a simplifying assumption of independent features, and works well for text.
  • Neural networks: learn complex patterns, and dominate in images, speech and language.
  • Gradient boosting: builds many small trees in sequence and is a leading method for tabular data.

Evaluating a classifier

Accuracy, the share of correct predictions, is the simplest measure, but it can mislead. If only one percent of transactions are fraud, a model that always predicts not fraud is ninety-nine percent accurate and useless. A confusion matrix breaks results into true positives, false positives, true negatives and false negatives. From it come more informative measures.

  • Precision: of the items predicted positive, how many really are.
  • Recall, or sensitivity: of the truly positive items, how many were found.
  • F1 score: the harmonic mean of precision and recall.
  • ROC curve and AUC: show the trade-off between catching positives and raising false alarms across all thresholds.

The right measure depends on the cost of errors. A cancer screening test should have high recall, since missing a case is costly, while a spam filter should have high precision, since hiding a real email is harmful.

Decision thresholds and calibration

Most classifiers output a probability, and a threshold turns it into a decision. Moving the threshold trades one kind of error for another. Lowering it catches more positives but also raises false alarms. Calibration checks whether predicted probabilities match real frequencies, so that items scored at eighty percent are positive about eighty percent of the time. Calibrated probabilities matter when decisions depend on risk, as in lending or medicine.

Handling imbalanced data

When one class is rare, models tend to ignore it. Remedies include collecting more examples of the rare class, oversampling it or generating synthetic examples with methods such as SMOTE, undersampling the common class, giving errors on the rare class a higher weight in the loss and choosing metrics such as precision, recall and area under the precision-recall curve in place of accuracy.

Applications of classification

Classification is used in email filtering, medical diagnosis from images and tests, credit approval and default prediction, fraud detection, sentiment analysis of reviews, document and topic tagging, image recognition, speech command recognition, quality inspection in factories, customer churn prediction and species identification. In many products, a classifier is a small part of a larger pipeline, with humans reviewing uncertain cases.

Advantages and challenges

Classification is well understood, measurable and widely supported by tools, and it suits many practical decisions. Challenges include the need for good labeled data, class imbalance, label noise, overfitting, changing data over time and fairness. A classifier trained on biased data can treat groups unequally, so teams check error rates by group and document the model’s intended use and limits.

A worked example

A bank builds a model to flag fraudulent card payments. It collects a year of transactions labeled by investigators and creates features such as amount, merchant type, time since the last purchase and distance from home. Because fraud is rare, the team weights fraud cases more heavily and judges the model by recall at a fixed false alarm rate. A gradient boosting model is trained and tested on the most recent month. The bank sets the threshold so that analysts receive a manageable number of alerts each day, and it retrains regularly as fraud patterns change.

Classification in deep learning and language models

Neural networks classify by passing an input through layers of learned features and ending with a layer that outputs one probability per class, usually through a softmax function. Convolutional networks classify images, and transformer models classify text, from sentiment to legal topics. Large language models can also classify with no training examples when given instructions in a prompt, as in zero-shot classification. Whatever the method, the same discipline applies: split the data properly, choose metrics that match the cost of errors and test on data that resemble real use.

Common mistakes to avoid

Frequent errors include judging a model by accuracy alone on imbalanced data, letting information from the test set leak into training, ignoring changes in the data after deployment and failing to review the model for bias across groups. Keeping a simple baseline, such as logistic regression, helps show whether a complex model is truly better.

FAQs

What is the difference between classification and clustering? Classification uses labeled examples and predefined classes. Clustering finds groups in unlabeled data without predefined classes.

Is logistic regression regression or classification? It is used for classification, because it predicts class probabilities, although it is named after the underlying regression technique.

Which metric should I use? It depends on the cost of mistakes. Precision, recall and F1 are better than accuracy when classes are imbalanced.

Key facts

  • Classification is a supervised learning task in which the output is a discrete category label.1
  • Common evaluation measures include accuracy, precision, recall, the F1 score and the area under the ROC curve.2
  • Classification can be binary, multiclass or multilabel.1

Related concepts

In the news

Quick checkWhat is the difference between classification and regression?Show answer

Classification predicts a category, such as spam or not spam, while regression predicts a continuous number, such as a price.

Sources

  1. scikit-learn, Supervised learning and classification, https://scikit-learn.org/stable/supervised_learning.html
  2. Google, Machine Learning Crash Course: Classification, https://developers.google.com/machine-learning/crash-course/classification
  3. Wikipedia, Statistical classification, https://en.wikipedia.org/wiki/Statistical_classification

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue