A support vector machine, or SVM, is a supervised machine learning algorithm that classifies data by finding the boundary that separates the classes with the widest possible margin. Among all the lines that could divide two groups of points, the SVM chooses the one that stays as far as possible from the nearest points on both sides. That wide gap tends to give better results on new data. SVMs can handle complex, curved boundaries through a technique called the kernel trick, and they were among the most successful methods in classification before deep learning took over many fields. They remain valuable for small and medium datasets.
History of support vector machines
The underlying ideas came from statistical learning theory developed by Vladimir Vapnik and Alexey Chervonenkis in the Soviet Union during the 1960s and 1970s. The linear maximal margin classifier was described in 1963. In 1992, Bernhard Boser, Isabelle Guyon and Vapnik showed how to use kernels to create nonlinear classifiers. In 1995, Corinna Cortes and Vapnik introduced the soft margin version, which tolerates errors, and the name support-vector networks. During the late 1990s and 2000s, SVMs set records on tasks such as handwritten digit recognition and text categorization.
How a support vector machine works
Imagine two classes of points on a plane. Many straight lines can separate them. The SVM picks the line, or in higher dimensions the hyperplane, that has the largest margin, the distance to the closest point of either class. The points that lie on the edge of the margin are the support vectors. Only these points determine the boundary. Points farther away could move or disappear without changing it. This makes SVMs efficient in memory, since the model stores only the support vectors, and it reflects a principle of structural risk minimization, which favors simple, well-separated solutions.
Hard margin and soft margin
A hard margin SVM demands that every point be on the correct side, which works only if the classes are perfectly separable and is sensitive to outliers. Real data are noisy, so the soft margin SVM allows some points to be inside the margin or misclassified, with a penalty. The parameter C controls the trade-off. A large C punishes errors heavily and gives a narrow margin that fits the training data closely, with a risk of overfitting. A small C allows more errors and gives a wider margin, which is more tolerant and may generalize better.
The kernel trick
When classes cannot be separated by a straight line, an SVM can map the data into a higher-dimensional space where a straight boundary exists. Computing the mapping explicitly would be costly, but the kernel trick avoids it. The algorithm needs only inner products between points, and a kernel function computes the inner product in the higher space directly from the original inputs. Common kernels include:
- Linear kernel: no mapping, good for high-dimensional sparse data such as text.
- Polynomial kernel: captures interactions up to a chosen degree.
- Radial basis function (RBF) kernel: a flexible default that creates smooth, localized boundaries, controlled by a parameter called gamma.
- Sigmoid kernel: related to neural networks, used less often.
Choosing the kernel and tuning C and gamma by cross-validation is the main work of using SVMs.
Support vector regression and other uses
Support vector regression, or SVR, applies the same ideas to predict numbers. It tries to fit a function that stays within a tube of tolerance around the data, ignoring small errors and penalizing larger ones. One-class SVMs learn the boundary around normal data and are used for anomaly detection. SVMs can handle multiple classes by training several binary classifiers, using one-versus-one or one-versus-rest strategies.
Preparing data for SVMs
SVMs are sensitive to the scale of features, because the margin is measured by distance. Standardizing each feature to a similar range is essential. Missing values must be handled, and categorical variables must be encoded. Training time grows quickly with the number of examples, roughly between quadratic and cubic, so SVMs are best on datasets up to tens of thousands of examples. For larger data, linear SVMs trained by stochastic gradient descent scale better.
Applications of support vector machines
SVMs have been used for text and spam classification, image and handwriting recognition, face detection, bioinformatics tasks such as classifying proteins and cancer types from gene expression, financial prediction, remote sensing, fault detection and speech tasks. They perform well when the number of features is large compared with the number of examples, as in genomics and text.
Advantages of SVMs
- Effective in high-dimensional spaces and with limited data.
- Memory efficient, since only support vectors are stored.
- Flexible through kernels for different types of boundaries.
- A convex optimization problem, so training finds the global optimum without getting stuck.
- Strong theoretical foundations in statistical learning theory.
Disadvantages of SVMs
- Training is slow on very large datasets.
- Performance depends on choosing the kernel and parameters carefully.
- No direct probability outputs. Probabilities require extra calibration such as Platt scaling.
- Less interpretable with nonlinear kernels.
- Can be outperformed by gradient boosting on tabular data and by deep networks on images, audio and text at large scale.
A worked example
A team classifies news articles into topics. Each article is turned into TF-IDF features, giving tens of thousands of dimensions and a few thousand examples. A linear SVM is trained, with C chosen by cross-validation. It reaches high accuracy and trains in seconds. When the team tests an RBF kernel, accuracy does not improve, which suggests that the classes are already nearly linearly separable in this feature space. They keep the simpler linear model and review the largest weights to see which words drive each topic.
SVM vs. other classifiers
Compared with logistic regression, SVMs focus on boundary points and maximize margin, while logistic regression uses all points and gives probabilities. Compared with decision trees and random forests, SVMs need scaling and give smooth boundaries. Compared with neural networks, they need less data and tuning for small problems but scale less well.
FAQs
What does SVM stand for? Support vector machine.
When should I use an SVM? When you have a small or medium dataset with many features, such as text or gene data, and want a strong classifier with a clear margin.
What is a kernel? A function that measures similarity between points as if they were mapped to a higher-dimensional space, which lets the SVM draw curved boundaries.
More in Machine Learning
Key facts
- A support vector machine chooses the decision boundary that maximizes the margin between the nearest points of each class, called support vectors.1
- The kernel trick lets SVMs find nonlinear boundaries by implicitly mapping data to a higher-dimensional space.2
- The soft margin, controlled by the parameter C, allows some misclassified points to improve generalization.2
Related concepts
Quick checkWhat are support vectors?Show answer
The training points closest to the decision boundary, which alone determine where the boundary is placed.
Sources
- scikit-learn, Support Vector Machines, https://scikit-learn.org/stable/modules/svm.html
- Cortes and Vapnik, Support-vector networks, https://link.springer.com/article/10.1007/BF00994018
- Wikipedia, Support vector machine, https://en.wikipedia.org/wiki/Support_vector_machine
Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue
