When your phone unlocks the instant it sees your face, it isn’t comparing photos the way you would. It’s converting the geometry of your face into a string of numbers and checking whether that number matches the one it already has on file — a process that has nothing to do with recognizing you the way a human does.

Step One: Finding a Face

Before any identification can happen, the system first has to detect that a face is present at all and isolate it from the background. Detection algorithms scan an image looking for face-like patterns, then crop tightly around just the face itself, discarding hair, clothing, and background clutter that would otherwise confuse later steps.

Mapping the Landmarks

Once a face is isolated, the software identifies dozens of specific reference points on it — many systems use around 68 facial landmarks marking the corners of the eyes, the tip of the nose, the edges of the mouth, and the jawline. These landmarks let the software normalize the face: rotating, scaling, and aligning it so the same person looks mathematically similar whether they’re facing the camera head-on or turned slightly to one side.

Turning a Face Into a Number

The aligned face is then fed into a deep neural network trained specifically to produce a faceprint, a numerical vector, often 128 numbers long, that represents the geometry and texture of that face in a mathematical space. Crucially, the network isn’t trained to output a name or a category. It’s trained using a technique called triplet loss, shown two photos of the same person and one photo of someone else at a time, and adjusted, millions of repetitions later, so that two photos of the same person always land close together in that mathematical space while photos of different people land far apart.

Matching, Not Recognizing

Identification then becomes simple arithmetic: the system measures the mathematical distance between a new face’s vector and the vectors already stored in a database, and if the distance falls below a set threshold, it calls that a match. This is why systems compare a “confidence score,” not a certainty — lighting, angle, aging, glasses, and image quality all nudge a face’s vector slightly, and a system tuned too loosely will misidentify strangers as matches, while one tuned too strictly will fail to recognize the right person.

Accuracy Isn’t Even Across Faces

A landmark 2019 study by the National Institute of Standards and Technology tested 189 algorithms from 99 developers against more than 18 million images and found that most facial recognition algorithms perform measurably worse on some demographic groups than others. Depending on the algorithm, false-positive rates for Asian and African American faces were found to be 10 to 100 times higher than for white faces in one-to-one verification tests, though the researchers noted that some of the most accurate algorithms overall were also among the most demographically consistent — evidence that accuracy and fairness aren’t necessarily a tradeoff, but that plenty of widely deployed systems haven’t gotten there yet.