When your phone unlocks the instant it sees your face, it is not comparing photos the way a person would. It is converting the geometry of your face into a string of numbers and checking whether that number is close enough to the one already on file — a process that has almost nothing in common with how a human actually recognizes someone.

Step One: Finding the Face

Before any identification can happen, the system first has to detect that a face is present at all and isolate it from everything around it. Detection algorithms scan an image looking for face-like patterns, then crop tightly around just the face itself, discarding hair, clothing, and background clutter that would otherwise confuse the steps that come next.

Mapping the Landmarks

Once a face is isolated, the software identifies dozens of specific reference points on it — many systems use around 68 facial landmarks marking the corners of the eyes, the tip of the nose, the edges of the mouth, and the jawline. These landmarks let the software normalize the face: rotating, scaling, and aligning it so the same person looks mathematically similar whether they are facing the camera head-on or turned slightly to one side. Skip this step and the same face photographed at two different angles could look like two different people to everything downstream.

Turning a Face Into a Number

The aligned face is then fed into a deep neural network (software loosely modeled on how layers of neurons process information, trained here on millions of face images rather than programmed with explicit rules) built specifically to produce a faceprint — a numerical vector, often 128 numbers long, that represents the geometry and texture of that face as a point in mathematical space. Crucially, the network is not trained to output a name or a category. It is trained using a technique called triplet loss: shown two photos of the same person and one photo of someone else at a time, then adjusted, millions of repetitions later, so that two photos of the same person always land close together in that mathematical space while photos of different people land far apart.

Matching, Not Recognizing

Identification then becomes simple arithmetic: the system measures the mathematical distance between a new face’s vector and the vectors already stored in a database, and if the distance falls below a set threshold, it calls that a match. This is why these systems report a “confidence score” rather than a certainty — lighting, angle, aging, glasses, and image quality all nudge a face’s vector slightly, and a system tuned too loosely will misidentify strangers as matches, while one tuned too strictly will fail to recognize the right person.

Accuracy Is Not Even Across Faces

That threshold matters because accuracy is not uniform. A landmark 2019 study by the National Institute of Standards and Technology tested 189 algorithms from 99 developers against more than 18 million images and found that most facial recognition algorithms perform measurably worse on some demographic groups than others. Depending on the algorithm, false-positive rates for Asian and African American faces were found to be 10 to 100 times higher than for white faces in one-to-one verification tests. The practical consequence is real: a system with an uneven false-positive rate is more likely to wrongly flag people from the groups it performs worse on. The researchers did note that some of the most accurate algorithms overall were also among the most demographically consistent, evidence that accuracy and fairness are not necessarily a tradeoff — but plenty of widely deployed systems have not gotten there yet.