Television has created a very unrealistic perception of facial recognition technology that many interpret as reality. Big Brother is watching you with a camera, all crimes are solved in an hour, and a satellite camera positioned in the far reaches of outer space can produce an instantaneous facial recognition match. Throw in some special effects and we immediately have someone’s birth certificate, high school graduation photos, and know he likes to eat pizza on Thursday nights. A misrepresentation of a powerful technology that is proven very effective when used properly. Understanding this technology through research, education, and communication often sways the misguided perceptions of many. Unfortunately, a great deal still remains prevalent, even in higher levels of society: those holding leadership positions at the state and national levels of government.

Just last month, the Government Accountability Office (GAO) criticized the FBI’s “facial recognition database” for violating civil liberties and privacy. The GAO fears are really based on misinformation and a misunderstanding of what happens when law enforcement agencies use this technology every day. The report states facial recognition technology raises potential concerns as an effective tool for aiding law enforcement investigations, the protection of privacy, and individual civil liberties.

As a detective who was assigned to the New York Police Department’s (NYPD’s) Real Time Crime Center, technology and data were two critical components to any investigation. We quickly learned that facial recognition was simply a technology tool. Not an absolute science like DNA but a biometric technology which could be used to identify unknown suspects and generate leads. Facial recognition technology, in particular, is not a smoking gun. It’s only one component in the investigative process that detectives can use to help close cases faster, solve crimes effectively, and ensure public safety.

What exactly can facial recognition do?

As I go across the country, or travel to various parts of the world discussing facial recognition technology, I have found it mission critical to first ensure that there is a full understanding of what facial recognition is, and clearly define what it is not. To show those in attendance by example what it can, and cannot do. Because of the misguided Hollywood perception that dominates this space, I always remind everyone this is nothing like the movie Minority Report or the TV show CSI. It’s a hands on process which requires more than just a computer application to find a Possible Match. It still requires a fair amount of good old fashioned police work.

It is really about matching photos or sketches of unknown suspects to galleries of known faces in photos – in our case mug shots – to try and find an investigative lead that will enable us to ID a perpetrator in a timely manner. The photos or sketches of suspects received are often blurry or grainy, requiring a significant amount of enhancement work to make them usable. It’s not so definitive and greatly lacks the mystic Hollywood makes it out to be.

Facial Recognition vs. Facial Identification

There are two distinct processes when it comes to using faces for investigations: facial recognition and facial identification. Facial recognition is the automated process that enables investigators to compare a probe photo against their entire photo database and helps reduce thousands of possibilities to dozens, and dozens to one. Yes, I said one. As a matter of fact, it should always be one single candidate match. Upon search, many candidates are usually returned depending on the parameters set by the user.

But how does an analyst sift through millions images and find a face? Apply search filters. Most facial recognition applications have filtering capabilities and allow an analyst to constrict searches down to levels of specificity. Once, the large mugshot gallery or “haystack of images” is reduced to a few hundred “bundles” of possibilities, analysts must undertake the critical aspect of this process known as facial identification. Here, the analyst physically inspects each face in the gallery and looks for physical similarities and differences between the probe and each of the returned candidates. This process allows for higher probabilities of success in locating a possible candidate match that resides in the returned list because the reliance is not left solely on the software. Hoping to find that single “needle”, that one face in the gallery, requires an effort from the analyst to carefully review each face.

Tedious and time consuming to say the least, but essential to the investigation. This is the differentiator for higher accuracy rates and makes for a credible and successful facial recognition program. Implementing this into a facial recognition workflow, makes it easier to obtain one possible candidate match from thousands, or even millions of faces in a gallery.

Most importantly, it is critical to remember that facial recognition technology can ONLY be used to aid in the investigation. It is not considered “sufficiently reliable to be admissible at trial,” according to one New York State Supreme Court ruling. Arrests cannot be made solely based on a facial recognition possible match. The onus still falls on the agency to establish probable cause for an arrest.

What about the false positives?

Often, you will hear about the fear of false positives routinely being generated in facial searches. This was also cited in the GAO FBI Face Recognition Technology Report and a major focal point of criticism. While it may sound interesting and plausible, it is not. If the facial recognition application returns faces which are similar, why wouldn’t an agency implement a process to physically inspect each face in the returned gallery?

The report states the FBI allows law enforcement agencies to request between 2 and 50 photos be returned from any NGI-IPS face recognition search. The report further states this is a completely automated process with no human analysis. Is the criticism by the GAO justified? Yes. The reason for higher rates of false positives is because there is no physical inspection performed by an analyst. Another reason for high rates of false positives is because the list of candidates set by the FBI is far too low at 50.

When searching against galleries that are in the millions, many factors prevent algorithms from reading images effectively. More faces mean more similarities to cull through. If a submitted image is high quality and meets all the necessary requirements of a good probe in pose, lighting, and expression, then a return in a Top 50 is likely. But what happens when images are less than ideal by facial recognition standards?

Because of this, these images will never return in a Top 50 ranking. In order for this to work correctly, the gallery must be smaller and the returned list of candidates should be set higher I always set my returns between 250 to 500 candidates. When images are less than ideal with regard to pose, lighting, etc. the possible candidate most often resides deeper in the returned list. So setting a return at 50 is counter-productive because in a gallery of hundreds, the probe to candidate match may be found at rank 51, 150, or 500.

Every image is different and algorithms read each face uniquely and will interpret results based on the number of faces it is searching against. Because of this, there is a definite need for the analyst to individually examine each face, and once a subject has been identified as a possible through physical attributes, an immediate background investigation should be performed to validate the candidate as a viable suspect in the investigation. Once the physical characteristics and background checks match up, a possible match report is provided by the agency to verify the single facial recognition match. A properly constructed possible match report document will clearly state in sum and substance, that probable cause must be established by other means to effect an arrest. No arrest can be made solely because of the facial recognition possible match report.

Automating a manual task.

As is the case with most forms of newer technology, facial recognition is not necessarily helping us do something different. It’s simply enabling us to do something better. What was once a manual drawn out process of viewing mugshot images to determine someone’s identity, or conducting neighborhood canvasses knocking on doors to determine someone’s true identity from a photo, facial recognition has streamlined the process by returning investigative results quicker and efficiently. We are not re-writing the rules of law enforcement or privacy rights. Just generating valuable leads which lead to identifications which are critical for criminal investigations.

It is more important to not view facial recognition as an absolute scientific breakthrough or a tool which infringes on our constitutional right to privacy. Rather, to view it as a valuable resource in our lives, as we continue to live within the parameters set forth by our forefathers when they wrote the U.S. Constitution. We now simply have better tools to capture the criminals who are preventing all of us from pursuing life, liberty and the pursuit of happiness.

Roger Rodriguez
Roger Rodriguez joined Vigilant Solutions after serving over twenty years with the NYPD where he spearheaded the NYPD’s first dedicated facial recognition unit and helped start up the Real Time Crime Center. Both are recognized as world models in law enforcement data analytics and facial recognition used in criminal investigations. Today, Roger drives the Facial Recognition, License Plate Reader, and Mobile Companion product lines for Vigilant Solutions as Director of Business Development. As subject matter expert and author, he shares his experiences through thought leadership presentations, media interviews, publications, and hundreds of law enforcement agencies around the world have benefitted from them.
PreviousRetaining Images of Lower Quality for Facial Recognition
NextThe Ferguson Effect. Is It Real?