Enhancing Images to Generate Investigative Leads
The facial recognition process is made very easy for an analyst or investigator when the probe image is “controlled” and meets the criteria for searching. Controlled by definition in facial recognition terminology means the image and face are ideal for searching against a gallery of images. The subject in the captured image has a frontal pose, no expression, facial features are clearly visible and unobstructed, the capture distance is not far from the lens, and lighting is very adequate. The image overall is ideal for enrollment into the facial recognition software and a candidate will most certainly return if they reside in the mugshot gallery. However, the world of public safety faces many challenges – no pun intended. Labeled as “faces in the wild” or “uncontrolled images”, these image types remain prevalent in the public safety space because criminals do not pose for photographs when committing crimes. Most faces captured by CCTV cameras, or other similar settings are less than ideal for facial recognition and prove to be extremely frustrating for detectives, investigators, and analysts.
As someone who conducted thousands of facial recognition searches during my years as a detective at the NYPD, I lived that frustration first hand. Initially, when we ventured into the new world of facial recognition we solely relied on the software to return a possible match. It became routine for investigators to reject images of lower quality if the probe image did not meet the criteria for facial recognition searching. We looked at the image, rejected the request, and very quickly moved on to the next case.
Image Quality for Facial Recognition
During our investigations we soon realized that most facial recognition systems had problems recognizing differences in lighting, pose, facial expressions, and preferred higher image quality. We desperately sought a way to leverage facial recognition technology on lower quality images to generate more leads. For us, it all started with basic image editing software. When probe images being introduced for enrollment were too dark, we made them brighter. If probe images were heavily pixelated, we slightly blurred the pixels making the photo appear smoother and the face easier to see within the image. If the subject stood at a distance, we would crop the face and enlarge the overall size of the cropped image. Very often, basic image editing enhancements made to less than ideal images became standard in our workflow. These enhanced images were then reintroduced into the facial recognition software. Doing so, produced many possible match results from lower quality probe images which would normally be rejected. Why? We simply searched our mugshot gallery system with a “better” probe image.
After four years of operation our metrics indicated at least 80% of our probe images were identified as “uncontrolled” and required some type of image processing before any facial recognition analysis. This was a critical component to our operation as a dedicated facial recognition unit in public safety. Enhancements made on lower quality probes always increased our success rate for a return of a possible match from our mugshot gallery. Identifying problematic images and leveraging technology was critical to the success of our facial recognition unit.
The triage process also identified certain images which had to be rejected because they did not meet the criteria for enrollment into the facial recognition system. These images had faces which appeared distorted or skewed. Most captured by a “fisheye” lens cameras found in bank ATM machines, CCTV cameras, and antiquated camera systems. Many unacceptable facial images displayed occlusion (blocking any part of the face) with hats, hoods, scarves, hands, hair, or shadows would cover select areas of a face. Profile images, or severely angled poses of faces looking down or up were also classified as unacceptable for searching against our mugshot system. However, no request was ever discarded and every case was prioritized. From a homicide investigation down to a petit larceny investigation, every image submitted for analysis was triaged in our small office. As the caseload grew and more requests arrived daily, we opted to use the external image editing software on these types of challenging probe images.
As the years progressed, the members of our dedicated unit became very proficient in identifying, classifying, and enhancing images for facial recognition. We knew when to apply specific image enhancements using external image editing software. We learned to leverage both technologies and often made something from nothing. What was deemed a reject image for facial recognition was now a “face” we could work with by making a few image enhancements.
Faces which were distorted or skewed were brought back to scale. We “stamped or painted out”, the occlusion found on facial images. Profiled faces were imported into the facial recognition software, corrected with pose correction tools, exported into our external image editing software, and imported back into the facial recognition application for searching. A very hands-on, laborious, time consuming process to say the least, but extremely effective for retaining valuable images of lower quality for facial recognition investigations. As a result, many investigations were closed because possible matches were made from low quality images.
I realize not every agency has the luxury of a dedicated staff for facial recognition. I also realize not everyone has proficiency in graphic design image editing software. There are others who may find navigation of applications very cumbersome. More importantly, as a former detective who worked many investigations, there are those who simply do not have the time to do this in the real world settings of the investigative process. Time is a valuable commodity and it is often devoted to other avenues in the investigation.
The Vigilant Solution
Because of this, Vigilant has developed a breakthrough in facial recognition for public safety. We have added a set of specialized image editing tools within FaceSearch™ which enable any user to enhance images of lower quality without requiring an extensive understanding of image editing software. More importantly, these tools are easy to learn and use. Drawn from my real life user experiences in facial recognition investigations, many of the time consuming image editing processes have been automated and scaled down for an easy user experience. FaceSearch™ is designed with the end user in mind.
The detective, investigator, or analyst can now edit lower quality images with just a few clicks of a mouse, or sliding a navigation bar left or right. An easy to use interface replicating the functionality of what many external and costly image editing software applications can do today. Lower quality images can now be retained for your investigations and anyone in your agency can use the tools to return a possible match!
To learn more specifics on how these images are enhanced be sure to read my complete whitepaper, which also includes tips and tricks for pre-processing images. You will also learn best practices for facial recognition and how your agency can leverage this technology as a lead generation tool with great success!