Tag Archives: facial recognition

Biometrics enhances accuracy without adversely impacting timeliness (DHS ICR part three)

This post is adapted from Bredemarket’s November 10, 2021 submitted comments on DHS-2021-0015-0005, Information Collection Request, Public Perceptions of Emerging Technology. See my first and second posts on the topic.

DHS asked respondents to address five questions, including this one:

(2) will this information be processed and used in a timely manner;

Here is part of my response.

I am answering this question from the perspective of a person crossing the border or boarding a plane.

During the summer of 2017, CBP conducted biometric exit facial recognition technical demonstrations with various airlines and airports throughout the country. Here, CBP Officer Michael Shamma answers a London-bound American Airlines passenger’s questions at Chicago O’Hare International Airport. Photo by Brian Bell. From https://www.cbp.gov/frontline/cbp-biometric-testing

From this perspective, you can ask whether the use of biometric technologies makes the entire process faster, or slower.

Before biometric technologies became available, a person would cross a border or board a plane either by conducting no security check at all, or by having a human conduct a manual security check using the document(s) provided by an individual.

  • Unless a person was diverted to a secondary inspection process, automatic identification of the person (excluding questions such as “What is your purpose for entering the United States?”) could be accomplished in a few seconds.
  • However, manual security checks are much less accurate than technological solutions, as will be illustrated in a future post.

With biometric technologies, it is necessary to measure both the time to acquire the biometric data (in this case a facial image) and the time to compare the acquired data against the known data for the person (from a passport, passenger manifest, or database).

  • The time to acquire biometric data continues to improve. In some cases, the biometric data can be acquired “on the move” as the person is walking toward a gate or other entry area, thus requiring no additional time from the person’s perspective.
  • The time to compare biometric data can vary. If the source of the known data (such as the passport) is with the person, then comparison can be instantaneous from the person’s perspective. If the source of the known data is a database in a remote location, then the speed of comparison depends upon many factors, including network connections and server computation times. Naturally, DHS designs its systems to minimize this time, ensuring minimal or no delay from the person’s perspective. Of course, a network or system failure can adversely affect this.

In short, biometric evaluation is as fast if not faster than manual processes (provided no network or system failure occurs), and is more accurate than human processes.

Automated Passport Control kiosks
located at international airports across
the nation streamline the passenger’s
entry into the United States. Photo Credit: 
James Tourtellotte. From https://www.cbp.gov/travel/us-citizens/apc

A world without biometric collection is a world with increased bias and less security and privacy (DHS ICR part two)

This post is adapted from Bredemarket’s November 10, 2021 submitted comments on DHS-2021-0015-0005, Information Collection Request, Public Perceptions of Emerging Technology. See yesterday’s post for additional thoughts on bias, security, and privacy.

By Cleanup by Andrew_pmk (talk · contribs); straightened and cropped by Holek (talk · contribs) – http://www.9-11commission.gov/press/911report_cover_HIGHRES.jpg, Public Domain, https://commons.wikimedia.org/w/index.php?curid=2376314

Because of many factors, including the 9/11 tragedy that spurred the organization of the Department of Homeland Security (DHS) itself, DHS has been charged to identify individuals as a part of its oversight of customs and border protection, transportation security, and investigations. There are many ways to identify individuals, including:

  • What you know, such as a password.
  • What you have, such as a passport or token.
  • What you are, such as your individual face, fingers, voice, or DNA.
  • Where you are.

Is it possible to identify an individual without use of computerized facial recognition or other biometric or AI technologies? In other words, can the “what you are” test be eliminated from DHS operations?

Some may claim that the “what you have” test is sufficient. Present a driver’s license or a passport and you’re identified.

  • However, secure documents are themselves secured by the use of biometrics, primarily facial recognition.
  • Before a passport is issued, many countries including the U.S. conduct some type of biometric test to ensure that a single person does not obtain two or more passports.
  • Similar tests are conducted before driver’s licenses and other secure documents are issued.

In addition, people attempt to forge secure documents by creating fake driver’s licenses and fake passports. Thus, all secure documents need to be evaluated, in part by confirming that the biometrics on the document match the biometrics of the person presenting the document.

In short, there is no way to remove biometric identification from the DHS identification operation. And if you did, who knows how each individual officer would judge whether a person is who they claim to be?

Thoughts on bias, security, and privacy (DHS ICR part one)

This post is adapted from Bredemarket’s November 10, 2021 submitted comments on DHS-2021-0015-0005, Information Collection Request, Public Perceptions of Emerging Technology.

From https://www.regulations.gov/comment/DHS-2021-0015-0006

The original DHS request included the following sentence in the introductory section:

AI in general and facial recognition in particular are not without public controversy, including concerns about bias, security, and privacy.

Even though this was outside of the topics specifically requiring a response, I had to respond anyway. Here’s (in part) what I said.

The topics of bias, security, and privacy deserve attention. Public misunderstandings on these topics have the capability of scuttling all of DHS’ efforts in customs and border protection, transportation security, and investigations.

Regarding bias, it is imperative upon government agencies, biometric vendors, and other interested parties (including myself as a biometric consultant) to educate and inform the public about issues relating to bias. In the interests of brevity, I will confine myself to two critical points.

  • There is a difference between identification of individuals and classification of groups of individuals.
    • The summary at the top of the Gender Shades website http://gendershades.org/ clearly frames the question asked by the study: “How well do IBM, Microsoft, and Face++ AI services guess the gender of a face?” As the study title and its summary clearly state, the study only attempted to classify the genders of faces.
    • This is a different problem than the problem addressed in customs and border protection, transportation security, and investigations applications: namely, the identification of an individual. If someone purporting to be me attempts to board a plane, DHS does not care whether I am male, female, gender fluid, or anything else related to gender. DHS only cares about my individual identity.
    • It is imperative that any discussion of bias as related to DHS purposes confine itself to the DHS use case of identification of individuals.
  • Different algorithms exhibit different levels of bias (and different types of bias) when identifying individuals.
    • While Gender Shades did not directly address this issue, it turns out that it is possible to identify differences in individual identification between different genders, races, and ages.
    • The National Institute of Standards and Technology (NIST) has conducted ongoing studies of the accuracy and performance of face recognition algorithms. In one of these tests, the FRVT 1:1 Verification Test (at the https://pages.nist.gov/frvt/html/frvt11.html URL), each tested algorithm is examined for its performance among different genders, races (with nationality used as a proxy for race), and ages.
    • While neither IBM nor Microsoft (two of the three algorithm providers studied in Gender Shades) have not submitted algorithms to the FRVT 1:1 Verification Test, over 360 1:1 algorithms have been tested by NIST.
    • It is possible to look at the data for each individual algorithm to see detailed information on the algorithm’s performance. Click on each 1:1 algorithm to see its “report card,” including demographic results.

However, even NIST tests are just that – tests. Performance of a research algorithm on a NIST test with NIST data does not guarantee the same performance of an operational algorithm in a DHS system with DHS data.

As DHS implements biometric systems for its purposes of customs and border protection, transportation security, and investigations, DHS not only needs to internally measure the overall accuracy of these systems using DHS algorithms and data, but also needs to internally measure accuracy when these demographic factors are taken into account. While even highly accurate results may not be perceived as such by the public (the anecdotal tale of a single inaccurate result may outweigh stellar statistical accuracy in the public’s mind), such accuracy measurements are essential for the DHS to ensure that it is fulfilling its mission.

Regarding security and privacy, which are intertwined in many ways, there are legitimate questions regarding how the use of biometric technologies can detract or enhance the security and privacy of individual information. (I will confine myself to technology issues, and will not comment on the societal questions regarding knowledge of an individual’s whereabouts.)

  • Data, including facial recognition vectors or templates, is stored in systems that may themselves be compromised. This is the same issue that is faced by other types of data that may be compromised, including passwords. In this regard, the security of facial recognition data is no different than the security of other data.
  • In some of the DHS use cases, it is not only necessary to store facial recognition vectors or templates, but it is also necessary to store the original facial images. These are not needed by the facial recognition algorithms themselves, but by the humans who review the results of facial algorithm comparisons. As long as we continue to place facial images on driver’s licenses, passports, visas, and other secure identity documents, the need to store these facial images will continue and cannot be avoided.
  • However, one must ensure that the storage of any personally identifiable information (including Social Security Numbers and other non-biometric data) is secure, and that the PII is only available on a need-to-know basis.
  • In some cases, the use of facial recognition technologies can actually enhance privacy. For example, take the moves by various U.S. states to replace their existing physical driver’s licenses with smartphone-based mobile driver’s licenses (mDLs). These mDL applications can be designed to only provide necessary information to those viewing the mDL.
    • When a purchase uses a physical driver’s license to buy age-restricted items such as alcohol, the store clerk viewing the license is able to see a vast amount of PII, including the purchaser’s birthdate, full name, residence address, and even height and weight. A dishonest store clerk can easily misuse this data.
    • When a purchaser uses a mobile driver’s license to buy age-restricted items, most of this information is not exposed to the store clerk viewing the license. Even the purchaser’s birthdate is not exposed; all that the store clerk sees is whether or not the purchaser is old enough to buy the restricted item (for example, over the age of 21).
    • Therefore, use of these technologies can actually enhance privacy.

I’ll be repurposing other portions of my response as new blog posts over the next several days.

Perhaps our privacy is REALLY threatened

Technology often advances more quickly than our society’s ability to deal with the ramifications of technology.

For example, President Eisenhower’s effort to improve our national defense via construction of a high-speed interstate highway system led to a number of unintended consequences, including the devastation of city downtown areas that were now being bypassed by travelers.

There are numerous other examples.

The previously unknown consequences of biometric technology

One way in which technology has outpaced society is by developing tools that unintentionally threaten individual privacy. For Bredemarket clients and potential clients, one relevant example of this is the ability to apply biometric technologies to previously recorded photographic, video, and audio content. (I won’t deal with real-time here.)

Hey, remember that time in 1969 that you were walking around in a Ku Klux Klan costume and one of your fraternity buddies took a picture of you? Back then you and your buddy had no idea that in future decades someone could capture a digital copy of that picture and share it with millions of people, and that one of those millions of people could use facial recognition software to compare the face in the picture with a known image of your face, and positively determine that you were the person parading around like a Grand Wizard.

Of course, there are also positive applications of biometric technology on older material. Perhaps biometrics could be used to identify an adoptee’s natural birth mother from an old picture. Or biometrics could be used to identify that a missing person was present in a train station on September 8, 2021 in the company of another (identified) person.

But regardless of the positive or negative use case, biometric identification provides us with unequalled capability to identify people who were previously recorded. Something that couldn’t have been imagined years and years ago.

Well, it couldn’t have been imagined by most of us, anyway.

Enter Carl Sagan (courtesy Elena’s Short Wisdom)

As a WordPress user (this blog and the Bredemarket website are hosted on WordPress), I subscribe to a number of other WordPress blogs. One of these blogs is Short Wisdom, authored by Elena. Her purpose is to collect short quotes from others that succinctly encapsulate essential truths.

Normally these quotes are of the inspirational variety, but Elena posted something today that applies to those of us concerned with technology and privacy.

This is a quote from Carl Sagan.

From https://shortwisdom.wordpress.com/2021/11/05/science-quote-by-carl-sagan-might-it-be/

“Might it be possible at some future time, when neurophysiology has advanced substantially, to reconstruct the memories or insight of someone long dead?…It would be the ultimate breach of privacy.”

The quote is taken from Broca’s Brain: Reflections on the Romance of Science, originally published in 1979.

The future is not now…yet

Obviously such technology did not exist in 1979, and doesn’t exist in 2021 either.

Even biometric identification of living people via “brain wave” biometrics isn’t substantively verified to any large degree; last month’s study only included 15 people. Big whoop.

But it’s certainly possible that this ability to reconstruct the memories and insights of the deceased could exist at some future date. Some preliminary work has already been done in this area.

If this technology ever becomes viable and the memories of the dead can be accessed, then the privacy advocates will REALLY howl.

And the already-deceased privacy advocates will be able to contribute to the conversation. Perhaps Carl Sagan himself will posthumously share some thoughts on the ongoing NIST FRVT results.

He can even use technology to sing about the results.

https://www.youtube.com/watch?v=zSgiXGELjbc

Pangiam is flying high on its acquisitions

Pangiam, the company that acquired both the Metropolitan Washington Airports Authority product veriScan and the Trueface company and product, is continuing to establish itself in the airport market (while pursuing other markets).

By Atlantacitizen – w:Image:Album 13 006.jpg, Public Domain, https://commons.wikimedia.org/w/index.php?curid=2608406

Forbes recently published this article:

Delta Airlines, the Transportation Security Administration (TSA), and a travel tech company called Pangiam have partnered up to bring facial recognition technology to the Hartsfield–Jackson Atlanta International Airport (ATL). 

As of next month, Delta SkyMiles members who use the Fly Delta app and have a TSA PreCheck membership will be able to simply look at a camera to present their “digital ID” and navigate the airport with greater ease. In this program, a customer’s identity is made up of a SkyMiles member number, passport number and Known Traveler Number.

Of course, TSA PreCheck enrollment is provided by three other companies…but I digress. (I’ll digress again in a minute.)

Forbes goes on to say that this navigation will be available at pre-airport check in (on the Fly Delta app), bag drop (via TSA PreCheck), security (again via TSA PreCheck), and the gate.

Incidentally, this illustrates how security systems from different providers build upon each other. Since I was an IDEMIA employee at the time that IDEMIA was the only company that performed TSA PreCheck enrollment, I was well aware (in my super-secret competitive intelligence role) how CLEAR touted the complementary features of TSA PreCheck in its own marketing.

From https://www.clearme.com/support/tsa-precheck/how-does-clear-work-with-tsa-precheck

Now I have no visibility into Pangiam’s internal discussions, but the company obviously has a long-term growth plan that it is executing.

So what will its next step be?

So who is Cubox?

Some people like to look at baseball statistics or movie reviews for fun.

Here at Bredemarket, we scan the latest one-to-many (identification) results from NIST’S Ongoing Face Recognition Vendor Test (FRVT).

Hey, SOMEBODY has to do it.

Dicing and slicing the FRVT tests

For those who have never looked at FRVT before, it does not merely report the accuracy results of searches against one database, but reports accuracy results for searches against eight different databases of different types and of different sizes (N).

  • Mugshot, Mugshot, N = 12000000
  • Mugshot, Mugshot, N = 1600000
  • Mugshot, Webcam, N = 1600000
  • Mugshot, Profile, N = 1600000
  • Visa, Border, N = 1600000
  • Visa, Kiosk, N = 1600000
  • Border, Border 10+YRS, N = 1600000
  • Mugshot, Mugshot 12+YRS, N = 3000000

This is actually good for the vendors who submit their biometric algorithms, because even if the algorithm performs poorly on one of the databases, it may perform wonderfully on one of the other seven. That’s how so many vendors can trumpet that their algorithm is the best. When you throw in other qualifiers such as “top five,” “best non-Chinese vendor,” and even “vastly improved,” you can see how dozens of vendors can issue “NIST says we’re the best” press releases.

Not that I knock the practice; after all, I myself have done this for years. But you need to know how to interpret these press releases, and what they’re really saying. Remember this when you read the vendor announcement toward the end of this post.

Anyway, I went to check the current results, which when you originally visit the page are sorted in the order of the fifth database, the Visa Border database. And this is what I saw this morning (October 27):

From https://pages.nist.gov/frvt/html/frvt1N.html on 10/27/2021.

For the most part, the top five for the Visa Border test contain the usual players. North Americans will be most familiar with IDEMIA and NEC, and Cloudwalk and Sensetime have been around for a while.

A new algorithm from a not-so-new provider

But I had never noticed Cubox in the NIST testing before. And the number attached to the Cubox algorithm, “000,” indicates that this is Cubox’s first submission.

And Cubox did exceptionally well, especially for a first submission.

As you can see by the superscripts attached to each numeric value, Cubox had the second most accurate algorithm for the Visa Border test, the most accurate algorithm for the Visa Kiosk test, and placed no lower than 12th in the six (of eight) tests in which it participated. Considering that 302 algorithms have been submitted over the years, that’s pretty remarkable for a first-time submission.

Well, as an ex-IDEMIA employee, my curious nature kicked in.

Who is Cubox?

I’ll start by telling you who Cubox is not. Specifically, Cubox is not CuBox the low-power computer.

The Cubox that submitted an algorithm to NIST is a South Korean firm with the website cubox.aero, self-described as “The Leading Provider in Biometrics” (aren’t they all?) with fingerprint and face solutions. Cubox competes in the access control and border control markets.

From http://www.cubox.aero/en_index.php#fproduct.

Cubox’s ten-year history and “overseas” page details its growth in its markets, and its solutions that it has provided in South Korea, Mongolia, and Vietnam.

And although Cubox hasn’t trumpeted its performance on its own website (at least in the English version; I don’t know about the Korean version), Cubox has publicized its accomplishment on a LinkedIn post.

From https://www.linkedin.com/posts/cuboxkr_nist-frvt-rank-activity-6856877642839339008-_Paf/

Why NIST tests aren’t important

But before you get excited about the NIST results from Cubox, Sensetime, or any of the algorithm providers, remember that the NIST test is just a test. NIST cautions people about this, I have cautioned people about this (see the fourth point in this post), and Mike French has also discussed this.

However, it is also important to remember that NIST does not test operational systems, but rather technology submitted as software development kits or SDKs. Sometimes these submissions are labeled as research (or just not labeled), but in reality it cannot be known if these algorithms are included in the product that an agency will ultimately receive when they purchase a biometric system. And even if they are “the same”, the operational architecture could produce different results with the same core algorithms optimized for use in a NIST study.

The very fact that test results vary between the NIST databases explicitly tells you that a number one ranking on one database does not mean that you’ll get a number one ranking on every database. And as French reminds us, when you take an operational algorithm in an operational system with a customer database, the results may be quite different.

Which is why French recommends that any government agency purchasing a biometric system should conduct its own test, with vendor operational systems (rather than test systems) loaded with the agency’s own data.

Incidentally, if your agency needs a forensic expert to help with a biometric procurement or implementation, check out the consulting services offered by French’s company, Applied Forensic Services.

And if you need help communicating the benefits of your biometric solution, check out the consulting services offered by my own company, Bredemarket. After all, I am a biometric content marketing expert.

Canada’s IRCC ITQ B7059-180321/B and the biometric proposals chess match

In a competitive bid process, one unshakable truth is that everything you do will be seen by your competitors. This affects what you as a bidder do…and don’t do.

My trip to Hartford for a 30 minute meeting

I saw this in action many years ago when I was the product manager for Motorola’s Omnitrak product (subsequently Printrak BIS, subsequently part of MorphoBIS, subsequently part of MBIS). Connecticut and Rhode Island went out to bid for an two-state automated fingerprint identification system (AFIS). As part of the request for proposal process, the state of Connecticut scheduled a bidders’ conference. This was well before online videoconferencing became popular, so if you wanted to attend this bidders’ conference, you had to physically go to Hartford, Connecticut.

The Mark Twain House in Hartford. For reasons explained in this post, I spent more time here than I did at the bidders’ conference itself. By Makemake, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=751488

So I flew from California to Connecticut to attend the conference, and other people from other companies made the trip. That morning I drove from my hotel to the site of the conference (encountering a traffic jam much worse than the usual traffic jams back home), and I and the competitors assembled and waited for the bidders’ conference to begin.

The state representative opened the floor up to questions from bidders.

Silence.

No one asked a question.

We were all eyeing each other, seeing what the other people were going to ask, and none of us were willing to tip our hands by asking a question ourselves.

Eventually one or two minor questions were asked, but the bidders’ conference ended relatively quickly.

There are a number of chess-like tactics related to what bidders do and don’t do during proposals. Perhaps some day I’ll write a Bredemarket Premium post on the topic and spill my secrets.

But for now, let’s just say that all of the bidders successfully kept their thoughts to themselves during that conference. And I got to visit a historical site, so the trip wasn’t a total waste.

And today, it’s refreshing to know that things don’t change.

When the list of interested suppliers appears to be null

Back on September 24, the Government of Canada issued an Invitation to Qualify (B7059-180321/B) for a future facial recognition system for immigration purposes. This was issued some time ago, but I didn’t hear about it until Biometric Update mentioned it this morning.

Now Bredemarket isn’t going to submit a response (even though section 2.3a says that I can), but Bredemarket can obviously help those companies that ARE submitting a response. I have a good idea who the possible players are, but to check things I went to the page of the List of Interested Suppliers to see if there were any interested suppliers that I missed. The facial recognition market is changing rapidly, so I wondered if some new names were popping up.

So what did I see when I visited the List of Interested Suppliers?

An invitation for me to become the FIRST listed interested supplier.

That’s right, NO ONE has publicly expressed interest in this bid.

A screen shot of https://buyandsell.gc.ca/procurement-data/tender-notice/PW-XS-002-39912/list-of-interested-suppliers as of the late morning (Pacific time) on Monday, October 11.

And yes, I also checked the French list; no names there either.

There could be one of three reasons for this:

  1. Potential bidders don’t know about the Invitation to Qualify. This is theoretically possible; after all, Biometric Update didn’t learn about the invitation until two weeks after it was issued.
  2. No one is interested in bidding on a major facial recognition program. Yeah, right.
  3. Multiple companies ARE interested in this bid, but none wants to tip its hand and let competitors know of its interest.

My money is on reason three.

Hey, bidders. I can keep your secret.

As you may have gathered, as of Monday October 11 I am not part of any team responding to this Invitation to Qualify.

If you are a biometric vendor who needs help in composing your response to IRCC ITQ B7059-180321/B before the November 3 due date, or in framing questions (yes, there are chess moves on that also), let me know.

I won’t tell anybody.

Under the lens…and under many other things: Ambarella

You’ll notice that while I do style myself as an expert on some things, I never claim that I know everything…because I obviously don’t.

This became clear to me when I was watching the Paravision Converge 2021 video and noticed its emphasis on optimizing Paravision’s recognition algorithms for Ambarella.

Ambarella-related announcements from https://www.paravision.ai/converge2021/.

I had never heard of Ambarella.

I should have heard of it.

Even in my own little corner of the technology world, Ambarella has made an impact:

We power a majority of the world’s police body cams.

We were the first to enable HD and UHD security with low power; we revolutionized image processing for low-light and high-contrast scenes; and we are an industry leader in next-generation AI video security solutions.

Video has been a key component of face detection, person detection, and face recognition for years. (Not really of iris recognition…yet.) In certain use cases, it’s extremely desirable to move the processing out from a centralized server system to edge devices such as body cams, smart city cameras, and road safety cameras, and Ambarella (and its software partners) optimize this processing.

In addition to professional (and consumer) security, Ambarella is also a player in automotive solutions including autonomous vehicles, non-security consumer applications, and a variety of IoT/industrial/robotics applications.

All of these markets are supported via Ambarella’s specialized chip architecture:

Our CVflow® chip architecture is based on a deep understanding of core computer vision algorithms. Unlike general-purpose CPUs and GPUs, CVflow includes a dedicated vision processing engine programmed with a high-level algorithm description, allowing our architecture to scale performance to trillions of operations per second with extremely low power consumption.

I’ve always been of the opinion that technology is moving away from specialized hardware to COTS hardware. For example, the fingerprint processing and matching that used to require high-end UNIX computers with custom processor boards in the 1990s can now be accomplished on consumer-grade smartphones.

However, the reason that these consumer-grade devices can now perform these operations is because specialized technologies have been miniaturized and optimized for incorporation into the consumer grade devices, such as Yi home video cameras.

From https://www2.yitechnology.com/yi-1080p-home-camera-2

Ambarella itself is US-based (in Santa Clara, California), was founded in 2004, is traded on NASDAQ, and is a $200+ million/year company (although revenues and profits have declined over the last few years). While much smaller than more famous semiconductor companies, Ambarella obviously fills a critical niche for (among others) professional security product firms.

So if you, like me, had never heard of Ambarella…now you have.

A tool is not a way of…bad things

For years I’ve uttered the phrase “a tool is not a way of life,” and a recent statement from Rank One Computing reminded me of this fact. In a piece on the ethical use of facial recognition, Rank One Computing stated the following in passing:

[Rank One Computing] is taking a proactive stand to communicate that public concerns should focus on applications and policies rather than the technology itself.

I emphatically believe that all technologies are neutral. They can be used for good, or they can be used for…bad things.

And yes, facial recognition has been misused.

It is an undeniable fact that a police jurisdiction used a computerized facial recognition result as a justifiable reason for arrest, rather than as an investigative lead that would need to be supported by additional evidence.

From https://www.aclu.org/letter/aclu-michigan-complaint-re-use-facial-recognition

But that incident, or ten incidents, or one hundred incidents, does NOT mean that ALL uses of facial recognition should be demonized, or even that SELECTED uses of facial recognition should be demonized (Amazon bad; Apple good).

Policies are not foolproof

Now I will grant that establishment of a policy or procedure does NOT necessarily mean that people will always act in compliance with that policy/procedure.

As an example, one accepted practice in lineup generation is double-blind lineup generation, in which you have different people involved in different parts of the lineup generation and witness viewing process. For example, these two roles can be distinct:

  • A person who knows who the arrested individual is creates the lineup (with additional safeguards to ensure that the created lineup isn’t biased).
  • A second person who DOESN’T know who the arrested individual is shows the lineup to the witness and records what the witness says and doesn’t say when viewing the lineup. The reason for the presence of a separate person is to ensure that the person administering the lineup doesn’t provide subconscious (or conscious) hints as to who the “right” person would be.

Now you can set up your police department’s procedures to require this, and your software vendor could design its software to support this. But that doesn’t prevent a corrupt Chief of Police from saying, “Jane, I want you to create the lineup AND show it to the witness. And make sure the witness chooses the RIGHT guy!”

But policy-based facial recognition is better than no facial recognition at all

But…if I may temporarily allow myself to run a tired cliché into the ground, that doesn’t mean you throw out the baby with the bathwater.

From 1512. Old clichés are old. Public Domain, https://commons.wikimedia.org/w/index.php?curid=689179

Rather than banning facial recognition, we should concentrate on defining ethical uses.

And there’s one more thing to consider. If you ban computerized facial recognition, how are you going to identify people? As I’ve noted elsewhere, witness (mis)identification is rampant with biases that make even the bottom-tier facial recognition algorithms seem accurate.

More on the Israeli master faces study

Eric Weiss of FindBiometrics has opined on the Tel Aviv master faces study that I previously discussed.

Oops, wrong “Faces.” Oh well. By Warner Bros. Records – Billboard, page 18, 14 November 1970, Public Domain, https://commons.wikimedia.org/w/index.php?curid=27031391

While he does not explicitly talk about the myriad of facial recognition algorithms that were NOT addressed in the study, he does have some additional details about the test dataset.

The three algorithms that were tested

Here’s what FindBiometrics says about the three algorithms that were tested in the Israeli study.

The researchers described (the master faces) as master keys that could unlock the three facial recognition systems that were used to test the theory. In that regard, they challenged the Dlib, FaceNet, and SphereFace systems, and their nine master faces were able to impersonate more than 40 percent of the 5,749 people in the LFW set.

While it initially sounds impressive to say that three facial recognition algorithms were fooled by the master faces, bear in mind that there are hundreds of facial recognition algorithms tested by NIST alone, and (as I said earlier) the test has NOT been duplicated against any algorithms other than the three open source algorithms mentioned.

…let’s look at the algorithms themselves and evaluate the claim that results for the three algorithms Dlib, FaceNet, and SphereFace can naturally be extrapolated to ALL facial recognition algorithms….NIST’s subsequent study…evaluated 189 algorithms specially for 1:1 and 1:N use cases….“Tests showed a wide range in accuracy across developers, with the most accurate algorithms producing many fewer errors.”

In short, just because the three open source algorithms were fooled by master faces doesn’t mean that commercial grade algorithms would also be fooled by master faces. Maybe they would be fooled…or maybe they wouldn’t.

What about the dataset?

The three open source algorithms were tested against the dataset from Labeled Faces in the Wild. As I noted in my prior post, the LFW people emphasize some important caveats about their dataset, including the following:

Many groups are not well represented in LFW. For example, there are very few children, no babies, very few people over the age of 80, and a relatively small proportion of women. In addition, many ethnicities have very minor representation or none at all.

In the FindBiometrics article, Weiss provides some additional detail about dataset representation.

…there is good reason to question the researchers’ conclusion. Only two of the nine master faces belong to women, and most depicted white men over the age of 60. In plain terms, that means that the master faces are not representative of the global public, and they are not nearly as effective when applied to anyone that falls outside one particular demographic.

That discrepancy can largely be attributed to the limitations of the LFW dataset. Women make up only 22 percent of the dataset, and the numbers are even lower for children, the elderly (those over the age of 80), and for many ethnic groups.

Valid points to be sure, although the definition of a “representative” dataset varies depending upon the use case. For example, a representative dataset for a law enforcement database in the city of El Paso, Texas will differ from a representative dataset for an airport database catering to Air France customers.

So what conclusion can be drawn?

Perhaps it’s just me, but scientific entities that conduct studies are always motivated by the need for additional funding. After a study is concluded, it seems that the entities always conclude that “more research is needed”…which can be self-serving, because as long as more research is needed, the scientific entities can continue to receive necessary funding. Imagine the scientific entity that would dare to say “Well, all necessary research has been conducted. We’re closing down our research center.”

But in this case, there IS a need to perform additional research, to test the master faces against different algorithms and against different datasets. Then we’ll know whether this statement from the FindBiometrics article (emphasis mine) is actually true:

Any face-based identification system would be extremely vulnerable to spoofing…