Why I find “race recognition” problematic

Let me start by admitting to my, um, bias.

For the last twenty-five plus years, I have been involved in the identification of individuals.

  • Who is the person who is going through the arrest/booking process?
  • Who is the person who claims to be entitled to welfare benefits?
  • Who is the person who wants to enter the country?
  • Who is the person who is exiting the country? (Yes, I remember the visa overstay issue.)
  • Who is the person who wants to enter the computer room in the office building?
  • Who is the person who is applying for a driver’s license or passport?
  • Who is the person who wants to enter the sports stadium or concert arena?

These are just a few of the problems that I have worked on solving over the last twenty-five plus years, all of which are tied to individual identity.

From that perspective, I really don’t care if the person entering the stadium/computer room/country whatever is female, mixed race, Muslim, left handed, or whatever. I just want to know if this is the individual that he/she/they claims to be.

If you’ve never seen the list of potential candidates generated by a top-tier facial recognition program, you may be shocked when you see it. That list of candidates may include white men, Asian women, and everything in between. “Well, that’s wrong,” you may say to yourself. “How can the results include people of multiple races and genders?” It’s because the algorithm doesn’t care about race and gender. Think about it – what if a victim THINKS that he was attacked by a white male, but the attacker was really an Asian female? Identify the individual, not the race or gender.

From http://gendershades.org/. Yes, http.

So when Gender Shades came out, stating that IBM, Microsoft, and Face++ AI services had problems recognizing the gender of people, especially those with darker skin, my reaction was “so what”?

(Note that this is a different question than the question of how an algorithm identifies individuals of different genders, races, and ages, which has been addressed by NIST.)

But some people persist in addressing biometrics’ “failure” to properly identify genders and races, ignoring the fact that both gender and race have become social rather than biological constructs. Is the Olympian Jenner male, female, or something else? What are your personal pronouns? What happens when a mixed race person identifies with one race rather than another? And aren’t we all mixed race anyway?

The latest study from AlBdairi et al on computational methods for ethnicity identification

But there’s still a great interest in “race recognition.”

As Jim Nash of Biometric Update notes, a team of scientists has published an open access paper entitled “Face Recognition Based on Deep Learning and FPGA for Ethnicity Identification.”

The authors claim that their study is “the first image collection gathered specifically to address the ethnicity identification problem.”

But what of the NIST demographic study cited above? you may ask. The NIST study did NOT have the races of the individuals, but used the individuals’ country of origin as a proxy for race. Then again, it is possible that this study may have done the same thing.

Despite the fact that there are several large-scale face image databases accessible online, none of these databases are acceptable for the purpose of the conducted study in our research. Furthermore, 3141 photographs were gathered from a variety of sources. Specifically, 1081, 1021, and 1039 Chinese, Pakistani, and Russian face photos were gathered, respectively. 

From https://www.mdpi.com/2076-3417/12/5/2605/htm

There was no mention of whether any of the Chinese face photos were Caucasian…or how the researchers could tell that they were Caucasian.

Anyway, if you’re interested in the science behind using Deep Convolutional Neural Network (DCNN) models and field-programmable gate arrays (FPGAs) to identify ethnicity, read the paper. Or skip to the results.

The experimental results reported that our model outperformed all the methods of state-of-the-art, achieving an accuracy and F1 score value of 96.9 percent and 94.6 percent, respectively.

From https://www.mdpi.com/2076-3417/12/5/2605/htm

But this doesn’t answer the question I raised earlier.

Three possible use cases for race recognition, two of which are problematic

Why would anyone want to identify ethnicity or engage in race recognition? Jim Nash of Biometric Update summarizes three possible use cases for doing this, which I will address one by one. TL;DR two of the use cases are problematic.

The code…could find a role in the growing field of race-targeted medical treatments and pharmacogenomics, where accurately ascertaining race could provide better care.

From https://www.biometricupdate.com/202203/identifying-ethnicity-problematic-so-scientists-write-race-recognition-code

Note that in this case race IS a biological construct, so perhaps its use is valid here. Regardless of how Nkechi Amare Diallo (formerly Rachel Dolezal) self-identifies, she’s not a targeted candidate for sickle cell treatment.

It could be helpful to some employers. Such as system could “use racial information to offer employers ethnically convenient services, then preventing the offending risk present in many cultural taboos.”

From https://www.biometricupdate.com/202203/identifying-ethnicity-problematic-so-scientists-write-race-recognition-code

This is where things start to get problematic. Using Diallo as an example, race recognition software based upon her biological race would see no problem in offering her fried chicken and watermelon at a corporate function, but Diallo might have some different feelings about this. And it’s not guaranteed that ALL members of a particular race are affected by particular cultural taboos. (The text below, from 1965, was slightly edited.)

Godfrey Cambridge. Retrieved from https://www.imdb.com/name/nm0131387/

People used to think of (blacks) as going around with fried chicken in a paper bag, (Godfrey) Cambridge says. But things have changed. “Now,” he says, “we carry an attache case—with fried chicken in it. We ain’t going to give up everything just to get along with you people.”

From http://content.time.com/time/subscriber/article/0,33009,839260,00.html. Yes, http.

While some employees may be pleased that they receive a particular type of treatment because of their biological race, others may not be pleased at all.

So let’s move on to Nash’s third use case for race recognition. Hold on to your seats.

Ultimately, however, the broadest potential mission for race recognition would be in security — at border stations and deployed in public-access areas, according to the report.

From https://www.biometricupdate.com/202203/identifying-ethnicity-problematic-so-scientists-write-race-recognition-code

I thought we had settled this over 20 years ago. Although we really didn’t.

From https://www.youtube.com/watch?v=_rkmIAnfDVY

While President Bush was primarily speaking about religious affiliation, he also made the point that we should not judge individuals based upon the color of their skin.

Yet we do.

If I may again return to our current sad reality, there have been allegations that Africans encountered segregation and substandard treatment when trying to flee Ukraine. (When speaking of “African,” note that concerns were raised by officials from Gabon, Ghana, and Kenya – not from Egypt, Libya, or Tunisia. Then again, Indian students also complained of substandard treatment.)

Many people in the United States and western Europe would find it totally unacceptable to treat people at borders and public areas differently by race.

Do we want to encourage this use case?

And if you feel that we should, please provide your picture. I want to see if your concerns are worthy of consideration.

Canada’s IRCC ITQ B7059-180321/B and the biometric proposals chess match

In a competitive bid process, one unshakable truth is that everything you do will be seen by your competitors. This affects what you as a bidder do…and don’t do.

My trip to Hartford for a 30 minute meeting

I saw this in action many years ago when I was the product manager for Motorola’s Omnitrak product (subsequently Printrak BIS, subsequently part of MorphoBIS, subsequently part of MBIS). Connecticut and Rhode Island went out to bid for an two-state automated fingerprint identification system (AFIS). As part of the request for proposal process, the state of Connecticut scheduled a bidders’ conference. This was well before online videoconferencing became popular, so if you wanted to attend this bidders’ conference, you had to physically go to Hartford, Connecticut.

The Mark Twain House in Hartford. For reasons explained in this post, I spent more time here than I did at the bidders’ conference itself. By Makemake, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=751488

So I flew from California to Connecticut to attend the conference, and other people from other companies made the trip. That morning I drove from my hotel to the site of the conference (encountering a traffic jam much worse than the usual traffic jams back home), and I and the competitors assembled and waited for the bidders’ conference to begin.

The state representative opened the floor up to questions from bidders.


No one asked a question.

We were all eyeing each other, seeing what the other people were going to ask, and none of us were willing to tip our hands by asking a question ourselves.

Eventually one or two minor questions were asked, but the bidders’ conference ended relatively quickly.

There are a number of chess-like tactics related to what bidders do and don’t do during proposals. Perhaps some day I’ll write a Bredemarket Premium post on the topic and spill my secrets.

But for now, let’s just say that all of the bidders successfully kept their thoughts to themselves during that conference. And I got to visit a historical site, so the trip wasn’t a total waste.

And today, it’s refreshing to know that things don’t change.

When the list of interested suppliers appears to be null

Back on September 24, the Government of Canada issued an Invitation to Qualify (B7059-180321/B) for a future facial recognition system for immigration purposes. This was issued some time ago, but I didn’t hear about it until Biometric Update mentioned it this morning.

Now Bredemarket isn’t going to submit a response (even though section 2.3a says that I can), but Bredemarket can obviously help those companies that ARE submitting a response. I have a good idea who the possible players are, but to check things I went to the page of the List of Interested Suppliers to see if there were any interested suppliers that I missed. The facial recognition market is changing rapidly, so I wondered if some new names were popping up.

So what did I see when I visited the List of Interested Suppliers?

An invitation for me to become the FIRST listed interested supplier.

That’s right, NO ONE has publicly expressed interest in this bid.

A screen shot of https://buyandsell.gc.ca/procurement-data/tender-notice/PW-XS-002-39912/list-of-interested-suppliers as of the late morning (Pacific time) on Monday, October 11.

And yes, I also checked the French list; no names there either.

There could be one of three reasons for this:

  1. Potential bidders don’t know about the Invitation to Qualify. This is theoretically possible; after all, Biometric Update didn’t learn about the invitation until two weeks after it was issued.
  2. No one is interested in bidding on a major facial recognition program. Yeah, right.
  3. Multiple companies ARE interested in this bid, but none wants to tip its hand and let competitors know of its interest.

My money is on reason three.

Hey, bidders. I can keep your secret.

As you may have gathered, as of Monday October 11 I am not part of any team responding to this Invitation to Qualify.

If you are a biometric vendor who needs help in composing your response to IRCC ITQ B7059-180321/B before the November 3 due date, or in framing questions (yes, there are chess moves on that also), let me know.

I won’t tell anybody.

Pangiam acquires something else (in this case TrueFace)

People have been coming here to find this news (thanks Google Search Console) so I figured I’d better share it here.

Remember Pangiam, the company that I talked about back in March when it acquired the veriScan product from the Metropolitan Washington Airports Authority? Well, last week Pangiam acquired another company.

TYSONS CORNER, Va., June 2, 2021 /PRNewswire/ — Pangiam, a technology-based security and travel services provider, announced today that it has acquired Trueface, a U.S.-based leader in computer vision focused on facial recognition, weapon detection and age verification technologies. Terms of the transaction were not disclosed….

Trueface, founded in 2013 by Shaun Moore and Nezare Chafni, provides industry leading computer vision solutions to customers in a wide range of industries. The company’s facial recognition technology recently achieved a top three ranking among western vendors in the National Institute of Standards and Technology (NIST) 1:N Face Recognition Vendor Test. 

(Just an aside here: companies can use NIST tests to extract all sorts of superlatives that can be applied to their products, once a bunch of qualifications are applied. Pay attention to the use of the phrase “among western vendors.” While there may be legitimate reasons to exclude non-western vendors from comparisons, make a mental note when such an exclusion is made.)

But what does this mean in terms of Pangiam’s existing product? The press release covers this also.

Trueface will add an additional capability to Pangiam’s existing technologies, creating a comprehensive and seamless solution to satisfy the needs of both federal and commercial enterprises.

And because Pangiam is not a publicly-traded company, it is not obliged to add a disclaimer to investors saying this integration might not happen bla bla bla. Publicly traded companies are obligated to do this so that investors are aware of the risks when a company speculates about its future plans. Pangiam is not publicly traded, and the owners are (presumably) well aware of the risks.

For example, a US government agency may prefer to do business with an eastern vendor. In fact, the US government does a lot of business with one eastern vendor (not Chinese or Russian).

But we’ll see what happens with any future veriTruefaceScan product.

Pangiam, a new/old player in biometric boarding

Make vs. buy.

Businesses are often faced with the question of whether to buy a product or service from a third party, or make the product or service itself.

And airports are no exception to this.

The Metropolitan Washington Airports Authority (MWAA), the entity that manages two of the airports in the Washington, DC area, needed a biometric boarding (biometric exit) solution. Such solutions allow passengers to skip the entire “pull out the paper ticket” process, or even the “pull out the smartphone airline app” process, and simply stand and let a camera capture a picture of the passenger’s face. While there are several companies that sell such solutions, MWAA decided to create its own solution, veriScan.


And once MWAA had implemented veriScan at its own airports, it started marketing the solution to other airports, and competing against other providers who were trying to sell their own solutions to airports.

Well, MWAA got out of the border product/service business last week when it participated in this announcement:

ALEXANDRIA, Va., March 19, 2021 /PRNewswire/ — Pangiam, a technology-based security and travel services provider, announced today that it has acquired veriScan, an integrated biometric facial recognition system for airports and airlines, from the Metropolitan Washington Airports Authority (“Airports Authority”). Terms of the transaction were not disclosed.

Pangiam is clearly the new kid on the block, since the company didn’t even exist in its current form a year ago. Late last year, AE Industrial Partners acquired and merged the decade-old Linkware and the newly-formed Pangian (PRE LLC) “to form a highly integrated travel solutions technology platform providing a more seamless and secure travel experience.”

But in a sense, Pangiam ISN’T new to the travel industry, once you read the biographies of many of the principals at the company.

  • “Most recently (Kevin McAleenan) served as Acting Secretary of the U.S. Department of Homeland Security (DHS)….”
  • “Prior to Pangiam, Patrick (Flanagan) held roles at U.S. Customs & Border Protection (CBP), the U.S. Navy, the National Security Staff, the Transportation Security Administration (TSA), and the Department of Homeland Security (DHS).”
  • “Dan (Tanciar) previously served as the Executive Director of Planning, Program Analysis, and Evaluation in the Office of Field Operations (OFO) at U.S. Customs and Border Protection (CBP).”
  • “Prior to Pangiam, Andrew (Meehan) served as the principal adviser to the Acting Secretary for external affairs at the Department of Homeland Security (DHS).”
  • “(Tom Plofchan) served as a National Security Advisor to the Department of Energy’s Pacific Northwest National Laboratory before entering government to serve as the Counterterrorism Advisor to the Commissioner, U.S. Customs and Border Protection, and as Counterterrorism Counselor to the Secretary, U.S. Department of Homeland Security.”

So if you thought that veriScan was well-connected because it was offered by an airport authority, consider how well-connected it appears now because it is offered by a company filled with ex-DHS people.

Which in and of itself doesn’t necessarily indicate that the products work, but it does indicate some level of domain knowledge.

But will airports choose to buy the Pangiam veriScan solution…or make their own?