Pangiam, CLEAR, and others make a “sporting” effort to deny (or allow) stadium access

Back when I initially entered the automated fingerprint identification systems industry in the last millennium, I primarily dealt with two markets: the law enforcement market that seeks to solve crimes and identify criminals, and the welfare benefits market that seeks to make sure that the right people receive benefits (and the wrong people don’t).

Other markets simply didn’t exist. If I pulled out my 1994-era mobile telephone and looked at it, nothing would happen. Today, I need to look at my 2020-era mobile telephone to obtain access to its features.

And there are other biometric markets also.

Pangiam and stadium bans

Back in 1994 I couldn’t envision a biometrics story in Sports Illustrated magazine. But SI just ran a story on how facial recognition can be used to keep fans out of stadiums who shouldn’t be there.

Some fans (“fanatics”) perform acts in stadiums that cause the sports teams and/or stadium authorities to officially ban them from the stadium, sometimes for life.

John Green is the man in the blue shirt and white baseball cap to Artest’s left. By Copyright 2004 National Basketball Association. – Television broadcast of the Pacers-Pistons brawl on ESPN., Fair use, https://en.wikipedia.org/w/index.php?curid=6824157

But in the past, these measures were ineffective.

For a long time, those “measures” were limited at best. Fans do not have to show ID upon entering arenas. Teams could run checks on all the credit cards to purchase tickets to see whether any belonged to banned fans, but those fans could easily have a friend buy the tickets. 

But there are other ways to enforce stadium bans, and Sports Illustrated quoted an expert on the matter.

“They’ve kicked the fan out; they’ve taken a picture—that fan they know,” says Shaun Moore, CEO of a facial-recognition company called Trueface. “The old way of doing things was, you give that picture to the security staff and say, ‘Don’t let this person back in.’ It’s not really realistic. So the new way of doing it is, if we do have entry-level cameras, we can run that person against everyone that’s coming in. And if there’s a hit, you know then; then there’s a notification to engage with that person.”

This, incidentally, is an example of a “deny list,” or the use of a security system to deny a person access. We’ll get to that later.

But did you notice the company that was mentioned in the last quote? I’ve mentioned that company before, because Trueface was the most recent acquisition by the company Pangiam, a company that has also acquired airport security technology.

But Pangiam/Trueface isn’t the only company serving stadium (and entertainment) venues.

CLEAR and stadium entry

Most of the time, sports stadiums aren’t concentrating on the practice of DENYING people entry to a stadium. They make a lot more money by ALLOWING people entry to a stadium…and allowing them to enter as quickly as possible so they can spend money on concessions.

One such company that supports this is CLEAR, which was recently in the news because of its Initial Public Offering. Coincidentally, CLEAR also provides airport security technology, but it has branched out from that core market and is also active in other areas.

For example, let’s say you’re a die-hard New York Mets fan, and you head to Citi Field to watch a game.

By Chris6d – Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=101751795

The Mets don’t just let anyone into the stadium; you have to purchase a ticket. So you need to take your ticket out of your pocket and show it to the gate staff, or you need to take your smartphone out of your pocket and show your digital ticket to the gate staff.

What if you could get into the stadium without taking ANYTHING out of your pocket? Well, you can.

In the CLEAR Lane, your fingerprint is all you need to use tickets in your MLB Ballpark app – no need to pull out your phone or printed ticket as you enter the game.

Now that is really easy.

Pangiam and CLEAR aren’t the only companies in this space, as I well know. But there’s the possibility that biometrics will be used more often for access to sports games, concerts, and similar events.

Two articles on facial recognition

Within the last hour I’ve run across two articles that discuss various aspects of facial recognition, dispelling popular society notions about the science in the process.

Ban facial recognition? Ain’t gonna happen

The first article was originally shared by my former IDEMIA colleague Peter Kirkwood, who certainly understood the significance of it from his many years in the identity industry.

The article, published by the Security Industry Association (SIA), is entitled “Most State Legislatures Have Rejected Bans and Severe Restrictions on Facial Recognition.”

Admittedly the SIA is by explicit definition an industry association, but in this case it is simply noting a fact.

With most 2021 legislative sessions concluded or winding down for the year, proposals to ban or heavily restrict the technology have had very limited overall success despite recent headlines. It turns out that such bills failed to advance or were rejected by legislatures in no fewer than 17 states during the 2020 and 2021 sessions: California, Colorado, Hawaii, Kentucky, Maine, Maryland, Massachusetts, Michigan, Minnesota, Montana, Nebraska, New Hampshire, New Jersey, New York, Oregon, South Carolina and Washington.

And the article even cited one instance in which public safety and civil libertarians worked together, proving such cooperation is actually possible.

In March, Utah enacted the nation’s most comprehensive and precise policy safeguards for government applications. The measure, supported both by the Utah Department of Public Safety as well as the American Civil Liberties Union, establishes requirements for public-sector and law enforcement use, including conditions for access to identity records held by the state, and transparency requirements for new public sector applications of facial recognition technology.

This reminds me of Kirkwood’s statement when he originally shared the article on LinkedIn: “Targeted use with appropriate governance and transparency is an incredibly powerful and beneficial tool.”

NIST’s biometric exit tests reveal an inconvenient truth

Meanwhile, the National Institute of Standards and Technology, which is clearly NOT an industry association, continues to enhance its ongoing Facial Recognition Vendor Test (FRVT). As I noted myself on Facebook and LinkedIn:

With its latest rounds of biometric testing over the last few years, the National Institute of Standards and Technology has shown its ability to adapt its testing to meet current situations.

In this case, NIST announced that it has applied its testing to the not-so-new use case of using facial recognition as a “biometric exit” tool, or as a way to verify that someone who was supposed to leave the country has actually left the country. The biometric exit use case emerged after 9/11 in response to visa overstays, and while the vast, vast majority of people who overstay visas do not fly planes into buildings and kill thousands of people, visa overstays are clearly a concern and thus merit NIST testing.

Transportation Security Administration Checkpoint at John Glenn Columbus International Airport. By Michael Ball – Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=77279000

But buried at the end of the NIST report (accessible from the link in NIST’s news release) was a little quote that should cause discomfort to all of those who reflexively believe that all biometrics is racist, and thus needs to be banned entirely (see SIA story above). Here’s what NIST said after having looked at the data from the latest test:

“The team explored differences in performance on male versus female subjects and also across national origin, which were the two identifiers the photos included. National origin can, but does not always, reflect racial background. Algorithms performed with high accuracy across all these variations. False negatives, though slightly more common for women, were rare in all cases.”

And as Peter Kirkwood and many other industry professionals would say, you need to use the technology responsibly. This includes things such as:

  • In criminal cases, having all computerized biometric search results reviewed by a trained forensic face examiner.
  • ONLY using facial recognition results as an investigative lead, and not relying on facial recognition alone to issue an arrest warrant.

So facial recognition providers and users had a good day. How was yours?

Is your home your castle when you use consumer doorbell facial recognition?

For purposes of this post, I will define three entities that can employ facial recognition:

  • Public organizations such as governments.
  • Private organizations such as businesses.
  • Individuals.

Some people are very concerned about facial recognition use by the first two categories of entities.

But what about the third category, individuals?

Can individuals assert a Constitutional right to use facial recognition in their own homes? And what if said individuals live in Peoria?

Concerns about ANY use of facial recognition

Let’s start with an ACLU article from 2018 regarding “Amazon’s Disturbing Plan to Add Face Surveillance to Your Front Door.”

Let me go out on a limb and guess that the ACLU opposes the practice.

The article was prompted by an Amazon 2018 patent application which involved both its Rekognition facial recognition service and its Ring cameras.

One of the figures in Amazon’s patent application, courtesy the ACLU. https://www.aclunc.org/docs/Amazon_Patent.pdf

While the main thrust of the ACLU article concerns acquisition of front door face surveillance (and other biometric) information by the government, it also briefly addresses the entity that is initially performing the face surveillance: namely, the individual.

Likewise, homeowners can also add photos of “suspicious” people into the system and then the doorbell’s facial recognition program will scan anyone passing their home.

I should note in passing that ACLU author Jacob Snow is describing a “deny list,” which flags people who should NOT be granted access such as that pesky solar power salesperson. In most cases, consumer products tout the use of an “allow list,” which flags people who SHOULD be granted access such as family members.

Regardless of whether you’re discussing a deny list or an allow list, the thrust of the ACLU article isn’t that governments shouldn’t use facial recognition. The thrust of the article is that facial recognition shouldn’t be used at all.

The ACLU and other civil rights groups have repeatedly warned that face surveillance poses an unprecedented threat to civil liberties and civil rights that must be stopped before it becomes widespread.

Again, not face surveillance by governments, but face surveillance period. People should not have the, um, “civil liberties” to use the technology.

But how does the tech world approach this?

The reason that I cited that particular ACLU article was that it was subsequently referenced in a CNET article from May 2021. This article bore the title “The best facial recognition security cameras of 2021.”

Let me go out on a limb and guess that CNET supports the practice.

The last part of author Megan Wollerton’s article delves into some of the issues regarding facial recognition use, including those raised by the ACLU. But the bulk of the article talks about really cool tech.

As I stated above, Wollerton notes that the intended use case for home facial recognition security systems involves the creation of an “allow list”:

Some home security cameras have facial recognition, an advanced option that lets you make a database of people who visit your house regularly. Then, when the camera sees a face, it determines whether or not it belongs to someone in your list of known faces. If the recognition system does not know who is at the door, it can alert you to an unknown person on your property.

Obviously you could repurpose such a system for anything you want, provided that you can obtain a clear picture of the face of the pesky social power salesperson.

Before posting her reviews of various security systems, and after a brief mention (expanded later in the article) about possible governmental misuse of facial recognition, Wollerton redirects the conversation.

But let’s step back a bit to the consumer realm. Your home is your castle, and the option of having surveillance cameras with facial recognition software is still compelling for those who want to be on the cutting edge of smart home innovation.

“Your home is your castle” may be a distinctly American concept, but it certainly applies here as organizations such as, um, the ACLU defend a person’s right against unreasonable actions by governments.

Obviously, there are limits to ANY Constitutional right. I cannot exercise my Fourth Amendment right to be secure in my house, couple that with my First Amendment right to freely exercise my religion, and conclude that I have the unrestricted right to perform ritual child sacrifices in my home. (Although I guess if I have a home theater and only my family members are present, I can probably yell “Fire!” all I want.)

So perhaps I could mount an argument that I can use facial recognition at my house any time I want, if the government agrees that this right is “reasonable.”

But it turns out that other people are involved.

You knew I was going to mention Illinois in this post

OK, it’s BIPA time.

As I previously explained in a January 2021 post about the Kami Doorbell Camera, “BIPA” is Illinois’ Biometric Information Privacy Act. This act imposes constraints on a private entity’s use of biometrics. (Governments are excluded in Illinois BIPA.) And here’s how BIPA defines the term “private entity”:

“Private entity” means any individual, partnership, corporation, limited liability company, association, or other group, however organized. A private entity does not include a State or local government agency. A private entity does not include any court of Illinois, a clerk of the court, or a judge or justice thereof.

Did you see the term “individual” in that definition?

So BIPA not only affects company use of biometrics, such as use of biometrics by Google or by a theme park or by a fitness center. It also affects an individual such as Harry or Harriet Homeowner’s use of biometrics.

As I previously noted, Google does not sell its Nest Cam “familiar face alert” feature in Illinois. But I guess it’s possible (via location spoofing if necessary) for someone to buy Nest Cam familiar face alerts in Indiana, and then sneak the feature across the border and implement it in the Land of Lincoln. But while this may (or may not) get Google off the hook, the individual is in a heap of trouble (should a trial lawyer decide to sue the individual).

Let’s face it. The average user of Nest Cam’s familiar face alerts, or the Kami Doorbell Camera, or any other home security camera with facial recognition (note that Amazon currently is not using facial recognition in its consumer products), is probably NOT complying with BIPA.

A private entity in possession of biometric identifiers or biometric information must develop a written policy, made available to the public, establishing a retention schedule and guidelines for permanently destroying biometric identifiers and biometric information when the initial purpose for collecting or obtaining such identifiers or information has been satisfied or within 3 years of the individual’s last interaction with the private entity, whichever occurs first.

I mean it’s hard enough for Harry and Harriet to get their teenage son to acknowledge receipt of the Homeowner family’s written policy for the use of the family doorbell camera. And you can forget about getting the pesky solar power salesperson to acknowledge receipt.

So from a legal perspective, it appears that any individual homeowner who installs a facial recognition security system can be hauled into civil court under BIPA.

But will these court cases be filed from a practical perspective?

Probably not.

When a social media company violates BIPA, the violation conceivably affects millions of individuals and can result in millions or billions of dollars in civil damages.

When the pesky solar power salesperson discovers that Harry and Harriet Homeowner, the damages would be limited to $1,000 or $5,000 plus relevant legal fees.

It’s not worth pursuing, any more than it’s worth pursuing the Illinois driver who is speeding down the expressway at 66 miles per hour.

My LinkedIn article “Don’t ban facial recognition”

By TapTheForwardAssist – Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=98670006

This post serves as a pointer to an article that I just published on LinkedIn, “Don’t ban facial recognition.”

If you’re going to prohibit use of a particular tool, you may want to check the alternatives to that tool to see if the alternatives are better…or worse.

To read the article, go here.

Pangiam acquires something else (in this case TrueFace)

People have been coming here to find this news (thanks Google Search Console) so I figured I’d better share it here.

Remember Pangiam, the company that I talked about back in March when it acquired the veriScan product from the Metropolitan Washington Airports Authority? Well, last week Pangiam acquired another company.

TYSONS CORNER, Va., June 2, 2021 /PRNewswire/ — Pangiam, a technology-based security and travel services provider, announced today that it has acquired Trueface, a U.S.-based leader in computer vision focused on facial recognition, weapon detection and age verification technologies. Terms of the transaction were not disclosed….

Trueface, founded in 2013 by Shaun Moore and Nezare Chafni, provides industry leading computer vision solutions to customers in a wide range of industries. The company’s facial recognition technology recently achieved a top three ranking among western vendors in the National Institute of Standards and Technology (NIST) 1:N Face Recognition Vendor Test. 

(Just an aside here: companies can use NIST tests to extract all sorts of superlatives that can be applied to their products, once a bunch of qualifications are applied. Pay attention to the use of the phrase “among western vendors.” While there may be legitimate reasons to exclude non-western vendors from comparisons, make a mental note when such an exclusion is made.)

But what does this mean in terms of Pangiam’s existing product? The press release covers this also.

Trueface will add an additional capability to Pangiam’s existing technologies, creating a comprehensive and seamless solution to satisfy the needs of both federal and commercial enterprises.

And because Pangiam is not a publicly-traded company, it is not obliged to add a disclaimer to investors saying this integration might not happen bla bla bla. Publicly traded companies are obligated to do this so that investors are aware of the risks when a company speculates about its future plans. Pangiam is not publicly traded, and the owners are (presumably) well aware of the risks.

For example, a US government agency may prefer to do business with an eastern vendor. In fact, the US government does a lot of business with one eastern vendor (not Chinese or Russian).

But we’ll see what happens with any future veriTruefaceScan product.

When biometric readers are “magic” (it’s a small face after all)

The news coming across the wire is that Disney’s Magic Kingdom in Florida is testing facial recognition. (H/T International Biometrics + Identity Association.)

“At Walt Disney World Resort, we’re always looking for innovative and convenient ways to improve our guests’ experience—especially as we navigate the impact of COVID-19. With the future in mind and the shift in focus to more touchless experiences, we’re conducting a limited 30-day test using facial recognition technology.”

If the test is successful and facial recognition is implemented, it would be a replacement for (touch) fingerprint technology, which the Disney parks suspended last July for health reasons. (Although touchless fingerprint options are available.)

Disney’s biometric history extends back to 2006, when it used hand geometry.

Pangiam, a new/old player in biometric boarding

Make vs. buy.

Businesses are often faced with the question of whether to buy a product or service from a third party, or make the product or service itself.

And airports are no exception to this.

The Metropolitan Washington Airports Authority (MWAA), the entity that manages two of the airports in the Washington, DC area, needed a biometric boarding (biometric exit) solution. Such solutions allow passengers to skip the entire “pull out the paper ticket” process, or even the “pull out the smartphone airline app” process, and simply stand and let a camera capture a picture of the passenger’s face. While there are several companies that sell such solutions, MWAA decided to create its own solution, veriScan.

https://www.airportveriscan.com/

And once MWAA had implemented veriScan at its own airports, it started marketing the solution to other airports, and competing against other providers who were trying to sell their own solutions to airports.

Well, MWAA got out of the border product/service business last week when it participated in this announcement:

ALEXANDRIA, Va., March 19, 2021 /PRNewswire/ — Pangiam, a technology-based security and travel services provider, announced today that it has acquired veriScan, an integrated biometric facial recognition system for airports and airlines, from the Metropolitan Washington Airports Authority (“Airports Authority”). Terms of the transaction were not disclosed.

Pangiam is clearly the new kid on the block, since the company didn’t even exist in its current form a year ago. Late last year, AE Industrial Partners acquired and merged the decade-old Linkware and the newly-formed Pangian (PRE LLC) “to form a highly integrated travel solutions technology platform providing a more seamless and secure travel experience.”

But in a sense, Pangiam ISN’T new to the travel industry, once you read the biographies of many of the principals at the company.

  • “Most recently (Kevin McAleenan) served as Acting Secretary of the U.S. Department of Homeland Security (DHS)….”
  • “Prior to Pangiam, Patrick (Flanagan) held roles at U.S. Customs & Border Protection (CBP), the U.S. Navy, the National Security Staff, the Transportation Security Administration (TSA), and the Department of Homeland Security (DHS).”
  • “Dan (Tanciar) previously served as the Executive Director of Planning, Program Analysis, and Evaluation in the Office of Field Operations (OFO) at U.S. Customs and Border Protection (CBP).”
  • “Prior to Pangiam, Andrew (Meehan) served as the principal adviser to the Acting Secretary for external affairs at the Department of Homeland Security (DHS).”
  • “(Tom Plofchan) served as a National Security Advisor to the Department of Energy’s Pacific Northwest National Laboratory before entering government to serve as the Counterterrorism Advisor to the Commissioner, U.S. Customs and Border Protection, and as Counterterrorism Counselor to the Secretary, U.S. Department of Homeland Security.”

So if you thought that veriScan was well-connected because it was offered by an airport authority, consider how well-connected it appears now because it is offered by a company filled with ex-DHS people.

Which in and of itself doesn’t necessarily indicate that the products work, but it does indicate some level of domain knowledge.

But will airports choose to buy the Pangiam veriScan solution…or make their own?

I really want to know (if this song is truly related to crime scene investigation)

I was performing some website maintenance this afternoon, and decided to add a page dedicated to Bredemarket’s services for identity firms. I was trying to think of an introductory illustration to go with the page, since the town crier can only go so far. So, claiming fair use, I decided that this image made perfect sense.

“Who Are You” by The Who. Fair use, https://en.wikipedia.org/w/index.php?curid=11316153

Now while use of the “Who Are You” album cover on a Bredemarket identity page makes perfect sense to me, it may not make sense to 6.9 billion other people. So I guess I should explain my line of thinking.

The link between human identification and the song “Who Are You” was established nearly two decades ago, when the television show “C.S.I. Crime Scene Investigation” started airing on CBS. TV shows have theme songs, and this TV show adopted a (G-rated) excerpt from the Who song “Who Are You” as its theme song. After all, the fictional Las Vegas cops were often tasked with identifying dead bodies or investigating crime scene evidence, so they would be expected to ask the question “who are you” a lot.

Which reminds me of two stories:

  • I actually knew a real Las Vegas crime scene investigator (Rick Workman), but by the time I knew him he was working for the neighboring city of Henderson.
  • CSI spawned a number of spinoffs, including “CSI:Miami.” When I was a Motorola product manager, CSI:Miami contacted us to help with a storyline involving a crime scene palm print. While Motorola software was featured in the episode, the GUI was jazzed up a bit so that it would look good on TV.

So this song (and other Who songs for the CSI spinoffs) is indelibly associated with police crime scene work.

But should it be?

After all, people think that “When a Man Loves a Woman” is a love song based upon its title. But the lyrics show that it’s not a love song at all.

When a man loves a woman
Down deep in his soul
She can bring him such misery
If she is playin’ him for a fool

So are we at fault when we associate Pete Townshend’s 1970s song “Who Are You” with crime scene investigation?

Yes, and no.

While the “who are you” question has nothing to do with figuring out who committed a crime, it DOES involve a policeman.

This song is based on a day in the life of Pete Townshend….

Pete left that bar and passed out in a random doorway in Soho (a part of New York). A policeman recognized him (“A policeman knew my name”) and being kind, woke him and and told him, “You can go sleep at home tonight (instead of a jail cell), if you can get up and walk away.” Pete’s response: “Who the f–k are you?”

Because it was the 1970s, the policeman did not try to identify the drunk Townshend with a mobile fingerprint device linked to a fingerprint identification system, or a camera linked to a facial recognition system.

Instead, the drunk Townshend questioned the authority of the policeman. Which is what you would expect from the guy who wrote the line “I hope I die before I get old.”

Speaking of which, did anybody notice that on the album cover for “Who Are You,” Keith Moon is sitting on a chair that says “Not to Be Taken Away”? Actually, they did…especially since the album was released on August 18, 1978 and Moon died on September 7.

While Moon’s death was investigated, no crime scene investigators were involved.

My entry for the Spilled Coffee Story Challenge

All the cool kids are doing online social media challenges. Some of these challenges, such as the Ice Bucket Challenge, are very beneficial to society. Others, such as the Tide Pod Challenge, are not.

I believe that this challenge, the Spilled Coffee Story Challenge, falls somewhere between the two. It won’t cure any debilitating diseases, but it won’t kill you either.

Before continuing, I want to emphasize that this is the Spilled Coffee STORY Challenge, not the Spilled Coffee Challenge. The Spilled Coffee Challenge could be very dangerous, because coffee is hot. So DON’T do that.

By Julius Schorzman – Own work, CC BY-SA 2.0, https://commons.wikimedia.org/w/index.php?curid=107645

Now most of you have never heard of the Spilled Coffee Story Challenge. That’s because I just made it up based upon an online conversation. So I’ll start by explaining how the Spilled Coffee Story Challenge came to be, and then I’ll tell my spilled coffee story.

How the Spilled Coffee Story Challenge came to be

Not too long ago, Sumair Abro and Rhonda Salvestrini were on a podcast together, talking about storytelling. To illustrate the importance of storytelling, Abro proceeded to…tell a story. It’s a story that he overheard about a woman who spilled coffee. By the end of the story, we all knew that…well, I’ll let Abro tell his story. The video can be found here.

After telling the story, Abro mentioned three points:

  1. “When you tell a story from your personal experience – people are genuinely interested.”
  2. “Don’t show all your cards immediately – have an element of surprise.” (Abro’s story DEFINITELY had a surprise at the end, revealing how spilling coffee could be a wonderful event for a particular person.)
  3. “Tell your story to the right audience.”

Salvestrini then chimed in, noting how stories need to be engaging and relevant.

Before going on, the brief clip that I linked above is actually part of a longer conversation between Abro and Salvestrini, which I mentioned before in this blog post.

But in this case, we’re only talking about the short excerpt on storytelling. I shared this excerpt myself on my Bredemarket LinkedIn page, making the following comment as I did so:

But my coffee-spilling story, in which I almost spilled coffee on a customer (but thankfully didn’t), would be hard to spin into a wonderful business truth.

This prompted a response from Rhonda Salvestrini:

Coffee-spilling stories are authentic and let our audience know that we are human. I’m sure you can spin it into a wonderful business truth. Let’s try!

Sumair Abro also chimed in:

hahaha..you dont need to spin it. It’s authentic as mentioned by Rhonda

Well, Rhonda and Sumair…CHALLENGE ACCEPTED.

My Spilled Coffee Story

My spilled coffee story took place a few years ago, when I was working for MorphoTrak. MorphoTrak was a merger of two former competitors that combined their operations—including their previously separate user conferences. I had been involved with the old Motorola User Conferences, so I knew the customers from that side of the company. And as time went on, I got to meet the customers from the non-Motorola side of the company (the Sagem Morpho side).

Me at a User Conference, several years after the coffee incident.

One of the ex-Sagem Morpho customers was from Hawaii. Specifically, the Hawaii Criminal Justice Data Center. This customer not only used MorphoTrak’s fingerprint identification technology, but also used its facial recognition technology, providing Hawaii law enforcement with the ability to use faces as an investigative lead when solving crimes.

Several years ago, the Hawaii Criminal Justice Data Center was represented on the Users Conference Executive Board by Liane Moriyama. Moriyama is a key figure in Hawaii criminal justice, since she was present when Hawaii established its first automated fingerprint identification system in 1990, and was also present for the establishment of Hawaii’s facial recognition system in 2013. But she is proudest of her accomplishments for vulnerable populations:

“We realized that we needed to help the non-criminal justice communities by using the technology and the biometrics (to protect) our vulnerable populations, our children, our disabled and our elderly through licensing and background checks. That really does protect the common citizen, and the culmination of all of that is when I was elected chair of the National Crime Prevention and Privacy Compact Council. I served two terms as the chair nationally and we have made tremendous strides in keeping the vulnerable populations safe.”

Liane Moriyama, Women in Biometrics 2017 Award recipient, quoted in Secure ID News

So Moriyama was a key customer for MorphoTrak, and a nationally recognized public security figure. Oh, and she’s a wonderful woman also (she gave away more macadamia nuts than the guy from Magnum P.I.).

All of this was very true when I was walking down the hall one fateful day. The Users Conference Executive Board was in town planning the next Users Conference. I was not involved in Users Conference planning at the time, but I would usually see Liane and the other customers when they were in the facility.

USUALLY I’d see them.

I didn’t see her one day when I went to the lunchroom to get some coffee, then exited the lunchroom and turned the corner.

Only THEN did I see her, as I turned the corner and found her right in front of me.

And disaster struck, and I spilled my coffee.

Luckily, I spilled it on MYSELF, and DIDN’T spill it on Liane.

She was extremely concerned about the fact that I had spilled coffee on myself, and I was incredibly relieved that I hadn’t spilled coffee on her.

Because if you have the choice, it’s better for you to suffer a mishap than for the client to suffer one.

So all ended well. Liane didn’t have to incur a dry cleaning bill while traveling, I took care of my own clothes, and she still gave me macadamia nuts in the future.

So now I’ll ask you: is “if you have the choice, it’s better for you to suffer a mishap than for the client to suffer one” a wonderful business truth?

Facial recognition and the U.S. Capitol attack

This post examines a number of issues regarding the use of facial recognition. Specifically, it looks at various ways to use facial recognition to identify people who participated in the U.S. Capitol attack.

By TapTheForwardAssist – Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=98670006

Let’s start with the technological issues before we look at the legal ones. Specifically, we’ll look at three possible ways to construct databases (galleries) to use for facial recognition, and the benefits and drawbacks of each method.

What a facial recognition system does, and what it doesn’t do

The purpose of a one-to-many facial recognition system is to take a facial image (a “probe” image), process it, and compare it to a “gallery” of already-processed facial images. The system then calculates some sort of mathematical likelihood that the probe matches some of the images in the gallery.

That’s it. That’s all the system does, from a technological point of view.

Although outside of the scope of this particular post, I do want to say that a facial recognition system does NOT determine a match. Now the people USING the system could make the decision that one or more of the images in the gallery should be TREATED as a match, based upon mathematical considerations. However, when using a facial recognition system in the United States for criminal purposes, the general procedure is for a trained facial examiner to use his/her expertise to compare the probe image with selected gallery images. This trained examiner will then make a determination, regardless of what the technology says.

But forget about that for now. I want to concentrate on another issue—adding data to the gallery.

Options for creating a facial recognition “gallery”

As I mentioned earlier, the “gallery” is the database against which the submitted facial image (the “probe”) is compared. In a one-to-many comparison, the probe image is compared against all or some of the images in the gallery. (I’m skipping over the “all or some” issue for now.)

So where do you get facial images to put in the gallery?

For purposes of this post, I’m going to describe three sources for gallery images.

  • Government facial images of people who have been convicted of crimes.
  • Government facial images of people who have not necessarily been convicted of crimes, such as people who have been granted driver’s licenses or passports.
  • Publicly available facial images.

Before delving into these three sources of gallery images, I’m going to present a use case. A few of you may recognize it.

Let’s say that there is an important government building located somewhere, and that access to the building is restricted for security reasons. Now let’s say that some people breach that access and illegally enter the building. Things happen, and the people leave. (Again, why they left and weren’t detained immediately is outside the scope of this post.)

Now that a crime has been committed, the question arises—how do you use facial recognition to solve the crime?

A gallery of government criminal facial images

Let’s look at a case in which the images of people who trespassed at the U.S. Capitol…

Whoops, I gave it away! Yes, for those of you who didn’t already figure it out, I’m specifically talking about the people who entered the U.S. Capitol on Wednesday, January 6. (This will NOT be the only appearance of Captain Obvious in this post.)

Anyway, let’s see how the images of people who trespassed at the U.S. Capitol can be compared against a gallery of images of criminals.

From here on in, we need to not only look at technological issues, but also legal issues. Technology does not exist in a vacuum; it can (or at least should) only be used in accordance with the law.

So we have a legal question: can criminal facial images be lawfully used to identify people who have committed crimes?

In most cases, the answer is yes. The primary reason that criminal databases are maintained in the first place is to identify repeat offenders. If someone habitually trespasses into government buildings, the government would obviously like to know when the person trespasses into another government building.

But why did I say “in most cases”? Because there are cases in which a previously-created criminal record can no longer be used.

  • The record is sealed or expunged. This could happen, for example, if a person committed a crime as a juvenile. After some time, the record could be sealed (prohibiting most access) or expunged (removed entirely). If a record is sealed or expunged, then data in the record (including facial images) shouldn’t be available in the gallery.
  • The criminal is pardoned. If someone is pardoned of a crime, then it’s legally the same as if the crime were never committed at all. In that case, the pardoned person’s criminal record may (or may not) be removed from the criminal database. If it is removed, then again the facial image shouldn’t be in the gallery.
  • The crime happened a long time ago. Decades ago, it cost a lot of money to store criminal records, and due to budgetary constraints it wasn’t worthwhile to keep on storing everything. In my corporate career, I’ve encountered a lot of biometric requests for proposal (RFPs) that required conversion of old data to the new biometric system…with the exception of the old stuff. It stands to reason that if the old arrest record from 1960 is never converted to the new system, then that facial image won’t be in the gallery.

So, barring those exceptions, a search of our probe image from the U.S. Capitol could potentially hit against records in the gallery of criminal facial images.

Great, right?

Well, there’s a couple of issues to consider.

First, there are a lot of criminal databases out there. For those who imagine that the FBI, and the CIA, and the BBC, BB King, and Doris Day (yes) have a single massive database with every single criminal record out there…well, they don’t.

  • There are multiple federal criminal databases out there, and it took many years to get two of the major ones (from the FBI and the Department of Homeland Security) to talk to each other.
  • And every state has its own criminal database; some records are submitted to the FBI, and some aren’t.
  • Oh, and there are also local databases. For many years, one of my former employers was the automated fingerprint identification system provider for Bullhead City, Arizona. And there are a lot of Bullhead City-sized databases; one software package, AFIX Tracker (now owned by Aware) has over 500 installations.

So it you want to search criminal databases, you’re going to have to search a bunch of them. Between the multiple federal databases, the state and territory databases, and the local databases, there are hundreds upon hundreds of databases to search. That could take a while.

Which brings us to the second issue, in which we put on our Captain Obvious hat. If a person has never committed a crime, the person’s facial image is NOT in a criminal database. While biometric databases are great at identifying repeat offenders, they’re not so good at identifying first offenders. (They’re great at identifying second offenders, when someone is arrested for a crime and matches against an unidentified biometric record from a previous crime.)

So even if you search all the criminal databases, you’re only going to find the people with previous records. Those who were trespassing at the U.S. Capitol for the first time are completely invisible to a criminal database.

So something else is needed.

A gallery of government non-criminal facial images

Faced with this problem, you may ask yourself (yes), “What if the government had a database of people who hadn’t committed crimes? Could that database be used to identify the people who stormed the U.S. Capitol?”

Well, various governments DO have non-criminal facial databases. The two most obvious examples are the state databases of people who have driver’s licenses or state ID cards, and the federal database of people who have passports.

(This is an opportune time to remind my non-U.S. readers that the United States does not have national ID cards, and any attempt to create a national ID card is fought fiercely.)

I’ll point out the Captain Obvious issue right now: if someone never gets a passport or driver’s license, they’re not going to be in a facial database. This is of course a small subset of the population, but it’s a potential issue.

There’s a much bigger issue regarding the legal ability to use driver’s license photos in criminal investigation. As of 2018, 31 states allowed the practice…which means that 19 didn’t.

So while searches of driver’s license databases offer a good way to identify Capitol trespassers, it’s not perfect either.

A gallery of publicly available facial images

Which brings us to our third way to populate a gallery of facial images to identify Capitol trespassers.

It turns out that governments are not the only people that store facial images. You can find facial images everywhere. My own facial image can be found in countless places, including a page on the Bredemarket website itself.

There are all sorts of sites that post facial images that can be accessible to the public. A few of these sites include Facebook, Google (including YouTube), LinkedIn (part of Microsoft), Twitter, and Venmo. (We’ll return to those companies later.)

In many cases, these image are tied to (non-verified) identities. For example, if you go to my LinkedIn page, you will see an image that purports to be the image of John Bredehoft. But LinkedIn doesn’t know with 100% certainty that this is really an image of John Bredehoft. Perhaps “John Bredehoft” exists, but the posted picture is not that of John Bredehoft. Or perhaps “John Bredehoft” doesn’t exist and is a synthetic identity.

But regardless, there are billions of images out there, tied to billions of purported identities.

What if you could compare the probe images from the U.S. Capitol against a gallery of those billions of images—many more images than held by any government?

It turns out that you CAN perform that comparison, and that law enforcement did perform that comparison.

Clearview AI’s…facial-recognition app has seen a spike in use as police track down the pro-Trump insurgents who descended on the Capitol on Wednesday….

Clearview AI CEO Hoan Ton-That confirmed to Gizmodo that the app saw a 26% jump in search volume on Jan. 7 compared to its usual weekday averages….

Detectives at the Miami Police Department are using Clearview’s tech to identify rioters in images and videos of the attack and forwarding suspect leads to the FBI, per the Times. Earlier this week, the Wall Street Journal reported that an Alabama police department was also employing Clearview’s tech to ID faces in footage and sending potential matches along to federal investigators.

But now we need to return to the legal question: is “publicly available” equivalent to “publicly usable”?

Certain companies, including the aforementioned Facebook, Google (including YouTube), LinkedIn (part of Microsoft), Twitter, and Venmo, maintain that Clearview AI does NOT have permission to use their publicly available data. Not because of government laws, but because of the companies’ own policies. Here’s what two of the companies said about a year ago:

“Scraping people’s information violates our policies, which is why we’ve demanded that Clearview stop accessing or using information from Facebook or Instagram,” Facebook’s spokesperson told Business Insider….

“YouTube’s Terms of Service explicitly forbid collecting data that can be used to identify a person. Clearview has publicly admitted to doing exactly that, and in response, we sent them a cease-and-desist letter.”

For its part, Clearview AI maintains that its First Amendment government rights supersede the terms of service of the companies.

But other things come in play in addition to terms of service. Lawsuits filed in 2020 allege that Clearview AI’s practices violate the California Consumer Privacy Act of 2018, and the even more stringent Illinois Biometric Information Privacy Act of 2008. BIPA is so stringent that even Google is affected by it; as I’ve previously noted, Google’s Nest Hello Video Doorbell’s “familiar face” alerts is not available in Illinois.

Between corporate complaints and aggrieved citizens, the jury is literally still out on Clearview AI’s business model. So while it may work technologically, it may not work legally.

And one more thing

Of course, people are asking themselves, why do we even need to use facial recognition at all? After all, some of the trespassers actually filmed themselves trespassing. And when people see the widely-distributed pictures of the trespassers, they can be identified without using facial recognition.

Yes, to a point.

While it seems intuitive that eyewitnesses can easily identify people in photos, it turns out that such identifications can be unreliable. As the California Innocence Project reminds us:

One of the main causes of wrongful convictions is eyewitness misidentifications. Despite a high rate of error (as many as 1 in 4 stranger eyewitness identifications are wrong), eyewitness identifications are considered some of the most powerful evidence against a suspect.

The California Innocence Project then provides an example of a case in which someone was inaccurately identified due to an eyewitness misidentification. Correction: it provided 11 examples, including ones in which the witnesses were presented to the viewer in a controlled environment (six-pack lineups, similar backgrounds).

The FBI project, in which people look at images captured from the U.S. Capitol itself, is NOT a controlled environment.