omnigarde Archives - Bredemarket

Does Your 1:N Facial Algorithm Meet Minimum 1:1 Accuracy?

Minimum 1:1 accuracy enforcement. Google Gemini..

Starting November 30, NIST is enforcing a minimum 1:1 accuracy requirement for developers who wish to participate in 1:N testing.

As a reminder, NIST conducts a Face Recognition Technology Evaluation (previously known as the Face Recognition Vendor Tests). This testing comes in two flavors: 1:1 verification and 1:N identification, known as FRTE 1:1 and FRTE 1:N, respectively.

The 1:1 testing is conceptually less difficult than 1:N, though it’s difficult enough. For 1:1 testing, you simply compare one biometric sample against another biometric sample to determine if they came from the same person. For 1:N testing, you compare a single biometric sample against an entire database of biometric samples. Up to 12 million in one of the NIST tests. Many, many more in the real world.

Would you rather compare against one biometric sample, or 12 million?

Changes to minimum 1:1 accuracy (time may change you)

For years, NIST allowed anyone to submit their algorithms for NIST testing, no matter how inaccurate they were. The results would be publicly displayed for all to see, so there was a disincentive to apply if you know your algorithm is poor.

But inaccurate algorithm providers submitted anyway. To those not familiar with NIST test results, an False Negative Identification Rate (FNIR) of 1.0000 is not good. As of this year, NEC has achieved an FNIR on the visa-bordrt test of 0.0012, which is…much better.

But as facial recognition algorithms improve in accuracy, NIST has decided to enforce some minimum standards on 1:N test participation. Specifically, you need to reach a minimum accuracy in the 1:1 test.

“Prerequisite Requirement (New Participants): Participants must first submit at least one algorithm to the FRVT 1:1 track, and its accuracy must exceed at least one of the following:

“Visa-Border dataset: FNMR of 0.025 or less at FMR=0.000001”

Yes, there was an editing error somewhere along the way, since there is ONLY one accuracy requirement. But there is a timing requirement that does not concern me here.

And this minimum requirement only applies to new participants. So the assumed workflow is that a new participant submits to 1:1 first, then once they clear the accuracy hoop they can submit to 1:N.

The relationship between 1:1 and 1:N

Years ago, a Bredemarket client asked me to review its (then) FRVT 1:1 results. I reviewed them; they performed excellently. But I pointed out to the client that for its particular market 1:1 was meaningless, and the company REALLY needed to submit to FRVT 1:N.

The company then revealed that it had also submitted its algorithm for 1:N testing. Eventually NIST released those results, and the company performed excellently for 1:N as well.

But today, even though 1:1 and 1:N address different problems, new NIST test participants must pursue 1:1 accuracy first to make sure they can participate in 1:N.

Reach for the stars

So if you’re new to this industry, set your priorities. Get your 1:1 house in order.

Of course, some “new” participants aren’t that new. Omnigarde had never written a facial recognition algorithm before, the principal having specialized in fingerprint algorithms through decades of corporate life. But that experience helped Omnigarde develop its facial recognition algorithm, which is in the upper tier of providers. (2025 1:N results here.)

But what will happen to facial accuracy in the future? Yes, it’s been improving substantially over the past two decades. But it may get much better.

NIST Implements Color Coding to Visually Identify High False Positive Facial Recognition Values

With the exception of colorblind people, the use of colors in dashboards makes information more accessible, particularly in populations where green means “good” and red means “bad.”

(Even if your name IS Bamber.)

The National Institute of Standards and Technology understands the importance of consistent colors, having worked on traffic light colors since the National Bureau of Standards days (PDF).

For more modern applications such as biometrics, NIST recently incorporated a color coding display change to one of its tabs for the “Face Recognition Technology Evaluation (FRTE) 1:N Identification” results. Specifically, the “Demographics: False Positive Dependence” tab.

The change, announced in an email, is as follows:

“The false positive identification error rate tables now include color-coding to highlight anomalously high values.”

In this context, “anomalously high” is bad, or red. (Actually dark pink, but close enough.)

But let’s explain WHY and HOW NIST made this change.

Why does NIST highlight demographic false positive dependence?

NIST has of course explored the demographic effects of face recognition for years, and the “Demographics: False Positive Dependence” tab provides additional tracking for this.

Why does NIST do this?

“False positives occur when searches return wrong identities. Such outcomes have application-dependent consequences, which can be serious.”

Serious as in arresting and jailing innocent people. I previously mentioned Robert Williams.

How does NIST highlight demographic false positive dependence?

Anyway, NIST created the “Demographics: False Positive Dependence” tab.

“The table shows false positive identification rates (FPIR), the fraction of searches that should not return gallery entries above a threshold, but do. The threshold is set for each algorithm to give a FPIR of 0.002 (1 in 500) or less on searches of women born in Eastern Europe.”

And for algorithms that have “anomalously high values” in other demographic populations, NIST has added the color coding.

“A cell is shaded by how much larger FPIR is than that: yellow if FPIR is 20 times larger; pink if FPIR is 40 times larger; and dark pink if FPIR is 80 times larger.”

What does the highlighting look like?

Let me illustrate this with the results from the three algorithms Omnigarde submitted.

Omnigarde’s first two algorithms, submitted in 2023 and 2024, exhibited high FPIR values for south Asian females, and the second algorithm also exhibited a high FPIR value for east Asian females. See the color coding.
The third algorithm, submitted in 2025, had lower FPIR values for these populations and thus no yellow color coding.

Even the less-stellar algorithms show improvement over time.

Data captured April 8, 2026. Anonymized (but you can figure it out if you’re curious).

Final thoughts

Both vendors and customers/prospects can rightfully question whether this is helpful or hurtful. I lean toward “helpful,” because if the facial recognition algorithm you use provides high false positives for certain popularions, you need to know.

And as always, law enforcement in the United States should NEVER solely rely on facial recognition results as the basis for an arrest…even for Eastern European females. They should ONLY be an investigative lead.

In the meantime, take care of yourself, and each other.

Jerry Springer. By Justin Hoch, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=16673259.

Omnigarde Peter Lo Biography: I Need to Steal This Idea

As we approach 2026, advanced biometric firm Omnigarde has released new marketing materials. One of these is a video biography of Omnigarde’s principal, Dr. Peter Lo.

Dr. Peter Lo.

Of all the videos I’ve created, I’ve never created a “Who I Am” video. Not that I have the industry recognition that Dr. Lo has…