Who must comply with NYC Local Law 144?

Any employer or employment agency using an automated employment decision tool (AEDT) to screen candidates for a job located in New York City, regardless of where the employer is headquartered. They must run an independent bias audit within the prior year, publish the summary, and notify candidates at least 10 business days in advance.

When do EU AI Act obligations apply to hiring systems?

AI systems used for recruitment and employee selection are high-risk under Annex III 4(a). High-risk obligations for hiring systems apply from August 2, 2026, covering risk management, data governance, human oversight, transparency, logging, and post-market monitoring.

An Automated Employment Decision Tool is any computational process derived from machine learning, statistics, or AI that produces a score, classification, or recommendation used to substantially assist or replace discretionary hiring or promotion decisions.

What are the penalties for NYC Local Law 144 non-compliance?

Fines start at $500 for a first violation and reach $1,500 for each subsequent violation. Each day of non-compliant use and each missed candidate notice counts as a separate violation.

Can our AI vendor perform our bias audit?

No. NYC Local Law 144 requires an independent auditor with no role in building, selling, or using the tool and no financial interest in it. Vendors can supply data but cannot serve as the independent auditor.

How Independent AI Hiring Bias Audits Work Under NYC Local Law 144

Summary: A bias audit under Local Law 144 is an independent statistical evaluation of an automated hiring tool. It measures how often the tool selects candidates from different demographic groups, then compares those rates using an impact ratio. The audit must be run by an independent party, must cover sex, race and ethnicity, and intersectional categories, and must be repeated every year. The output is a summary that employers publish so candidates can see how the tool performs before they apply.

Key takeaways

A bias audit measures selection rates and impact ratios across protected groups, not the internal design of the model.
The four-fifths rule is the common benchmark: an impact ratio below 0.8 signals a group is being selected at a notably lower rate.
The auditor must be genuinely independent, with no role in building, selling, or using the tool.
Data quality is the make-or-break input. Missing demographic data forces the use of estimation methods and weakens the result.
The audit is annual, so it should be treated as a recurring program, not a one-off project.

What a bias audit actually measures

A bias audit does not open up the model and inspect its logic. It measures outcomes. Specifically, it asks how often the tool advances candidates from each demographic group, and whether any group is advanced at a meaningfully lower rate than the group that fares best.

This outcome-based approach is deliberate. Two tools can be built very differently and still produce the same disparity in who moves forward. What matters to a candidate, and to the law, is the result. The audit therefore focuses on selection rates and the ratios between them.

Selection rate and impact ratio, explained

The selection rate for a group is the share of people in that group who receive a favorable outcome from the tool, such as being scored highly enough to advance. If 100 women are evaluated and 40 are advanced, the selection rate for women is 40 percent.

The impact ratio compares the selection rate of one group to the selection rate of the most-selected group. If men are advanced at 50 percent and women at 40 percent, the impact ratio for women is 40 divided by 50, which is 0.8.

The widely used benchmark is the four-fifths rule. An impact ratio below 0.8 is treated as a signal that the tool may be selecting one group at a substantially lower rate than another. It is not an automatic verdict of illegality, but it is a flag that warrants investigation and, often, remediation.

Which groups the audit must cover

Local Law 144 requires the audit to calculate these metrics across sex categories, race and ethnicity categories, and intersectional categories that combine the two, such as the selection rate for Hispanic women or Asian men. Intersectional analysis matters because a tool can look balanced on sex alone and on race alone, yet still disadvantage a specific combined group. Averages hide this. Intersectional breakdowns reveal it.

Who can perform the audit

The auditor must be independent. That means no involvement in building the tool, no involvement in selling it, no role in using it, and no financial interest in its continued use. The purpose is to remove the conflict of interest that would arise if the party being evaluated also graded the exam.

This requirement trips up more companies than any other part of the law. The most convenient source of audit data is usually the vendor, and the vendor is precisely the party who cannot serve as the independent auditor. A vendor can provide the underlying data, but an independent third party has to run and sign the analysis.

What data you need to be audit-ready

The audit runs on your historical selection data broken down by demographic group. In practice you need records of who was evaluated by the tool, what outcome each candidate received, and the sex and race or ethnicity of those candidates.

The common obstacle is missing demographic data, because many candidates decline to self-identify. When self-reported data is thin, auditors can use test data the tool's developer provides, or apply statistical estimation methods. Both are permitted, but both are weaker than clean historical data. The single most useful thing you can do before an audit is improve the completeness of your demographic collection through voluntary, well-designed self-identification.

Interpreting the result

An audit rarely comes back as a clean pass or fail. More often it produces a set of impact ratios, some comfortably above 0.8 and some below. A ratio below the threshold does not mean you must abandon the tool. It means you have identified a disparity that needs explanation and, where the disparity is not job-related and consistent with business necessity, mitigation.

This is where the audit becomes genuinely useful rather than a compliance tax. A disparity in the data is often pointing at something fixable in how the tool is configured or how the role is defined. Treating the audit as diagnostic, not just defensive, is what separates companies that improve from companies that merely file paperwork.

Turning the audit into a summary

Once complete, the audit results are distilled into a summary that you publish on your hiring page, alongside the date the tool was first used. The summary needs to be findable by candidates. Publishing it in a hard-to-reach legal document defeats the transparency purpose the law was written to serve.

Frequently asked questions

What is the four-fifths rule? A benchmark that flags a possible adverse impact when a group's selection rate is less than four-fifths, or 0.8, of the most-selected group's rate.

Can the vendor run our bias audit? No. The auditor must be independent and cannot have built, sold, or used the tool, or have a financial stake in it.

What if we do not have demographic data on our candidates? Auditors can use developer test data or statistical estimation, but clean historical data produces a stronger result. Improving voluntary self-identification before the audit is worthwhile.

Does a failing impact ratio mean the tool is illegal? Not automatically. It flags a disparity that requires investigation and, where the disparity is not justified by business necessity, mitigation.

How often do we audit? Every year, for as long as the tool is in use. The most recent audit must fall within the prior 12 months.

A bias audit is most valuable when it is more than a box to tick. Run well, it tells you exactly where your hiring tools treat people differently and what to do about it. If you want an independent audit that produces both the compliant summary and a clear remediation path, that is the work we do.