New York City Issues Final Rules and New Enforcement Date for its Artificial Intelligence (AI) Law

The New York City Department of Consumer and Worker Protection (“DCWP”) published updated rules to implement its Artificial Intelligence (AI) law, New York City Local Law 144 of 2021. The AI law places restrictions on employers and employment agencies' use of Automated Employment Decision Tools ("AEDT”) used to screen candidates or employees for employment decisions.

The DCWP initially published proposed rules on September 23, 2022, and after a public hearing on November 4, 2022, the DCWP published an updated version of the proposed rules on December 23, 2022. After considering feedback from a hearing held on January 23, 2022, the DWCP published a final version of these rules on April 6, 2023 (the “Final Rules”).

Although the AI law was originally set to go into effect on January 1 of this year, the DCWP has again indicated that it pushed back enforcement to July 5, 2023.  While the Final Rules are dense and very detailed, below are key considerations for employers as they prepare for compliance. 

What is an AEDT?

The NYC law applies to AEDTs that “substantially assist or replace discretionary decision making.” As in its earlier iteration, the Final Rule defines this phrase to mean:

• to rely on a simplified output (score, tag, classification, ranking, etc.), with no other factors considered; or
• to use a simplified output as one of a set of criteria where the simplified output is weighted more than any other criterion in the set; or
• to use a simplified output to overrule conclusions derived from other factors including human decision-making.

What must employers do?

The first task for employers is to understand what tools, if any, they use to “substantially assist or replace discretionary decision making” as further clarified by the Final Rules.  For any of those covered tools, the Final Rules clarify the requirements for conducting a bias audit as well as the related publication and notice requirements.

To comply with New York City’s AI law, employers must:

• Subject AEDTs to an independent bias audit within one year of use;
• Provide a publicly available summary of the audit;
• Give notice (described below) to candidates and employees about the use of the AEDT;
• Allow candidates or employees to request alternative evaluation processes as an accommodation, if available (the proposed rules do not create an obligation to create an independent alternative selection process); and
• Make available information about the source and type of data collected, unless disclosure would violate the law/interfere with a law enforcement investigation (in which case the employer or employment agency must provide an explanation of why disclosure of such information would violate law/interfere with a law enforcement investigation).

Who may conduct the audit?

One significant open question that these Final Rules seek to address is who can conduct the necessary bias audit.  The Final Rules specify that the audit must be conducted by an “independent auditor,” which is a person or group “capable of exercising objective and impartial judgment.” Importantly, the auditor must not have been “involved in using, developing, or distributing the AEDT” and must not have an employment relationship or a financial interest with the employer seeking to use the AEDT or with the vendor that has developed or distributed the AEDT. 

What data must employers calculate?

The Final Rules also provide additional details on what data needs to be analyzed to conduct a compliant bias audit.  At a high level, the Final Rules explain that:

• When an AEDT selects candidates for employment or employees to be considered for promotion the auditor must calculate the impact ratio for all “sex categories,” ”"race/ethnicity categories,” and “intersectional categories of sex, ethnicity, and race” (the “demographic categories”).
• The audit must calculate the (i) median score for the full sample of applicants; (ii) scoring rate for individuals in each demographic category; (iii) the impact ratio for each demographic category; and (iv) separately calculate the impact on each demographic category (i.e. separately analyze the impact ratio for selection of female v. male candidates to satisfy the required analysis for sex categories).
• Indicate the number of individuals not included in the analysis because their demographic categories are unknown.
• The auditor may exclude a category that represents less than 2% of the data used for the bias audit when calculating the impact ratio. However, the summary of the results “must include the independent auditor’s justification for the exclusion” and “the number of applicants and scoring or selection rate for the excluded category.”
• Use historical data unless there is not sufficient historical data (i.e. it is a new AEDT), in which case “test data” can be used in the bias audit.
• The audit may use historical data from other employers or employment agencies if the employer or employment agency (1) provides such data from its own use of the AEDT or (2) if the employer or employment agency has not previously used the AEDT.

What are the notice and publication requirements?

Once the employer identifies its covered AEDTs and receives the results of the independent bias audit for using the AEDT, the employer must make publicly available on its website (or provide a link to an external website) the following information:

• the date of the most recent bias audit;
• the summary of results, including the source and explanation of data used;
• the number of individuals the AEDT assessed that fell in an unknown category;
• the number of candidates;
• the selection, scoring, and impact rates, as applicable; and
• the distribution date.

Employers may provide a hyperlink to an active website with these results and this information must be posted for at least 6 months after the latest use of the AEDT.

With respect to notice, employers must provide at least 10 business days’ notice to candidates and employees that an AEDT will be used.  The Final Rules clarify that to comply with the law, an employer may be required to provide a reasonable accommodation and allow a candidate or employee to request an alternative selection process but that nothing “requires an employer or employment agency to provide an alternative selection process.”

***

For advice on complying with this law and the Final Rules, please contact a member of Orrick’s Employment team.