tracking-ll144-bias-audits

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

tracking-ll144-bias-audits

★ 19 Open GitHub ↗

A crowd-sourced public tracker of bias audits of automated employment decision tools (AEDTs) released by employers related to NYC's Local Law 144.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Tracking Automated Employment Decision Tool Bias Audits

Many employers commonly use automated or AI-based tools as part of their [hiring processes](https://www.aclu.org/know-your-rights/know-your-digital-rights-digital-discrimination-in-hiring), including to advertise job opportunities, collect and filter applications, review resumes, conduct interviews, and more. These tools can enable or exacerbate discrimination in hiring based on race, sex, disability, or other protected characteristics, and applicants may not even know they are being used in the first place.

Because of a [recently enacted law](https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page), employers in New York City using these technologies are now required to work with independent auditors to conduct “bias audits” of some of the tools they use and share certain results publicly. While employers are required to provide a link to relevant bias audits on their website, these audits can be very difficult to find, and some employers have posted and later removed this information. We think that for algorithmic audits to provide meaningful transparency, they should be easily accessible. That's why the ACLU and NYCLU created this repository, where you can find examples of bias audits employers have posted publicly, as well as links to [Perma.cc](https://perma.cc/) captures of employers' bias audits.

If you know of Local Law 144 bias audits that employers have posted that we missed, you can [email us](mailto:analytics_inquiry@aclu.org) to let us know.

# Crowd-sourced Tracker

| Deploying Entity* | Tool Vendor or Name |  Auditor | Date of Audit | Original | Perma.cc Capture | Other Entities That Have Posted the Audit | 
| ----------------- | ----------- | ------------- |-------------- | --------- | -------------------- | -------------------- |
| ADP | ADP Profile Relevance and Candidate Relevancy | BLDS, LLC | 04/2023 | Removed from employer website | [Capture from 11/06/23](https://perma.cc/KF5Y-5XY7) | |
| ADP | ADP Profile Relevance and Candidate Relevancy | BLDS, LLC | 04/2024 | [From ADP.com](https://www.adp.com/-/media/adp/no-cache/candidate-relevancy/candidate-relevancy-faq.pdf?) | [Capture from 7/31/24](https://perma.cc/XFY4-BPSC) | | 
| Bloomberg | PLUM Assessment | Resolution Economics, LLC | 7/3/2023 | [From CDN](https://assets.bbhub.io/company/sites/51/2023/07/20230703-BLP-Bias-Audit-for-AEDT.pdf) | [Capture from 11/03/23](https://perma.cc/2EMX-WNYU) | |
| BNP Paribas | pymetrics inc. (Harver) Soft Skills Platform | BABL AI | 6/29/2023 | [From BNP Paribas Job Posting (see link to bias audit at the bottom of the page)](https://bnppus.tal.net/vx/lang-en-GB/mobile-0/appcentre-1/brand-7/xf-e678f51d2aa2/candidate/so/pm/1/pl/2/opp/809-2024-Full-Time-Analyst-CIB-Functions-Change-and-Solutions-Internal-Consulting/en-GB) | [Capture from 1/03/24](https://perma.cc/9KAQ-QXC3) |  |
| Dover | Dover | Varun Ganesan | 6/23/23 | [From CDN](https://cdn.dover.io/compliance/Dover%20-%20Audit_Result.pdf) | [Capture from 11/03/23](https://perma.cc/C4E7-XP63) | |
| Eightfold.AI | Eightfold Matching Model | BABL AI | 6/16/2023                | Removed from employer website as of check on 9/30/24       | <ul><li>[Capture of summary page from 11/02/23](https://perma.cc/TU38-FW6U)</li><li>[Capture of bias audit from 11/02/23](https://perma.cc/3JGK-7H76)</li></ul> | |
| Eightfold.AI | Eightfold Matching Model | BABL AI | 4/03/2023 | [From Eightfold.AI](https://eightfold.ai/nyc-eightfold-matching-model/) | [Capture from 8/28/24](https://perma.cc/M5Q5-X9DW) | | 
| HackerRank  | HackerRank Plagiarism Detection System | BABL AI |  6/30/2023 | Removed from employer website as of check on 9/30/24 | [Capture from 11/02/23](https://perma.cc/V5D7-42ZK) | |
| HackerRank | HackerRank Plagiarism Detection System | BABL AI | 7/22/2024 | [From HackerRank.com](https://support.hackerrank.com/hc/en-us/articles/18060171781523-Summary-of-Bias-Audit-Results-of-the-HackerRank-s-Plagiarism-Detection-System-for-New-York-City-s-Local-Law-144) | [Capture from 9/13/24](https://perma.cc/NU4S-7UD2) | | 
| HackerRank | HackerRank Image Analysis System | BABL AI | 6/30/2023 | Removed from employer website as of check on 9/30/24 | [Capture from 11/02/23](https://perma.cc/7XQ4-UXLE) | |
| HackerRank | HackerRank Image Analysis System | BABL AI | 7/22/2024 | [From HackerRank.com](https://support.hackerrank.com/hc/en-us/articles/18059959675539-Summary-of-Bias-Audit-Results-HackerRank-s-Image-Analysis-System) | [Capture from 9/13/24](https://perma.cc/B4FS-G7QC) | | 
| The Hartford | Unknown/Not Disclosed      | DCI Consulting Group | 7/5/2023 | Removed from employer website as of check on 11/02/23| [Capture from 8/11/23](https://perma.cc/562P-52U4) | |
| HiredScore  | HiredScore    | Unknown | 7/2/2023                | Removed from employer website as of check on 10/1/24 | [Capture from 11/02/23](https://perma.cc/96ZU-U5DE) | |
| HiredScore  | HiredScore    | Unknown | 2/12/2024               | [From HiredScore.com](https://www.hiredscore.com/nyc-legal-law) | [Capture from 10/01/24](https://perma.cc/8TGZ-7NND) | |
| JetBlue | HireVue (various assessments) | DCI Consulting Group | 7/5/2023 | Removed from employer website as of check on 10/1/24 | [Capture from 11/06/23](https://perma.cc/6Z94-QW5E) | |
| JetBlue | HireVue (various assessments) | DCI Consulting Group | 7/5/2024 | [From JetBlue.com](https://www.jetblue.com/legal/airport-operations-bias-audit) | [Capture from 10/1/24](https://perma.cc/8T9A-Y2NG) | | 
| Morgan Stanley | Eightfold Matching Model | BLDS, LLC | 7/5/2023 | Removed from employer website as of check on 10/1/24 | [Capture from 11/03/23](https://perma.cc/J844-MEBQ) | |
| Morgan Stanley | Eightfold Matching Model | BLDS, LLC | 7/31/2024 | [From MorganStanley.com](https://www.morganstanley.com/content/dam/msdotcom/en/disclaimers/Bias_Audit_of_Morgan_Stanleys_Use_of_Eightfold_Model_For_Scoring_Applicants.pdf) | [Capture from 10/1/24](https://perma.cc/A8TT-MN38) | |
| NBCUniversal | SmartRecruiters SmartAssistant | Conductor AI | 7/11/2023 | Removed from Conductor AI's website as of check on 10/16/24 | [Capture from 11/03/23](https://perma.cc/YZ8T-CK5M) | [AdTalem](https://careers.adtalem.com/aedt-notice)|
| NYU Langone Health | Talent Plus | Ambrose Consulting, LLC | 2023 | [From CDN](https://cdn-static.findly.com/wp-content/uploads/sites/641/2023/06/29155721/2023-Audit-Results-002.pdf) | [Capture from 1/03/24](https://perma.cc/83UA-BVJ9) | |
| Paradox | Paradox's Olivia Chatbot | Holistic AI | 6/5/2023 | [From Paradox.AI](https://trust.holisticai.com/nyc-bias-audit/eyJjbGllbnRJZCI6InBhcmFkb3gtcWpkbGEiLCJlbnRpdHlJZCI6IjMwODRmNjdmLTE0ZjEtNDM1Yy04ZmJiLTQyNDQ2ZWUwYzM4ZCIsInN5c3RlbUlkIjoiMzA4NGY2N2YtMTRmMS00MzVjLThmYmItNDI0NDZlZTBjMzhkIn0) | [Capture from 11/02/23](https://perma.cc/CSA8-S2RC) | |
| Paradox | Paradox's Conversational AI Platform | Holistic AI | 5/6/2024 | [From Paradox.AI](https://trust.holisticai.com/nyc-bias-audit/eyJjbGllbnRJZCI6InBhcmFkb3gtcWpkbGEiLCJlbnRpdHlJZCI6IjU2MWM5MjZiLWQ5MzYtNGZlNy1iM2U2LThjZDEzZDcyY2ExOCIsInN5c3RlbUlkIjoiNTYxYzkyNmItZDkzNi00ZmU3LWIzZTYtOGNkMTNkNzJjYTE4In0) | [Capture from 9/27/24](https://perma.cc/UU6V-DX56) | |
| Paradox | Paradox's Traitify Tool | Holistic AI | 6/12/2023 | [From Paradox.AI](https://trust.holisticai.com/nyc-bias-audit/eyJjbGllbnRJZCI6InBhcmFkb3gtcWpkbGEiLCJlbnRpdHlJZCI6Ijk5MjA2OTA1LThiZGUtNDlhNy1iODhjLWM3NWZmYjkxZDk2YiIsInN5c3RlbUlkIjoiOTkyMDY5MDUtOGJkZS00OWE3LWI4OGMtYzc1ZmZiOTFkOTZiIn0) | [Capture from 11/02/23](https://perma.cc/SK87-ZFY5) | |
| Paradox | Paradox's Traitify Tool | Holistic AI | 5/6/2024 | [From Paradox.AI](https://trust.holisticai.com/nyc-bias-audit/eyJjbGllbnRJZCI6InBhcmFkb3gtcWpkbGEiLCJlbnRpdHlJZCI6IjAwZmM3MzM2LTU2NmQtNGU5NS1iNDQ4LWMyOTM0ZjgzMjM3OCIsInN5c3RlbUlkIjoiMDBmYzczMzYtNTY2ZC00ZTk1LWI0NDgtYzI5MzRmODMyMzc4In0) | [Capture from 9/27/24](https://perma.cc/CE76-L7L4) | |
| Pfizer | HireVue (various assessments) | DCI Consulting Group | 7/5/2023 | [From CDN](https://cdn.pfizer.com/pfizercom/2023-09/HireVue%202023%20Bias%20Report_2023Aug14.pdf) | [Capture from 11/03/23](https://perma.cc/4EQP-MLYA) | |
| Plum | Plum Platform | Holistic AI | 8/23/2023 | [From use.plum.io](https://use.plum.io/hubfs/Resources/Audit/2022-2023/2022-2023_Plum_Audit_Report_Summary.pdf) | <ul><li>[Capture of executive summary from 1/03/24](https://perma.cc/955M-P4EU)</li><li> [Capture from 10/1/24](https://perma.cc/M8PS-4QMG)</li> | |
| Reejig | Reejig Ethical Talent AI | Holistic AI | 11/30/2023 | [From Reejig.com](https://www.reejig.com/ethical-talent-ai-audit-results) | [Capture from 1/03/24](https://perma.cc/MTD7-NTLM) | |
| RippleMatch | RippleMatch Fit Score Algorithm | BABL AI | 6/22/2023 | Removed from employer websites as of check on 10/1/24 | [Capture from 11/03/23](https://perma.cc/BXW6-7EMA) | |
| RippleMatch | RippleMatch Fit Score Algorithm | BABL AI | 6/27/2024 | [From Ripplematch.com](https://ripplematch.com/wp-content/uploads/2023/06/RippleMatch-Fit-Score-Algorithm-%E2%80%93-Public-Summary.pdf) | [Capture from 9/24/2024](https://perma.cc/A637-PUTT) | |
| Sheppard Mullin | Suited Law Assessment | Holistic AI | 5/26/2023 | Removed from employer websites as of check on 10/1/24 | [Capture from 11/02/23](https://perma.cc/8MA2-P6VB) | |
| Sheppard Mullin | Suited Law Assessment | Holistic AI | 3/11/2024 | [From WellSuited](https://wellsuited.notion.site/Audit-Results-c01787a838974d53ae93e8ab5cbee291) | [Capture from 10/1/2024](https://perma.cc/GZY5-UKZN) | |
| Wells Fargo | Unknown | DCI Consulting Group | 7/5/2023 | [From WellsFargoJobs.com](https://www.wellsfargojobs.com/media/afddvztf/teller-impact-ratio-analysis.pdf) | [Capture from 11/03/23](https://perma.cc/D7SW-K7WQ) | |
| Covey | Covey | Conductor AI | 8/10/2023 | Removed from employer website as of check on 9/30/24 | [Capture from 2/1/24](https://perma.cc/WMG2-N8HH) | [Vannevar Labs](https://perma.cc/B4V6-QAMP) |
| Covey | Covey | Idiro Analytics | 7/29/2024 | [From getcovey.com](https://getcovey.com/nyc-local-law-144) | [Capture from 9/27/24](https://perma.cc/79YC-5PJT) | | 
| Vervoe | Vervoe Assessment | Holistic AI | 3/21/2024 | [From vervoe.com](https://vervoe.com/ai-bias-audit-results/) | [Capture from 4/24/24](https://perma.cc/2G85-KUSG) |  |
| ZoomInfo | ZoomInfo | Unknown | 5/15/2023 | [From ZoomInfo.com](https://www.zoominfo.com/legal/nyc-local-law-144-notice) | [Capture from 4/24/24](https://perma.cc/YHV4-XVFC) | |
| Burlington | HireVue (various assessments for Intern or new college graduate jobs) | DCI Consulting Group | 7/5/2023 | [From CDN](https://dn9tckvz2rpxv.cloudfront.net/burlington/pdf/AI%20audit%20for%20EC.pdf) | [Capture from 4/24/24](https://perma.cc/9BK7-X4UB) |
| Burlington | HireVue (various assessments for Intern or new college graduate jobs) | DCI Consulting Group | 7/5/2024 | [From CDN](https://seo.nlx.org/burlington/pdf/Burlington%20DCI%20Audit%20Results%20%28002%29.pdf) | [Capture from 10/1/24](https://perma.cc/RD7H-V44G) |
| Burlington | HireVue assessment used for Allocation Analyst role | DCI Consulting Group | 11/1/2023 | [From CDN](https://dn9tckvz2rpxv.cloudfront.net/burlington/pdf/NYC%20Audit%20AA.pdf) | [Capture from 4/24/24](https://perma.cc/VL7X-5T5L) |
| Burlington | HireVue assessment used for Intern role | DCI Consulting Group | 11/1/2023 | [From CDN](https://dn9tckvz2rpxv.cloudfront.net/burlington/pdf/NYC%20Audit%20Intern.pdf) | [Capture from 4/24/24](https://perma.cc/LA9R-UYXP) | |
| Bryq | Bryq Talent Intelligence | Holistic AI | 5/15/2023 | [From CDN](https://framerusercontent.com/assets/VJW6JIM9p7ZG7H3gbRthHc3UFM.pdf) | [Capture from 4/24/24](https://perma.cc/NNR6-9DW3) | |
| Bryq | Bryq Talent Intelligence | Holistic AI | 5/17/2024 | [From CDN](https://framerusercontent.com/assets/li09IQfzjxCZjmNyKNCEnZy5fM.pdf) | [Capture from 10/1/24](https://perma.cc/3TJD-EQE4) | |
| Citizens | HireVue (various assessments) | DCI Consulting Group | 7/5/2023 | Removed from employer website as of check on 10/1/24 | [Capture from 7/10/24](https://perma.cc/GD69-APXU) | | 
| Citizens | HireVue (various assessments) | DCI Consulting Group | 7/5/2024 | [From CitizensBank.com](https://jobs.citizensbank.com/Annual-AI-Bias-Audit) | [Capture from 10/1/24](https://perma.cc/CT4J-PRQZ) | | 
| Cigna | Unknown | Unknown | 10/18/2023 | [From Cigna.com](https://www.cigna.com/static/www-cigna-com/docs/nyc-aedt-bias-audit-2023.pdf) | [Capture from 10/16/24](https://perma.cc/8FUT-SLLJ) | | 
| Alidoro | Landed | Conductor AI | 6/27/2023 | [From CDN](https://assets-global.website-files.com/646e99830281b54b8ee06d17/649cab95673936630f6fbfe7_Alidoro%20Bias%20Audit.pdf) | [Capture from 10/16/24](https://perma.cc/HMT5-Q6Y4) | | 
| SmartRecruiters | SmartAssistant | Conductor AI | 8/02/24 | [From CDN](https://nyc-144-audits.s3.amazonaws.com/2024/smartrecruiters.pdf) | [Capture from 10/16/24](https://perma.cc/T9RX-KV79) | | 
| Landed | Landed | Conductor AI | 6/26/2024 | [From Employer.GotLanded.com](https://employer.gotlanded.com/nyc-audit-2024) | [Capture from 10/16/24](https://perma.cc/9GZ9-UMGZ) | | 
| Phenom | Phenom Fit Score | Conductor AI | 4/04/2024 | <ul><li>[Phenom Fit Score Report 2024](https://assets.phenom.com/hubfs/02_Assets/ebook/Phenom_Fit_Score_2024_Audit_Report.pdf)</li><li> [Phenom Bias Audit](https://go.phenom.com/hubfs/Phenom%20Fit%20Score%202024%203rd%20Party%20Bias%20Audit.pdf)</li> | <ul><li>[Capture of Phenom Fit Score Report 2024 from 10/16/24](https://perma.cc/B74J-AV5S)</li><li> [Capture of Phenom Bias Audit 10/16/24](https://perma.cc/238P-47MT)</li>
| Beamery | AI Talent Match | Warden AI | 09/17/2024 | <ul><li>[Beamery AI Assurance Dashboard](https://trust.warden-ai.com/beamery)</li><li>[Beamery AI Assurance Dashboard Results](https://trust.warden-ai.com/beamery/ai-talent-match/bias)</li><li>[Beamery Bias Audit Report](https://trust.warden-ai.com/beamery/ai-talent-match/nyc)</li> | <ul><li>[Capture of Beamery AI Assurance Dashboard from 10/29/24](https://perma.cc/AWA7-QCMW)</li><li>[Capture of Beamery AI Assurance Dashboard Results from 10/29/24](https://perma.cc/R6TN-3M7T)</li><li>[Capture of Beamery Bias Audit Report from 10/29/24](https://perma.cc/VB3U-BZ6M)</li> 
| Sense | Sense Candidate Matching | Holistic AI | Unknown | [From SenseHQ.com](https://www.sensehq.com/blog/sense-passes-bias-audit-to-comply-with-nyc-hiring-law) | [Capture from 11/05/24](https://perma.cc/M8DB-N6XX)
| Sense | Sense Chatbot | Holistic AI | Unknown | [From SenseHQ.com](https://www.sensehq.com/blog/sense-passes-bias-audit-to-comply-with-nyc-hiring-law) | [Capture from 11/05/24](https://perma.cc/M8DB-N6XX)

*The term "deploying entity" generally refers to the entity that posted the associated audit report on their website, which may be a tool vendor or an employer using a vendor's tool.

# License

The tracker in this repository is licensed under [CC BY-NC 4.0](http://creativecommons.org/licenses/by-nc/4.0/).

tracking-ll144-bias-audits

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README