AI FAIRNESS
GLOBAL LIBRARY
Resources by origin
North America
AI Fairness 360
Bias and Fairness Audit Toolkit
This extensible open source toolkit can help you examine, report, and mitigate discrimination and bias in machine learning models throughout the AI application lifecycle. We invite you to use and improve it.
Open source tech tool for auditing the data and predictions of machine learning solutions for their fairness. Requires relatively light technology background (can be used by non-developers)
Produced mainly for: Tech teams
Origin: North America
Language: English
Type: Tech tool
Creator: IBM
Origin: North America
Language: English
Type: Tech tool
Creator: Chicago University
%2019_10_15.png)
Audit-AI
Open sourced bias testing tool useful for developers, that seems active for updating
Produced mainly for: Tech teams
Origin: North America
Language: English
Type: Tech tool
Creator: Pymetrics
Themis™ o Themis-ML
A library that implements fairness-aware machine learning algorithms
Produced mainly for: Tech teams
Origin: North America
Language: English
Type: Tech tool
Creator: Massachussets University -
FairTest
Enables developers or auditing entities to discover and test for unwarranted associations between an algorithm's outputs and certain user subpopulations identified by protected features.
Produced mainly for: Tech teams
Origin: North America
Language: English
Type: Tech tool
Creator: Universidad de Columbia, Stanford, EPFL, Saarland Univ., Cornell Tech, Jacobs Institute
AI For Everyone Course
Povide AI practitioners and non-technical professionals with the necessary tools to go all the way from foundational basics to advanced application, empowering them to build an AI-powered future.
Produced mainly for: For everyone
Origin: North America
Language: English
Type: Course
Creator: Andrew Ng
Fairness Flow
Engages in cutting-edge research that can improve and power new product experiences at a huge scale for our community. Building on Facebook AI's key principles of openness, collaboration, excellence, and scale, we make big, bold research investments focused on building social value and bringing the world closer together.
Produced mainly for: Non - tech teams
Origin: North America
Language: English
Type: Guide or manual
Creator: Facebook
Ethical Toolkit for Engineering/Design Practice
Santa Clara University - Multi set of tools implementing ethical reflection, deliberation, and judgment into engineering and design workflows
Origin: North America
Language: English
Type: Tech tool
Creator: Markkula Center for Applied Ethics – Santa Clara University
.jpg)
Fairness Toolkit (LiFT)
Linkedin - open source software library designed to enable the measurement of fairness in AI and machine learning workflows.
Origin: North America
Type: Tech tool
Creator: LinkedIn
Real World AI: A Practical Guide to Responsible Machine Learning
This practical guide to deploying AI lays out a human-first, responsible approach that has seen more than three times the success rate when compared to the industry average.
Produced mainly for: C Level, Manager, Tech Team & Non - Tech Team
Origin: North America
Language: English
Type: Guide
Creator: Wilson Pang, CTO at Appen, and Alyssa Simpson Rochwerger, Director of Product at Blue Shield of California and Former VP of Data and AI at Appen
AI Ethics for Nonprofits toolkit
Participants will learn some of the fundamentals of AI ethics and then immediately get to practice applying ethical considerations related to the principle of Fairness in the context of several humanitarian and international development use cases
Produced mainly for: C Level, Manager, Tech Team, Non Tech Team
Origin: North America
Language: English
Type: Tool tech & Guide
Creator: NetHope (consortium of 60 INGOs) in partnership with MIT D-Lab and USAID
.jpg)
Ethics & Algorithms Toolkit
A practical toolkit for cities to use to help them identify the risks of using an AI algorithm, and maps out the mitigating measures for different risks.
Origin: North America
Language: English
Type: Tech tool
Creator: GovEx, the City and County of San Francisco, Harvard DataSmart, Data Community DC
%2011_10_37.png)
Model Cards
Short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups and intersectional groups that are relevant to the intended application domains.
Produced mainly for: C levels & Managers
Origin: North America
Language: English
Type: Guide or manual
Creator: Google
Bias in machine learning and ethical implications
The report outlines ways in which financial institutions are cautiously implementing Machine Learning into their processes, such as in credit risk assessment and anti-money laundering, and the mathematical as well as social components of fairness to be considered.
Produced mainly for: C leve & Managers
Origin: North America
Language: English
Type: Guide
Creator: Institute of International Finance
Responsible AI from Pilot to Production
A biased model that works for some users, and not others, is a failed model. Or a model that wasn’t sourced responsibly can be a poor reflection of company values and a nightmare with the media. It’s helpful to remember that AI reflects the people and the company that build it: when something goes wrong, it shows something may also be wrong internally.
Produced mainly for: C Level, Manager, Tech Team & Non - Tech Team
Origin: North America
Language: English
Type: Guide
Creator: Appen
The Impact of Ethics and Bias on Artificial Intelligence
AI is the study and research of providing software and hardware that attempts to emulate a human being. Cognitive computing is computing focused on reasoning and understanding that is inspired by human cognition. It is a subset of AI.
Produced mainly for: C levels & Managers
Origin: North America
Language: English
Type: Framework
Creator: David Schubmehl, Research Director, Cognitive/AI Systems
Ethical OS Toolkit
For visualizing and anticipating future risk of technology products, acknowledging that once technology is released and reaches scale it may be used for purposes beyond the original intention.
Origin: North America
Language: English
Type: Guide or Manual
Creator: The Institute for the Future, Omidyar Network‘s Tech and Society Solutions Lab
Asia
RCModel, a Risk Chain Model for Risk Reduction in AI Services
Model AI Governance Framework
The risk chain model (RCModel) supports AI service providers in proper risk assessment and control, and offers rpolicy recommendations
Provides detailed and readily-implementable guidance to private sector organisations to address key ethical and governance issues when deploying AI solutions.
Origin: Asia
Language: English
Type: Guide or manual
Creator: University of Tokyo
Produced mainly for: Managers & C levels
Origin: Asia
Language: English
Type: Guide
Creator: PDPC
From Principles to Practice: use cases for implementing responsible AI in financial services
Artificial intelligence - Governance framework model. Second Edition
Findings from work by a group of partners in a project to explore the implementation of responsible AI principles, including Deutsche Bank, Linklaters, Microsoft, Standard Chartered, and Visa.
Addition of industry examples in each section to illustrate how organizations have implemented AI governance practices.
Produced mainly for: Managers & C levels
Origin: Asia
Language: English
Type: Guide
Creator: Microsoft, based on MAS FEAT principles
Produced mainly for: Government
Origin: Asia
Language: English
Type: Guide
Creator: S Iswaran- Minister for Communications and Information- Singapore
Model Artificial Intelligence Governance Framework Second edition
Incorporates the experiences of organizations that have adopted AI, and feedback from our participation in leading international platforms, such as the European Commission’s High-Level Expert Group and the OECD Expert Group on AI.
Produced mainly for: Manager
Origin: Asia
Language: English
Type: Guide or manual
Creator: SG:D, IMDA, PDPC (Singapore)
Veritas Initiative
Framework for financial institutions to promote the responsible adoption of Artificial Intelligence and Data Analytics. It has developed a fairness assessment methodology in credit risk scoring and customer marketing, and has published whitepapers on the fairness assessment methodology and the open source code of these two use cases.
Produced mainly for: Tech teams
Origin: Asia
Language: English
Type: Tech tool
Creator: Monetary Authority of Singapore
Europe
Review into bias in algorithmic decision-making
It's more an educational publication than a tool (as the name suggests: "Review of.."). However, also provides some (high-level) recommendations for governments and regulators.
Origin: Europe
Language: English
Type: Guide
Creator: Centre for Data Ethics and Innovation
Guide of Algorithmic Auditing
An algorithmic audit identifies, addresses and corrects algorithmic bias so that you can act on informed and vetted AI-powered business recommendations.
Produced mainly for: Companies using AI
Origin: Europe
Language: English & Spanish
Type: Tech tool
Creator: Eticas
Explaining decisions made with AI
Mainly focused on explaining decisions made with AI, but it contains fairness issues in the model
Origin: Europe
Language: English
Type: Framework
Creator: Information Commissioner's Officer (ICO)
.jpg)
AI Principles
Promote use of AI that is innovative and trustworthy and that respects human rights and democratic values. Adopted in May 2019, they set standards for AI that are practical and flexible enough to stand the test of time.
Produced mainly for: Government and Ministerial Levels
Origin: Europe
Language: English
Type: Guide
Creator: OECD
Fairness Compass
Common definitions of fairness and ways of calculating the performance of a machine learning model. Mathematical tension across different fairness definitions makes it impossible to achieve "complete fairness." It helps stakeholders identify the most appropriate fairness definition for a specific use case via a decision tree.
Produced mainly for: Managers, Tech teams & Non - tech teams
Origin: Europe
Language: English
Type: Guide
Creator: Boris Ruf and Marcin Detyniecki (AI Research at AXA)
Fair Pricing in Financial Services: summary of responses and next steps
Summarices the main themes in the submissions we received and, where appropriate, provide our responses. Provides further clarification on how we will apply our Framework in practice.
Origin: Europe
Language: English
Type: Guide
Creator: Financial Conduct Authority (FCA)
Latin America and the Caribbean
¿Cómo implementar la debida diligencia en derechos humanos en el desarrollo de tecnología? El impacto en la privacidad
Privacidad y datos personales: una mirada desde el periodismo
Helps identifying how the technological development may harm people's right to privacy. Either directly by the use of such technology, or indirectly as a result of the use of a third party.
Produced mainly for: Managers
Origin: Latin America and the Caribbean
Language: Spanish
Type: Guide
Creator: Asociación por los Derechos Civiles. Argentina-based NGO. Written by Xavier Ibarreche
To offer journalists and communicators basic notions about privacy and protection of personal data, the political and economic implications for democracy and the importance of preserving individuals' rights when reporting events of public interest.
Produced mainly for: Non - tech teams
Origin: Latin America and the Caribbean
Language: Spanish
Type: Guide
Creator: Asociación por los Derechos Civiles. Written by Leandro Ucciferri, with the independent consultants Agustina Bendersky and Denisse Cufré
IA Responsable: Manual técnico: Ciclo de vida de la inteligencia artificial
The purpose of this manual is to provide technical recommendations and technical best practices in order to avoid contrary results (often unexpected) to the decision-makers' expectations.
Produced mainly for: Tech teams
Origin: Latin America and the Caribbean
Language: Spanish
Type: Manual
Creator: Felipe Gonzalez, Teresa Ortiz & Roberto Sánchez Ávalos for BID
Uso responsable de IA para política pública: manual de formulación de proyectos
This manual is part of a series of documents and tools developed by the fAIr LAC initiative to guide policy makers and their technical teams in mitigating the challenges inherent in AI-based decision support systems and promoting their responsible use.
Origin: Latin America and the Caribbean
Language: Spanish
Type: Manual
Creator: Gabriea Denis, María Hermosilla, Claudio Aracena, Roberto Sánchez Ávalos, Natalia González Alarcón & Cristina Pombo for BID
Caja de Herramientas Humanísticas
Philosophical insights aimed at clarifying the nature of artificial intelligence, its relationship with human intelligence, as well as the various types of developments that have taken place in this field and the questions and challenges they raise.
Origin: Latin America and the Caribbean
Language: Spanish
Type: Guide
Creator: Guia.ai collaborating with CeTyS, BID & fAIr LAC
Sesgo e Inferencia en Redes Neuronales ante el Derecho
To approach the phenomenon of bias generated through neural networks, either in their training or in the design of their objective function, and to analyze some of its possible legal implications.
Origin: Latin America and the Caribbean
Language: Spanish
Type: Guide
Creator: Carlos Amunátegui Perelló 1* Raúl Madrid for CeTyS
Ocenia/Pacific
NZ Algorithm Charter
Demonstrates a commitment to ensuring New Zealanders to have confidence in how government agencies use algorithms. The charter is one of many ways that the government demonstrates transparency and accountability in the use of data.
Produced mainly for: Government agencies
Origin: Pacific
Language: English & Maori
Type: Framework
Creator: Aotearoa/NZ Government
New Zealand research on Trust and Identity 2020
Is a purpose driven, inclusive, membership funded organization, whose members have a shared passion for the opportunities that digital identity can offer. Digital Identity NZ supports a sustainable, inclusive and trustworthy digital future for all New Zealanders. It is part of the NZ Tech Alliance.
Produced mainly for: C Level & Non Tech Team
Origin: Pacific
Language: English
Type: Guide
Creator: Digital Identity NZ
.jpg)
Data Privacy and Pou
The pou define why we exist, who we serve, what our goals are and how we make good decisions as kaitiaki for whānau.
Origin: Pacific
Language: English
Type: Tech tool
Creator: Hiria Te Rangi and Amber Craig
Worldwide
Data Ethics Charter
.jpg)
AI Blindspot
The IIF Data Ethics Charter outlines a set of principles for the ethical handling of customer data in the financial services industry and larger economy. Principles and examples of practice provide an overview of how financial institutions responsibly manage, protect, share, and use customer data.
Origin: Worldwide
Language: English
Type: Guide
Creator: Institute of International Finance (IIF)
Principles for Accountable Algorithms and a Social Impact Statement for Algorithms
A discovery process tool for product teams to spot unconscious biases and structural inequalities in AI systems.
Produced mainly for: Tech teams
Origin: Worldwide
Language: English
Type: Tech tool
Creator: Berkman Klein Center and MIT Media Lab
.jpg)
Responsible Data for Children
Aims to help developers and product managers design and implement algorithmic systems in publicly accountable ways. Accountability in this context includes an obligation to report, explain, or justify algorithmic decision-making as well as mitigate any negative social impacts or potential harms.
Produced mainly for: Managers & Tech teams
Origin: Worldwide
Language: English
Type: Guide or manual
Creator: Fairness, Accountability, and Transparency in Machine Learning Conference (FAT/ML)
The work is intended to address practical considerations across the data lifecycle, including routine data collection and one-off data collections. It provides guidance, principles and practical case studies.
Origin: Worldwide
Language: English
Type: Guide, principles, case studies
Creator: UNICEF and GovLab
The Case for Better Governance of Children’s Data: A Manifesto
Sets aspirational benchmarks to guide governments, the private sector and international organizations in developing data governance that take full account of children’s issues and rights, including in AI systems.
Origin: Worldwide
Language: English
Type: Guide
Creator: UNICEF