top of page
gorSAys_xZ41NHvYilLR05jhmealg5wC6tAVWbmPcq4.jpg

AI FAIRNESS 
GLOBAL LIBRARY

Resources

All resources
Diseño sin título.jpg

AI Fairness 360

This extensible open source toolkit can help you examine, report, and mitigate discrimination and bias in machine learning models throughout the AI application lifecycle. We invite you to use and improve it.

Produced mainly for: Tech teams

Origin: North America

Language: English

Type: Tech tool

Creator: IBM

Captura de Pantalla 2021-09-27 a la(s) 19.10.15.png

Audit-AI

Open sourced bias testing tool useful for developers, that seems active for updating

Produced mainly for: Tech teams

Origin: North America

Language: English

Type: Tech tool

Creator: Pymetrics

Diseño sin título.jpg

Bias and Fairness Audit Toolkit

Open source tech tool for auditing the data and predictions of machine learning solutions for their fairness. Requires relatively light technology background (can be used by non-developers)

Origin: North America

Language: English

Type: Tech tool

Creator: Chicago University

Diseño sin título.png

FairTest

Enables developers or auditing entities to discover and test for unwarranted associations between an algorithm's outputs and certain user subpopulations identified by protected features.

Produced mainly for: Tech teams

Origin: North America

Language: English

Type: Tech tool

Creator: Universidad de Columbia, Stanford, EPFL, Saarland Univ., Cornell Tech, Jacobs Institute

1.jpg

Themis™ o Themis-ML

A library that implements fairness-aware machine learning algorithms

Produced mainly for: Tech teams

Origin: North America

Language: English

Type: Tech tool

Creator: Massachusetts University

1.jpg

Responsible AI

The toolkit presents key risks associated to AI, including those related to bias & fairness. It offers a free responsible AI Diagnostic tool and a PDF version of pwc's Practical Guide to Responsible AI.

Language: English

Type: Guide

Creator: PwC

Diseño sin título (2).jpg

InterpretML

Open-source package for training interpretable (explainable) ML models

Language: English

Type: Tech tool

Creator: Microsoft

1.jpg

Responsible Innovation: A Best Practices Toolkit

Responsible Innovation is a toolkit for developers that provides a set of practices in development, for anticipating and addressing the potential negative impacts of technology on people. It covers applying Microsoft's AI principles of fairness, and approaches for harms modeling and community jury-style reviews.

Language: English

Type: Tech tool

Creator: Microsoft

Diseño sin título (4).jpg

Empowering AI Leadership – An Oversight Toolkit for Boards of Directors

A guide and toolkit for broader public and business leaders to consider not only AI fairness but also business strategy, governance and responsibility.

Language: English & Spanish

Type: Guide

Creator: World Economic Forum

1.jpg

TOOLBOX: Dynamics of AI Principles

The resource provides a list and a geographical worl map pointing ethical principle declarations and documents

Language: English

Type: Guide or manual 

Creator: AI Ethics Lab

1.jpg

Algorithmic Accountability Policy Toolkit

This toolkit includes resources for advocates interested in or currently engaged in work to uncover where algorithms are being used and to create transparency and accountability mechanisms

Produced mainly for: Non - tech teams

Language: English

Tool: Guide or manual

Creator: AI Now Institute

1.jpg

RCModel, a Risk Chain Model for Risk Reduction in AI Services

The risk chain model (RCModel) supports AI service providers in proper risk assessment and control, and offers rpolicy recommendations

Origin: Asia

Language: English

Type: Guide or manual

Creator: The University of Tokyo

1.jpg

From Principles to Practice – An interdisciplinary framework to operationalize AI ethics

The paper offers concrete guidance to decision-makers in organizations developing and using AI on how to incorporate values into algorithmic decision-making, and how to measure the fulfillment of values using criteria, observables and indicators combined with a context dependent risk assessment.

Language: English

Tool: Guide

Creator: AIEI Group

1.jpg

Guidelines for Quality Assurance of Machine Learning-based Artificial Intelligence

The Guidelines for the Quality Assurance of AI Systems offers a comprehensive technical assessment of quality measures for AI systems, but it is not strictly speaking a document on AI Fairness. It is updated periodically in its original Japanese version, but an informal English translation is available too.

Language: Japanese

Type: Guide or manual

Creator: QA4AI

Diseño sin título (8).jpg

Fairness feature testing

Diseño sin título (9).jpg

CERTIFAI

Allows you to flag protected features in your dataset and then actively guides you through the selection of the best fairness metric to fit the specifics of your use case. 

Produced mainly for: Tech teams

Language: English

Type: Tech tool

Creator: Data Robot

Tool developed by Cognitive Scale for data scientists to evaluate their AI models for robustness, fairness, and explainability, and allows users to compare different models or model versions for these qualities.

Language: English

Type: Tech tool

Creator: Cognitive Scale - Cortex

1.jpg

ML Fairness Gym

Open source development tool for building simple simulations that explore the potential long-run impacts of deploying machine learning-based decision systems.

Language: English

Type: Tech tool

Creator: Google

Diseño sin título (11).jpg

Ethics Canvas

Helps you structure ideas about the ethical implications of the projects you are working on, to visualize them and to resolve them.

1.jpg

Fairness-indicators: Tensorflow's Fairness Evaluation and Visualization Toolkit

Designed to support teams in evaluating, improving, and comparing models for fairness concerns in partnership with the broader Tensorflow toolkit. Perspective AI is provided as a content moderation case study.

Language: English

Type: Tech tool

Creator: Google

Diseño sin título (10).jpg

Ethically Aligned Design

Identifies specific verticals and areas of interest and helps provide highly granular and pragmatic papers and insights as a natural evolution of our work.

Produced mainly for: Tech teams

Language: English

Type: Guide or manual

Creator: IEEE Global A/IS Ethics Initiative

Produced mainly for: Managers

Language: English

Type: Tech tools

Creator: ADAPT Centre

1.jpg

Review into bias in algorithmic decision-making

It's more an educational publication than a tool (as the name suggests: "Review of.."). However, also provides some (high-level) recommendations for governments and regulators.

Origin: Europe

Language: English

Type: Guide

Creator: Centre for Data Ethics and Innovation

1.jpg

Data Ethics Charter

The IIF Data Ethics Charter outlines a set of principles for the ethical handling of customer data in the financial services industry and larger economy. Principles and examples of practice provide an overview of how financial institutions responsibly manage, protect, share, and use customer data.

Origin: Worldwide

Language: English

Type: Guide

Creator: Institute of International Finance (IIF)

Diseño sin título (14).jpg

AI Blindspot

A discovery process tool for product teams to spot unconscious biases and structural inequalities in AI systems.

Produced mainly for: Tech teams

Origin: Worldwide

Language: English

Type: Tech tool

Creator: Berkman Klein Center and MIT Media Lab

Diseño sin título (15).jpg

Responsible Data for Children

The work is intended to address practical considerations across the data lifecycle, including routine data collection and one-off data collections. It provides guidance, principles and practical case studies.

Origin: Worldwide

Language: English

Type: Guide, principles, case studies

Creator: UNICEF and GovLab

1.jpg

Model AI Governance Framework

Provides detailed and readily-implementable guidance to private sector organisations to address key ethical and governance issues when deploying AI solutions.

Produced mainly for: Managers & C levels

Origin: Asia

Language: English

Type: Guide

Creator: PDPC

1.jpg

AI For Everyone Course

Povide AI practitioners and non-technical professionals with the necessary tools to go all the way from foundational basics to advanced application, empowering them to build an AI-powered future.

Produced mainly for: For everyone

Origin: North America

Language: English

Type: Course

Creator: Andrew Ng

1.jpg

Explaining decisions made with AI 

Mainly focused on explaining decisions made with AI, but it contains fairness issues in the model

Origin: Europe

Language: English

Type: Framework

Creator: Information Commissioner's Officer (ICO)

Guide of Algorithmic Auditing

1.jpg

An algorithmic audit identifies, addresses and corrects algorithmic bias so that you can act on informed and vetted AI-powered business recommendations.

Produced mainly for: Companies using AI

Origin: Europe

Language: English & Spanish

Type: Tech tool

Creator: Eticas

1.jpg

Principles for Accountable Algorithms and a Social Impact Statement for Algorithms

Aims to help developers and product managers design and implement algorithmic systems in publicly accountable ways. Accountability in this context includes an obligation to report, explain, or justify algorithmic decision-making as well as mitigate any negative social impacts or potential harms.

Produced mainly for: Managers & Tech teams

Origin: Worldwide

Language: English

Type: Guide or manual

Creator: Fairness, Accountability, and Transparency in Machine Learning Conference (FAT/ML)

1.jpg

The Case for Better Governance of Children’s Data: A Manifesto

Sets aspirational benchmarks to guide governments, the private sector and international organizations in developing data governance that take full account of children’s issues and rights, including in AI systems.

Origin: Worldwide

Language: English

Type: Guide

Creator: UNICEF

1.jpg

From Principles to Practice: use cases for implementing responsible AI in financial services

Findings from work by a group of partners in a project to explore the implementation of responsible AI principles, including Deutsche Bank, Linklaters, Microsoft, Standard Chartered, and Visa.

Produced mainly for: Managers & C levels

Origin: Asia

Language: English

Type: Guide

Creator: Microsoft, based on MAS FEAT principles

Diseño sin título (19).jpg

AI Principles

Promote use of AI that is innovative and trustworthy and that respects human rights and democratic values. Adopted in May 2019, they set standards for AI that are practical and flexible enough to stand the test of time.

Produced mainly for: Government and Ministerial Levels

Origin: Europe

Language: English

Type: Guide

Creator: OECD

1.jpg

Artificial intelligence - Governance framework model. Second Edition

Addition of industry examples in each section to illustrate how organizations have implemented AI governance practices.

Produced mainly for: Government

Origin: Asia

Language: English

Type: Guide

Creator: S Iswaran- Minister for Communications and Information- Singapore

1.jpg

¿Cómo implementar la debida diligencia en derechos humanos en el desarrollo de tecnología? El impacto en la privacidad

Helps identifying how the technological development may harm people's right to privacy. Either directly by the use of such technology, or indirectly as a result of the use of a third party.

Produced mainly for: Managers

Origin: Latin America and the Caribbean

Language: Spanish

Type: Guide

Creator: Asociación por los Derechos Civiles. Written by Leandro Ucciferri, with the independent consultants Agustina Bendersky and Denisse Cufré

1 (1).jpg

Ethics & Algorithms Toolkit

A practical toolkit for cities to use to help them identify the risks of using an AI algorithm, and maps out the mitigating measures for different risks.

Origin: North America

Language: English

Type: Tech tool

Creator: GovEx, the City and County of San Francisco, Harvard DataSmart, Data Community DC

Model Artificial Intelligence Governance Framework Second edition

1.jpg

Incorporates the experiences of organizations that have adopted AI, and feedback from our participation in leading international platforms, such as the European Commission’s High-Level Expert Group and the OECD Expert Group on AI.

Produced mainly for: Manager

Origin: Asia

Language: English

Type: Guide or manual

Creator: SG:D, IMDA, PDPC (Singapore)

1 (3).jpg

Fairness Toolkit (LiFT)

Linkedin - open source software library designed to enable the measurement of fairness in AI and machine learning workflows.

Origin: North America

Type: Tech tool

Creator: LinkedIn

1.jpg

Bias in machine learning and ethical implications

The report outlines ways in which financial institutions are cautiously implementing Machine Learning into their processes, such as in credit risk assessment and anti-money laundering, and the mathematical as well as social components of fairness to be considered.

Produced mainly for: C leve & Managers

Origin: North America

Language: English

Type: Guide

Creator: Institute of International Finance

1.jpg

Veritas Initiative

Framework for financial institutions to promote the responsible adoption of Artificial Intelligence and Data Analytics. It has developed a fairness assessment methodology in credit risk scoring and customer marketing, and has published whitepapers on the fairness assessment methodology and the open source code of these two use cases.

Produced mainly for: Tech teams

Origin: Asia

Language: English

Type: Tech tool

Creator: Monetary Authority of Singapore

Fair Pricing in Financial Services: summary of responses and next steps

1.jpg

Summarices the main themes in the submissions we received and, where appropriate, provide our responses. Provides further clarification on how we will apply our Framework in practice.

Origin: Europe

Language: English

Type: Guide 

Creator: Financial Conduct Authority (FCA)

1.jpg

Responsible AI from Pilot to Production

A biased model that works for some users, and not others, is a failed model. Or a model that wasn’t sourced responsibly can be a poor reflection of company values and a nightmare with the media. It’s helpful to remember that AI reflects the people and the company that build it: when something goes wrong, it shows something may also be wrong internally.

Produced mainly for: C Level, Manager, Tech Team & Non - Tech Team

Origin: North America

Language: English

Type: Guide

Creator: Appen

1.jpg

The Impact of Ethics and Bias on Artificial Intelligence

AI is the study and research of providing software and hardware that attempts to emulate a human being.  Cognitive computing is computing focused on reasoning and understanding that is inspired by human cognition. It is a subset of AI. 

Produced mainly for: C levels & Managers

Origin: North America

Language: English

Type: Framework

Creator: David Schubmehl, Research Director, Cognitive/AI Systems

1 (13).jpg

Data Privacy and Pou

The pou define why we exist, who we serve, what our goals are and how we make good decisions as kaitiaki for whānau.

Produced mainly for: Non - Tech teams

Origin: Pacific

Language: English

Type: Tech tool

Creator: Hiria Te Rangi and Amber Craig

1.jpg

Uso responsable de IA para política pública: manual de formulación de proyectos

This manual is part of a series of documents and tools developed by the fAIr LAC initiative to guide policy makers and their technical teams in mitigating the challenges inherent in AI-based decision support systems and promoting their responsible use.

Origin: Latin America and the Caribbean

Language: Spanish

Type: Manual

Creator: InterAmerican Development Bank (IDB)

1.jpg

Caja de Herramientas Humanísticas

Philosophical insights aimed at clarifying the nature of artificial intelligence, its relationship with human intelligence, as well as the various types of developments that have taken place in this field and the questions and challenges they raise.

Origin: Latin America and the Caribbean

Language: Spanish

Type: Guide

Creator: Guia.ai collaborating with CeTyS, BID & fAIr LAC

1.jpg

Privacidad y datos personales: una mirada desde el periodismo

To offer journalists and communicators basic notions about privacy and protection of personal data, the political and economic implications for democracy and the importance of preserving individuals' rights when reporting events of public interest.

Produced mainly for: Non - tech teams

Origin: Latin America and the Caribbean 

Language: Spanish

Type: Guide

Creator: Asociación por los Derechos Civiles. Argentina-based NGO. Written by Xavier Ibarreche

1.jpg

Fairness Flow

Engages in cutting-edge research that can improve and power new product experiences at a huge scale for our community. Building on Facebook AI's key principles of openness, collaboration, excellence, and scale, we make big, bold research investments focused on building social value and bringing the world closer together.

Produced mainly for: Non - tech teams

Origin: North America

Language: English

Type: Guide or manual

Creator: Facebook

1.jpg

Ethical Toolkit for Engineering/Design Practice

Santa Clara University - Multi set of tools implementing ethical reflection, deliberation, and judgment into engineering and design workflows

Origin: North America

Language: English

Type: Tech tool

Creator: Markkula Center for Applied Ethics – Santa Clara University

Captura de Pantalla 2021-09-27 a la(s) 11.10.37.png

Model Cards

Short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups and intersectional groups that are relevant to the intended application domains.

Produced mainly for: C levels & Managers

Origin: North America

Language: English

Type: Guide or manual

Creator: Google

1 (4).jpg

NZ Algorithm Charter

Demonstrates a commitment to ensuring New Zealanders to have confidence in how government agencies use algorithms. The charter is one of many ways that the government demonstrates transparency and accountability in the use of data.

Produced mainly for: Government agencies

Origin: Pacific

Language: English & Maori

Type: Framework

Creator: Aotearoa/NZ Government

fairness_compass_logo_small.png

Fairness Compass

Common definitions of fairness and ways of calculating the performance of a machine learning model. Mathematical tension across different fairness definitions makes it impossible to achieve "complete fairness." It helps stakeholders identify the most appropriate fairness definition for a specific use case via a decision tree.

Produced mainly for: Managers, Tech teams & Non - tech teams

Origin: Europe

Language: English

Type: Guide

Creator: Boris Ruf and Marcin Detyniecki (AI Research at AXA)

1 (6).jpg

The Word Embeddings Fairness Evaluation Framework

Open source library for measuring bias in word embedding models. It generalizes many existing fairness metrics into a unified framework and provides a standard interface for: encapsulating existing fairness metrics from previous work and designing new ones, encapsulating the test words used by fairness metrics into standard objects called queries and computing a fairness metric on a given pre-trained word embedding model using user-given queries.

Produced mainly for: Tech teams

Language: English

Type: Tech tool

Creator: Millennium Institute for Foundational Research on Data (IMFD).

1.png

Real World AI: A Practical Guide to Responsible Machine Learning

This practical guide to deploying AI lays out a human-first, responsible approach that has seen more than three times the success rate when compared to the industry average.

Produced mainly for: C Level, Manager, Tech Team & Non - Tech Team

Origin: North America

Language: English

Type: Guide

Creator: Wilson Pang, CTO at Appen, and Alyssa Simpson Rochwerger, Director of Product at Blue Shield of California and Former VP of Data and AI at Appen

1 (10).jpg

AI Ethics for Nonprofits toolkit

Participants will learn some of the fundamentals of AI ethics and then immediately get to practice applying ethical considerations related to the principle of Fairness in the context of several humanitarian and international development use cases

Produced mainly for: C Level, Manager, Tech Team, Non Tech Team

Origin: North America

Language: English

Type: Tool tech & Guide

Creator: NetHope (consortium of 60 INGOs) in partnership with MIT D-Lab and USAID

1.jpg

New Zealand research on Trust and Identity 2020

Is a purpose driven, inclusive, membership funded organization, whose members have a shared passion for the opportunities that digital identity can offer. Digital Identity NZ supports a sustainable, inclusive and trustworthy digital future for all New Zealanders. It is part of the NZ Tech Alliance.

Produced mainly for: C Level & Non Tech Team

Origin: Pacific

Language: English

Type: Guide

Creator: Digital Identity NZ

IA Responsable: Manual técnico: Ciclo de vida de la inteligencia artificial

1.jpg

The purpose of this manual is to provide technical recommendations and technical best practices in order to avoid contrary results (often unexpected) to the decision-makers' expectations.

Produced mainly for: Tech teams

Origin: Latin America and the Caribbean

Language: Spanish

Type: Manual

Creator: Felipe Gonzalez, Teresa Ortiz & Roberto Sánchez Ávalos for BID

1.jpg

Sesgo e Inferencia en Redes Neuronales ante el Derecho

To approach the phenomenon of bias generated through neural networks, either in their training or in the design of their objective function, and to analyze some of its possible legal implications.

Origin: Latin America and the Caribbean

Language: Spanish

Type: Guide

Creator: Carlos Amunátegui Perelló 1* Raúl Madrid for CeTyS

WioT1hbSBiVtZI2Im0Lc_1-Nf4oSCjYB8XV9vlvl7oMLg-300x300.jpg

Ethical OS Toolkit

For visualizing and anticipating future risk of technology products, acknowledging that once technology is released and reaches scale it may be used for purposes beyond the original intention.

Origin: North America

Language: English

Type: Guide or Manual

Creator: The Institute for the Future, Omidyar Network‘s Tech and Society Solutions Lab

bottom of page