An update on our work in responsible innovation

Projects such as research methods for evaluating misinformation and datasets that need more diverse representation tend to receive conditions to proceed toward a launch. A recurring condition given to teams is to engage in ProFair testing with people from a diversity of backgrounds, often in partnership with our central Product Inclusion and Equity team. This year, the number of ProFair consultations increased annually by 100%. A recurring approach is to create and release detailed documentation in the form ofdata cards and model cards for transparency and accountability. The number of AI Principles reviews with model or data card mitigations increased 68% in the last year.

As we’ve stated, we’ve embedded customized AI governance and review committees within certain product areas (like Cloud and Health). As a result, both the Health Ethics Committee and Cloud make decisions with specialized expertise, such as establishing policies for potentially winding down the Covid-19 Community Mobility Reports and the Covid-19 Forecaster, respectively, if situations arise that might cause the data quality to degrade. This year, we extended this specialized approach and created a dedicated consumer hardware AI Principles review process.

It’s important to note that product teams across Google engage in everyday responsible AI practices even if not in formal reviews. YouTube is leveraging a more targeted mix of classifiers, keywords in additional languages, and information from regional analysts. This work is a result of collaboration with our researchers who focus on new tools for AI fairness. The Photos team participated in an Equitable AI Research Roundtable (EARR) with a group of external advisors on potential fairness considerations. And the Gboard team deployed a new, privacy-by-design approach to federated machine learning. These examples did not stem from AI Principles reviews, but reflect the adoption of the AI Principles across Google.

Tools and research

In early 2022, to offer easier access to our publications on responsible AI, we curated an external collection of more than 200 research papers focused on the topic. We continue to launch, refine and consolidate technical resources, including proactive tools like:

• The Monk Skin Tone Scale, developed by Harvard University Sociology Professor Dr. Ellis Monk. The scale offers a spectrum of skin tones from all around the world for use in evaluating and addressing fairness considerations in AI.
• The Know Your Data tool (KYD), which helps developers with tasks such as quickly identifying issues in fairness, and which has integrated the Monk Scale to help developers examine skin tone data for unfair bias.
• The Language Interpretability Tool, or LIT, to help developers probe an ML model, now with a new method to better understand, test and debug its behaviors.
• Counterfactual Logit Pairing, which helps ensure that a model’s prediction doesn’t change when sensitive attributes or identity terms referenced in an example are removed or replaced, now added to the TensorFlow Model Remediation Library (see the research paper for more).
• And to help teams measure their progress against the AI Principles, we’re piloting an internal tool to help teams assess how ML models were developed in accordance with emerging smart practices, previous reviews, and our growing body of ethics, fairness, and human-rights work.

Many responsible AI tools developed by researchers are actively in use by product teams at Google. For example, Photos, Pixel and Image Search are leveraging the Monk Skin Tone Scale.

External engagement

Ensuring the responsible development and deployment of AI is an ongoing process. We believe it should be a collaborative one, too, so we remain deeply engaged with governments across Europe, the Middle East and Africa, Latin America, Asia Pacific, and the U.S. to advocate for AI regulation that supports innovation around the world for businesses of all sizes. We share our approach to responsible AI and recommendations, comments and responses to open requests for information. We also initiated and are leading an effort with the International Standards Organization (ISO/IEC PWI TS 17866) to share best practice guidance for the development of AI.

As these efforts look toward the future, Responsible AI needs to be supported across industries today. So for current Google Cloud Partners and customers seeking best practices to help with the responsible implementation and AI governance in their organization, we added responsible AI prerequisites to the Google Cloud Partner Advantage ML Specialization, including a newly-released training, “Applying AI Principles with Google Cloud.”

To help nurture the next generation of responsible AI practitioners, we launched a free introduction to AI and machine learning for K-12 students. And we continue to develop an external Responsible Innovation Fellowship program in the U.S. for students at Historically Black Colleges and Universities (HBCUs).

Our approach to responsible innovation also means keeping an eye on emerging markets where AI is being developed. We launched a new AI research center in Bulgaria and expanded support for African entrepreneurs whose businesses use AI through our Google for Startups Accelerator: Africa.

The examples we’re sharing today are a sampling of our ongoing commitment to responsible innovation. They also reflect our ability to change and keep setting a high bar for trustworthy AI standards for our company. We remain dedicated to sharing helpful information on Google’s journey, as recommended practices for responsible AI continue to emerge and evolve.