• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

Mark Cuban says he has the solution to growing income inequality, and it's to reward every employee—from CEO to janitor—with company stock

2

‘I want to die broke’: Billionaire philanthropist Denny Sanford dies after giving away $4 billion

3

OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation

1

Mark Cuban says he has the solution to growing income inequality, and it's to reward every employee—from CEO to janitor—with company stock

2

‘I want to die broke’: Billionaire philanthropist Denny Sanford dies after giving away $4 billion

3

OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation
ConferencesBrainstorm AI

Fighting bias in A.I. means acknowledging it exists

By
Dan Reilly
Dan Reilly
Down Arrow Button Icon
By
Dan Reilly
Dan Reilly
Down Arrow Button Icon
November 9, 2021, 4:55 PM ET
Add Fortune on Google for similar content.

While artificial intelligence can reduce human error and work at speeds that no individual can replicate, the fact remains that programs and algorithms are built by people with their own sets of biases. Overt or internalized prejudices can seep into how A.I. and machine learning systems are constructed and perform, potentially leading to systems that only work for the portion of the population that’s representative of the creators. 

It’s an issue that many tech corporations are grappling with now, especially as there’s been an increased awareness of the issues surrounding race, gender, and economic imbalances in society. So how can companies ensure that the A.I. products they build are as free from bias as possible? At Sony AI, which was founded in 2020 and deals with everything from image sensors to music and movies, the key is trying to identify potential biases at the earliest stages of product development. 

“There’s no such thing as a perfectly unbiased algorithm, so I always tell business units, ‘The fact that we’re pushing you to conduct fairness assessments is not because we think that you’ve done something wrong or that your product, in particular, is really problematic. Instead, it’s something we always want to check for,’” said Sony Group’s head of A.I. ethics office Alice Xiang at this week’s Fortune Brainstorm A.I. conference in Boston. “Whether it be something relatively innocuous, like an autofocus feature that finds faces or finds eyes, to when we’re talking about robotics, frequently computer vision is a major part of that. If you have a self-driving car, then you need to think about being able to detect pedestrians and ensure that you can detect all sorts of pedestrians and not just people that are represented dominantly in your training or test set.”

“Business units often understand when you explain to them what the issue is, broadly, of bias,” Xiang continued. “No one wants to produce products that are biased, but it’s actually quite difficult in practice to figure out the right benchmarks for testing for bias, the right techniques to mitigate bias. That’s where research really plays a key role. Because this is a really new space, there are  constantly new methods being developed, and we really want to be on the cutting edge in terms of the techniques that we’re employing.”

A key issue in this specific area is identifying where bias already exists and how to weed that out from becoming a part of an algorithm or system. It’s certainly an area of concern for Dr. Margaret Mitchell, the chief ethics scientist of Hugging Face, which focuses heavily on A.I. language processing.

“We find that when we have these large language models training on tons and tons of data … most of it is sourced from the web, where we see a lot of racism and sexism and ableism and ageism,” she said. “[It’s] largely sourced from Wikipedia, which is primarily written by men, white men, between something like 20 to 30 or so, and single and PhD, higher-level education, which means that the kind of topics that are covered, that are then scraped in training the language models, reflect those knowledge bases, reflect those backgrounds.”

To illustrate this point, Mitchell cited a search result from Google, the same company that fired her in early 2021 following her open criticism of the company’s lack of diversity and inclusion (Google says her termination was due to a breach of its code of conduct and security policies). “If you try and do a search for ‘Black history,’ you’ll be redirected to African-American history, which is American-centric and not quite understandable about the whole history of people who are Black,” she said. “It’s a really key issue when it comes to what the language models regurgitate as a function of the normal skews, just in who is talking and who’s being scraped on the web, as well as the inherent racism and sexism, etc. that gets expressed. These end up coming out in what’s generated and what’s suggested.”

“AI is never going to be perfect,” said Dr. Haniyeh Mahmoudian, global AI ethicist at DataRobot, which works in machine learning automation and development. “It comes to having a very thorough understanding of what are the risks of using the system, having a thorough risk assessment of the process, understanding if you have data quality concerns — everything that goes along the way of building an AI system. And based on that, then we can understand if there is a need for mitigation.”

Basically, she said, it’s about monitoring A.I. at every step of the way to make sure bias isn’t becoming part of the program. “We can take on some mitigation tasks along the way, or understand who is going to be impacted if this system, at some point, makes mistakes, if it does something that is unexpected. This actually helps us understand if you’re ready to build and put this system in production or if we need to hold back and collect better data until you’re ready to deploy.”

As Xiang said, it’s there that transparency in the process is key to eliminating biases. “There is more of a sense that we can at least hold a human accountable if something goes wrong, or at least ask them for the rationale behind their decision making,” she said. “From that perspective, I think it’s very important for folks not only to think about, ‘How do we make this as good as possible and do the relevant risk assessment?’ but also carefully document the failure models of the product, make that very clear, and have mechanisms in place to be able to detect failures in deployment then act upon them. Because that’s the major place where it can be very risky to move forward with A.I. versus humans.”

More must-read business news and analysis from Fortune:

  • From Delta to Southwest, the airlines in the best—and worst—shape going into a chaotic holiday season
  • How a risky bet on the Shiba Inu coin made this warehouse manager a millionaire
  • Patagonia doesn’t use the word ‘sustainable.’ Here’s why
  • Will monthly child tax credit payments continue in 2022? Their future rests on Biden’s Build Back Better bill
  • ‘I’m afraid we’re going to have a food crisis’: The energy crunch has made fertilizer too expensive to produce, says Yara CEO
Subscribe to Fortune Daily to get essential business stories straight to your inbox each morning.
About the Author
By Dan Reilly
See full bioRight Arrow Button Icon
Add Fortune on Google for similar content.

Latest from our Conferences

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • TikTok icon
  • YouTube icon

Latest from our Conferences

At Fortune Brainstorm Tech 2026, Chris Bedi, Chief Customer Officer and Enterprise AI Advisor, ServiceNow; China Widener, Vice Chair and US Technology, Media & Telecommunications Industry Leader, Deloitte; and Phil Wiser, Chief Technology Officer, Paramount, speak on a panel with Kristin Stoller, Fortune editorial director.
NewslettersFortune Workplace Innovation
This tech CEO fired 80% of his workforce over AI resistance. Here’s what he’s learned since then
By Kristin StollerJune 15, 2026
1 month ago
Courtney Robinson, head of policy and communications, at Akoya speaks on a panel at Fortune Brainstorm Tech 2026.
RetailBrainstorm Tech
AI shopping agents are coming. No one is ready for them
By Jeremy KahnJune 12, 2026
1 month ago
The head of Claude Code hasn’t ‘written a line of code by hand’ in 8 months
ConferencesBrainstorm Tech
The head of Claude Code hasn’t ‘written a line of code by hand’ in 8 months
By Nick LichtenbergJune 11, 2026
1 month ago
Sarah Franklin, Chief Executive Officer of Lattice, and Francine Katsoudas, EVP and Chief People, Policy and Purpose Officer at Cisco, speak at Fortune's COO Summit with Kristin Stoller, Editorial Director at Fortune.
NewslettersFortune Workplace Innovation
AI disruption arrived 6 years early—now executives are drawing the line
By Kristin StollerJune 8, 2026
1 month ago
Fortune Brainstorm Tech 2026 livestream
ConferencesBrainstorm Tech
Fortune Brainstorm Tech 2026 livestream
By Fortune EditorsJune 8, 2026
1 month ago
dw
ConferencesCOO Summit
This CEO has had 6 major jobs in Silicon Valley: How Dennis Woodside built a career on saying yes to hard problems
By Nick LichtenbergJune 3, 2026
2 months ago

Most Popular

Mark Cuban says he has the solution to growing income inequality, and it's to reward every employee—from CEO to janitor—with company stock
Success
Mark Cuban says he has the solution to growing income inequality, and it's to reward every employee—from CEO to janitor—with company stock
By Sasha RogelbergJuly 20, 2026
1 day ago
‘I want to die broke’: Billionaire philanthropist Denny Sanford dies after giving away $4 billion
Success
‘I want to die broke’: Billionaire philanthropist Denny Sanford dies after giving away $4 billion
By Sydney LakeJuly 20, 2026
2 days ago
OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation
Cybersecurity
OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation
By Jeremy Kahn and Emily ForliniJuly 21, 2026
8 hours ago
'Dr. Doom' Nouriel Roubini says we're headed for universal basic income or 'some form of socialism' as AI revolutionizes work—He calls that optimistic
AI
'Dr. Doom' Nouriel Roubini says we're headed for universal basic income or 'some form of socialism' as AI revolutionizes work—He calls that optimistic
By Jason MaJuly 18, 2026
3 days ago
Mathematicians grapple with a ‘very rapid and very unsettling change’ as AI cracks yet another century-old problem
AI
Mathematicians grapple with a ‘very rapid and very unsettling change’ as AI cracks yet another century-old problem
By Eva RoytburgJuly 21, 2026
11 hours ago
Despite a $156 million contract, Knicks star Jalen Brunson still calls his parents for financial advice any time he makes a big purchase
Success
Despite a $156 million contract, Knicks star Jalen Brunson still calls his parents for financial advice any time he makes a big purchase
By Emma BurleighJuly 21, 2026
12 hours ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.