Amazon Scraps AI Hiring Tool For Bias Against Women

A Reuters report published Tuesday revealed that an AI tool built by Amazon to recruit staff had to be scrapped after it was found that it discriminated against female job seekers.

RELATED:

Amazon Cuts Monthly Incentives, Stock Awards After Salary Increase

Amazon set up an engineering team in Edinburgh, Scotland in 2014 to find a way to automate its recruitment process.

They created 500 computer models to trawl through past candidates' resumes and pick up on around 50,000 key terms. The system would crawl the web to recommend candidates.

“Everyone wanted this holy grail,” one of the people said. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”

But by 2015 it was discovered that the system was not rating candidates for technical posts in a gender-neutral way.

The reason for this bias is due to how Amazon’s AI models were trained — they vetted applicants by observing patterns in resumes submitted to the company in the last 10 years, most of which came mostly from men.

This led the system to teach itself that male candidates were preferable and penalized resumes that included the word “women." It also downgraded graduates of two all-women colleges.

Amazon tried to edit the program to make it more gender neutral but not being able to guarantee non-discrimination, the company ultimately disbanded the team working on the system and limited its usage.

Amazon’s recruiters looked at the recommendations generated by the tool when searching for new hires, but never relied solely on those rankings as told to Reuters anonymously by one of the people associated with the system.

Another person said that a new team in Edinburgh has been formed to give automated employment screening another try, this time with a focus on diversity.

The system not only proved to be biased against women but also recommended unqualified candidates for various job posts.

The company did not comment on the AI recruiting tool but mentioned that they are committed to workplace diversity and equality.

The Reuters report came at a time when more and more companies are looking to use similar technologies for recruitment.

It also serves as a lesson to the growing list of large companies including Hilton Worldwide Holdings, Inc. (HLT.N) and Goldman Sachs Group, Inc. (GS.N) that are looking to automate portions of the hiring process.

Some 55 percent of U.S. human resource managers said artificial intelligence, or AI, would be a regular part of their work within the next five years, according to a 2017 survey by talent software firm CareerBuilder.

AI had seen widespread oppositions from various rights groups. The American Civil Liberties Union is presently challenging a law that allows criminal prosecution of researchers and journalists who test hiring website’s algorithms for discrimination.