14 DAY TRIAL // Amazon scrapped an internal and experimental artificial intelligence-powered recruiting tool after it taught itself to prefer job applications coming from men while penalizing resumes sent in by women, as reported by Reuters.
The recruiting tool was the result of Amazon's desire to automate the recruiting process, with their effort starting in 2014 when a development team was assembled to create software capable of sifting through, and rating resumes on its own.
“Everyone wanted this holy grail,” Reuters' report says. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”
Although the team managed to develop a machine learning-based recruiting engine which would rate CVs with scores based on a five-star scale, Amazon realized sometime during 2015 that their new automated recruiter was showing an obvious bias, penalizing resumes who came in from women.
According to the report, the recruiting tool taught itself to be biased against women after analyzing patterns in the applications submitted during a 10-year long period.
Amazon reportedly scrapped its biased recruiting engine before actually using it during the recruiting process
Seeing that the majority of the resumes were being received from male applicants, the AI powering Amazon's axed recruiting engine concluded that males were to be preferred.
Moreover, the recruiting software was effectively penalizing resumes containing the "women’s” term and giving lower scores to women who attended two women-only schools.
Although Amazon tried to tweak the recruiting engine to be neutral, it eventually gave up given that there was no guaranteed that the machine learning-based automated recruiter will not fall into the same bad habits sometime in the future.
"Amazon’s recruiters looked at the recommendations generated by the tool when searching for new hires, but never relied solely on those rankings," Reuters' anonymous sources also stated.