Loading…
17
Question about how long it took me to spot the bias in a hiring tool my cousin's company uses
My cousin works at a mid size insurance company in Columbus and they started using an AI tool to screen resumes about 8 months ago. She asked me to look at it since I do hiring for my own small crew and she thought something felt off, and it took me almost 3 weeks of comparing old resumes to figure out what it was doing. The tool was basically downgrading anyone who had a gap in their work history of more than 6 months, even if the gap was from caring for a sick parent or a layoff. She pulled 40 old applications and 29 of the ones it auto rejected had gaps like that, and most were women. Nobody at her company had questioned it because the tool claimed it was just looking at "job fit" and they trusted the number it gave. My question is who should even be responsible here, the company selling the tool or the company using it? Has anyone else dug into one of these screening systems and figured out where the bias was actually coming from?
1 comments
Log in to join the discussion
Log In1 Comment
joel60313d ago
That is such a gut punch to read, and honestly good on you for actually sitting down and doing the work instead of just trusting the number like everyone else did. Three weeks of digging through old resumes is no joke, and the fact that 29 out of 40 auto rejects had gaps, with most being women, is not a coincidence, that is the tool doing exactly what it was built to do, just quietly. As for who's responsible, I'd say both, but the company using it carries more of the blame since they're the ones making the actual hiring decisions, and "the software told us to" stopped being a real excuse a long time ago. The thing that gets me is nobody questioned it because it said "job fit," which is just a fancy label that hides whatever rule the system is really using. Your cousin is lucky she had you to pull the curtain back, because most places never even look.
7