I watched a facial recognition system at my old job flag the wrong person three times in one afternoon.

The security team argued it was a necessary risk for safety, but I saw it cause real panic for a guy who just looked similar to someone on a watchlist, so where do we draw the line between security and wrongful accusation?

3 comments

3 Comments

drew_park1mo agoTop Commenter

Hot take: Tech that can't tell people apart is bad tech. Longer response: The line gets drawn when innocent people start getting harassed. That panic you saw is the real cost. These systems need to be way more accurate before they get used everywhere. Otherwise we're just trading safety for a different kind of fear.

the_rose1mo ago

Trading safety for a different kind of fear" is exactly it. They're building these systems that freak out over the wrong person. That panic you mentioned isn't some small bug, it's a total system failure. We're putting this stuff in schools and airports when it can't even do its one job right. It's reckless.

hugo_schmidt29d ago

Honestly, I've seen something similar happen where a guy got pulled aside at a train station because his face matched a blurry photo. What worked for me was just calmly saying "that's not me" and showing them my ID right away, no arguing. But the system should have never put him in that spot in the first place, it's on them to get it right before ruining someone's day.