Watched a developer argue that AI models don't need ethical guidelines because they're just math

Some guy in my local coding meetup kept saying you can't put morals on algebra and walked out when people pushed back. Six months later his company's chatbot started recommending dangerous medical advice because nobody set boundaries on what it could generate. Has anyone else run into this "it's just code" attitude and seen it blow up later?

3 comments

3 Comments

rubyschmidt21d ago

That chatbot story actually changed my mind on this. I used to be one of those people who thought ethics guidelines were just extra paperwork, but seeing how fast a "it's just math" attitude can turn into real harm made me realize we all need those boundaries in place.

victorb1718d ago

Well damn, @veram99 you nailed it with that car analogy. But here's the thing nobody's talking about - what happens when those ethics guidelines end up being written by the same people who wanted to skip them in the first place? I've seen this play out in construction. The same contractors who complain about safety regs end up writing the new rules after something goes wrong, and suddenly the rules are just loose enough to keep them out of court but not tight enough to actually protect anyone. That chatbot guy probably helped shape his own guidelines after his mess made headlines. Who's watching the watchmen, right?

veram9921d ago

Oh man, "you can't put morals on algebra" is gonna be my new favorite quote to laugh at when things go sideways. Yeah, sure, the math itself doesn't have ethics, but the people building the whole system sure do. It's like saying a car's engine has no morals so you don't need brakes or a steering wheel. That chatbot giving out bad medical advice is exactly the kind of predictable mess you get when people pretend their code exists in a vacuum. The math doesn't care, but the person who gets hurt by the output sure will. Guess that guy learned the hard way that "just code" still needs grownups to watch over it.