Overheard a data scientist say 'garbage in, garbage out' and it clicked about my AI side project

I was at a coffee shop last week (the one on 5th with the terrible WiFi) and this guy was explaining to his friend why their model kept failing. He kept saying that phrase over and over. It finally hit me - I spent 3 months tweaking hyperparameters on my little chatbot project when the real problem was my training data was full of typos and weird formatting. Cleaned up the dataset last night, retrained, and the results are night and day different. Has anyone else wasted time on the wrong part of their AI project?

3 comments

3 Comments

joseph_adams6628d ago

Spent two weeks tuning a model before realizing my dataset had duplicate entries.

margaret_kelly5528d ago

Oh man, my coworker spent a week debugging the same error lol

the_thomas28d agoMost Upvoted

And a whole week? No way. I feel for that person so hard. @joseph_adams66 two weeks on duplicate entries makes me want to cry too. Like how do you not notice that until way later. I bet they felt so dumb after realizing it.