-1

My team's AI model for spotting roof damage from drone photos went off the rails last month

We were testing a new version in Austin, Texas, and it started flagging every single dark shingle as 'severe hail damage'. The training data had a bias because most of our labeled 'damage' photos came from storms with dark clouds, so the model learned to link shadow color with problems. We had to pull the whole system for a week and add over 2,000 new images with varied lighting to fix it. Now I'm wondering, is it better to scrap a flawed model and start fresh, or is retraining with better data always the right move? What would you do?

3 comments

3 Comments

felix1471mo agoProlific Poster

You said you're wondering if you should scrap the flawed model. Honestly, starting fresh sounds like a huge waste. You already put in the work to find the lighting bias, and you fixed it with those 2000 new images. That retraining probably solved the core issue. A brand new model would just be guessing again and might find a different weird bias to latch onto. Your current model has already learned a ton about roofs, just not the shadows. Keep building on what you have.

kelly_nelson951mo ago

Read a case study where a team kept retraining their flawed drone model. After the third data fix, it finally got good at spotting power line damage without the weather confusion.

jade_singh1mo ago

Stick with the fixed model, it's way smarter now. We had a sensor model that kept failing on overcast days until we just fed it more cloudy data. Throwing out all that learned structure feels like starting a puzzle over because one piece was bent.