RAG systems beat fine-tuning for customer support bots

I spent 3 months trying to fine-tune a Llama model on our support tickets. Switched to a simple RAG pipeline with vector search last week. Why do people keep pushing fine-tuning when retrieval gives better answers without retraining?

2 comments

2 Comments

the_kevin25d ago

Ha, welcome to the club. I went through the same exact thing, spent weeks trying to squeeze better answers out of a fine-tuned model only to realize I was basically just making it memorize our FAQ page (badly). The second I threw a vector search on top of our actual documentation everything just worked, no retraining needed. It's honestly kind of funny how many blog posts still say fine-tuning is the magic bullet for support bots when it's actually the slow, painful way to get to the same place. RAG just makes sense for stuff that changes often, like product updates or new features, because you don't have to rebuild the model every Tuesday. Fine-tuning has its place I guess, but for customer support it feels like using a sledgehammer to hang a picture frame.

lunakim25d ago

I read somewhere that even the big AI companies are starting to shift their recommendations toward RAG over fine-tuning for customer-facing stuff, and that made me go "oh, so I wasn't just doing it wrong.