Question about choosing between A/B testing tools and going with gut instinct

So last month I had to decide between running a proper A/B test on two landing page headlines or just going with the one my boss said felt right. I spent about 3 days setting up the test through Google Optimize, making sure everything was tagged correctly and traffic split was even. The test ran for 2 weeks and the winning headline ended up boosting our click-through rate by 8%. But honestly, the whole process ate up a ton of time I could have spent on other stuff like getting new content out. My boss still gives me a hard time about "overthinking it." Has anyone else dealt with pushing for data-driven choices when higher-ups just want to go with their gut?

2 comments

2 Comments

alice3361mo ago

Oh I heard a podcast where someone said gut feelings are just fast pattern recognition from past experience!

alicesingh1mo ago

Oh wow that's a tough spot. So you put in all that work and your boss still doesn't see the value? Makes me wonder how much that 8% actually matters to them in the big picture. Did they ever say why they didn't trust the data after it proved them wrong? Like were they just embarrassed or did they have a real reason? Sometimes higher ups get scared of change even when the numbers back it up. I guess my real question is did you ever get to show them the dollar amount that 8% translated to? That's usually the only thing that shuts people up.