Research your Research Options

There’s a website in Ireland that we all use and we all hate. It’s not one of those ‘love to hate’ gripes like the budget airline that will remain nameless, but in fairness has improved over the years. No, it’s one of those ‘have to use’ because they control most of their market and if you don’t use it, you don’t get it.

The thing about monopolies is that they don’t have to worry about customer satisfaction or the user experience – like it or lump it – we have no choice 😛

Not long ago, I was clunking around the site trying to buy, in way too many clicks that was acceptable even in the 90s, when a pop-up survey assailed me. I was more shocked that the site was interested in my user experience than the fact that it was yet another barrier to completing the purchase.

While completing the survey, 3 things jumped out at me:

1. They didn’t spare any expense in the survey software. I can tell you that the entry level subscription to this software is c. €20k p.a. – no monkeys here!
2. The questions they asked will provides lots of data but very little information to actually implement change: the usual suspects – the stock multiple choice and Likert scales that will tell them how many respondents ‘disagree’ with sentences like ‘it’s easy to purchase…’ , ‘I would recommend this site to a friend..’ but not WHY? Nor what needs to change! 
   Everyone thinks they can design a survey and they just throw together a few stock questions but there is a method to our madness – there are objectives, logic, sequence, wording that doesn’t lead, exploratory research to arrive at those multi choice responses – we don’t just make them up. Then we test, do pilot analysis, revise, test again then run.
3. What was the point of the research exercise?
   • Has the head of research or anyone in the marketing department no friends?
   • Are they not consumers themselves?
   • Don’t they already know their site sucks?

Research Opportunity Lost

There was only one open-ended question that allowed me to freestyle my opinion and I suggested that instead of spending €20k on a survey software licence that will confirm (empirically) that their user experience needs attention, why wouldn’t they spend that money on usability testing – you know track respondents as they perform tasks, do some A/B testing on maybe some wireframes – find out specifically what’s not working and test alternatives.

Research your Research Options

So often, research output is limited by the limited experience and knowledge of those commissioning the research: ‘We need some research, let’s do an online survey.’ 

There are as many choices of research designs as there are ethnic cuisines – get off the meat and two veg!