There’s a pattern here, of sorts. The research industry pushes out boring surveys. Clients want innovation. Agencies complain about timescales and budgets. We all get together at conferences to vent… and then we go back to the office and forget and about it until next year.
The good news? WARC’s online research conference offered up some tangible solutions we can start using today.
The line-up was varied, combining leading agencies and a peppering of interesting clients. Our chair for the day was Wired’s Editor at Large Ben Hammersley. His wry humour kept things moving along apace.
Jon Puleston talked about Designing surveys that get results. Jon is what I’d call a researchers’ researcher, whose experiments show how asking questions in different ways gets different responses.
Online surveys are formulaic, boring, packed full of strongly agree – strongly disagree grid questions. The problem? We are focussing on what we want to know rather than how we want people to respond.
The solution? Inject more creativity.
Visual design is easier than it ever has been. Let’s get rid of the clipart and get make visual appeal a priority.
We should also warm people up rather than going straight to the nitty gritty. A fun question up front can inject enthusiasm into the remainder of a survey. As he put it:
“Our industry has no sense of foreplay…we’ve got to think about our chat up lines.”
We should treat respondents like consultants, not fools. Empower them with tasks that capture their imagination. Take media planning. Asking Imagine you had £1million to spend on advertising to reach you – how would you do it, what media would you use? gets much more out of people than a typical “pick from a list” approach.
Feedback also has a role to play. We don’t share results with respondents. Ask them an interesting question – then tell them what other people think. They like it when we do. It’s easy. So let’s do it.
I agree with Jon, but suggest practical challenges remain. Do researchers now need copywriting and art direction skills to write a survey? This is easy at an integrated agency like Brass, but maybe a challenge for the data-factories like…
Peter Harrison’s talk Playing with Behavioural Economics continued the theme.
Behavioural economics has changed how researchers think about thinking. We know emotions drive much of our decision making (see Kahneman’s two-system view of cognition) but we lack a methodology to back it up. We know routine decision-making is heavily influenced by context, like the environment we are in and the choices presented to us. Most research is de-contextualised: asking people to make theoretical decisions about previously unconsidered future scenarios. Often the survey context is one of boredom and frustration. It’s what he calls the context gap.
He offers some interesting ideas to create context in research.
- Games are good. A playful task can change the way we think, feel and behave. Creativity not boredom.
- Splitting people into teams and set them into competition also works. Easily possible on online communities.
- He also suggests restricting time. Many FMCG purchases are split second. Getting people to choose e.g. NPD from a shelf mock up in a limited timeframe is more reliable in his experience.
- Getting people to empathise and role play e.g. medical patients playing the role of doctor to brainstorm treatment solutions.
- Thoughtful introductions also get people in the frame of mind they would be when making the decision the survey relates to.
Ray Pointer’s client debate was entertaining, but perhaps generated more heat than light. Quality and robustness concerns over online research remain but it doesn’t stop clients like Nokia doing all their tracking research this way. It’s more cost effective, innit?
The accusation that clients are responsible for bad surveys was firmly rebutted. Sure, they may want the moon on a stick, yesterday. What is it going to take for agencies to push back? Martin Silcock from Nokia said “I rate my agencies by the quality of the questions they pose”. Food for thought.
Clients want new approaches to get closer to customers like mobile research. Costs are a barrier. John Boreham from Mothercare’s view? Take a punt on a new way of doing things once a year and see what happens. Nokia’s agencies have to offer tasters to help counter internal resistance. Their online community started this way.
A final word from Martin Silcock at Nokia:
“The thing I like form an agency is that they are slightly experimenting on you, pushing the boundaries of what is possible.”
Agreed. See you all next year.