WARC conference: Online research now & next

Pic courtesy of Jan Chipchase

There’s a pattern here, of sorts. The research industry pushes out boring surveys. Clients want innovation. Agencies complain about timescales and budgets. We all get together at conferences to vent… and then we go back to the office and forget and about it until next year.

The good news? WARC’s online research conference offered up some tangible solutions we can start using today.

The line-up was varied, combining leading agencies and a peppering of interesting clients. Our chair for the day was Wired’s Editor at Large Ben Hammersley. His wry humour kept things moving along apace.

Jon Puleston talked about Designing surveys that get results. Jon is what I’d call a researchers’ researcher, whose experiments show how asking questions in different ways gets different responses.

Online surveys are formulaic, boring, packed full of strongly agree – strongly disagree grid questions. The problem? We are focussing on what we want to know rather than how we want people to respond.

The solution? Inject more creativity.

Visual design is easier than it ever has been. Let’s get rid of the clipart and get make visual appeal a priority.

We should also warm people up rather than going straight to the nitty gritty. A fun question up front can inject enthusiasm into the remainder of a survey. As he put it:

“Our industry has no sense of foreplay…we’ve got to think about our chat up lines.”

We should treat respondents like consultants, not fools. Empower them with tasks that capture their imagination. Take media planning. Asking Imagine you had £1million to spend on advertising to reach you – how would you do it, what media would you use? gets much more out of people than a typical “pick from a list” approach.

Feedback also has a role to play. We don’t share results with respondents. Ask them an interesting question – then tell them what other people think. They like it when we do. It’s easy. So let’s do it.

I agree with Jon, but suggest practical challenges remain. Do researchers now need copywriting and art direction skills to write a survey? This is easy at an integrated agency like Brass, but maybe a challenge for the data-factories like…

Peter Harrison’s talk Playing with Behavioural Economics continued the theme.

Behavioural economics has changed how researchers think about thinking. We know emotions drive much of our decision making (see Kahneman’s two-system view of cognition) but we lack a methodology to back it up. We know routine decision-making is heavily influenced by context, like the environment we are in and the choices presented to us. Most research is de-contextualised: asking people to make theoretical decisions about previously unconsidered future scenarios. Often the survey context is one of boredom and frustration. It’s what he calls the context gap.

He offers some interesting ideas to create context in research.

  • Games are good. A playful task can change the way we think, feel and behave. Creativity not boredom.
  • Splitting people into teams and set them into competition also works. Easily possible on online communities.
  • He also suggests restricting time. Many FMCG purchases are split second. Getting people to choose e.g. NPD from a shelf mock up in a limited timeframe is more reliable in his experience.
  • Getting people to empathise and role play e.g. medical patients playing the role of doctor to brainstorm treatment solutions.
  • Thoughtful introductions also get people in the frame of mind they would be when making the decision the survey relates to.

Ray Pointer’s client debate was entertaining, but perhaps generated more heat than light. Quality and robustness concerns over online research remain but it doesn’t stop clients like Nokia doing all their tracking research this way. It’s more cost effective, innit?

The accusation that clients are responsible for bad surveys was firmly rebutted. Sure, they may want the moon on a stick, yesterday. What is it going to take for agencies to push back? Martin Silcock from Nokia said “I rate my agencies by the quality of the questions they pose”. Food for thought.

Clients want new approaches to get closer to customers like mobile research. Costs are a barrier. John Boreham from Mothercare’s view? Take a punt on a new way of doing things once a year and see what happens. Nokia’s agencies have to offer tasters to help counter internal resistance. Their online community started this way.

A final word from Martin Silcock at Nokia:

“The thing I like form an agency is that they are slightly experimenting on you, pushing the boundaries of what is possible.”

Agreed. See you all next year.

About Simon Shaw

I'm a Director at an insight consultancy. I'm interested in marketing, market research & consumer psychology. The views expressed are not necessarily those of my employer.
This entry was posted in Conferences, Marketing Research, Technology, WARC and tagged , , , , , . Bookmark the permalink.

2 Responses to WARC conference: Online research now & next

  1. Simon Riley says:

    Sounds an interesting day. On the behavioural economics stuff, it sometimes seems the questions survey researchers are grappling with now are the same ones that confronted them with the rise of qualitative research methods in the mix 20 or 30 years ago. That is, trying to bring nobbly psychological realities into the methods of an industry based on the necessarily reductive disciplines of measurement. The smart ones got that qual was doing a load of things survey research couldn’t do. But qual remains unsatisfying to many in that part of the industry because it isn’t in the business of providing numbers and objective measures.

    BE is so important, I think, because it can provide the bridge. Because it’s the product of economists, it can talk to the numbers-based researchers in a way qual can’t. But it is essentially giving them the message qual has tried to give for decades: a lot of what matters is about what goes on below the ‘rational’ radar. Peter Harrison’s ideas sound like classic qual research thinking. I do hope it gets through to everyone this time!

    Did you sense general acceptance of the basic principles from the audience, or scepticism? I’m guessing the big question might have been more, how do we implement the new learnings from BE in our surveys? To which one might reply, the same way you have adjusted to what qual practitioners have been showing for the last 40 years (embarrassed silence descends upon the room). Sorry, I speak as a former big agency qual researcher here, so I’ve seen some great and some definitely not so great examples of quant researchers’ attitudes to qual data. Anything that brings the smell of coffee to those still slumbering is welcome.

    • Simon Shaw says:

      Thanks for your thoughts and suggestions Simon, really helpful.

      For me the overriding sentiment from the day was “Online data collection has serious flaws. We need to change.”

      Your point on BE being the bridge is entirely right in my view. There are people out there that get it who are making it work. These people tend to be in smaller entrepreneurial agencies who are empowered to think, straddling qual / quant / NewMR.

      I’d guess that the average delegate passively agreed with what Jon and Peter were saying. But even if every delegate vowed to start afresh, the industry supertanker isn’t going to turn around anytime soon.

      The starting point for most questionnaires is a previous questionnaire, copy & paste.
      The starting point for most briefs is a previous brief, copy & paste.
      The starting point for most board reports is a set of KPIs, copy & paste.

      Change is risk. It will take those on the agency vanguard to evidence the business benefits of new approaches before the data factories sit up and listen.

      So you might say the meta-message from the day was actually “Online data collection has serious flaws. We need to change. You first…”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s