Stick or twist? Supermarket buying patterns from 5 years of Clubcard data

A team of psychologists and data scientists from UCL worked together to interrogate Tesco Clubcard data, exploring hidden patterns in how people buy. It is powerful study because of its:

  • Scale (283,000 people);
  • Scope (longitudinal data covering 5 years, 89 store visits per person on average)
  • Remit (purchase data, as opposed to attitudinal data)

I’m always keen to get a meta-level perspective of buying behaviour, stretching horizons beyond short-term client projects to long-term trends. The study promises a great deal.

The findings are a mixture of the obvious and the new. It’s a reminder that you can throw terabytes of computing power at an ocean of data and end up mostly confirming what we already know. That’s OK – it’s why it is called research.

Clubcard buying patterns

The academic paper is dense and has lots of jargon. The diagram makes the pattern clear. Each dot is a shopping trip: green dots are when people twist and blue dots are when people stick.

Coherency maximising_Clubcard

I find all this stuff useful for a couple of reasons:

A reminder that real-world decision making differs from the lab

The authors describe how lab experiments modelling similar choices often show people are systematic and rational. They exhibit system 2 strategies in these system 2 conditions, relying on objective criteria in comparable trade-off environments.

The real world is full of subjective evaluations (“ooh that looks tasty”) in varied trade-off environments (e.g. stores that change layout or merchandise).

As Professor Dilip Soman put it in his book The Last Mile:

“When you’re in a store, absolutely everything around you could influence what you buy: the display, the price presentation… the presence of crowds… What’s more, these factors could interact with each other… Theory can show us the way but without testing… we risk failure because of something in the background context that trips up the effectiveness of our intervention.”  

Data can only take us so far into the subjective world of human taste. How do you reverse engineer a whim?

Hidden patterns that shoppers may not be aware of  

People like things more the more often they choose them (the authors call this ‘coherency-maximising’). They hypothesise that “people come to like what they purchase, out of a need to “make sense” and explain their choices to themselves and others.”

This aligns with other sources – like Sharp’s How Brands Grow – showing how people bring their attitudes into line with their behaviour. As Sharp writes “…since brands aren’t very important to us, brand buying tends to have a strong effect on our rather weak attitudes.” 

About Simon Shaw

I'm a Director at an insight consultancy. I'm interested in marketing, market research & consumer psychology. The views expressed are not necessarily those of my employer.
This entry was posted in Big data, Market Research and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s