Playing games, finding answers

Playing games, finding answers

When you’re researching, how do you find truth and reality, instead of finding what people want to project or think you want to hear?

[Note: This blog has been amended on 18th December 2012, adding input from Mark Earls, a.k.a. The Herdmeister. I’m really grateful for his expert input, so I’ve added Mark’s comments verbatim in italics and square brackets below the body text. If you want to know more about what Mark does, his book “Herd” is well worth buying]

It’s widely agreed now that the way isn’t through focus groups. They really suck. A marketeer with large automobile clients told me that when new car manufacturers sit men in a room together, men are very clear on what makes them buy a car: its specifications. The higher the spec, the more likely to buy. In focus groups, the overwhelming evidence from men, was that they buy based on factors like higher mileage, better audio, ABS and traction control.

The only problem with this is that it’s not true. Anonymous one-to-one polling and other post-purchase evidence showed that these were often the last reasons why a purchase was made. In focus groups, in front of other men, it was next to impossible to get them to give the real reasons why they buy a car. The real reasons were just too, well, embarrassing. Real reasons ranged from: “My wife likes it” to “It’ll impress the neighbours” to “my brother in law’s got one – and my family think he’s a big shot”

It’s the same when you ask someone obese to keep a food diary. The cookies they say they consume too often reflect their perception of who they’d like to be, rather than who they are and how thy behave. Far better to count the cookies left in the jar at the end of the week.

I’ve found this phenomenon myself, both conducting my own research and in the research I’ve encountered by clients. As a result, I’m always looking out for new ways of finding answers and I’m constantly trying to use new, more creative ways of exploring behaviour.

This is the story of something i tried recently with the a client in Africa. I think it worked.

The task
The task the client set me was to find insights that could guide their strategy to economically empower girls in the developing world. It was a super-exciting brief, but after a lot of desk research reviewing five years of their existing reporting and research, I left for Rwanda (where I’d be commissioning a team of local researchers and translators to undertake qualitative and quantitative research that I had designed) pretty confident that the survey questions we’d asking the 10-19 year old girls were the right ones, but unconvinced that me and my client-partner on this project had cracked how we would conduct our qualitative research, and how our method would lead to more honest answers, and avoid all the standard focus group or cookie-munching pitfalls.

Africa teaches agility
I’ve worked in sixteen countries across sub-saharan Africa, and one thing that I’ve learned is the need to stay agile. When people tell me that things in Africa are slow moving or hard to navigate, I say that mostly I’ve found something different. Working in Africa sometimes reminds me of playing the old arcade game Frogger. At first the vehicles crossing your path in front seem to form an impenetrable wall between you and your destination, but if you watch them closely, learn their different rhythms and wait patiently, suddenly they align, opening a gap that allows you to just walk calmly across. In Africa you need to be open to these moments and agile enough to take them.

Luckily for us, this gap appeared just a day after we arrived.

When we arrived in Kigali, one of the first people we met, who worked in the local office of the client’s charity had developed a task in the charity’s local language magazine to encourage local girls’ engagement with money. Phoebe conceded that her attempt was rushed and unfinished, but she was definitely on to something. With only a day to go before we went up-country to start the research, my research partner and I decided to try and create a game, using Phoebe’s task as our starting point, to test girls’ behaviour with and around money.

We both thought that this could be a great foil for the largely quantitative survey we had designed and had the potential to side-step the focus group traps that other researchers had fallen into.

What’s in a game?
My guiding thought was a game of Monopoly. You know that moment when all the property is sold? When even some fool has bought the Electric Company and Water Works? Well, that’s when the real game starts. The shyster’s a shyster. The person who’d steal from the bank will try to steal from the bank. The soft touch will let you pay rent after you’ve passed Go and collected £200. Basically, Monopoly mirrors real life. If we could create a game that could do this in Rwanda, we had the potential of observing a simulation of real-life behaviour.

We quickly designed a game based on a village market. There were items for sale in the market that girls could buy (like shoes, food, sanitary towels, medicine, a chicken, school books). Each girl got 2000 Rwandan francs at the beginning of the game to spend on these items. Girls were then invited to buy items before the game (proper) started and save money either at home or in a bank. Once the game started, real-life scenarios were read from cards to which created jeopardy to the girls were invited to react: to do or not do; buy or not buy (“tomorrow’s Sunday and you have to go to church”, “your mother’s ill and needs medicine”, “your period has just started”, “if you’ve bought a chicken, it’s laid six eggs” and many others).

The game we made for the field was as cheap and cheerful as it could have been. Colour photocopies of various items from Ni Niyampinga’s market task were stuck onto card and cards that created jeopardy scenarios were handwritten on the bus as we travelled. The money was a printed, hand-cut Word document. We produced no playing board – My research partner played the game with a translator and the girls just asking questions as she went – but it was all we needed to get the girls imagination working and to create another world that could mirror their real one.

The point of the game was clear: use it to simulate real-life behaviour, listen and watch the girls reactions and draw insights from what we heard and saw.

Playing the market game in Rwanda

We knew that we’d got it right when two things happened. The first was when we played the game with 10-12 year old girls. They spent a large part of their 2000 francs on sweets. This was despite both the sweets and the money being notional. When asked why, the girls said that that’s what they’d do if they had 2000 francs to spend. One even pointed out to us that they were 10 year olds: what were they supposed to do? The other thing that happened was when we played with an older group. No sooner had we distributed the money to each of them, than they passed it all to one girl amongst them. When asked why, they answered that they were part of a group savings plan, and that’s how they’d usually manage money, so they’d like to play the game as a group. That was when we knew that their behaviour in the game was as close to real life as we were going to get!

Once I was back in the UK, coding and collating near to 30,000 or so survey responses then mixing and comparing some of the data (basically, rudimentary regression analysis) I kept going back to the findings of the game as a touch-stone. When an aggregate of girls were telling us something in the surveys, how did this compare to the behaviour of the girls playing the game? More often than not, the behaviour complemented the survey aggregates.

Playing the game – absorbing the learning
I was going to be presenting the results of my work in front of the charity’s team, and other friends of the project, at a five day offsite they were having. When I started the project, one of my first thoughts was that we should find ways of metaphorically bringing the girls from Rwanda to the offsite. Often these kind of affairs can feel quite remote from the customers or subjects that they’re ultimately aimed to serve. Finding ways of bringing the girls’ individual characters with windows into the realities of the lives they lead would keep the Foundation empathising with the girls’ needs, informing better choices of how to serve them.

I thought the game would be perfect for this. I would get the charity staff to play the game.

My initial board game sketch

Imagist’s final game design

Building on what we had learned in the field, I sketched out a board for the game, then worked with Imagist (who had designed and manufactured the beautiful career cards I’d taken with me as a prop for the surveys) to make an actual board game. We played the game in prototype and continued to refine the scenario cards, rules and set up pretty much until the day the game was played (even enlisting a local Portland Kinko to produce more cards with adjusted questions). My research partner then wrote a dozen personas based on the Rwandese girls who had played the games, and girls who had completed the survey. Personas like:

“My name is Francine. I am 13 years old and live with my mother in Musanze. My father is in prison, there without trial since 199. My favourite possession is my school notebooks. What I want to be when I grow up is a teacher. What I am most likely to become is a market trader. If I had 2000 francs I would buy airtime for my cellphone and save the rest.”

Each had a scenario card and had to play like the girl. At the end of the game, they opened an envelope to reveal how girls that age, in their circumstances, had actually played the game. Foundation staff that played the game (apart from having fun) said that one of the biggest insights for them was how paralysing the poverty was. In the game there were choices throughout. But the choices were so difficult, that for a lot of the game they just went round and round, without committing to spending money at all. This is what we had found in the field.

Playing the Market Game

The game went down so well, the charity is now developing it to use as an internal management tool.

In all the talk of “engagement marketing” and “gamification”, I think that the little cardboard board game that I developed with my partner serves as a template for how real insight can be drawn from customers or from groups that are hard to reach. Be agile, listen, keep it simple, keep it open and make it fun.

This way we can learn and produce better stuff that people really want and need.

[Mark Earls:– the problem is that traditional market research presumes that we are reliable witnesses to our own lives and motivations and actions. All the latest cognitive and behavioural science points to the fact that we are mostly “unreliable witnesses” to ourselves.

– it also suffers from directness (see the industry’s obsession with brainscience and going beyond what people tell us). What you’re talking about is oblique research strategies (which have a really strong academic pedigree). By getting people to play games, we can understand a whole lot more about how they really do things than by asking them. See this e.g. . Not least because games deal with behaviour rather than opinions (and how we’d like to think we act).

– the other big issue (says HERDmeister) is that much of our lives are lived in the company of others. Asking individuals about themselves is really not that realistic – it presumes people do what they do independently of those around them. Not true. Games not only embrace this context but also tend to help us understand interactions…

– the “truth” is a little misleading as an idea. Why not just say ‘go beyond this’ and “find out what’s really going on”.

– I think you underplay the importance of game as a medium for learning by the research users (clients). This is a big insight and one which allows us to transcend time and space and make research so much more valuable…]