Blog

Why You Should Not (Only) Look For “Representative Results” in Consumer Research

If you’re running consumer research, you’re probably used to getting “representative results” from your research agency. I think it’s worth looking into how they get there, what it actually means, and why we are proposing a different way that we think has a lot more value to clients.

Let’s start by looking at what it is you are getting when your agency talks about “representative results” or “your results are nationally representative”.

What it means is that the results you look at are taken from a sample of respondents that, in its composition, is representative of the whole population. That’s a great concept and can be informative, but it’s important to look at how the agency arrives at it.

Every respondent costs money. Traditional agencies have a high cost per respondent (due to the manual, phone-based system still being one of the dominant ways of data collection), which means they do not want to waste money on a respondent that won’t be counted into the overall result. That means they arrive at the overall result by using a matrix table that consists of the respondent parameters that they want to control for.

Let’s use a simple example where a study should be nationally representative in terms of age and gender, and the population consists of the same % for each cell in the matrix. The matrix for a study of 1000 overall respondents (a typical delivery for a brand tracker for example) would look something like that:

 

Men
Women
18-25
100 responses 100 responses
26-35
100 responses 100 responses
36-45
100 responses 100 responses
46-55
100 responses 100 responses
+55
100 responses 100 responses

 

This setup, and the costs associated with every respondent, means that your agency will stop recruiting for 18-25 year-old men the moment they have 100 respondents.

You can probably already see the problem: While the overall result might be representative and somewhat informative, it won’t be very actionable. The results and responses to whatever survey was used will be different between 18 year-olds and 60 year-olds, for example. The problem is: When you want to look at the results for each cell specifically, you are left with a low number of responses for each and statistically not reliable data.

At Trybe, we’ve gone a different way: We recruit everyone, without an underlying matrix. We do this because we think that you need to be able to look at granular segments of the population, with a high number of respondents in each, in order to get actionable data. Now, most researchers will scream “But it needs to be representative!!” from the top of their lungs. I’ll get to that. First, here is what the same matrix might look like when we deliver a brand tracker:

 

Men
Women
18-25
324 responses 343 responses
26-35
432 responses 345 responses
36-45
345 responses 239 responses
46-55
243 responses 324 responses
+55
343 responses 432 responses

 

You can already see that there are a lot more respondents in each cell, which means that when you look at each segment (=each cell in the matrix) on its own you get a lot more actionable and reliable results. Of course, the problem is that the overall result is not representative anymore, because some segments are over- and some under-represented. Just looking at the overall results without any adjustments would skew the data.

We counter this by giving you the option to MAKE it representative, after the fact. In all our dashboards you have the ability to weight the data in exactly the way you want to look at it. That will re-calculate the results of your study instantly to make it representative, but it will give you an additional benefit: It makes it possible to make the results representative vs. whatever population you want, not just the population of a country. Let me explain.

In many cases and for many products it can be interesting to look at a nationally representative sample. But in most cases, a nationally representative population is not your addressable market or your market of interest. You might have a product that is distributed in places that are 75% male and 25% female. Or 100% aimed at <30 years old. Or any other combination of the demographic data points you collect. Yet, with traditional research you are stuck with a nationally representative study or need to radically bring down the number of total responses in order to mirror your actual market. We think this is too big of a tradeoff, and from the beginning have decided that we want to go another way.

At Trybe, we allow you to weight the data any way you want. You can mirror the total market, a specific segment of it, a certain age composition, etc. Anything you want to look at. And because we deliver you a minimum of 2000 respondents for each research (and often a multiple of that), you will always have enough respondents to get actionable insights out of it.

Weighting feature on Trybe’s Dashboard

If this sounds complicated: It isn’t. You simply activate or deactivate weighting and are free to adjust the weighting to whatever it is you want to look at, for any metric included in our brand trackers.

If you want to know how it works, feel free to reach out to us and we will run you through the metrics we deliver, how the dashboard looks like and what the benefits of this approach are for you.

 

 


Related Blog Posts
Consumer Insights: An Introduction to The Basics of What It Takes
What Is Syndicated Research And What Are Its Benefits?