4 Reasons Not To Rely On Market Research Alone

May 9, 2008 at 5:51 am | Posted in Advertising, Books, Communication, General, Gladwell, Malcolm - Blink, Marketing, Ogilvy, David - On Advertising, Online marketing, ROI, Usability, Web 2.0 | 2 Comments
Tags: , , , , , , , , ,

I was freezing my tush off a couple of weeks ago at Wrigley Field and inquired to my good friend why he had made the unlikely (in my mind, at least) switch from marketing to insurance. It seemed to me that he was turned off by the manipulative and predictive nature of old-school marketing – as though statistics and market research would tell exactly how someone would behave.

Then, just yesterday, I read both David Oglivy’s chapter “18 Miracles of Research” in On Advertising and Hank Williams’ post Who Needs Market Research. The stars seem aligned to answer a few questions about market research, including: Why can I not rely solely on market research and how can the online channel help?David Ogilvy

Sure, research is helpful to some extent. As Ogilvy said, “Advertising people who ignore research are as dangerous as generals who ignore decodes of enemy signals. (pg. 158)” But you are making a severe mistake if you expect focus groups, polls, and testing to divine your strategy like a Magic 8-ball.

Market research (especially customer-focused research) must be taken with a sizable grain of proverbial salt. Here are four reasons why:

1. While I think there is some use of market research, I agree with Hank Williams’ hypothesis that content and experience are much more important. People cannot articulate an experience they’ve never had. Focus on producing good content and a good experience – not whether people claim that they understand how they think they will respond to a hypothetical situation. And even if you have the product or advertisement, do you really think people will respond the same way to it during a focus group at the mall as they would in their own homes?

2. Often times, people can’t articulate their feelings about a product at all. Malcolm Gladwell has a few intriguing examples in Blink. From Gladwell: “It [the Aeron chair] looked different. There was nothing familiar about it. Maybe the word ‘ugly’ was just a proxy for ‘different’ ” [said Bill Dowell, research lead on the Aeron]. The problem with market research is that often it is simply too blunt an instrument to pick up this distinction between the bad and the merely different. (pg. 174)”Malcolm Gladwell

Not only can someone not tell you about an experience they have had, they often can’t articulate one they have had. (Think about all the stories of witnesses picking the wrong suspect in a line-up.)

3. Market research produces a false sense of certainty. Many businesspeople are cowards (admit it, you’ve seen them). They want to keep their job rather than do their job, so they spend all day making sure they don’t get in trouble (read: take risks).

Listen up, college students: Marketing does not bode well for the risk-adverse. And market research is often the tool of the risk adverse. It excuses the peon’s work to the manager, the manager’s decisions to the VP, and the VP’s guidance to the President.

Gladwell goes on, this time using television shows as an example.

“Viewer didn’t actually hate [All in the Family and The Mary Tyler Moore Show]. They were just shocked by them. And all the ballyhooed techniques used by the armies of marketer researchers at CBS utterly failed to distinguish between these two very different emotions.

But testing products or ideas that are truly revolutionary is another matter, and the most successful companies are those that understand that in those cases, the first impressions of their consumers need interpretation. We like market research because it provides certainty – a score, a prediction; if someone asks us why we made the decision we did, we can point to a number. But the truth is that for the most important decisions, there can be no certainty. (pg. 175 – my emphasis)”

4. People knowingly or unknowingly lie or give the answer they think they ought to. It’s an unpleasant truth. Spend any time in politics and you will become a believer too.

From Ogilvy: “Respondents do not always tell the truth to interviewers. I used to start my questionnaires by asking, ‘Which would you rather hear on the radio tonight – Jack Benny or a Shakespeare play?’ If the respondent said Shakespeare, I knew he was a liar and broke off the interview. (pg. 164)”

In addition to saying what they think you want to hear (as in Ogilvy’s example), remember that many people even today carry deep biases. Watch the exit polls after this year’s Presidential campaign. I guarantee the exit poll results will be significantly different from the actual voting in favor of McCain. While people want to say they will vote for a woman or an African-American, things change when they’re alone in the booth. The same behavior applies to products and advertisements

Why The Online Channel Improves This Process:

People experience websites and the metrics (time on page, pageviews, clicks, bounce rate, etc) prove their interests. You can be certain of this because metrics don’t lie.

Despite all this, we have spent decades believing in and promoting the old way of market research. It was to be expected – we had nothing else to go on.

Now, however, web analytics free us from market research. Instead of asking “What would you do in this situation,” we can actually measure behavior, with certainty, in real time.

There is no reason to run a focus group at the mall or pay for phone interviews. Almost every demographic is well represented online. The results are more accurate and it costs far less. Why would you do it the old way?

If your company still partakes in these practices, re-read #3. Someone there needs to be assuaged and reassured. They are so uncertain of the product or ad that they cling to something tangible: people gathered, surveys marked in pencil, spreadsheets checked and cross-checked.

Sure, sometimes it is a useful exercise to do the old customer-focused market research. It can sometimes serve a purpose – whether in extracting customer opinion or forcing businesspeople to solidify their positions. But market research ought not be a manipulative tool. You cannot neither predict the future nor truly influence behavior with just market research. My friend from Wrigley understood the distastefulness of this.

Build a relationship with your customer. Listen. Engage them. Foster trust. Develop these skills instead and you’ll hit it out of the park time.

Advertisements

2 Comments »

RSS feed for comments on this post. TrackBack URI

  1. Market research can provide actionable information if only a company’s Core and Must-Have Customers are surveyed. (Core Customers are a company’s most loyal customers who are willing to pay a fair resources. Must-Have Customers are people who could become Core Customers, but they currently do business with the competition.)

    To find out exactly why Core and Must-Have Customers value a company’s products or services they have to be asked through research. There’s a huge difference between mediocre research and great research. Mediocre research yields “nice-to-know” information but nothing relevant that can be acted on. Great research is insightful and provides the basis for effective action.

    It’s extremely important to confine research to Core and Must-Have Customers. Asking anyone else will taint the data and could send the company down the completely wrong—and potentially very dangerous—path. Be sure to ask “knock-out” questions as early in the process as possible. Getting all the way through a survey, only to find out that the person isn’t a Core or Must-Have Customer, is a tremendous waste of time and resources.

    In order to formulate the questions, it’s critical to have a good idea of the kind of answers that the research is looking for. For instance, a local company selling high-end electronics wants to know how to attract customers away from a national big box electronics company. With that as the goal, the research questions can be structured in a way that will produce information that can actually be used.

  2. “There is no reason to run a focus group at the mall or pay for phone interviews. Almost every demographic is well represented online. The results are more accurate and it costs far less. Why would you do it the old way?”

    Couldn’t agree more, DJ. I’ve been preaching to my clients – for months now – that their cheapest way to test promos, offers and new messages is to run a test on Google AdWords. If you can get a sample of 200-400 impressions, the data is pretty dang good. Granted, AdWords is a biased crowd for a lot of different reasons..however, it works well in certain niches (technology, for one, where users are trained on google in the first place.)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.
Entries and comments feeds.

%d bloggers like this: