It sounds like a good idea: survey customers to find out what they want, and then let their answers shape your product development and marketing.

But in reality, it’s often a bust.

A case in point: according to an article in BusinessWeek (8/1/05, p. 38), in 2002 the Gap began an intensive program of focus groups, surveys, and other market research.

But in the fiscal quarter ending April 30 of this year, sales fell 4 percent. Analysts expect them to drop another 2 percent for the quarter ending on July 30, 2005.

The reason: eight former employees and two analysts say the Gap “has shifted too far toward research and away from the instinct and emotion favored by many successful clothing merchandisers.”

I see an insidious trend today, especially among traditional marketers turning toward direct marketing: they sincerely believe they can, through customer surveys, confirm that consumers will respond to the product, price, offer, headline, and copy before producing and mailing the promotion.

They mistakenly believe that if they get enough survey responses, they are guaranteed a winning promotion simply by stringing the consumer responses together into copy.

Unfortunately, experienced direct marketers know that there is a huge gap between what consumers say they will buy and what they actually buy.

And for this reason, the only accurate way to determine what will sell is through testing: selling a real product with a real offer to real consumers who vote their preferences not by answering a survey, but with their purchases.

“In my 10+ years experience in DM publishing, I’ve found that customer surveys can be very misleading,” says Julie. “Recipients tend to give answers they think you want to hear.

“Case in point: a company spent over $100,000 developing a product that customer survey respondents said they would purchase. At the end of the day, they didn’t purchase. And the company lost a bundle.

“My advice: treat survey responses as minor gauges of interest. Tread lightly. And test, test, test!”

“A lot of the time customers really don’t know what they want,” says Peter. “The masses simply clamor about waiting for the next best thing. Not exactly the best group of folks to ask.

“This kind of thing permeates video game design discussions quite frequently. Everyone wants to predict what the next blockbuster title will be based on current trends, when in reality, you could spend millions of dollars on focus groups of current title enthusiasts and come up with an ultra-generic flop.”

“A few years ago an insurance company hired me to create a direct mail package. When the design and copy were completed to draft form, my client had a brilliant idea: ‘Let’s show this to a focus group of potential customers,’” says copywriter Steve Slaunwhite. “Nothing I said could dissuade him.

“Based on feedback from the group, my client made sweeping changes to the package, which included eliminating all references to the word FREE. ‘Too trite,’ they said. The package was mailed. And it flopped.

“What went wrong? Not the focus group, according to my client. ‘The focus group can’t be wrong,’ he asserts. ‘After all, they’re the customers!’”

One reason that customer surveys are particularly ineffective for evaluating promotional copy is that consumers like to believe they are immune to direct marketing techniques.

A focus group will loudly proclaim that you “buy one, get one free offer” won’t work with them because they don’t like gimmicks. Then, in an actual test, that same offer will perform like gangbusters, beating the pants off the straightforward offer the focus group members all said they preferred.

So the next time someone at your company says “Let’s do a customer survey,” keep these facts in mind:

1. Survey results are not terribly accurate predictors of what products and offers will sell, the price consumers will pay, or the headlines, copy, or promotions they will respond to.

2. Surveys, in particular focus groups, are useful for discovering the language that consumers use to talk about a particular product, need, or problem.

3. The Internet allows large numbers of consumers to be surveyed quickly and at low cost, with online tools such as www.surveymonkey.com.