Yes, we do market research.  In fact, for many years I have published industry benchmarking and best practices research in the area of Social Media.  So it might be surprising to read a post about when – and why – market research should not be done.

At its core, the rationale for market research is to help make better decisions.  Market research can discover, validate, and reduce risk.  It can improve ROI, improve management confidence, and expose competitive threat.  And it can also be the worst thing that an organization can do.

Even when done perfectly, traditional market research – both qualitative (focus groups, interviews, etc) and quantitative (surveys) – has one inherent flaw:  it takes time.  The market research instrument is designed, tested, and approved.  The respondent sample set is recruited and surveyed.  The results are tabulated, tested for statistical significance, and analyzed.  Finally, a report is written, presented, and discussed.  It takes time and costs money.

Traditional market research works by examining a sample, and then extrapolating these findings to the “universe”.  There is no absolute certainty that a particular sample actually does represent the actual universe.  And even if it did, testing only a sample means that invaluable “outlier” findings would not appear.  Today’s competitive dynamic and sense of urgency has meant that organizations are now adopting two alternative approaches:  DIY (Do-It-Yourself) Market Research, and Agile Crowdsourcing.

DIY Market Research:  In this approach, the organization turns to the free: Survey Monkey, Informal web polls, Likes, and blog or Twitter feedback.  The rationale is that some data is better than none, and therefore it is best to ask questions and see the results.  While this sounds reasonable, it introduces what market researchers call bias, and can result in precisely the wrong decisions being made.

For example, a web poll asking about future service preferences will only capture results from those who already have at least some affinity to the organization.  (They did, after all, make it to the web site, and they did spend the time and effort filling in the web poll.)  But is it not just as important to learn what others think – especially those who didn’t ever get to the site?  If you put this concern aside, market research always exists in context: who are the respondents, anyway?  Are they from a geography that the organization doesn’t serve?  Are they children?  Are they machine-based spammers?

Mechanical bias can also skew results.  Consider a survey delivered using Survey Monkey,  Are the results biased because of the order of the questions?  Or because of the order of the answers within each question?  Or is the sample set too small, or non-representative?  Are the results biased because certain types of respondents (senior management, non-native English speakers) get frustrated by the survey length, and drop out prematurely?

Yes, DIY market research when done properly, can help. Sadly, though, the results can mislead. When the care taken to remove bias in traditional market research is neglected in favor of urgency, this isn’t surprising.

Agile Crowdsourcing:  This approach is different.  Instead of traditional market research, the organization pilots an early version directly in the marketplace.  Based on real feedback, it makes mid-course improvements, releasing better and better versions, or more and more focused products and services.  This approach, made popular by Google and their forever-in-beta products, is powerful.  It posits that by the time traditional market research would have provided basic insights, far more can be learned by going directly to the market and  experiencing first-hand feedback plus several product development-release-feedback cycles.

This same iterative direct-to-market approach is at the core of a Pay-per-click advertising strategy.  Skip the market research, and directly test which of multiple ads generate greater click-through.  Then iterate by dropping the poor performers, and release another generation of ads that are variants of the best.  Then repeat.  This same technique – typically called A-B testing – is also at the core of improving email response and web page conversion.

While big data and Agile Crowdsourcing are exceptionally appealing, there are downsides.  For example, making changes based on observed behavior does not explore the underlying motivation to transact.  It also doesn’t evaluate the motivation of those who abandon their journey along the way – or those who are not even aware of the product or service’s existence.  Agile Crowdsourcing cannot explore the “could be” for products or services not yet on the market.  And finally, using this approach gives your competitors a sneak-peek at your capabilities and direction, reducing your competitive advantage.

Does this topic resonate? Reach out to Randall: he can present it to your group.  (More presentation topics)
Download Randall’s professional credentials: Speaker credentials one-sheet or Management Advisory credentials.

Content Authenticity Statement: 100% original content: no AI was used in creating this content.

@RandallCraig (Follow me for daily insights)
www.RandallCraig.com: Professional credentials site.