(The names have been changed to protect the innocent.)

Over the past couple months an enterprise software vendor has been running a series of blind focus groups designed to ascertain what the market view of their products really is.  They’ve gone the full monty on it, with one way glass mirrors, professional moderators, and have worked with a truly representative cross section of consumers, not just their own customers.

Prior to the focus group, this company considered themselves top in the industry, in terms of technology, in terms or market perception, and in terms of customer support.  They walked tall, chests puffed out proudly.

Unfortunately, that isn’t what the focus groups (and there were many across the country) told them.  Universally they heard their brand associated with words like “second tier,” “outdated technology” and “poor customer service.”

This was truly heart wrenching stuff, which echos completely across the organization, not just in the marketing department, but through the produt teams, technology, customer support and  especially in the halls of upper management.

One telling statistic came through: when ranking products, one competitor continually came up on top, especially in terms of technology where they were ranked as heads and tails above all others.  That competitors product?  A white label version of the vendors application, the same exact software that the groups had also tagged as “outdated technology.”  Calling all the results into question.

The lessons here are legion:

• You need to do blind focus groups to guage market opinion – talking to your customer base alone will give you warped view of the world.  Typically, someone says “let’s talk to the customers” so they get all the sales people to give them contacts.  The problem is those contacts represent the most tame of the existing customer base, and also they represent exactly the people who have a vested professional interest in your product. Yesterday I wrote about Branded Communities, and this is yet another reason they don’t make sense for focus groups, as you will only hear from the people who are proponents of your product
• This is really Opinion Research – opinions stink and everyone has one.  The problem is that no one in your market is going to be a real expert on all the products, especially when it comes to enterprise software.  So when asked, they fall back on guesses, what they remember from that 10 minute demo at the tradeshow 2 years ago (which isn’t much, although they do remember that hangover, which was truly epic).  
• A blind focus group is a loaded gun – you can expect your going to hear some very bad things.  Your baby is ugly.  Remember that you’re seeing a cross section of the market, which is going to be a warts and all experience.  You’d probably do best to throw out the best 10% as well as the worst 10% of views just to get a more realistic view.  The problem is, at some point your CEO will be looking at this research, and they have notoriously thin skins when it comes to hearing about problems.

Personally, I think that the fact that the same bit of software was listed at the bottom and the top of the solutions in this vertical speaks to an obvious marketing problem with this vendor.  If their competitor can sell it as “the best of the breed” then why would their own customers think it is outdated?

Okay, question for the masses: Have you done a blind focus group?  Would you consider doing one? How would you deal with the message these guys got?