Industry analysts are covered fairly regularly on this blog. If you work in the technology business, industry research firms are a very important audience. In an industry filled with hyperbole and spurious claims on ‘speeds and feeds’, research firms provide end-users with a haven of independent, rational and informed information that strips away the hype to provide solid, understandable and reliable advice – that’s the theory – and in the majority of cases that’s the practice.
When you consider the budgets involved in IT projects across every industry, in every country, you begin to see just how important research firms are to end-users and vendors alike.
Recently David Berlind‘s quest for media transparency turned the spotlight on industry analysts.
He quotes a recent attack on Gartner by NetsEdge Research Group, specifically regarding Gartner’s famed Magic Quadrants. For the uninitated, Gartner’s Magic Quadrants use four boxes to graphically illustrate the relative positions of technology vendors in a given market segment based on the analysis of the relevant Gartner analysts. These quadrants are hugely influential and poor placement can often adversely influence customer prospects.
Berlind also analyses a recent press release from Check Point Software that touts favorable “independent” research – that of course they paid for.
He offers the following advice:
“First, any pitches by vendors or public relations personnel to the press, analysts, or customers that cite research must absolutely disclose any relationship that those entities have with the provider(s) of the research being cited. Second, there needs to be a review and consensus of what language can be used in these pitches.”
On the one hand I applaud David turning the spotlight on the industry analyst business – it needs to be questioned. But on the other hand I can’t help but think he’s missing a bigger piece of the puzzle.
Gartner is the largest industry research firm by a country mile and its acquisition of the third largest research firm (META Group) further dwarfs the next largest competitor, namely Forrester. As a result, I accept that as the industry leader they should be questioned and subject to scutiny. However, in my dealings with them over fifteen years I’ve never experienced the sales pitch that he outlines in his post. Indeed in the course of a recent conversation I had with one of Gartner’s senior analysts, he bemoaned the Magic Quadrant issue. He felt it didn’t provide an accurate picture of the total market – but he felt under pressure from end-users to create one.
Any media industry that has strong influence on high value purchasing decisions will be much sought after. This influence will also provide money generating opportunities for unscrupulous firms. Of that there is no question.
But we have to be careful. We can’t simply categorize a research firm’s bona fides based on their size. The vast majority of analysts and firms I have dealings with are fair and independent. Furthermore the analysts are knowledgable and understand the difference between vendor marketing and reality – and communicate that difference to their clients.
Some of the most independent firms that I’ve come across are small boutiques with broad market knowledge. In addition, while David points out dodgy research results which have been paid for by vendors, there are a growing number of research firms that pump the reputation of subscribing vendors without any disclosure that the vendors they’re promoting are also paying them.
This is a more sinister development. While most medi-savvy individuals will spot a branded research whitepaper as paid-for collateral, when an “independent” report is published only hyping customer vendors – with no disclosure – that’s a far more complex and misleading piece of collateral.
The majority of industry analysts are good, honest and intelligent professionals who do a great job helping end-users find the best technology for their needs. As with all professions, and PR people need no reminder of this fact, there is a subset who have no scruples making money any way they can.
As long as marketing people are willing to finance the cowboys, they’ll always be with us.
So what’s the answer? It’s a difficult one.
The problem is that the ecosystem of analyst firms who conduct pay-for-play research have a ready and willing audience of vendors with big budgets. Money talks so don’t expect these people to disapear, they won’t. Just like PR’s latest affliction, astroturfing, the research industry has its own practitioners of the dark arts.
There has recently been some talk of new research models such as those proposed by Red Monk and the New Rowley Group which aim to re-evaluate the best means of interaction between vendors and end users.
These new approaches may eventually have a positive effect, but in the meantime we should all support David Berlind’s calls for more transparency around analyst reports and advice. The fastest way to achieve this is to educate end-users about the research business. They need to ask the questions about which vendors are subscribers, which vendors sponsored a report etc. When the ultimate customer is asking those questions the business will become self-regulated.
In the meantime, I’d love to hear your views on the matter.
Footnote:
Steve Rubel has a good post on the emergence of two different models of business blogging, namely:
Murphy's Law 