Often copied, frequently challenged, and sometimes derided, Gartner’s Magic Quadrant has long been the king of the hill when it comes to major evaluations by industry analysts. Yet, the landscape is changing, and analyst evaluations are slowly moving away from ranking vendors to focus more on markets themselves. 

For decades, the Gartner Magic Quadrant has enjoyed an extraordinary reign as the definitive way of evaluating vendors in any market space – to the point where improving your performance in an MQ has become a mantra: entire books have been written on the subject, and some advisors earn big bucks in peddling what are essentially commonsense tips to vendors on the dark art of moving their dot up and to the right.  

No other analyst evaluation has come even close to making a dent in the MQ, nor has any analyst firm been able to get anywhere near the number of MQs – 123 of them, at the last count. And almost every day, there is a slew of press releases from happy vendors proclaiming that they’ve been recognized as a leader …  

MQs are still going strong when it comes to services – essentially anything you can’t buy in small volumes, test, and then walk away if it doesn’t meet your needs. Today, there are a lot of MQs for “management” and “systems”, as well as “platforms”. 

MQs have their shortcomings 

One shortcoming of MQs has always been that, once a market reaches maturity, there’s little point in continuing to evaluate vendors. Any vendors still in the game must have something going for them. There are hardly any MQs left when it comes to physical products that enterprises can buy, although Gartner is still evaluating storage systems.  

When it comes to helping vendors navigate emerging technologies, Magic Quadrants are no use, either.  Most technologies on the climb in Gartner’s legendary hype cycles are too immature to warrant an MQ. For example, there is not yet any MQ on quantum computing, artificial intelligence, or blockchain. 

For analysts, MQs are major projects that take months. Gartner also stipulates that its analysts cannot engage on advisory with vendors under consideration in an MQ. Since established vendors are the most likely to want to spend money with Gartner analysts to refine their value proposition, a research window of up to six months is a lengthy blackout period where lucrative paid engagements are out of bounds. 

Although it’s a badge of honor for a Gartner analyst to be listed as the author of a Magic Quadrant, the process is also painful. Vendors will comb through qualification criteria and over-analyze every question and every capability. Following a peer review process, it finally comes to sending out excerpts of the MQ for review – and as you can expect, vendors naturally contest every caution in attempts to move their dot by nanometers. 

In general, year-on-year, dots don’t move very fast – which is why asking or hoping for a big jump is an exercise in futility. 

All too often, MQs are delayed by vendors. A couple of years ago, one MQ was delayed by four months … by a vendor that wasn’t even listed: for that very reason. Gartner has an Ombudsman – whose job is to adjudicate on escalations that have not reached a point where both sides have agreed (usually to differ). The process is painful for both sides. Vendors don’t want to damage the relationship with their most-influential analysts by calling them out as wrong. In contrast, analysts need to justify their evaluations, with their judgment being called into question. 

Another problem is that by the time it is published, an MQ is already out of date. Again, a couple of years ago, one of the vendors listed in an MQ as a visionary (bottom right) had gone bust by the time the MQ appeared in print. This is why Gartner executives claim that MQs are an outlook for the next 12 months, while Peer Insights – Gartner’s customer review platform – provides the rear-view mirror.  Since MQs are refreshed only every year or so, they need to be forward-looking enough to convince enterprises to spend money on paying for something up to 12 months old. 

The Magic Quadrant albatross
These are all good reasons why the Magic Quadrant has become something of an albatross for Gartner. 

It’s also why Gartner has introduced Market Guides – which look more at the market requirements and list only “sample vendors”. There’s no vendor evaluation, so the only thing vendors can contest is whether they should have been listed. Forrester is following suit with its Tech Tide series. 

These guides move the delicate process of discussing the strengths and weaknesses of leading vendors into the lucrative world of inquiry calls … something Gartner continues to push its analysts to prioritize. Even when traveling on business, analysts are now required to field inquiries – and are awarded post-inquiry points by participants. Across Gartner, the average – average! – score for all analysts from inquiries is around 4.6/5. That’s pretty high, considering there are more than 100,000 inquiries a year from end-users alone. 

Since these inquiries are conducted behind closed doors, and there’s no record of what was discussed, there are no tiresome vendor objections to get in the way.  

Now, it takes only a few words from an analyst to exclude a vendor from an RFP: “I’m not convinced” / “they haven’t been able to show me” / “customers I speak to aren’t delighted”.  

For AR professionals, it’s more of an imperative than ever to make sure your top tier analysts are up-to-speed and to know what they’re saying about you. If you aren’t asking those tough questions – ideally in inquiry calls or paid engagements like Gartner SAS – then you’re never going to get to the root cause of your problem. 

The shift towards inquiries also places a greater emphasis on the imperative to access your Tier One analysts. If you don’t have paid access, you will struggle to find the right moment to ask the analysts what they think. 

Although the Magic Quadrant is by no means dead, the new battlefield for analyst relations professionals is to find out – and try to influence – what your top analysts are saying in these black box inquiry calls. And if you’d like to find out more, we’re here to help.