Who Made You King?

Kiplinger's Personal Finance Magazine, April, 2001 by Kimberly Lankford

ONLINE | Peek behind the curtain at GOMEZ, the self-appointed arbiter of what's good and what's bad on the Web.

YOU'VE SEEN the announcement splashed across various Web sites: Ranked #1 by Gomez. The victors spew out press releases proclaiming their superiority. The losers lick their wounds, often tweaking their sites in hopes of rising in the rankings. Web-company CEOs make the pilgrimage to Boston to meet with the Gomez staff and search for insight into the process that might give them the winning edge. Less well-known is that many firms that fall into Gomez's sights pay big bucks for insider advice on how to improve their scores. As you will see, how Gomez makes money raises some eyebrows, but many also-rans are afraid to criticize the company on the record.

Given the sweeping influence that Gomez rankings have, a couple of questions leap to mind: Who are these guys? And how do Julio Gomez and his staff determine which banking or brokerage or insurance or whatever Web site is best for you?

When Gomez launched his firm from his suburban Boston home three and a half years ago, the number of Web retailers had just begun to explode. Because many of the firms had names nobody had ever heard of, Gomez foresaw a market for consumer road maps. Although his nascent firm was also an unknown, the supremely confident Gomez dubbed it "the e-commerce authority."

The company's claim to fame is its Internet Scorecards. Available at Gomez.com, they now rank e-tailers in 32 industries--including brokers, insurance marketplaces, mortgage lenders, drugstores, toy and pet-supply dealers, and car-buying sites. While the results of Gomez's studies get lots of press, the rating criteria are rarely explained. The more you understand about those criteria, the better you can judge the value of the rankings--and their limitations.

In the beginning

PRINCETON grad Julio Gomez dreamed up the company when he was an analyst at Forrester Research, an e-commerce research firm. He had already founded a securities firm in his early thirties, sold it to join Forrester, and was well-known among financial-services companies for analyzing the impact of the Internet on their businesses.

Part of his job at Forrester was to study long-term trends for online brokerages. As he focused on where the industry might be in five years, Gomez found himself bombarded with "questions about what was going on right now." He was stunned to discover that there weren't any good answers.

Sensing a void, Gomez left Forrester to fill it. He joined a former colleague, John Robb, and a former client, Alexander Stein--each with complementary backgrounds in finance and technology--to found Gomez Advisors in May 1997. The trio, all in their mid thirties, commuted together from the suburbs to a small rent-a-space in downtown Boston. The company's first project was a scorecard rating online-brokerage firms, which Julio Gomez unveiled with great fanfare in June 1997, as the keynote speaker at the Securities Industry Association conference in New York City.

The scorecard caused quite a stir because it was the first time anyone had lined up all the brokerage Web sites and assessed the quality of the online experience they offered consumers.

"We said, we don't care who you are," says Chris Musto, one of Gomez's first hires and now the company's exuberant head of research. "If you qualify, we will rate you against the same objective criteria based on what consumers want and need, not based on what seems like a good marketing gimmick to you, or your market share, your ads or your profitability."

Only 28 firms qualified for that first list--the only requirement being that they let customers complete a trade online; many big-name brokerages were excluded because they did not allow online trading. Established discount brokers were ranked alongside unknown upstarts.

PC Financial Network (which later became DLJDirect and recently morphed into CSFBDirect) earned the highest score on that first scorecard, followed by Schwab and E*Trade.

As e-commerce took off, so did Gomez's business. The company added a scorecard for online banking in June 1998, then began delivering verdicts on travel and book-selling sites a few months later. Online-auction and auto sites came aboard in the spring of 1999. Thirteen more industries were added in August and September. Meanwhile, the staff exploded to more than 150 employees.

But what do they do?

THE METHODOLOGY Gomez created for the first scorecard became the model for everything that followed. He and his partners approach the process scientifically, trying to come up with objective measures for judging how well a Web site serves its visitors.

The Gomez score is based entirely on analysts' experiences testing the site, not customers'. An insurance company may have a great site, but you won't know from the score alone if it hassles people on auto claims.

Gomez, Robb and Stein drew upon their own experiences to develop 30 to 50 yes-or-no questions that could measure success in each of the five categories: ease of use, customer confidence, on-site resources, relationship resources (which measure features that build an ongoing relationship with a customer) and overall cost.