Back when I was applying for college, what powered my generation's choices was the Guide to American Colleges, produced by Barron's Educational Services. Barron's stood out by bravely trying to rate colleges, putting them, as I recall, into five broad categories such as "highly selective" and "competitive." The categorization was largely driven by SAT admission scores, and no attempt was made to rank colleges within a particular category. By today's standards, it was pretty tame stuff, but at the time, it was revolutionary.

The Guide to American Colleges is still being published, but has nowhere near the influence for which I remember it. That honor now appears to belong to America's Best Colleges, published by U.S. News and World Report which is so dominant and so important, it not only changed the fortunes of U.S. News, it appears to have radically altered the higher education landscape as well. Wherein lies its power? Its straight numerical ranking of colleges, from the very best on down and down and down.

What qualified U.S. News to rank our institutions of higher education? Nothing really. It just decided it wanted to do it. It selected a number of measurements it decided were proxies for quality, crunched the numbers and published the results.

In the beginning, colleges played along. Their listings gave them free publicity, and the high ranking colleges wasted no time asserting their bragging rights. When reports started coming out that some colleges were actually altering their admissions processes purely to improve their U.S. News rankings, that spelled trouble. And now a growing number of colleges have concluded that the U.S. News rankings have gotten too powerful and announced a boycott.

Colleges are a wonderful example of the difficulties inherent in trying to rate services. To a large extent, only you can assess the value of a particular service you receive. Because a service is something you can't see or touch, its value is based on your own personal perceptions, and value is often wrapped up in how you choose to make use of the service you receive.

Perhaps not surprisingly, interest in rating something seems inversely related to the difficulty of rating it. Because it's so difficult to tell what you're getting for your education dollar is precisely why ratings of educational institutions have such enormous appeal and influence.

Implications for data publishers? Ratings systems are extraordinarily powerful things to offer. But when you rate, you create winners and losers. And the losers will often hate you more than the winners will love you. Too, while anyone can rate anything, the most powerful ratings systems are invariably fueled by strong brands, so when you rate something your brand equity is on the line. That means at the very least you want your rating methodology to be as bulletproof as possible. And you may also want to re-consider the Barron's approach to ratings that created value for the user while engendering less competitive tension. The lesson here for data publishers is that it is sometimes smart to strike a balance between utility and controversy.

Labels: America's Best Colleges, Barrons, Guide to American Colleges, rankings, ratings, U.S. News and World Report