Just recently, industry visionary Esther Dyson published an open letter to Yahoo suggesting a new strategic direction. The article, entitled "Release 0.9: What Should Yahoo! Do?" offers two separate ideas, but it's the first one that caught my eye.

Dyson suggests that Yahoo can truly distinguish itself from Google by returning to its roots and becoming a directory again. In this context, Dyson means classifying websites against a taxonomy. She likens a Google search to using a searchlight in a darkened room. It highlights specific things, but you have no idea what you're not seeing. A classification system for websites removes this constraint by grouping similar websites together, through some human or automated effort.

This immediately brought back memories of a 1996 article in Wired magazine, that discussed Yahoo's categorization activities in fascinating detail. Although Yahoo co-founder Jerry Yang admitted that Yahoo was already falling behind in the job of categorizing the web (imagine a staff of 20 indexers trying to categorize every website), both Yang and the article's author seemed to agree the real challenge lay in creating an ontology - a classification scheme for all human knowledge. Ultimately, Yahoo abandoned its focus on site classification in favor of an automated keyword index. Another company (starts with "G") came along with an indexing approach that produced more relevant search results, and the rest is history.

There are two important insights here for data publishers:

First, while many data publishers view Google as an overwhelming and unstoppable competitor, its one-size-fits-all offering is both its great strength and weakness. You search on Google's terms and while it delivers a lot of value quickly, reliably and free, you never know what you are not seeing. There continues to be significant opportunity for those who can refine what Google does by improving on discoverability, by expediting the research process, by returning more relevant results in a more convenient format, and by coupling content with tools. And for most of us, it's best not to confront Google broadly; better to pick a specific niche and mine it deeper and better.

Second, while doing away with human indexers in favor of automated keyword indexing, Yahoo was able to vastly expand the number of websites it indexed, but ultimately, this high level of automation left it exposed to competitors with better technology. Had Yahoo continued to manually categorize websites (perhaps coupled with some level of automation and maybe a user-generated assist of some sort), I believe that today it would be an important alternative to Google, not an also-ran to Google. Further, investment in people to categorize websites, while expensive, would have yielded not just significant competitive differentiation, but also significant competitive defense.This game's not over; it's just beginning. And those who are smart about how to gather and organize information are on the winning team.

Model of Excellence Awards


We are pleased to announce that
Panjiva Inc.is a finalist for an InfoCommerce2009 Model of Excellence award.

Review Panjiva's Model of Excellence profile here Hear Panjiva CEO Josh Green at DataContent 09 DataContent 09: All Roads Lead to Data. Full program here.