The Unintentional Vehicle for Secret Formulas
posted Tuesday, 1 November 2005
We've all noticed that many of the the most successful Web sites, particularly commercial ones, tend to have highly proprietary algorithms powering their functional cores. Algorithms whose details, or even their broad outlines in some cases, are often protected in secretive ways very reminicent of the famous Coca-Cola soda formula.
Many well-known Web behemoths use private algorithms for strategic capabilities, like Google with their PageRank, or Amazon and their product recommendation engine. Newer, smaller players are increasingly making a name with them too, like Flickr with their new "interestingness" and particularly Tech.Memeorandum with its extremely successful recipe for reducing the most recent posts on 20 million blogs down to a handful of relevant ones. Like Peter Cashmore says in the title of his post this week, it's the humans vs. the algorithms: who should edit Web 2.0?
The premise of Web 2.0 is a largely populist one however. If you buy into it, all of this Internet stuff supposedly gets a lot better if we start plugging ourselves into two way relationships with the Web, contributing our most meaningful data, and (implicitly or explicitly) enriching and annotating the other information we find. As it turns out, a lot of this activity feeds and provides life and value to proprietary mechanisms that lie deep within many of these sites. This has led to numerous discussions lately on who owns the results of this work. But this is actually a separate issue from the propretiary mechanisms piece.
Certainly the very successful Web 2.0 poster children such as Wikipedia and BitTorrent use entirely open and well-documented methods. Wikipedia is fueled by MediaWiki, a freely available open source framework for managing user contributed content. BitTorrent, for its part, is a completely open source protocol with incarnations so numerous they might never be listed exhaustively. One big difference between these two and the others is that they are not attempting to be financially successful. This means they have nothing to lose and hence nothing to conceal. Disclaimer #1: I actually have no insight into whether Tech.Memeorandum is trying to make money in the long run, but it's likely. Disclaimer #2: I admit that concealing certain algorithims from network effect hacks like page rank gaming might actually be a public service, but that again obscures deeper potential problems.
The Web 2.0 practice set provides a solid toolkit of design patterns and business models that gives us several ways to monetize the techniques it describes. What's still unclear is the effects that certain ways of getting there will have, good or bad, on the Web community at large. Of course, free high quality search is a terrific boon to most of us. We truly love a good search engine. How many times did you Google something today for example? But becoming overdependent on anything you can't ultimately control is always a danger.
The Long Tail is the most famous example of Web 2.0-style monetization; the mass servicing of micromarkets has led to eBay and Amazon becoming worth billions. But it's the other big technique that is probably the one that's the shortest route to the biggest success. This is the development of secret algorithms that provide services so good that they are a powerful and ultimately irresistable draw to users in vast numbers. Then, flush with almost monopolistic power, one can make substantial financial withdrawals from the international bank of crowd wisdom.
This seems to be the business model that will be the most successful in the large, certainly Google has proven it through its incredible success story. And it all lies in having a big secret that you don't share. It really is of some concern. I am however, an optimist that believes it will work out in the end, though I do believe that Secret Formulas are probably a foster step-child Web 2.0 meme that no one likes to talk about much.
So, as with all technology, we do wonder what the risks are of utterly trusting and becoming dependent on concealed devices in our daily lives, particularly ones which are fundamentally outside of our control. There are unresolved issues here that need exploration and the answers that we get over the next few years will determine the direction that many Web 2.0 practitioners will eventually take.
What do you think? Will the Web become dominated by a few large players holding portfolios of secret formulas?
Technorati: web2.0
links: del.icio.us
Dion,
