On December 23rd, when the search engine scene was quiet and most people were getting ready for Christmas, Wikipedia co-founder Jimmy Wales revealed plans of a “people-powered” search engine in an interview with The Times.

The hype

Wales demonstrates again how talented he is at using the media. The interview was packed with very quotable statements and has been quoted again and again in the otherwise quiet days of Christmas. Here are a couple of them:

“Google is very good at many types of search, but in many instances it produces nothing but spam and useless crap. Try searching for the term ‘Tampa hotels’, for example, and you will not get any useful results.”

“Essentially, if you consider one of the basic tasks of a search engine, it is to make a decision: ‘this page is good, this page sucks’ […]Computers are notoriously bad at making such judgments, so algorithmic search has to go about it in a roundabout way. But we have a really great method for doing that ourselves. We just look at the page. It usually only takes a second to figure out if the page is good, so the key here is building a community of trust that can do that.”

The wiki inspired search engine project is run by Wikia, a for-profit company of which Wales is chairman. On the official project web site Search Wikia, you find some high grade hype, calling it “a project to create the search engine that changes everything” and stating that :

“Search is part of the fundamental infrastructure of the Internet. And, it is currently broken. Why is it broken? It is broken for the same reason that proprietary software is always broken: lack of freedom, lack of community, lack of accountability, lack of transparency.”

For as long as web search has existed, there has been a race to improve it. Maybe community and transparency can contribute to making a better search engine, but Wales has said little about how.

Some facts

Behind this hype, it is possible to find some facts:

• According to The Times, the project is called Wikiasari. Search Wikia says this is the name of an old project and will not be the name of the new search engine.
• The Times said the plan is to launch the new wiki-inspired search engine in the first quarter of 2007. In a Q&A session with Danny Sullivan over at Search Engine Land, Wales says we will probably have to wait a couple of years for the finished product. [Thanks for picking up the phone and talking to Wales, Danny! Your Q&A clarifies a lot.]
• Amazon, which has contributed money for Wikia, is not involved in the project.
• Jimmy Wales and Wikia are looking for “community members who would like to help build people-powered search results and developers to help us build an open-source alternative for web search”.
• Although the community referred to is most likely the Wikipedia community, the new search engine is a Wikia project that is not officially connected with Wikipedia.

The rest is unclear

It is not clear how Wales is going to harvest the power of human judgment. This is clearly more than a directory edited by humans, like the Open Directory Project (ODP). Wales needs a team that is able to cover more sites than the ODP.

Some of the quotations seem to indicate that we are talking about a ranking system which gives each page (or site?) included a quality score by one (or several?) of the volunteers. This quality score then replaces the Pagerank given to pages by Google.

A forum discussion at Search Wikia seems to indicate that the “Search Wiki” could make use of automatic search engine spiders.

Keep in mind, however, that Google’s Pagerank is not generated by a computer in isolation. The whole point of Pagerank is to make use of the collective human wisdom of the Web, giving pages points based on the number and quality of incoming links. Google’s ranking system is, like the one of the Search Wiki, ultimately based on human input as well.

What’s to come?

So what can we expect from the new search engine? One thing is certain: After starting out with such a massive hype, the risk of failure is great.

I do not agree with the statement that web search is broken. Sure, there are some problems, and spam is one of the biggest. But is wiki-inspired search going to fix that? The wiki model is exciting and Wikipedia is changing the Web. But Wikipedia has problems of its own and wiki vandalism is in many ways a parallel to search engine spamming. Wales and his team has not yet found a way to fix this and it is bound to be an important issue for the new search engine.

Another issue is reaching critical mass — the number of registered users who rates web sites. I am sure Wales has done the math and found out how many volunteers he needs to get an index that is large enough for a respectable search engine and that he is confident that the volunteers will arrive. Google’s advantage is that its computers can “review” millions of pages every day.

With all the hype this Christmas, scores of volunteers are probably already lining up to be part of “the search engine that changes everything”. But it might be harder to motivate people to use their spare time to do the basically invisible work of rating search results, compared to the fun of publishing articles in Wikipedia or the importance of being an ODP editor. Wales needs to find a way to do this.

I wish Wales and his team good luck. Even though web search is not in need of a knight in shining armour, like he wants us to believe, there is always need for search engine innovation.

The Times interview that started the hype
Search Wikia home, with some more hype and some interesting information
The Search Engine Land Q&A, which clarifies a lot