What will the search engine landscape look like in 2015?

Pandia is currently celebrating its 8 anniversary; our site is a very old one in Internet terms. However, we are not going to mark this occasion by writing long articles about what has happened since December 1998. Instead we are going to tell you what is going to happen during the next eight years.

Due to a freak accident, Pandia’s newsletter mailing service opened a digital wormhole in cyberspace in January 2015, sending us a very interesting issue of the Pandia Post. We are going to reprint several of the newsletter articles included during the next two weeks. An here’s the first one:

Search engine trends from 2007 to 2015

By Per and Susanne Koch, January 2015

Pandia would like to wish all its readers, listeners and viewers a happy new year!

2014 was definitely an exciting year for searching, and 2015 seems to become even more important. In this issue of the Pandia Post we will discuss the major developments in the online search sphere for the last eight years, and take a look at possible future developments. Let us get right down to the basics:

Google still going strong, Yahoo and Microsoft tie the knot

We have seen an impressive development of search technologies during the last eight years, and we will come back to the major innovations later in this newsletter. Still, it should be noted that the old fashioned text search we learned to love in 1990’s is still around. Google will still feed you blue links on a white web pages, HTML style, and people continue to click on text ads.

A searcher visiting from the year of 2007 will still find most of its his or her favorites. Google, Yahoo! and Ask are still there, and the only major newcomers on the European and American scenes are Chinese Baidu and Scandinavian Balder.com.

However, MSN Search aka Windows Live Search  is no more. It didn’t have to go that way. When Microsoft bought Yahoo! in 2008, they did actually discuss making Yahoo! a subsite of Windows Live and abandon the Yahoo! search engine technology.

Fortunately, experts convinced them otherwise. The experts pointed out that this would be the same mistake as Yahoo! had done when taking over AlltheWeb and AltaVista in 2003. At that time Yahoo! abandoned the best technology (AlltheWeb) and kept the weakest one (Inktomi).   Microsoft decided to keep  best search engine (Yahoo!) and discontinue Windows Live Search. The Microsoft team of search engineers became part of the Yahoo! Search team in 2009.

The fact that Yahoo! was kept as a separate identity turned out to be a great success. The new engineers and researchers from Microsoft found a new home where there was more room for creativity than in the Microsoft mastodon, and together the two teams managed to face some of the major problems search engines where facing those days: the size of the Web and the horrible amount of search engine spam.

Personalized search

Like Google, Yahoo! started focusing on personalized in earnest around 2007. Any user willing to sign up for an account  soon found that the search engine started to learn from his or her search behavior, identifying themes, topics and web site neighborhoods of particular interest. Relevant pages were given a boost in his or her search engine results.

At the same time the search engines started to weed out sites and pages he or she did not find interesting (for instance by measuring how much time the person spent on that page).The information gathered from registered users was also used to influence the results for non-registered users. Needless to say, scraper sites and pages of low quality found it hard to compete under such circumstances.

This development led to a renewed interest in so-called white hat, organic search engine marketing. It made no sense to aim for at “top 10 ranking” anyway, for the site that ranked as No. 1 for Betty, would rank as No. 42 in Kim’s search results. Instead the goal became to generate traffic and sales — which, of course, should have been the main objective all along.

Social search

The search engines also found other ways of identifying high quality pages that went beyond the “PageRank” methodology developed by Google in the 1990’s (i.e. where Google calculated the quality and relevance of a page based on the number and quality of inbound links).

Around 2006-2007 Yahoo! started in earnest to make use of information gathered from its two bookmarking services, MyWeb and del.icio.us (they were merged into Yahoo! del.icio.us in 2009). Sites that were favored by users of these services began to see a slight improvement in rankings.

Webmasters realized this, of course, and the black hat camp started spamming these services. However, by 2009 Yahoo! was getting so good at identifying “unnatural bookmarking,” that it managed to reduce the problem.

Google bought Magnolia and Simpy in 2007 (both were immediately merged with Google Bookmarks), and Ask acquired Bluedot in 2008. Now all the big three had efficient bookmarking services that could deliver input to search algorithms.

This did not mean that they abandoned the old ways of deciding page rank. On-page factors like keyword density and use of tags remained important, as did the link structure of the web. However, the search algorithms had now become so complicated that it became impossible for search engine marketers to “reverse engineer” them with any kind of accuracy.

Yahoo! managed to get through its 2006-2007 crisis, mostly due to its expertise in the social arena. Yahoo! had for a long time been better than Google in community building and Google’s acquisition of YouTube in 2006 did not change that. Yahoo! (now a part of Microsoft) could use this expertise in developing and buying even more social services, making its own visitors create even more searchable content.

Moreover, Google’s acquisition of YouTube did not become the success it had hoped for. YouTube experienced a tremendous growth in 2006 and 2007, mainly thanks to teenagers viewing “cool” video clips. Unfortunately, the attention span of a modern teenagers is shorter than an MTV video (now on average lasting 1.50 minutes), and they soon moved over to new and more fashionable places to hang out (triple X in 2009, monsterSUSHI in 2011 and Pluto the Dog in 2012).

Now, of course, teenagers have abandoned the Web all together, having moved over to virtual realities like 3DLife and Otherland. YouTube has evolved into a site for news-clips, 2010’s comedies and live family snap shots.

Office online

Google had a tremendous success with its online office strategy, however. Google was not the first in this arena. As early as in the 1990’s Microsoft and Yahoo! had presented online mail services (Hotmail and Yahoo! Mail) whereby users could read their mail from any computer in the world.

Google followed up on this paradigm with a large number of online services: mail (GMail), word processor (Google docs), spreadsheet (ditto), web site designer (Web Creator), web analytics, personal discussion forums (Orkut and Groups) and more. Most of these were available by 2006, but they were improved significantly in the years to come.

Moreover, as broadband became “broader” and more ubiquitous services were added. In 2007 Google added Gdrive (free online storage), in 2008 Google Presentations (a Powerpoint killer letting you use your browser for presentations), Google Podcaster in 2009 (for recording and distributing sound files) and Google Image and Video in 2010 (a photo and video editor).

All these services were funded by advertising. For a fee users could also upgrade to better and more advanced services and get rid of the ads.

(This development was not without controversy. On the 2010 New Dehli Webmaster World conference a Google representative let slip that they were using the content of personal mails and files to influence the search results of Google subscribers. This was another factor in the ongoing struggle to provide more accurate and spam-free SERPs. Some bloggers presented this as a violation of people’s privacy: “Google is reading your mail!” It took some time for Google to explain that no, they were not reading anyones mail, but yes, they did use automatic software to spider the content of mailboxes and file directories in order to map the interest of the user.)

Given that all of these Google office services could be used through any browser in any operating system, Microsoft realised it was in big trouble. Not only could you now perform nearly any regular office task without the Windows operating system. You did not have to buy Microsoft Office either.

By 2009 several computer manufacturers had started selling cheap office LINUX PCs that hid the complexity of LINUX for the user, booting directly up in a free version of the Firefox or Opera browsers, with all the relevant Google services presented in separate tabs. By 2011 you didn’t even need a PC. Who needed Vista?

Microsoft tried to meet this challenge with Windows Live, but they botched it, due to a confusing use of  brands, and the fact that they had given Google such a head start. By 2011 45 percent of American small- and medium sized companies had switched to Google Tools (as the package was called now), some 8 percent were using Yahoo! Toolbox and 5 percent other competing online office services.

Go to part 2: Search 2015: When media equals the Internet