What lies ahead for local search engine technology - Part 2

Read part 1 of this interview here.

[AB] Let’s look at commercial searches and informational searches; do you see the two becoming distinct categories?

[AF] No. A central theme behind classical information retrieval theories is that users are driven by an information need. More granular search log analyses over the past years have attempted to categorize queries as “transactional” (Commercial), “informational”, and “navigational”.

The immediate intent behind “navigational” queries is to reach a particular site; “informational” queries aim at acquiring information assumed to be present on web pages; while “transactional” queries usually result in some activity such as an online purchase. Andrei Broder, while chief scientist officer at AltaVista in the late 90’s demonstrated that queries at the time were roughly split equally among each category.

We don’t live in a binary world where queries (or content) are either inherently commercial or purely informational. The commercial-informational dichotomy looks more like a spectrum to me, where understanding user intent and the psychology of purchasing cycle is critical.

The definitions behind commercial and informational content are fuzzy and personal; content perceived as purely commercial by some might be informational to others and vice versa. Clearly, the query "1819 treaty manuscript" could be considered "informational" in nature, but leading to a book purchase at Amazon about the United States-Spain treaty of 1819, or even the schedule of a trip to Spain or Florida.

[AB] So what’s the answer?

[AF] In focus groups, users have told us unequivocally that they would much prefer a search engine display an array of content types that may be relevant to their query, rather than try to guess what their intent was. Users also appreciate having tools available to help them narrow their results. Based in part of this feedback, InfoSpace worked with Vivisimo last year to deploy a 'Refine Your Results' feature on our three owned and operated search properties -- Dogpile, WebCrawler sand Metacrawler.

The feature automatically organizes and groups results by category for every search, providing a comprehensive view of web search results and allowing users to more rapidly get to the information most relevant to them. For example, a search on "flowers" groups results into subcategories such as delivery, gardening, arts and crafts, and more.

[AB] If search engine users gave up a little of their privacy and allowed their search habits to be monitored, would this allow the search engines to provide better, customized results?

[AF] There is no doubt that sharing personal data with search engines would result in better individual search experiences. The quality of search results is a function of two sets of variables: i) the user query and ii) the content indexed.

Search engines are constantly crawling and indexing more web pages, more often, leveraging better entity extraction and concept recognition techniques, inferring document relationships in smarter ways. An enhanced understanding of user intents would certainly unlock more value from this semantic understanding of Web content.

Link analysis and other “off-the-page” ranking criteria have played an increasing role in relevancy algorithms over the past years. Monitoring navigation behavior at a user-level could conceivably be the basis to developing an understanding of users’ individual interests over time, in essence personalizing the equivalent of Google’s PageRank scores.

If you consistently browse music-related content, search engines should become smart enough to understand that your query “Prince” most probably relates to the singer than to the royal family. Personalizing search relevancy algorithms presents some major scalability and performance challenges, though. It takes days, if not weeks to process link analyses and compute authority scores for individual Web sites after a crawl.

[AB] Do you think search engine users will balk due to privacy fears?

[AF] Privacy concerns are certainly legitimate to some extent. I actually see some parallel between users’ reluctance to using their credit card online in the early e-commerce days and giving up personal information to search engines today. It’s a constant trade-off between privacy concerns and the added value extracted from that data.

In the meantime, IP-sniffing technology might take search engines a step closer to personalizing search results without requiring users to compromise on very personal information. IP-analytic software associate internet-connected devices to geographic areas, domains (.com, .edu, and .gov), ISPs, connection speed and browser types with some level of confidence.

Analyzing click popularity at an aggregate level along IP-associated parameters could be leveraged to extrapolate personalized ranking for clusters of users exhibiting similar behaviors. This technique would not be unlike Amazon’s implementation of collaborative filtering technology, in essence also reaching similar goals than social networks such as Eurekster.

[AB] InfoSpace also offers wireless data applications. Do you think that search has a future on a cell phone?

[AF] Sending local content such as yellow page listings, directions, maps and business ratings to mobile devices just makes sense. I remember looking up on my cellular phone the nearest ice-cream parlor from the park a couple years ago with my kid. It worked! The experience was far from optimal, though, scrolling through about 10 to 15 screens I could barely read.

Personalization features, geo-based services, faster networks, better handset resolution and color displays should significantly improve the experience over time. The navigation schema, whether search or browse modes, will be critical to make cellular phones a viable platform for both end-users and IYP advertisers.

About 90% of mobile phones will be Web-enabled by 2006, making it a more attractive platform for content providers, developers, and information architects to invest time on.

The opportunity to deliver Web search and online directory information to mobile devices is something InfoSpace is well positioned to capitalize on. InfoSpace was a wireless data pioneer in the US and our mobile division today powers wireless data applications for every major US provider with the exception of Nextel. Going forward, we see a significant opportunity to increasingly combine our mobile and search and directory assets to accelerate the adoption of thee services on wireless devices.

[AB] Thanks Arnaud for taking the time to share with us your thoughts on the future of search!

Clients include Real.com, Alaska Air, Peopleclick, Monica Lewinsky and NBC. You can reach Andy at andy@proranking.com and view his daily SEO blog at www.searchenginelowdown.com.

More search engine news...