The Answer Search Natural Language Search Engine

The Norwegian company Stochasto is getting ready to launch their natural language search engine, Answer Search, in English.

When you use a traditional search engine, the best you can hope for is normally an answer to the question "where can I find information about this or that?". You select the web search results that seems to be the most relevant and scan through the web page to find the information you are looking for.

The real question that lies behind your quest is very often much more specific: "When was Albert Einstein born?", "What is the capital of Malaysia?", "What is the meaning of life?".

Wouldn't it be great if the search engine could give you the answer right away, so that you would not have to read through large amounts of texts? Moreover, wouldn't it be great if you could ask this question using your natural language, and not complicated search terms that are hard to remember and hard to learn?

It has been done before -- sort of. Ask Jeeves used to claim that their search engine could understand natural language, and it certainly looked that way. However, their technology was based on an index compiled by human beings that matched the most popular questions with web sites.

Pandia has previously reported on other companies that have tried to design software that automatically interprets a questions, analyze a large number of web pages and extracts the relevant answer from these questions (see articles on Kozoru and BrainBoost .

However, these attempts are still in their preliminary phases. Now a Norwegian company claims that it has solved the problem, and that it will be able to deliver a natural English language answering machine early next year.

- This is not keyword search with a layer of natural language on top, says Stochasto CEO Jan Husby. - This is real natural language search based on an extensive database of vocabulary, conjugations, phrases, grammatical rules and semantic contexts.

Searching with this new technology is an interesting experience: Instead of getting a list of websites that might offer an answer to your query, you get to ask your question in natural language and are presented with an answer in plain English along with a link to the web site from which the answer originated.

Pandia has seen a preliminary version of the site. The database underlying the test site is still too small to give an accurate impression of how the final product will be, but it does indeed manage to give very accurate replies to questions like the ones listed above.

- This technology can make an attractive alternative to keyword based web search, says Husby and continues: - It also has the potential to simplify tasks for professionals in knowledge intensive areas, for instance lawyers needing to search through large volumes of laws and regulations.

Two versions of Answer Search will be available. One is a metasearch engine where Stochastic's technology is utilized on results from Google. This will be extended to include more search engines before the launch.

If you ask "When was Hamlet written?", you get answers reflecting the latest research, stating that Shakespeare wrote Hamlet around 1600, but that it is now beleived to be based on a lost play by an unknown author in the generation before Shakespeare. When we asked "Who is Susanne Koch?", the answer was: "Susanne and Per Koch are the owners and editors of Pandia.com." We like it!

The meta search engine will eventually be available not only as asearch site, but also as an application that you can install on your own computer, enabling you to searcht he web in natural language from your desktop.

Husby explains that Stochasto's web search would give even better results if it was used, not on meta search results, but on an index tailor made for the technology. Stochasto primarily supplies the techology, though, and are looking for a partner in this.

The main product, though, is based on indexes generated by Stochasto's technology. It will be available as desktop search, enterprise search and publisher search (for multimedia publications, including CD and DVD disks).

Answer Search is already available in Russian, marketed by Stochasto's partner Stocona. At the technology exhibition Softool 2004 in Moscow September 28 to October 2, Stochasto's Answer Search was awarded the prize for best search engine among 11 nominees.

Now a team of linguists are working on the the English language search technology. About 80% of the work is done and the product will be available in the first quarter of 2005.

The press is constantly looking out for the next Google. We guess that Stochasto will not try to launch a new global web search engine, but instead focus on the enterprise search market, selling solutions that help companies find their way through their own data.

The Russian version of the product may also be used for desktop search, however, i.e. for retrieving information from local hard disks and mail boxes.

Several companies, including Google, have recently launched such products, believing -- correctly in our opinion -- that power users need such tools. Some of these companies might find this technology to be an interesting supplement.

Moreover, it is a fair guess that companies like Microsoft, Google and Yahoo! are working on -- or at least considering -- the implementation of natural language/direct answer search on their own search sites. One way of getting this technology fast, is by licencing it from others.