The Pandia report on the Lycos InSite paid inclusion program continued.

InSite Site Search Customization

Unlike other paid inclusion programs, Lycos offer you a site search engine as part of the more expensive programs. This is a nice touch, as Web usability studies have shown that having a site search facility does help visitors find what they are looking for.

The question is, of course, whether the Lycos solution gives any benefits compared to other services available.

It is clearly much easier to implement these services than installing a site search script on your own Web server. Beta tests done by Fast and Lycos indicates that the majority of customers go for a rather modest level of customization. They select the ready made search box and stays with the default layout, instead of designing their own result pages using XML feeds.

This is why Lycos present rather few options as regards result page optimization. You may include your own logo and your own color scheme, but that's about it. Hence it will clearly show that these pages are not part of your site as such.

We suppose a lot of Web masters will find the lack of choice a relief. Others, however, will rather gravitate towards online services like Atomz, which will host up till 500 pages for free, and which gives you a nearly total control of the page design.

However, Baker made it perfectly clear that Lycos will offer more extensive customization in the future. "The key is to selectively expose users to new features," Wilde said. "First we will provide simple to use robust features. Later we will add more advanced features."

For site searches, FAST PartnerSite can return site search results in XML and no requirements or restrictions are placed on webmasters on formatting.

The meta description tag

We noted with interest that the listings on a demo result page did not make use of the meta description tag. The meta description tag is a hidden HTML tag that webmasters may use to tell the search engine how to describe a page in the search result listings.

Baker answered that Fast no longer will trust this tag automatically. The search engine will compare the meta tag description with the actual visible text content on a page, and control that it actually is a sentence.

If there is no connection between the tag and the content, Fast will go to the Open Directory to see if the site is listed there. If it is the search engine will present the Open Directory description. If it is not listed in the ODP, it will use the first 250 characters of the first text paragraphs.

And what if you leave the paid inclusion program?

Small site owners as well as search engine optimizers often worry what will happen if you leave programs like this. Will the search engine drop the listing entirely? Given that Fast tries to cover "all the web, all the time" we would guess that the URLs would stay in the database, although all of the other benefits would disappear (e.g. the 48 hours refresh rate).

This is actually not the case. Fast keeps track of whether the webpage was included in the Fast database prior to becoming part of the InSite or PartnerSite programs. If it was, they will move the URL from the special partner site database into the regular database -- i.e. it will be crawled by the regular Fast spider and continue to be included in search results.

If it was not present in the Fast database before being added to the InSite/PartnerSite programs, Fast will delete the entry. However, the regular Fast spider may find it later and include it in the same way as any other webpage.

Dynamic pages

In order to handle the information overflow of administering large Web sites, webmasters often make use of database driven scripts (CGI, Perl, ASP etc.) when publishing content. These scripts normally produce URLs (Web addresses) including signs like & and ?.

Search engine spiders normally avoid such pages, as they may be trapped in a loop where the databases automatically produces an endless number of pages. This is, of course, a problem for webmasters who would like to see such pages included in search engine results.

Baker said that Fast is now testing a new spider that can handle dynamic pages. However, it has not been fully deployed yet. Still, in the InSite/PartnerSite programs you may add each dynamic URL manually, and the pages will be included.

Fast and the Lycos network

Lycos presents InSite as "a comprehensive suite of search marketing services designed to help you promote your site to the 40 Million loyal users of the Lycos Network". The problem is that not all Lycos sites are powered by Lycos. A few, like Symptico.ca and some Asian versions get search results from Fast's most important competitor, Google.

Wilde pointed to the fact that local versions of Lycos were joint ventures, and that some of the local sites might -- for particular reasons -- chose another search engine provider. However, Lycos US is the "distributor of best practice", meaning that most local versions will follow the US version in providing Fast results.

We asked Baker what we could expect from Fast in the nearest future in the field of technological development.

He confirmed that Fast aims at presenting a 2 billion objects database by the end of March 2002. Fast will also continue to implement new linguistic technologies in order to improve search results, especially when applied to specific countries and languages.

We may also see the implementation of "visual relevancy", meaning that search results will be made more relevant through the highlighting of the query term in result lists.

Pay for placement

Like most of the major search portals, Lycos presents results from the Overture pay-per-click search engine. Sites like Lycos have been heavily criticized for not making it perfectly clear that these listings are in fact text ads, and not regular listings.

Wilde said that Lycos would now go on to mark these listings as paid result, naming them "Sponsored listings". And so they have.

Baker added that Fast had no intention of starting its own pay-per-click service.

Lycos InSite
Fast PartnerSite
LookSmart launches new paid inclusion program (Pandia June 2001)
AltaVista offers paid Express Inclusion Plan (Pandia June 2001)
Fast offers pay for inclusion program (Pandia June 2001)

The Pandia Post is our free bimonthly newsletter on Internet searching, search engines and directories.

We will give you the latest news from the world of Internet exploring, useful tips on how to improve your searching skills or search engine ranking, as well as information on the development of the Pandia Search Central.

We will never give your address to any other company or organization.