The hidden Internet

July 2000

When you are searching the Web using a regular search engine, you are only searching webpages, i.e. pages that are coded according to the HTML-standard. However, a lot of websites let you access databases beyond the Web. Various scripts interpret your request, query the database, and give you the results on a special webpage. Regular search engines like Alta Vista and Google do not query these databases, and consequently information from these sources will not show up in their search results.

According to a recent study made by BrightPlanet Public information on the "deep" or "hidden" Web is currently 400 to 550 times larger than the commonly defined World Wide Web . The deep Web contains 7,500 terabytes of information or nearly 550 billion individual documents, compared to 19 terabytes of information (1 billion documents) in the "surface" Web. BrightPlanet has found that more than half of the deep Web content resides in topic specific databases. A full 95% of the deep Web is publicly accessible information ´ not subject to fees or subscriptions.

• BrightPlanet's white paper on the deep Web
  
  
• SearchIQ specialized search engines
  
• Searchpower
  
• Searchability!
  
• The InvsibleWeb - the search engine of search engines
  
• A collection of special search engines
  
• Search Engine Guide
  
• FinderSeeker - for national and regional search engines
  
• Fossick.com Web Search Alliance
  
• Complete Planet
  
  
• ResearchBuzz has more on this story

More search engine news...