WebCrawler was originally a separate search engine with its own database.
Recently it has been repositioned as a metasearch engine, providing a composite of search returns from Yahoo!
en.wikipedia.org /wiki/WebCrawler (100 words)
WebCrawler Gets "Deportaled"(Site not responding. Last check: 2007-09-19)
WebCrawler has a new look and feel, returning to its roots and living up to its longtime motto "It's that Simple." The revamped search engine has ditched nearly all of its "portal" features and is now focusing exclusively on providing high-quality web search results.
WebCrawler is owned by Excite, though for years it used its own, much smaller index of web pages to provide search results.
WebCrawler's "de-portalization" is a welcome change to the venerable search engine, as is Excite's apparent commitment to paying attention to WebCrawler's users and tuning the search engine to better fit their needs.
Webcrawler Tips(Site not responding. Last check: 2007-09-19)
If you give WebCrawler several terms to search for, it will "OR" them together and provide a list of results which contain all or just some of the terms you provided.
With WebCrawler, relevance is determined by how many times a word appears in the document.
WebCrawler is aware of the problem, however, and is eliminating those pages from its indexes which engage in this practice.
WebCrawler News(Site not responding. Last check: 2007-09-19)
Although they mainly index index pages of a site, WebCrawler will also crawl deeper into some sites, particularly those with high "link popularity" which means other sites which have links to yours.
WebCrawler indexes each word on each page up to 1 megabyte of text.
WebCrawler suggests that you use a title which truly describes your page.
www.laisha.com /zine/webcrawler.html (210 words)
WebCrawler(Site not responding. Last check: 2007-09-19)
WebCrawler avoids CGI directories, so HTML documents that are generated on the fly present difficulties for indexing.
It is not clear whether WebCrawler indexes META tags or not.
(WebCrawler displays 60 characters maximum on the result page from the title tag) This is omportant because WebCrawler puts more weight to the title than the body text.
Webcrawler was originally owned by America Online (AOL) in 1995 and had become the preferred search engine for AOL users until it was sold to Excite the following year.
Webcrawler gained popularity with AOL users and has remained quite popular to this day.
Webcrawler will spider or follow all links it finds on your website and index them into their database.
Web Matrix: WebCrawler(Site not responding. Last check: 2007-09-19)
The WebCrawler was recently acquired by America Online, who promise to support it as an Internet service without censoring its content.
WebCrawler is an exclusively searchable database of Web documents, built on a custom software engine written by the author using C. Features and Limitations:
WebCrawler provides both Forms and Non-forms interfaces to the search engine, however Forms support is required for most of the search features.
WebCrawler's History(Site not responding. Last check: 2007-09-19)
WebCrawler spat out its first Top 25 list on March 15, 1994.
WebCrawler was fully supported by advertising on October 3, 1995 but maintained a strict separation between the advertising and the search results.
WebCrawler was initially supported by its own dedicated team within Excite, but that was eventually abandoned in favor of running both WebCrawler and Excite on the same back end.
Precision ANOVA of infoseek, lycos, and webcrawler Index Services(Site not responding. Last check: 2007-09-19)
Although I did not get an outstanding grade on the project (see Appendix C for the gist of the professor's comments), I feel that there are real results and that they are important enough to post on the Internet.
Webcrawler also did poorly, having the lowest alpha of all four in the top ten precision (see chart #4).
It is quite clear from a casual inspection that Webcrawler and Infoseek have the best response times, and the least variable ones.
www.winona.edu /library/webind.htm (4091 words)
WebCrawler News(Site not responding. Last check: 2007-09-19)
WebCrawler is the smallest of the major search engines.
If you neglect to create a meta description tag, WebCrawler will determine one for you, which is not always good or accurate.
For the pages that you submit to WebCrawler, they are not indexed by Excite, but instead other pages that you submit to Excite itself are there on WebCrawler; this means that you'd better to always submit your web pages to Excite, first, and then to WebCrawler.
Citations: Finding What People Want: Experiences with the WebCrawler - Pinkerton (ResearchIndex)(Site not responding. Last check: 2007-09-19)
Similarity metrics are derived from the vector space model [31] which represents each document or query by a vector with one dimension for each term and a weight along that dimension that estimates the term s contribution to the meaning of the document.
However, as it will be argued throughout this work, such an evaluation could prove to be inaccurate since there are cases where users may be dissatisfied with the output of such modules even if they contain references to resources of high relevancy to the corresponding query.
This algorithm was explored as early as 1994 in the WebCrawler
The way the WebCrawler sample handles this is in how it uses the StatusCode member of the LinkInfo class.
It is important to note that the WebCrawler sample keeps a master Hashtable (not shown in the above exampled) that holds all of the Urls that it discovers (regardless of whether or not it was able to successfully visit them) as well as a temporary Hashtable used while collecting links from a Url.
As we have seen, the WebCrawler uses it's master Hashtable to keep track of the Urls it has found and uses the data stored in the Urls associated LinkInfo object to determine if it has previously visited the link.
WebCrawler Add URLs(Site not responding. Last check: 2007-09-19)
If WebCrawler doesn't find a Web page that you think belongs in our index, you can submit the URL for that page and up to 9 others using the form below.
However, the URLs in our database that it visits may not be included in the WebCrawler index for up to two weeks.
You can use our URL Status form to see if a URL you submitted or know about is in our database and, if so, find out when it was last visited.
Search services: WebCrawler(Site not responding. Last check: 2007-09-19)
History: The WebCrawler Project began as Brian Pinkerton“s research project at the Department of Computer Science and Engineering at the University of Washington in Seattle, announced April 20, 1994.
Good transition from the browsing structure (Webrawler select) to the index but not from the index to the browsing structure.
It is not possible to search only the Webcrawler select directory.
It's got a funny history: as far as we know, AOL took it over and they tried to protect the word "crawler" as a trademark for a technology that is similar to another technology called "spider" which many search engines use.
Then, Excite got WebCrawler, and now @Home owns them.
Webcrawler was defunct for quite some time but now it looks like it's back up - what nostalgia!