|
Top/Computers/Internet/Searching/Search Engines/Robots |
|
Robots : |
| Web robots (also known as crawlers or spiders) are programs that traverse the Web automatically, and which are used by search engines to index the Web, or part of it. |
|
9 web sites matced for Robots |
|
Search Engine Robots and Other User Agents
|
John A. Fotheringham presents data in tabular form on the robots sent by search engines and other sites to read and index Web pages: their origins, names and IP addresses.
|
|
List of Robot Agent Strings
|
A list from PGTS of Web robots with the identifying data they leave in Web site logs.
|
|
All About Search Indexing Robots and Spiders
|
Search Tools Consulting explains how the search engine programs called "robots" or "spiders" work, and reviews related sites.
|
|
Robots, Spiders and Other User Agents: a Resource for WebMasters
|
José Luis Pellicer's searchable database of robots, spiders and other user agents for programs that surf the web.
|
|
Search Engine IP Addresses
|
Lists IP addresses of search engine spiders. Can be searched by IP address. Also links to resources on spiders.
|
|
Psychedelix List of User-Agents
|
Andreas Staeding's large list of search engine spiders, similar Web robots, and Web browsers: their web-log identification and links to their originators.
|
|
HTTP User Agent Index
|
An alphabetical list of user agents and the deployer behind them, compiled by Christoph Rüegg.
|
|
Spider Track
|
Tracks and displays real time hits and IP addresses of MSNbot, Yahoo's Slurp, and Googlebot. Provides aggregate hits of each spider over the last 30 days.
|
|
Robot IP Address
|
Brian Dunnintg provides a list of all the major search engine robot IP addresses, by full class C only.
|
|
| |
| Web Site Directory - powered by Dmoz |
Arts
Movies, Television, Music... |
Business
Jobs, Real Estate, Investing... |
Computers
Internet, Software, Hardware... |
Games
Video Games, RPGs, Gambling... |
Health
Fitness, Medicine, Alternative... |
Home
Family, Consumers, Cooking... |
News
Media, Newspapers, Weather... |
Regional
US, Canada, UK, Europe... |
Science
Biology, Psychology, Physics... |
Shopping
Auctions, Clothing, Gifts... |
Society
People, Religion, Issues... |
Sports
Baseball, Soccer, Basketball... |
| |
|
|
|
|
Sponcer Links
|
Live Chat
mioot Live chat system help you to track your web site visitor in real time and enable live chat features with your web site visitor.
|
|
miOOt Technologies is an Indian based software company offering complete onsite offshore software development solutions with high end technologies and standard. |
| |
|
|
|
