Search Engine Spiders
Results 1 to 6 of 6

Thread: Search Engine Spiders

  1. #1
    Senior Member
    Join Date
    Jan 2004
    Posts
    199

    Search Engine Spiders

    Can anyone explain (or have any links to sites explaining) how search engine spiders (bots, indexers, whatever you want to call them) work ? How to they find the sites? Where do they start looking ? etc etc
    -

  2. #2
    Senior Member
    Join Date
    Oct 2002
    Posts
    4,055
    Well, they kinda work similiar to search engine's. They use context and word's in the meta tag's and word's that are on the site. For instance, AntiOnline has word's dealing with security on the front page and in it's source code. Spider's will go to AntiOnline then find a site with those meta tag's and header's in it's source and on the website itself.
    Space For Rent.. =]

  3. #3
    Senior Member
    Join Date
    Jan 2004
    Posts
    199
    any idea how the search bot knows that antionline.com was here in the first place ?
    -

  4. #4

  5. #5
    Senior Member
    Join Date
    Jan 2004
    Posts
    199
    Thanks alot dude, that was extactly what i was looking for.
    -

  6. #6
    Senior Member
    Join Date
    Oct 2002
    Posts
    4,055
    Damn you Negative Smart bastard..
    Space For Rent.. =]

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •