This topic contains 5 replies, has 0 voices, and was last updated by  nila 6 years, 1 month ago.

  • Author
    Posts
  • #7521

    PeanutsRevenge
    Participant

    Why were the crawlers banned (google, msn, yahoo), place doesn’t get much attention as is, without blocking the main source of free avertising!

    “06:24:00AM: ArmorWraith says:
    Band the google msn and yahoo crawlers, dunno if itll stay that way though”

    #63704

    Laptops Daddy
    Participant

    shouldnt make any difference. thrax said something about giving them accounts for some reason or other, but they should still be able to index the forums without.

    normally youd just add Disallow rules to robots.txt

    #63705

    ArmorWraith
    Participant

    The matter has been resolved.. I was unaware they were beneficial.. I seen the word bot and had deleted them thinking they would clutter the forums..

    My bad. #-o

    Armor

    #63706

    PeanutsRevenge
    Participant

    @laptops Daddy wrote:

    shouldnt make any difference. thrax said something about giving them accounts for some reason or other, but they should still be able to index the forums without.

    normally youd just add Disallow rules to robots.txt

    Good Point!

    Membership is only required for submitting, not reading 😐

    #63707

    imported_gcamp
    Participant

    As far as I know the bot accounts get a much reduced page, with no graphics etc… so they can crawl the site more efficiently.

    #63708

    comedown
    Participant

    What a hilarious blunder.

    #63709

    nila
    Participant

    It is a interesting sharing………..

Viewing 7 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.