Results 1 to 3 of 3
  1. #1
    bagpiperdude90's Avatar
    Join Date
    Apr 2007
    Posts
    217
    Reputation
    10
    Thanks
    1

    Robots.txt exclude

    Hey, to forum / site admin(s):

    I'd suggest you add a robots.txt exclude file to the root web directory here, and have it exclude the /forum subfolder. If you think about it... do you really want people to be able to do a google search and come up with results and links coming right to the forums? Do you want google caching everything in here?

    Just an idea. I see neither the main site folder nor the forum subfolder have a robots.txt exclude file. That means google bot and all the other search engines will index every post in the forum. If you want that... then ok. But, I don't see why you'd want random google searches to link to some forum content.

  2. #2
    arunforce's Avatar
    Join Date
    Dec 2005
    Gender
    male
    Location
    A place for amigos
    Posts
    21,686
    Reputation
    2927
    Thanks
    9,510
    Quote Originally Posted by bagpiperdude90 View Post
    Hey, to forum / site admin(s):

    I'd suggest you add a robots.txt exclude file to the root web directory here, and have it exclude the /forum subfolder. If you think about it... do you really want people to be able to do a google search and come up with results and links coming right to the forums? Do you want google caching everything in here?

    Just an idea. I see neither the main site folder nor the forum subfolder have a robots.txt exclude file. That means google bot and all the other search engines will index every post in the forum. If you want that... then ok. But, I don't see why you'd want random google searches to link to some forum content.
    The more Google indexes, the more our rank goes up, so more people can find us when they search for the hacks they need. =)
    Isn't that the reason for search engines? Were helping Google find our content so that they can help searchers find results.




    v1.96 - Tutorials, Upgraded Town, Def Revamp
    v1.97 - Cutscenes/Quests, Friend List
    v1.98 - Classes, Necklace/Rings
    v1.99 - Guilds
    v2.00 - Open Beta


  3. #3
    bagpiperdude90's Avatar
    Join Date
    Apr 2007
    Posts
    217
    Reputation
    10
    Thanks
    1
    yeah, thats the reason for google.

    I was under the impression this place was more of a "grandfathered in" place... i.e. people hear of it through a friend, who got it through a friend, who got it through a friend. Kinda like the black market. You're not going to search google and get the 3rd hit to be a black market site selling crack.

    I guess I was under the wrong impression. Sorry! Go ahead and delete this topic if you want.

Similar Threads

  1. Robot Rage
    By -Joe- in forum Hack Requests
    Replies: 9
    Last Post: 03-07-2008, 06:29 AM
  2. Replies: 7
    Last Post: 01-04-2007, 04:33 PM
  3. QuaSI THE ROBOT
    By Dave84311 in forum General
    Replies: 2
    Last Post: 10-08-2006, 05:22 AM
  4. Best Robot Dance
    By arunforce in forum General
    Replies: 0
    Last Post: 07-15-2006, 01:10 PM
  5. [Request] Test.txt
    By WertyRO in forum Gunz General
    Replies: 28
    Last Post: 02-19-2006, 10:27 PM