Bush’s Robots.txt Vs. Obama’s Robots.txt

21 January, 2009

Current Affairs

If you run a blog or a website you probably know what a robots.txt file is, so you can skip the next line. It’s a small text file that tells search engine what to include and not to include in their indexing of your site.

As you might have noticed or read www.whitehouse.gov has been given an Obama make-over. The changes has aren’t just graphical and textual. Perhaps as a comparison to the previous government, the new robots.txt is a bit more welcoming.

This is Obama’s robots.txt:

User-agent: *
Disallow: /includes/

Yes that’s it!

Now compare it to George W. Bush’s 2400 line robots.txt (I won’t post it all)

  1. User-agent:     *
  2. Disallow:       /cgi-bin
  3. Disallow:       /search
  4. Disallow:       /query.html
  5. Disallow:       /omb/search
  6. Disallow:       /omb/query.html
  7. Disallow:       /expectmore/search
  8. Disallow:       /expectmore/query.html
  9. Disallow:       /results/search
  10. Disallow:       /results/query.html
  11. Disallow:       /earmarks/search
  12. Disallow:       /earmarks/query.html
  13. Disallow:       /help
  14. Disallow:       /360pics/text
  15. Disallow:       /911/911day/text
  16. Disallow:       /911/heroes/text
  17. Disallow:       /911/messages/text
  18. Disallow:       /911/patriotism/text
  19. Disallow:       /911/patriotism2/text

Let’s hope Obama keeps it this way. It’s a good start. So American…please don’t screw this up icon smile Bushs Robots.txt Vs. Obamas Robots.txt

If you enjoyed this post, make sure you subscribe to my RSS feed!
, , ,
  • http://www.lexliacnekit.com/ Acne products

    Nice… We'll see how fast this changes. Everyone keeps saying Obama will be our savior and lead the world into a peaceful harmony, I'm not saying he won't but what I am saying let's wait and see what happens and what he does before we go commending his actions. Regards!

  • http://www.myfreeps3.net PS3

    Nice observation!
    Hope they keep it up!

  • http://www.townsvillecomputerrepairs.com jamie

    Awesome post

  • http://inventorymanagementsoftwarezone.com/ inventory management software

    how did u manage to find out that robot.txt apply for that website..opss..i am newbie.

  • http://topfivesextoys.com adult toys


  • http://www.avanta.co.uk Office space

    Well, that could mean a number of things. Obviously Obama is more web savvy than Bush. Or probably Obama loves SEO. lol

  • http://www.paspaaditlem.com Paspaaditlem

    Good. Great you noticed this

  • http://www.onlinebankratings.com/bankonelocations.htm Josh

    I guess Obama can pretty much do no wrong.

  • http://www.colewire.com SO Cable

    People are forgetting that the old Robots.txt was disallowing links to junk pages and search results. But hey, transparency is transparency I guess.

  • http://masajesreynaldo.blogspot.com Masajes Reynaldo

    A greeting from Spain.

    Passed by this blog by chance and liked his ma and wave energy.

    Enjoy your meal and continue.


  • http://aromatherapycourse.blogspot.com Aromatherapy Course

    Who made seo For Them?

  • Sulumits Retsambew

    So which of these two is recommended robots.txt when it comes to seo. BTW love the new revamping, Site looks much better now :)

    Good luck to Obama