robot.txt

[MRGTB]

Cam somebody who knows how to use a robot.txt file give me an example of what should be excluded to robots on your site. To help index the site better.

I'm currently using this:

User-agent: * 
Disallow: /cgi-bin/
Disallow: /images/

Situated in my public_html folder

[dbembibre]

This is my robots.txt

User-agent: Mediapartners-Google* 
Disallow: /admincp/
Disallow: /modcp/
Disallow: /clientscript/
Disallow: /cpstyles/
Disallow: /customavatars/
Disallow: /customcode/
Disallow: /customprofilepics/
Disallow: /images/
Disallow: /includes/
Disallow: /signaturepics/
Disallow: /oivar/
Disallow: /club/
Disallow: /Movilnorte/
Disallow: /uploads/
Disallow: /gallery/
Disallow: /articulos/
Disallow: /geek/
Disallow: /ajax.php
Disallow: /attachment.php
Disallow: /announcement.php
Disallow: /avatar.php
Disallow: /calendar.php
Disallow: /cron.php
Disallow: /editpost.php
Disallow: /faq.php
Disallow: /joinrequests.php
Disallow: /global.php
Disallow: /image.php
Disallow: /inline.php
Disallow: /joinrequests.php
Disallow: /login.php
Disallow: /member.php
Disallow: /misc.php
Disallow: /moderator.php
Disallow: /newattachement.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /poll.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /register.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /sendtofriend.php
Disallow: /showgroups.php
Disallow: /showpost.php
Disallow: /subscription.php
Disallow: /subscriptions.php
Disallow: /threadrate.php
Disallow: /usercp.php

User-agent: Googlebot
Disallow: /*goto=
Disallow: /*mode=
Disallow: /*&pp=
Disallow: /*showthread.php?p=
Disallow: /*postcount=
Disallow: /*daysprune=
Disallow: /*&sort=

[Mazinger]

What's the use of User-agent: Mediapartners-Google* & User-agent: Googlebot?

[Wayne Luke]

Google uses different bots for different purposes. Googlebot is their primary search engine bot and they use others for advertising and content sharing purposes.