+ Reply to Thread
Results 1 to 2 of 2

Thread: How To Reduce Bandwidth Theft And Website Abuse

  1. #1
    Senior Member Jose Amaral Rego is a jewel in the rough Jose Amaral Rego is a jewel in the rough Jose Amaral Rego is a jewel in the rough Jose Amaral Rego's Avatar
    Join Date
    Feb 2005
    Location
    Edmonton, Alberta, Canada
    Age
    40
    Posts
    10,956

    How To Reduce Bandwidth Theft And Website Abuse

    To reduce bandwidth theft and website abuse from Spambots, Inline linking & Scrapers on your Apache Web Server. You would first have to be able to track their activities and identify them. You can use AWStats to see how much bandwidth is been used by each User-Agents. Using 'robots.txt' will work for search bots that follow the instruction, but most bad_bots ignor or use the list to scan what you do not want displayed. This method is not the best method nor is adding robot.txt.

    How To Ban Spambot(s) and or Bad Bots
    1. Identify troublesome USER_AGENTS
      • Mail address harvesting bot:
        spider visits your board and searches out symbol [@] or (X)HTML character entity [& #064;] or string [mailto:] to mail addresses to send spam later.
      • Misuse of robots.txt:
        bot reads /robots.txt and then deliberately jumps right into the Disallow:ed directory.
      • Scraper bot:
        they will duplicate the entire contents of your site, then set it up somewhere else and place Adwords on it. Using a Bot-trap will not protect your site against these bots because they usually follow robots.txt.
      • Ignoring robots.txt:
        bot reads /robots.txt but then during spidering forgets and ignores the Disallow: directive.
      • Not looking at robots.txt:
        bot starts spidering the site without even looking for a /robots.txt.
      • Corporate tattletales:
        these are good bots for most who do not violate any corporations use of their trademark, copyrights, openly criticize them and so forth.
      • Chinese spambot:
        some of the dumb and silly spam bots using UA strings "Indy Library" or "Internet Explore 5.x".
      • Guestbook harvester:
        bot spiders guestbook pages only, often very aggressively (several pages per second), and of course ignoring /robots.txt.
    2. Create or add to existing .htaccess file.
      How to create a .htaccess file extention
      You do not need to download and or install any software or hardware product to create an htaccess file. Basically all you want to use is your default systems text editor program such as 'Notepad'

      The only difficult part in saving file with extension name .htaccess is that if your system can accept a no-name filename with a long string extension.
      ".htaccess"

      # Open text editor 'Notepad'
      # Add your mod_rewrite for redirection header code (301)
      # From 'text editor 'Notepad' Toolbar - File - Save As...
      # File name: ".htaccess" - Note: Use double quotes around all characters
      # Save as type: All Files
      # Click [Save]
      # Upload file to your files directory.

      If this does not work for your current P.C., then you can upload file using a FTP program as ".htaccess.txt" and then use FTP to rename.
    3. Add the following to .htacces file.
      Code:
           #  Block bad_bots, spiders, crawlers and harvesters
           RewriteEngine on
           Options +FollowSymlinks
           RewriteBase /
           #  Block bad_bots by their own internet protocol(ip)
           RewriteCond %{REMOTE_HOST} ^209.189.115..* [OR]
           #  Bad_bots list
           RewriteCond %{HTTP_USER_AGENT} ^AESOP_com_SpiderMan [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Alexibot [OR] 
           RewriteCond %{HTTP_USER_AGENT} Anonymouse.org [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^asterias [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^attach [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^BackDoorBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^BackWeb [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Baiduspider [OR] 
           RewriteCond %{HTTP_USER_AGENT} Bandit [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^BatchFTP [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Bigfoot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Black.Hole [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
           RewriteCond %{HTTP_USER_AGENT} ^BlowFish [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
           RewriteCond %{HTTP_USER_AGENT} ^BotALot [OR] 
           RewriteCond %{HTTP_USER_AGENT} Buddy [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^BuiltBotTough [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Bullseye [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^BunnySlippers [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Cegbfeieh [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^CheeseBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
           RewriteCond %{HTTP_USER_AGENT} Collector [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} Copier [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^CopyRightCheck [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^cosmos [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Curl [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
           RewriteCond %{HTTP_USER_AGENT} ^DA [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^DIIbot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
           RewriteCond %{HTTP_USER_AGENT} ^DittoSpyder [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Download [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Download\ Devil [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Download\ Wonder [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Download.*Demon[NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} Downloader [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^dragonfly [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Drip [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^EasyDL [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^ebingbong [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
           RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
           RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
           RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
           RewriteCond %{HTTP_USER_AGENT} ^EroCrawler [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Exabot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
           RewriteCond %{HTTP_USER_AGENT} Extractor [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
           RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
           RewriteCond %{HTTP_USER_AGENT} ^FileHound [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
           RewriteCond %{HTTP_USER_AGENT} ^flunky [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Foobot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^FrontPage [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
           RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
           RewriteCond %{HTTP_USER_AGENT} ^gotit [OR] 
           RewriteCond %{HTTP_USER_AGENT} Grabber [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Harvest [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^hloader [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
           RewriteCond %{HTTP_USER_AGENT} ^httplib [OR] 
           RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
           RewriteCond %{HTTP_USER_AGENT} ^HTTrack [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^humanlinks [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR]  
           RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
           RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
           RewriteCond %{HTTP_USER_AGENT} ^InfoNaviRobot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^InfoTekies [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Intelliseek [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Iria [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Jakarta [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^JennyBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
           RewriteCond %{HTTP_USER_AGENT} ^JOC [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
           RewriteCond %{HTTP_USER_AGENT} ^JustView [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Jyxobot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Kenjin.Spider [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Keyword.Density [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
           RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
           RewriteCond %{HTTP_USER_AGENT} ^LexiBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^lftp [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^libWeb/clsHTTP [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^likse [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^LinkextractorPro [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^LinkScan/8.1a.Unix [OR]
           RewriteCond %{HTTP_USER_AGENT} ^LinkWalker [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^LNSpiderguy [OR]
           RewriteCond %{HTTP_USER_AGENT} ^LWP::Simple [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^lwp-trivial [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Magnet [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Mag-Net [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^MarkWatch [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Mata.Hari [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Memo [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Microsoft\ URL\ Control [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
           RewriteCond %{HTTP_USER_AGENT} ^MIIxpc [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Mirror [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Missigua\ Locator [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Mister.PiX [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
           RewriteCond %{HTTP_USER_AGENT} ^moget [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Mozilla/3.Mozilla/2.01 [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^NAMEPROTECT [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
           RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
           RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Netcraft [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^NetMechanic [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
           RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
           RewriteCond %{HTTP_USER_AGENT} ^NextGenSearchBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^NG [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^NimbleCrawler [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Ninja [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^NPbot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Offline.Explorer [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Openfind [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^OutfoxBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
           RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
           RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
           RewriteCond %{HTTP_USER_AGENT} ^PHP\ version\ tracker [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Pockey [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^ProPowerBot/2.14 [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^psbot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Pump [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^QueryN.Metasearch [OR] 
           RewriteCond %{HTTP_USER_AGENT} Recorder [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
           RewriteCond %{HTTP_USER_AGENT} Reaper [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
           RewriteCond %{HTTP_USER_AGENT} ^RepoMonkey [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^RMA [OR] 
           RewriteCond %{HTTP_USER_AGENT} sitecheck.internetseer.com [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
           RewriteCond %{HTTP_USER_AGENT} Siphon [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^SlySearch [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Snake [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Snapbot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Snoopy [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^sogou [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^SpaceBison [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^SpankBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^spanner [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Sqworm [OR] 
           RewriteCond %{HTTP_USER_AGENT} Stripper [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} Sucker [NC,OR] 
           RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
           RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
           RewriteCond %{HTTP_USER_AGENT} ^suzuran [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Szukacz/1.4 [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Teleport [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Telesoft [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^The.Intraformant [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^TheNomad [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^TightTwatBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Titan [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^toCrawl/UrlDispatcher [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^True_Robot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^turingos [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot/1.5 [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^URLy.Warning [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Vacuum [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^VCI [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Whacker [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Web.Image.Collector [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebBandit [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Webclipping.com [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^WebEnhancer [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Web.Image.Collector [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebmasterWorldForumBot [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebSite [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Website.Quester [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Webster.Pro [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Webster [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
           RewriteCond %{HTTP_USER_AGENT} ^WISENutbot [OR]
           RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^WWW-Collector-E [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Xaldon [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Xenu's [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Xenu [OR] 
           RewriteCond %{HTTP_USER_AGENT} ^Zeus [OR]
           RewriteCond %{HTTP_USER_AGENT} ^Zyborg
           #  Choose below one RewriteRule you can choose where to send bad_bots.
    4. Select final action
      • # Sends to 'Forbidden Error' page once a match is made [Forbidden 403 Page, Last Rule]
        RewriteRule ^.* - [F,L]
      • # Sends to any fake site or custom html page [Last Rule, Redirect]
        RewriteRule ^(.*)$ http://fake_link.com/ [L,R]
      • # Sends to any fake info custom html page [Last Rule, Redirect]
        RewriteRule /*$ http://www.yourdomainname.com [L,R]
        RewriteRule ^.* - [F]
        RewriteCond %{HTTP_REFERER} ^http://www.spam_back.com$
        RewriteRule !^http://[^/.]\.yourdomainname.com.* - [F]
      • # Sends to any fake info custom html page or origin spam URL[Last Rule, Redirect]
        RewriteRule /*$ http://www.spam_site-or-custom_page.com [L,R]
    Last edited by Jose Amaral Rego; Mon 2nd Jun '08 at 2:36pm. Reason: Updated list

  2. #2
    Senior Member Jose Amaral Rego is a jewel in the rough Jose Amaral Rego is a jewel in the rough Jose Amaral Rego is a jewel in the rough Jose Amaral Rego's Avatar
    Join Date
    Feb 2005
    Location
    Edmonton, Alberta, Canada
    Age
    40
    Posts
    10,956
    How To Stop Inline Linking And Bandwidth Leeching


    Many host will provide you with this option, but if you are not able to find or know how to install such a script, then try using this. Mod_rewrite is very powerful tool to use, but if you apply incorrect within the .htaccess file, you will recieve an '500 Internal Server Error "Page cannot be displayed"'.

    What is inline linking or bandwidth leeching?

    Website(s) who use your (URL) Uniform Resource Locator to display your images, movies, music on their site by linking directly to your site are engaging in 'inline linking' or 'bandwidth leeching'. This in turn will drain your total bandwidth allowable usage per month. (only if you have these restriction inplace)
    1. What needs to be done to stop inline linking.
      • Identify any odd activities by viewing 'Referer Logs' within host control panel.
      • Referrer logs are extra entries in your 'Access Logs' file informing you what site your visitor(s) originated from. By default referrer logs are not included with most small hosting accounts, but you may be able to install or upgrade to another hosting package.
      • You can use whitelists method, but then you may or will block all referers, legitimate requests from search engines, feed readers, etcetera except those you have listed.
      • You will also need the following
        • Apache web server account with mod_rewrite enabled.
        • Access to .htaccess and support from your hosting provider if you do not or could not activate mod_rewrite.

      Once you know you have mod_rewrite enabled and support from hosting provider, then we can start with how to block leechers.

    2. Create a small weight image
      It can be of any dimensions that is not offensive and upload to any folder i.e.: images/misc/stop_hotlink.gif

    3. Create or add to existing .htaccess file.

      How to create a .htaccess file extention
      You do not need to download and or install any software or hardware product to create an htaccess file. Basically all you want to use is your default systems text editor program such as 'Notepad'

      The only difficult part in saving file with extension name .htaccess is that if your system can accept a no-name filename with a long string extension.
      ".htaccess"

      # Open text editor 'Notepad'
      # Add your mod_rewrite for redirection header code (301)
      # From 'text editor 'Notepad' Toolbar - File - Save As...
      # File name: ".htaccess" - Note: Use double quotes around all characters
      # Save as type: All Files
      # Click [Save]
      # Upload file to your files directory.

      If this does not work for your current P.C., then you can upload file using a FTP program as ".htaccess.txt" and then use FTP to rename.
    4. Add the following to .htacces file.
      Code:
           RewriteEngine on
           #  Prevent inline(hot) linking from these specific listed sites
           RewriteCond %{HTTP_REFERER} ^http://(.+\.)?xanga.com/  [NC,OR]
           RewriteCond %{HTTP_REFERER} ^http://(.+\.)?myspace\.com/ [NC,OR]
           RewriteCond %{HTTP_REFERER} ^http://(.+\.)?blogspot\.com/ [NC,OR]
           RewriteCond %{HTTP_REFERER} ^http://(.+\.)?livejournal\.com/ [NC]
           RewriteCond %{REQUEST_URI} !/path/to/stop_hotlink.gif
           #  Choose below one RewriteRule you can choose for inline image linkers.
    5. Select final action
    Last edited by Jose Amaral Rego; Thu 13th Dec '07 at 8:46pm.

+ Reply to Thread

Similar Threads

  1. tips to reduce bandwidth?
    By mikeinjersey in forum vBulletin 3.5 'How Do I' Questions and Troubleshooting
    Replies: 2
    Last Post: Wed 8th Mar '06, 10:51pm
  2. reduce bandwidth
    By hollyboy in forum vBulletin 3.0 How Do I and Troubleshooting Forum
    Replies: 21
    Last Post: Sun 3rd Apr '05, 5:16pm
  3. How Do I - Reduce my Bandwidth
    By RobFitz in forum vBulletin 3.0 How Do I and Troubleshooting Forum
    Replies: 8
    Last Post: Thu 2nd Sep '04, 1:10pm
  4. Stop Bandwidth Theft?
    By RHarbison in forum vBulletin 2 'How Do I' and Troubleshooting
    Replies: 0
    Last Post: Wed 31st Mar '04, 2:00am
  5. How do I reduce bandwidth
    By MVaughn in forum vBulletin 2 'How Do I' and Troubleshooting
    Replies: 2
    Last Post: Fri 21st Nov '03, 3:53pm

Bookmarks

Posting Permissions

Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts