NOTE: The spider list was last updated on Thursday 14th February 2013 @ 06:15 PM AEDT and has a total of 644 spiders.
NOTE 2: As of Monday 1st July 2013, due to medical & life-style reasons I will no longer be updating/maintaining the spiders list, and will no longer participate in the vBulletin community. At that point I will add the latest .xml file for people to continue using.
Two Years On
In the last year (two years since the I launched this service) there have been an extra 41,000 downloads of the new spiders list, which means there have been an average of 122 downloads a day in the last year alone. This spiders list is still being added to and actively maintained.
One Year On
In the year since the I launched this service there have been 8,401 downloads of the new spiders list which is an average of 23 downloads a day, and I have added well over 100 new and updated spiders to the list since I took over from Dream.
WS Spiders List
This is a follow-on thread to Dream's vBulletin Spiders Directory (for updated spiders_vbulletin.xml files) thread.
As Dream has not updated his Spider List (which has 503 spiders) since early October 2009, I got the urge to update the spiders list..... and as I do not have access to Dream's source code I decided to write my own Spider List service from scratch that emulates the system that Dream created.
I would like to thank Dream for creating his service and maintaining the database for over a year and a half. My new service will hopefully pick up where his left off.
The free service I have created allows you to download the updated "spiders_vbulletin.xml" file (currently with 515 spiders being identified) and this will hopefully increase weekly (with your help by submitting spiders).
This is the official thread to submit spiders, and as long as the submitted spiders are not currently in the database, I will add them as soon as possible after they are submitted and I have confirmed they are valid.
I hope you enjoy this new free service.
Mosh Shigdar, former vBulletin Project Tools Developer.
If you have questions about how this works read the FAQ:
WS Spiders List FAQ
How to Submit a Spider here:
Post the following info in this thread:
- Spider name
- Spider Ident (aka user-agent) (It's not the IP address! please read here for more info)
- Spider Website (optional)
- Spider Contact Email (optional)
If you are not sure of the information to be submitted, then follow the instructions listed in the I want to submit a spider. What do you need? section of the FAQ.