Spam detection research

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Andy Huang
    Senior Member
    • Feb 2004
    • 4602

    Spam Spam detection research

    Some of you may have noticed my absence in the last couple weeks. This is because I just started my grad school studies and are finding course work to be heavier than originally anticipated. Currently, I am working on a research project about spam detection, which I am hoping I'd be able to one day bring a better spam detection system to vBulletin users -- you. As such, I am doing some quick field studies and I am hoping some of you could help me with some information.

    Please note that this research is not sponsored by Jelsoft, or Internet Brands, and is completely aside from vBulletin in general, so please don't feel obligated to participate in any means. If you have useful information but coming from a different forum software suite, please do feel free to include them, too. Just state which software you are using if that is relevant though. If you are interested in participating, please fire me an email at andy.huang-at-vbulletin...(you know the rest).

    The things I'm particularily interested are:
    1) Do you use internal spam detection services, where you would process messages / registration locally without submitting to third party servers? This includes things such as email validation, built in captcha/question & answer registration, or key word based spam detection.

    2) Do you use external spam detection services, where you would submit information to a third party server? Currently, vBulletin offers reCaptcha, and Akismet/Typepad antispam as built in solution. You may also have other third party addons, such as vBStop Forum Spam, etc. If you use a third party one, please also briefly describe how does it work.

    2.5) If you do not use external spam detection services, why? Company policy? Doesn't trust third party black boxes? Don't know about their existance? etc.

    3) After all the spam detection services you've put in place, approximately how much spam does your forum get on a daily / weekly / monthly basis, and do they get caught? IE: "I get ~50 spam posts a day, usually they're in moderation queue. I get around 1k new posts a week.", "I get ~10 spam posts a week, and they leak through my detectors. I get around 100 posts a week.", "I get no spam, because my forum requires manual approval.", etc.

    4) If you could, please provide some of these spam messages to me by copy and pasting them into separate .txt files (IE: one spam post per file) and zip them up for me so the mail server doesn't automatically detect your email to me as spam -- I think including a password on the zip might also help the process if for whatever reason the spam detector decides to go into zip contents...

    Thanks in advance!

    Edit: For those that scans the footnote for email address -- -- please email the response to andy.huang-at-vbulletin...(you know the rest). Thanks again!
    Last edited by Andy Huang; Tue 29 Sep '09, 11:03am.
    Best Regards,
    Andy Huang
  • Wayne Luke
    vBulletin Technical Support Lead
    • Aug 2000
    • 73979

    #2
    1) Yes... Currently use Email Verification, Length of time spent registering algorithms, and an internal keyword based stop list. I do not use any type of Captcha on registration.

    2) Yes... Currently all registrations are automatically checked against botscout.com and manually checked against stopforumspam.com. Automatic checks are done using the botscout.com api. The forum also has Akismet set up for checking posts.

    3) I have received 0 spam so far but the forum is fairly slow since it the intention is to focus on vBulletin 4.0 which is under NDA currently. Have had several potential spammers sign up but they were never able to post anything they were either stopped by email validation, keyword stoplist or manually verification. Have had a few false positives with Akismet though.
    Translations provided by Google.

    Wayne Luke
    The Rabid Badger - a vBulletin Cloud demonstration site.
    vBulletin 5 API

    Comment

    • Shamil.
      Senior Member
      • Feb 2008
      • 4755
      • 4.2.X

      #3
      1) Yes, I currently use email validation, and Q + A on a forum.

      2) I use reCaptcha at registration on one of my forums, Akismet to filter spam.

      2.5) I personally prefer all systems to be in my control, and will be removing reCaptcha soon and replacing with internal captcha or Q + A etc.

      3) Today, there was a sharp rise, by 6 x of what we usually get - I've had none in the past month.. today server was D/DoS(ed)... we're popular

      4), if I catch it, I'll throw it at you
      Shamil Nunhuck, - Radon Systems Ltd.
      VPS + Dedicated Server Hosting and Management
      vBulletin Hosting and Services
      Server / Website Consultation

      Comment

      • anthonyparsons
        Senior Member
        • Mar 2005
        • 2597

        #4
        1) Yes. Keyword & email ignore lists. I do not use email validation, instead I use custom profile questions.

        2) Yes. Recaptha on all available aspects for guest group.

        3) Around 5 - 10 posts weekly of spam get through the system with around 2000 - 3000 new posts being made weekly to the forum. These are all personal advertising, people spammers. We do not moderate new members, but restrict heavily. New members are limited to two forums which are not public or search engine accessible, no PM, group, album, profile posting abilities, etc on initial registration.

        Comment

        • MRGTB
          Senior Member
          • May 2005
          • 5454

          #5
          Originally posted by Wayne Luke
          1) Yes... Currently use Email Verification, Length of time spent registering algorithms, and an internal keyword based stop list. I do not use any type of Captcha on registration.

          2) Yes... Currently all registrations are automatically checked against botscout.com and manually checked against stopforumspam.com. Automatic checks are done using the botscout.com api. The forum also has Akismet set up for checking posts.

          3) I have received 0 spam so far but the forum is fairly slow since it the intention is to focus on vBulletin 4.0 which is under NDA currently. Have had several potential spammers sign up but they were never able to post anything they were either stopped by email validation, keyword stoplist or manually verification. Have had a few false positives with Akismet though.
          Jesus Wayne, I wasn't aware you was using all that on your site. I thought you was only using STOP FORUM SPAM. Talk about making it bullet proof, or in your case Spam Proof. Never known anybody use so many measures like that before.

          Anyway, back to subject at hand. I just use reCaptcha (vb.org mod) for my earlier vBulletin version, used only for registration. And so far it's been stopping all spam bots from automaticly registering themselves. But then my forum is very small, so I'm not a big target yet I guess.
          Last edited by MRGTB; Tue 29 Sep '09, 3:37pm.

          Comment

          • Andy Huang
            Senior Member
            • Feb 2004
            • 4602

            #6
            Been about 2 weeks and I've only gotten info from the replies here... Bumpty-bump-bump, please provide some information to help me help make spam filters better for your forums
            Best Regards,
            Andy Huang

            Comment

            • Wayne Luke
              vBulletin Technical Support Lead
              • Aug 2000
              • 73979

              #7
              You might try posting at sites like adminaddict.net, adminfuel.com, theadminzone.com, sitepoint.com and other more general sites.
              Translations provided by Google.

              Wayne Luke
              The Rabid Badger - a vBulletin Cloud demonstration site.
              vBulletin 5 API

              Comment

              • nexialys
                Senior Member
                • May 2004
                • 2183
                • 1.1.x

                #8
                and not to look stupid, but remember vbulletin.org ?!

                there is a lot of admins participating there too...
                oh no, i'm not going with Xenforo... come on, i'm better than that... i stick with Wordpress... rofl

                Comment

                Related Topics

                Collapse

                Working...