UPDATE: This article was updated to include information about LinkSleeve a few days after publication.
Google took special aim at low-value, high-volume links with its February 9, 2011 so-called “JC Penney Update”. They devalued tens of thousands of compromised Web forums and blog comment posts that had been flooded with spammy links from users of Xrumer, Scrapebox, and other spam link software. The link spam industry took a clear and sustained hit from that update but some hard-core spammers kept using the software in the hope of getting through Google’s defenses. I have no doubt there was sporadic success simply because the World Wide Web is so big.
If nothing else there are always new blogs and forums coming online and even now their default installation settings leave much to be desired when it comes to fighting Website spam. Fortunately, blog and forum operators can look to a few tools to help us keep the spammers off our Websites, or at least reduce the amount of spam that gets through. Now is probably a good time to review what I have found that I am using. I may have to upgrade my arsenal after this post, but I’m getting used to that.
Web spammers find your blog or forum by running searches on popular services like Bing and Google. Their software looks for phrases like “leave a comment”, “register now”, “Powered by WordPress”, “Powered by Vbulletin”, etc. Any text that is used across your Website because it is embedded in a default template or a popular template makes your Website a target for Web spammers. Sometimes you can upgrade from free or “Lite” versions to “Pro” versions that remove these expressions from your footers. Sometimes you can modify the templates yourself without violating the terms of service. It’s important — if you are using licensed software — that you understand what you are and are not allowed to do because you don’t want to have your license revoked.
Web spammers subscribe to proxy services that feed their HTTP requests to Websites from IP addresses around the world. The IP addresses may be leased from legitimate hosting companies but my feeling is that most of these IP addresses are coming from compromised computers — that is, computers that have been tunred into zombies by malware. Wherever these proxy IP addresses come from, for years we have only had to track addresses in the IPv4 format. Now I fear that life has just become much more complicated.
The Internet has now formally begun its conversion to IPv6 addressing. There are far more addresses available in the IPv6 space than in the IPv4 space and we should — as a worldwide Internet community — be able to connect our refrigerators, TV sets, washing machines, dryers, clock radios, and just about every other conceivable electronic device to home networks that are in turn connected to the rest of the Internet. Unfortunately, none of these IPv6 addresses are being filtered by current anti-spam measures and therefore we are in a race against time to see whether spammers will fill that gap in security with more spam or if developers will plug the gaping hole.
For now the vast majority of IPv6 addresses may be working in parallel with mapped IPv4 addresses. At some point, however, each Internet Service Provider will make the decision to stop honoring requests from IPv4 addresses and only accept requests from IPv6. It’s unlikely your ISP will tell you this. It will simply happen. So the question is, what happens next?
For now, if you’re running WordPress on a Website, you need to make sure you’re using two important plugins:
Akismet – This is the original blog anti-spam tool developed by Matt Mullenweg. It works pretty well although every now and then innocent people get caught in the filters. Fortunately, there is an appeals process so that someone at Auttomatic will review your information and give you a reprieve — if they decide you really have not been naughty. You should activate Akismet as soon as you install WordPress. Most Websites can get an Akismet API key free-of-charge and most of the remaining sites only have to pay a nominal fee to offset the cost of maintaining the service. Akismet will mark suspicious comments as spam and hold them for your review. You SHOULD review your spam queue at least once a week. You as the site administrator have the ability to unmark any comments you think are okay.
Stop Spammer Registrations – This plugin checks the database at Stop Forum Spam to see if someone leaving a comment has been tagged as a spammer. Sometimes there are false-positives but if people trying to leave comments on your blog are incorrectly blocked, if you make it easy for them to contact you, you can whitelist their email addresses and/or IP addresses. I have done this enough times to know it works. You can also set your plugin to maintain a local cache to reduce the amount of traffic between your blog and the database. You do NOT need an API key to use this plugin, but if you do get one you can report comment spam to the database and that actually helps you as the automated spam tools will keep coming back to your site and leaving more comments.
Stop Spammer Registrations also works with other services. According to standard text embedded in the plugin dashboard, the plugin “eliminates 99% of spam registrations and comments. Checks all attempts to leave spam against StopForumSpam.com, Project Honeypot, BotScout, DNSBL lists such as Spamhaus.org, Ubiquity Servers, disposable email addresses, and HTTP_ACCEPT header.”
Linksleeve – If you’re advanced enough to feel comfortable dinking with the code of your Website, you can a few lines of code developed by Virante to check the URLs left in comments against the various spam databases. Learn more at LinkSleeve.org.
Besides these plugins you should also manage your WordPress Discussion settings. I have chosen to close comments on my posts after certain periods of days. Some people are not happy about that decision but it greatly reduces the opportunity to attract comment spam because Web spammers also search for articles by keyword where they can leave “relevant” comments and links. If you have 1,000 posts on your blog and all are open to comments you may receive dozens or hundreds of spammy comments per day. If, however, you only publish a couple of posts per week and you close comments after 15 or 30 days the spammers can only attack a small percentage of your posts at any time. I also have any comment that contains 1 or more links held for moderation. On some blogs I require that all comments be approved by an administrator.
If you run Web forums like VBulletin or PhPBB then Web spammers will be interested in three kinds of opportunities:
Forum Profiles – All they have to do is register accounts and leave links in the profiles. They never come back to post anything. Some spammers realize that forum administrators are monitoring registrations so they leave the accounts alone for a while (days, weeks, sometimes months) and come back to update them later.
Forum Signatures – Some spammers use software to post a few vague, ambiguous comments (usually as new threads) in your discussions; or they use a service like FIVERR to hire cheap labor to go in and register accounts and leave posts in your discussions. It may look like you’re getting real visitors who want to participate in your community but they don’t really say anything, they don’t hang around long, and after the discussions have died down (or sooner if the spammers are inexperienced) the spammers come in and set up links in the signatures.
Forum Posts – There are still occasional spammers who come in and start whole new discussions to tell your community about their great Websites. Most of these people are not using software. They are just very naive about software. But if a spammer suspects a forum is “running on autopilot” the forum is added to a list of forums that can be used gratuitously for easy linking. They register accounts and fill the forum with all sorts of nonsense discussions, dropping links everywhere: in forum profiles, in signatures, and in the posts themselves.
Most if not all the forums can use plugins to check Stop Forum Spam or some other service. You don’t have to report spammers if you don’t want to but it helps everyone if you report spam to these services.
You can also maintain your own local list of blocked IP addresses, domains, and email addresses. It’s not very efficient to block by email address so I usually block the IP addresses.
You can set your forums to require administrative review before new accounts are allowed to post. This is tedious and I modified my registration template to tell people they may have to wait 3-4 days to be approved. I do that to give the spammers time to show up in the databases. I then block the IP addresses that have been tagged for spam elsewhere.
You can also limit the privileges you give to new users. Make them write 10, 20, or 50 posts before they can get a signature link.
Most forum software now prevents search engine robots from seeing signatures anyway (you may have to turn this on in your administrative settings).
And you should block your user profile pages from being crawled by the search engines in your robots.txt file. This way the pages will never be indexed even if spammers point links at them.
Some people still try to use CAPTCHAs to defeat spammers but unfortunately there are services that the spammers can plug their software into which overcome the CAPTCHAs. The spam software passes the CAPTCHA to one of the services and the service hands the CAPTCHA to a real human being who solves the problem and passes back the solution. The spammers sometimes use software that is capable of caching or remembering specific CAPTCHAs and their solutions, so CAPTCHAs really don’t work very well.
It’s regrettable that we as Webmasters have to go to these lengths to get rid of bogus comments. Unfortunately there are unscrupulous people out there who feel they have a right to make money at our expense without our permission or compensating us in any way. So it is necessary to take appropriate, prudent measures to stop these filthy dirt bags.
They, of course, blame Google or society for their problems but no one is forcing them to spam the Web. They made that choice themselves. Just report them and move on. You don’t owe them any sympathy — nor their customers.
And if a spammer is really pernicious and just won’t go away, then make a list of all the domains they are trying to get links for and report those domains to Google through their Web spam report form. Sooner or later the spammy domains will be removed from the index and the spammers will lose their money. To file a spam report with Google log in to Google Webmaster Tools and go here.
Read More about Search Engine Optimization
How Long Does It Take SEO To Work?
Outbound Links: Why Use Forward Links for SEO?
On-Page Optimization SEO Checklist
White Hat Link Earning Techniques
Follow Reflective Dynamics |
Click here to follow Reflective Dynamics on Twitter: @refdynamics. Click here to follow SEO Theory on Twitter: @seo_theory. Reflective Dynamics' RSS Feed (summaries only) |