Are you ready to block the Semalt and Buttons For Website bots yet? Over the past several months many people have noticed the Semalt.com and, to a lesser extent, the buttons-for-website.com domains showing up in their Analytics reports as frequent visitors (referrers) to their websites. This article will explain how to block the two of them on WordPress sites.

I have noticed the two crawlers on most of the WordPress sites that I manage for a month or two now. At first it was just one or two hits here and there but then it became every day and sometimes multiple hits from each one every day. This was enough to become annoying because I manage a lot of small business sites that serve local markets. These sites do not get a whole lot of traffic each day and so getting 2-6 hits per day from these two sites adds up over the course of a month.

The Semalt and Buttons For Website bots do not seem to do any actual harm to websites however their effect as traffic should not be ignored. If your website is getting 30, 50 or 120 visits per month from these bots it will affect your overall bounce rate on the site since the bounce rate on these bots is always 100% and the time spent on site is less than one second. This will make it seem as though visitors to your site are not finding the material they were looking for and, to the search engines, may decrease the perceived quality of your site and thereby effect your position on SERP (Search Engine Results Page). Especially on a small site this can undo a lot of the careful optimization you have done to improve the position of your site in organic search results.

Both of these crawlers are actually linked to what seem (at first glance) to be legitimate websites. Semalt appears to be a website analysis service like WooRank or Moz. Buttons For Website offers…sharing buttons for your website like +AddThis. This is where the legitimacy seems to end though. I have read various posts about these two sites and the consensus so far seems to be that both have somewhat nefarious goals and methods in play.

Semalt has been around a bit longer than Buttons For Website and seems to propagate itself a bit differently. There are two good analysis articles done on Semalt, you can read them here and here. The consensus is that Semalt has created a “Botnet”:

“Botnets sometimes compromise computers whose security defenses have been breached and control conceded to a third party. Each such compromised device, known as a “bot”, is created when a computer is penetrated by software from a malware (malicious software) distribution. The controller of a botnet is able to direct the activities of these compromised computers” – Wikipedia

Their Botnet involves hundreds or thousands of computers and too many IP addresses to be able to effectively bloc the crawler via IP Exclusion in Analytics. To see a list of Ip’s associated with Semalt go to this page and just hit go. It will return a long list of (at least hundreds) of IPs associated with Semalt. The Semalt bot is allegedly being distributed as a virus hidden within one or more programs. The previously mentioned articles say that a popular utility download called “Soundfrost” is one of them.

Blocking these sites like you would other crawlers/spiders in your robots.txt file may not be effective either since compliance with directives in the robots.txt file is voluntary and if you are running something Black Hat you probably do not care about complying with the wishes of others.

The Semalt botnet could potentially be used in many ways. One of them is to direct traffic to a particular website in order to attempt to artificially boost the SEO of that site. The Semalt crawler is configured so that it does not appear like a crawler and instead is perceived as an actual visitor to your website which is how it can skew your Analytics data. This is called “Link Spamming” and is an old Black Hat SEO technique that various upgrades to the algorithim of search engines has made much less effective than it used to be:

“Referrer spam (also known as log spam or referrer bombing) is a kind of spamdexing (spamming aimed at search engines). The technique involves making repeated web site requests using a fake referer URL to the site the spammer wishes to advertise. Sites that publish their access logs, including referer statistics, will then inadvertently link back to the spammer’s site. These links will be indexed by search engines as they crawl the access logs.”

“This benefits the spammer because the free link improves the spammer site’s search engine ranking owing to link-counting algorithms that search engines use.” – Wikipedia

Other ways that such a botnet could be used (according to the same Wikipedia article) would be “Botnets are exploited for various purposes, including denial-of-service attacks, creation or misuse of SMTP mail relays for spam, click fraud, mining bitcoins, spamdexing, and the theft of application serial numbers,login IDs, and financial information such as credit card numbers”. Obviously these possibilities make such traffic undesireable on your website.

Buttons For Website seems to be very similar in function (alleged to be a spambot/botnet) except that it uses a different delivery method. In this case the Buttons For Website site simply offers a handy sharing tool for you to install on your website. However, by installing the supplied code, you are potentially creating a way for a person to hijack (zombify) the web browser of visitors to your site.

According to one article I found javascript hijacking can also be used for nefarious purposes. Even though the article is about using javascript to create a botnet through online ads the same principle should work just as well with a permanent installation like sharing buttons.

“Adding arbitrary JavaScript to ads is easy to do and in the experience of the researchers wasn’t checked very closely by the ad network. To make it more convenient to change the malicious script, rather than placing the script itself in the ad, they put in the script source.” – NetworkWorld

Semalt And Buttons For Website Blocking

Since potentially both Semalt and Buttons For Website traffic is going to be coming from a large number of IP addresses (Semalt from infected computers and Buttons For Website from visitors to infected sites) the option of blocking this traffic by IP exclusion in Analytics would not be effective. An alternative, which is what I have used successfully on all of the WordPRess sites that I manage, is to block traffic from semalt.semalt.com and buttons-for-website.com in the .htacces file of each site.

To do this you have to have access to the files in the root directory on your web host that make up your WordPress site and be using an Apache system (most hosting providers do). If you have never worked with the files in the root directory of your site and/or are not familiar with editing the .htaccess file ask your webmaster to do it for you. If you make mistakes when editing or adding to the .htaccess file of your site the result can be your whole site crashing.

If you are comfortable with editing your .htaccess file then adding the following code to it should block both Semalt and Buttons For Website traffic to your site.

# block visitors referred from semalt.com
RewriteEngine on
RewriteCond %{HTTP_REFERER} semalt\.com [NC]
RewriteRule .* – [F]
# End semalt block
# block referer spam buttons for website
RewriteEngine On
RewriteCond %{HTTP_REFERER} buttons\-for\-website\.com
RewriteRule ^.* - [F,L]
# End buttons for website block

I have personally used this method to block Semalt and Buttons For Website traffic on over 25 WordPress sites that I manage and so far it has resulted in the total elimination of all traffic from these two sites from all of the managed websites. If you do not have a webmaster and are seeing traffic from these sources to your WordPress website I will be happy to help you with the problem. I can be contacted through email, the contact from on this site or via my Google Helpouts profile (fee applies in all cases).