- Search Engine Optimization Starter Guide (.pdf file)
- Google's article: "Search Engine Optimization (SEO)"
2010-04-24
Search Engine Optimization (SEO)
Here are some resources from Google to help you with Search Engine Optimization (SEO):
2010-04-05
Simple banner ad rotator using SSI
Here's a simple method to rotate multiple banner ads into one ad space on your webpage by using SSI (server-side-include) statements in your .html files on your linux server (note: does not work for .php files nor for Windows servers).
To keep things organized, create a directory in the root directory of your website and call it "/ads" (you can use any directory name or location you prefer; just make appropriate adjustments below).
For each banner ad that you want to show, create a text file for it and save it in the "/ads" directory. For example, you might have three ads: /ads/cookware.txt, /ads/cookbooks.txt, /ads/appliances.txt. Each file would have a bit of HTML code to show a banner ad; for example:
<a href="http://www.amazon.com/"><img src="amazon.gif"></a>
They should not contain <html> or <body> tags. They should just contain the basic HTML tags that are used to show the banner ad.
Create a text file named /ads/ads.txt that contains something like the following:
<!--#config timefmt="%S" -->
<!--#if expr="${DATE_LOCAL} <20" -->
<!--#include virtual="/ads/cookware.txt"-->
<!--#elif expr="${DATE_LOCAL} <40" -->
<!--#include virtual="/ads/cookbooks.txt"-->
<!--#else -->
<!--#include virtual="/ads/appliances.txt"-->
<!--#endif -->
The %S (that's a capital letter S) says that the server should look at the seconds hand of the server's internal clock to determine what ad to show. Since the probability of a user visiting the website during any given second (of 60 seconds) is the same as any other second, on average, the ads will be shown evenly over a reasonable amount of time.
The #if with "<20" tests to see if the seconds is less than 20 and causes the first ad to be show. The #elif (means "else if") with "<40" tests to see if the seconds is less than 40 and causes the second ad to be show. The #else (i.e.: all previous tests failed) causes the last ad to be shown.
Why "<20" and "<40"? Because in this example, three ads are being rotated. Take 60 seconds and divide by 3 ads to get 20 seconds per ad.
If you want to show a different number of ads then adjust the tests. For example, for five ads, you would use something like the following code:
<!--#config timefmt="%S" -->
<!--#if expr="${DATE_LOCAL} <12" -->
<!--#include virtual="/ads/ad1.txt"-->
<!--#elif expr="${DATE_LOCAL} <24" -->
<!--#include virtual="/ads/ad2.txt"-->
<!--#elif expr="${DATE_LOCAL} <36" -->
<!--#include virtual="/ads/ad3.txt"-->
<!--#elif expr="${DATE_LOCAL} <48" -->
<!--#include virtual="/ads/ad4.txt"-->
<!--#else -->
<!--#include virtual="/ads/ad5.txt"-->
<!--#endif -->
For different numbers of ads, here are the tests to use:
2 ads: <30
3 ads: <20, <40
4 ads: <15, <30, <45
5 ads: <12, <24, <36, <48
6 ads: <10, <20, <30, <40, <50
7 ads: <9, <17, <26, <34, <43, <51
8 ads: <8, <15, <23, <30, <38, <45, <53
9 ads: <7, <13, <20, <27, <33, <40, <47, <53
10 ads: <6, <12, <18, <24, <30, <36, <42, <48, <54
Integrating the ad rotator into your webpage is simple. Use an SSI include statement in your webpage where you want the ad rotator to display an ad:
<!--#include virtual="/ads/ads.txt" -->
This statement will cause the ad rotator code to be included into your webpage, and it will then display one of your ads based on the clock's seconds.
Note: You might need to include the following statement in your .htaccess file (located in your website's root directory) so that the server will process the SSI statements:
AddHandler server-parsed .html .txt
That's how to create a basic ad rotator using SSI. If you have multiple ad spaces on your webpage and you want a different ad rotator for each ad space, create a separate ad rotator for each one (e.g.: /ads/top.txt, /ads/left.txt, /ads/inline.txt).
If you need something more complex, then have a look at the free ad server software OpenX.
To keep things organized, create a directory in the root directory of your website and call it "/ads" (you can use any directory name or location you prefer; just make appropriate adjustments below).
For each banner ad that you want to show, create a text file for it and save it in the "/ads" directory. For example, you might have three ads: /ads/cookware.txt, /ads/cookbooks.txt, /ads/appliances.txt. Each file would have a bit of HTML code to show a banner ad; for example:
<a href="http://www.amazon.com/"><img src="amazon.gif"></a>
They should not contain <html> or <body> tags. They should just contain the basic HTML tags that are used to show the banner ad.
Create a text file named /ads/ads.txt that contains something like the following:
<!--#config timefmt="%S" -->
<!--#if expr="${DATE_LOCAL} <20" -->
<!--#include virtual="/ads/cookware.txt"-->
<!--#elif expr="${DATE_LOCAL} <40" -->
<!--#include virtual="/ads/cookbooks.txt"-->
<!--#else -->
<!--#include virtual="/ads/appliances.txt"-->
<!--#endif -->
The %S (that's a capital letter S) says that the server should look at the seconds hand of the server's internal clock to determine what ad to show. Since the probability of a user visiting the website during any given second (of 60 seconds) is the same as any other second, on average, the ads will be shown evenly over a reasonable amount of time.
The #if with "<20" tests to see if the seconds is less than 20 and causes the first ad to be show. The #elif (means "else if") with "<40" tests to see if the seconds is less than 40 and causes the second ad to be show. The #else (i.e.: all previous tests failed) causes the last ad to be shown.
Why "<20" and "<40"? Because in this example, three ads are being rotated. Take 60 seconds and divide by 3 ads to get 20 seconds per ad.
If you want to show a different number of ads then adjust the tests. For example, for five ads, you would use something like the following code:
<!--#config timefmt="%S" -->
<!--#if expr="${DATE_LOCAL} <12" -->
<!--#include virtual="/ads/ad1.txt"-->
<!--#elif expr="${DATE_LOCAL} <24" -->
<!--#include virtual="/ads/ad2.txt"-->
<!--#elif expr="${DATE_LOCAL} <36" -->
<!--#include virtual="/ads/ad3.txt"-->
<!--#elif expr="${DATE_LOCAL} <48" -->
<!--#include virtual="/ads/ad4.txt"-->
<!--#else -->
<!--#include virtual="/ads/ad5.txt"-->
<!--#endif -->
For different numbers of ads, here are the tests to use:
2 ads: <30
3 ads: <20, <40
4 ads: <15, <30, <45
5 ads: <12, <24, <36, <48
6 ads: <10, <20, <30, <40, <50
7 ads: <9, <17, <26, <34, <43, <51
8 ads: <8, <15, <23, <30, <38, <45, <53
9 ads: <7, <13, <20, <27, <33, <40, <47, <53
10 ads: <6, <12, <18, <24, <30, <36, <42, <48, <54
Integrating the ad rotator into your webpage is simple. Use an SSI include statement in your webpage where you want the ad rotator to display an ad:
<!--#include virtual="/ads/ads.txt" -->
This statement will cause the ad rotator code to be included into your webpage, and it will then display one of your ads based on the clock's seconds.
Note: You might need to include the following statement in your .htaccess file (located in your website's root directory) so that the server will process the SSI statements:
AddHandler server-parsed .html .txt
That's how to create a basic ad rotator using SSI. If you have multiple ad spaces on your webpage and you want a different ad rotator for each ad space, create a separate ad rotator for each one (e.g.: /ads/top.txt, /ads/left.txt, /ads/inline.txt).
If you need something more complex, then have a look at the free ad server software OpenX.
2010-03-13
Guide: Google Sitemaps
Beginners guide to what a Google Sitemap is, why you need it, where to get sitemap software to make one, and how to tell Google.com about your sitemap file.
Guide: Google Sitemaps
http://www.scribd.com/doc/28319477/Guide-Google-Sitemaps
Google Sitemaps generator software:
http://www.c3scripts.com/sitemap/
Guide: Google Sitemaps
http://www.scribd.com/doc/28319477/Guide-Google-Sitemaps
Google Sitemaps generator software:
http://www.c3scripts.com/sitemap/
2010-02-08
See a Need, Fill a Need
Social media advisor Chris Brogan wrote in his article "Pursue the Goal Not the Method" about the important difference between method and goals. About goals, he wrote:
When you are working on your affiliate marketing efforts, you have to always keep your goals in mind. You have to be flexible as to the method. The method is not as important as the goal.
And not only do you have to keep your goals in mind, you have to think about the goals and needs and wants of your target market (see "So What Do People Want, Exactly?").
I'm reminded of a line from the animated film "Robots
" (2005). The motto of the main character is, "See a need, fill a need." This succinct line summarizes all of Chris Brogan's "The goal is ..." statements. We should burn that bit of marketing wisdom into our minds.
So the next time you write a blog post or an article, or add something to your website, keep your goals and your visitors' goals in mind. In fact, even before you do any of that, ask yourself if that is the best method to reach those goals. Be flexible in your method and reach for the goal.
Remember ...
See a need, fill a need.
Seek the goal.
The goal is equipping people.
The goal is satisfying need.
The goal is seeking to better others.
The goal is to provide.
The goal is to make everything work better.
The goal is equipping people.
The goal is satisfying need.
The goal is seeking to better others.
The goal is to provide.
The goal is to make everything work better.
When you are working on your affiliate marketing efforts, you have to always keep your goals in mind. You have to be flexible as to the method. The method is not as important as the goal.
And not only do you have to keep your goals in mind, you have to think about the goals and needs and wants of your target market (see "So What Do People Want, Exactly?").
I'm reminded of a line from the animated film "Robots
" (2005). The motto of the main character is, "See a need, fill a need." This succinct line summarizes all of Chris Brogan's "The goal is ..." statements. We should burn that bit of marketing wisdom into our minds.So the next time you write a blog post or an article, or add something to your website, keep your goals and your visitors' goals in mind. In fact, even before you do any of that, ask yourself if that is the best method to reach those goals. Be flexible in your method and reach for the goal.
Remember ...
See a need, fill a need.
2010-02-07
Pay-Per-Click ROI Calculator
When you run Pay-Per-Click (PPC) Ad Campaigns such as Google AdWords, you have to keep an eye on your Return On Investment (ROI).
With any ad campaign, there are four important "input" numbers that you need to know so you can calculate "output" numbers such as ROI:
Rather than calculating all these numbers by hand, you can use this handyPPC ROI Calculator .
By the way, if you're using Google AdWords for your PPC ad campaigns, AdWords can be a great way of getting traffic to your website, but it can also get very expensive if you don't know what you're doing. I suggest that you check outBeating Adwords .
With any ad campaign, there are four important "input" numbers that you need to know so you can calculate "output" numbers such as ROI:
- Monthly Ad Spend: How much money you want to spend per month for ads. This is your monthly ad budget. You have control over this number.
- Conversion Rate (%): How many clicks convert into sales, expressed as a percent. Calculated as: number of sales divided by number of clicks, expressed as a percent. For example, 3 sales from 250 clicks is a 1.2% conversion (i.e.: 3 / 250 * 100 = 1.2). You can estimate this number by looking at the past performance of your ad campaigns. Conversion rate will vary depending on a wide variety of factors, including the products you're promoting, your marketing efforts, your competition, SEO, search engine placement, the season, the economy, and more. Use a realistic conversion rate; typically, less than 2.0% for online sales. Note: Past conversion rate is not a guarantee of future conversion rate.
- Cost Per Click ($): Cost per click (CPC) is how much you are being charged per click by the ad network. Typcially, when you create an ad, you set the maximum CPC you're willing to pay for that ad. You have control over this number.
- Average Revenue Per Sale ($): How much money you make (on average) per sale. You can estimate this number by looking at your past sales history. You have some control over this number by promoting higher priced items. Note: Past sales are not a guarantee of future sales.
- Estimated Clicks Per Month: Number of clicks in the month. Calculated as: ad budget divided by cost per click.
- Estimated Total Sales: Number of sales. Calculated as: number of clicks times the conversion rate. For example, 250 clicks with a conversion rate of 1.2% would be 3 sales (i.e.: 250 * 1.2 / 100 = 3). Note: The number of sales is only an estimate and is not guaranteed!
- Estimated Cost Per Sale: Cost for each sale. This number had better be less than the revenue per sale otherwise you're losing money per sale!
- Estimated Total Revenue: Total revenue from the sales. Calculated as: total number of sales * revenue per sale.
- Estimated Total Profit: Estimate of how much money you would make. Calculated as: Total revenue (income) minus monthly ad budget (expense). This number had better be positive otherwise you're losing money!
- Estimated ROAS: Estimated Return On Ad Spend. If this number is greater than 100%, you're making money; if exactly 100%, you're breaking even; if less than 100%, you're losing money.
- Estimated ROI: Estimated Return On Investment. If this number is greater than 0%, you're making money; if exactly 0%, you're breaking even; if less than 0%, you're losing money.
Rather than calculating all these numbers by hand, you can use this handy
By the way, if you're using Google AdWords for your PPC ad campaigns, AdWords can be a great way of getting traffic to your website, but it can also get very expensive if you don't know what you're doing. I suggest that you check out
2010-02-06
Recuva: Free File Recovery Software
Unlike my other posts, this one is not about affiliate marketing tips.
It's about a useful little utility program that I found that recovers deleted files on your Windows computer, Recycle Bin, digital camera card, or MP3 player.
It's called Recuva and it's free (the author does ask for a donation if you find it useful).
Recuva can recover pictures, music, Office documents, video, and all other file types too.
The thing is, you really should install Recuva on your Windows computer before you need. Because when you delete any file (whether accidentally or intentionally), the space that the file occupied is now freed up for any new files to use. So if you were to install Recuva at that point, the Recuva program itself might be written to that section on the hard drive thus overwriting the very file that you wanted to recover.
I have Recuva installed on all my computers just in case I need. And I recommend it to everyone I can.
So, install it now, before you need it. Check it out... Recuva
It's about a useful little utility program that I found that recovers deleted files on your Windows computer, Recycle Bin, digital camera card, or MP3 player.
It's called Recuva and it's free (the author does ask for a donation if you find it useful).
Recuva can recover pictures, music, Office documents, video, and all other file types too.
The thing is, you really should install Recuva on your Windows computer before you need. Because when you delete any file (whether accidentally or intentionally), the space that the file occupied is now freed up for any new files to use. So if you were to install Recuva at that point, the Recuva program itself might be written to that section on the hard drive thus overwriting the very file that you wanted to recover.
I have Recuva installed on all my computers just in case I need. And I recommend it to everyone I can.
So, install it now, before you need it. Check it out... Recuva
Blocking Bad Bots and Scrapers with .htaccess
Why block "bad bots"? Well, first let me say what a "bad bot" is. It is a search engine spider or email harvester spider that downloads pages from your website and does not obey your robots.txt file. The major of bad bots are a useless waste of your server's bandwidth and resources. That's why I ban them from my websites.
Most bad bots can be identified by their User-Agent value. By making your server check for known User-Agent values of bad bots, you can block those bad bots. Some bad bots forge their User-Agent value and pretend to be web browsers such as FireFox or IE; those bad bots cannot be blocked based on User-Agent (you may be able to block them based on IP address).
The testing for bad bots goes in your website's .htaccess file. If you have a dedicated server, you could put the testing in your webserver's http.conf file for greater efficiency.
I have the following statements in my website's root .htaccess file before all other Rewrite statements:
The statement "ErrorDocument 403 /403.html" causes all banned users to be redirected to /403.html so you should create a simple 403.html file that just reports the user has been banned. Keep it simple without any images or external CSS references (those files would be blocked).
This .htaccess file is based on a post at AskApache.com. The same concept is also mentioned in a post at JavasSriptKit.com.
It is also possible to ban spiders/spammers/users based on their IP address. For example, you could ban access from IP addresses located in China. I'll cover that topic in a subsequent post.
Most bad bots can be identified by their User-Agent value. By making your server check for known User-Agent values of bad bots, you can block those bad bots. Some bad bots forge their User-Agent value and pretend to be web browsers such as FireFox or IE; those bad bots cannot be blocked based on User-Agent (you may be able to block them based on IP address).
The testing for bad bots goes in your website's .htaccess file. If you have a dedicated server, you could put the testing in your webserver's http.conf file for greater efficiency.
I have the following statements in my website's root .htaccess file before all other Rewrite statements:
#--- 403 Forbidden file ---
ErrorDocument 403 /403.html
#--- enable Rewrite engine ---
RewriteEngine on
RewriteBase /
#--- block bad bots ---
# see: http://www.askapache.com/htaccess/blocking-bad-bots-and-scrapers-with-htaccess.html
RewriteRule ^403\.html$ - [L]
# IF THE UA STARTS WITH THESE
RewriteCond %{HTTP_USER_AGENT} ^(aesop_com_spiderman|alexibot|backweb|bandit|batchftp|bigfoot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(black.?hole|blackwidow|blowfish|botalot|buddy|builtbottough|bullseye) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cheesebot|cherrypicker|chinaclaw|collector|copier|copyrightcheck) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cosmos|crescent|curl|custo|da|diibot|disco|dittospyder|dragonfly) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(drip|easydl|ebingbong|ecatch|eirgrabber|emailcollector|emailsiphon) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(emailwolf|erocrawler|exabot|eyenetie|filehound|flashget|flunky) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(frontpage|getright|getweb|go.?zilla|go-ahead-got-it|gotit|grabnet) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(grafula|harvest|hloader|hmview|httplib|httrack|humanlinks|ilsebot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(infonavirobot|infotekies|intelliseek|interget|iria|jennybot|jetcar) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(joc|justview|jyxobot|kenjin|keyword|larbin|leechftp|lexibot|lftp|libweb) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(likse|linkscan|linkwalker|lnspiderguy|lwp|magnet|mag-net|markwatch) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mata.?hari|memo|microsoft.?url|midown.?tool|miixpc|mirror|missigua) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mister.?pix|moget|mozilla.?newt|nameprotect|navroad|backdoorbot|nearsite) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(net.?vampire|netants|netcraft|netmechanic|netspider|nextgensearchbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(attach|nicerspro|nimblecrawler|npbot|octopus|offline.?explorer) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(offline.?navigator|openfind|outfoxbot|pagegrabber|papa|pavuk) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(pcbrowser|php.?version.?tracker|pockey|propowerbot|prowebwalker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(psbot|pump|queryn|recorder|realdownload|reaper|reget|true_robot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(repomonkey|rma|internetseer|sitesnagger|siphon|slysearch|smartdownload) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(snake|snapbot|snoopy|sogou|spacebison|spankbot|spanner|sqworm|superbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(superhttp|surfbot|asterias|suzuran|szukacz|takeout|teleport) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(telesoft|the.?intraformant|thenomad|tighttwatbot|titan|urldispatcher) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(turingos|turnitinbot|urly.?warning|vacuum|vci|voideye|whacker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(libwww-perl|widow|wisenutbot|wwwoffle|xaldon|xenu|zeus|zyborg|anonymouse) [NC,OR]
# STARTS WITH WEB
RewriteCond %{HTTP_USER_AGENT} ^web(zip|emaile|enhancer|fetch|go.?is|auto|bandit|clip|copier|master|reaper|sauger|site.?quester|whack) [NC,OR]
# ANYWHERE IN UA -- GREEDY REGEX
RewriteCond %{HTTP_USER_AGENT} ^.*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures).*$ [NC]
# ISSUE 403 / SERVE ERRORDOCUMENT
RewriteRule . - [F,L]
(download this file)ErrorDocument 403 /403.html
#--- enable Rewrite engine ---
RewriteEngine on
RewriteBase /
#--- block bad bots ---
# see: http://www.askapache.com/htaccess/blocking-bad-bots-and-scrapers-with-htaccess.html
RewriteRule ^403\.html$ - [L]
# IF THE UA STARTS WITH THESE
RewriteCond %{HTTP_USER_AGENT} ^(aesop_com_spiderman|alexibot|backweb|bandit|batchftp|bigfoot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(black.?hole|blackwidow|blowfish|botalot|buddy|builtbottough|bullseye) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cheesebot|cherrypicker|chinaclaw|collector|copier|copyrightcheck) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cosmos|crescent|curl|custo|da|diibot|disco|dittospyder|dragonfly) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(drip|easydl|ebingbong|ecatch|eirgrabber|emailcollector|emailsiphon) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(emailwolf|erocrawler|exabot|eyenetie|filehound|flashget|flunky) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(frontpage|getright|getweb|go.?zilla|go-ahead-got-it|gotit|grabnet) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(grafula|harvest|hloader|hmview|httplib|httrack|humanlinks|ilsebot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(infonavirobot|infotekies|intelliseek|interget|iria|jennybot|jetcar) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(joc|justview|jyxobot|kenjin|keyword|larbin|leechftp|lexibot|lftp|libweb) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(likse|linkscan|linkwalker|lnspiderguy|lwp|magnet|mag-net|markwatch) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mata.?hari|memo|microsoft.?url|midown.?tool|miixpc|mirror|missigua) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mister.?pix|moget|mozilla.?newt|nameprotect|navroad|backdoorbot|nearsite) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(net.?vampire|netants|netcraft|netmechanic|netspider|nextgensearchbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(attach|nicerspro|nimblecrawler|npbot|octopus|offline.?explorer) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(offline.?navigator|openfind|outfoxbot|pagegrabber|papa|pavuk) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(pcbrowser|php.?version.?tracker|pockey|propowerbot|prowebwalker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(psbot|pump|queryn|recorder|realdownload|reaper|reget|true_robot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(repomonkey|rma|internetseer|sitesnagger|siphon|slysearch|smartdownload) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(snake|snapbot|snoopy|sogou|spacebison|spankbot|spanner|sqworm|superbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(superhttp|surfbot|asterias|suzuran|szukacz|takeout|teleport) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(telesoft|the.?intraformant|thenomad|tighttwatbot|titan|urldispatcher) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(turingos|turnitinbot|urly.?warning|vacuum|vci|voideye|whacker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(libwww-perl|widow|wisenutbot|wwwoffle|xaldon|xenu|zeus|zyborg|anonymouse) [NC,OR]
# STARTS WITH WEB
RewriteCond %{HTTP_USER_AGENT} ^web(zip|emaile|enhancer|fetch|go.?is|auto|bandit|clip|copier|master|reaper|sauger|site.?quester|whack) [NC,OR]
# ANYWHERE IN UA -- GREEDY REGEX
RewriteCond %{HTTP_USER_AGENT} ^.*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures).*$ [NC]
# ISSUE 403 / SERVE ERRORDOCUMENT
RewriteRule . - [F,L]
The statement "ErrorDocument 403 /403.html" causes all banned users to be redirected to /403.html so you should create a simple 403.html file that just reports the user has been banned. Keep it simple without any images or external CSS references (those files would be blocked).
<html>
<head><title>403 Forbidden</title></head>
<body>
<h1>403 Forbidden</h1>
<p>Access forbidden.</p>
</body>
</html>
(download this file)<head><title>403 Forbidden</title></head>
<body>
<h1>403 Forbidden</h1>
<p>Access forbidden.</p>
</body>
</html>
This .htaccess file is based on a post at AskApache.com. The same concept is also mentioned in a post at JavasSriptKit.com.
It is also possible to ban spiders/spammers/users based on their IP address. For example, you could ban access from IP addresses located in China. I'll cover that topic in a subsequent post.
Subscribe to:
Posts (Atom)
Posts
