Hotlinking: In Depth

As a follow up on the article on how to block Fusker, here’s a fairly comprehensive explanation blocking hotlinkers using .htaccess and mod_rewrite with reasonably full explanations. Please read it carefully, almost every little bit matters!

Obviously, your server must have mod_rewrite. I think they all do.

Keep in mind – servers can be different based on the version of Apache, various mods installed, etc. If you use any part of this, test it out carefully. In some cases, where I have something like “.*”, you might need to use “(.*)” in your specific server configuration.

I’d recommend creating a new subdirectory, drop in some content and new subdirectories, and test in there until you are sure everything works.

First, I want to point out that the Apache will start looking for .htaccess files in the root directory, then in each directory it has to follow to find the target file. If you were linking to www.example.com/images/banners/banners.jpg, Apache would look for .htaccess in /, /images, and /images/banners, and will process each one it finds. This is pretty bad when you want to leave a few directories open for hotlinking. The answer to this is the AllowOverride directive in the Apache config files. Every server I’ve used has had the AllowOverride set correctly in the apache configs, and this can be a very good thing. This basically says “let this specific .httaccess file control this specific directory”. If it finds one, it does not merge and process any others. If your server does not have this directive set this way, you can ask them to do it, or, be ready to manage your htaccess files very carefully! You can put .htaccess files in every directory that simply redirect requests for missing files (see the ErrorDocument directives later on), or put hotlink blocking .htaccess files in each directory that matters, eg; images/*, privatesPics/*, whatever.

Assuming you will block hotlinking throughout your web site (and the AllowOverride directive is set correctly), here’s how to leave a subdirectory open. Create an .htaccess file in the subdirectory you want to leave open, and add the following:

Options -Indexes
RewriteEngine On


Save the file, and you are done. This tells Apache to disallow direct browsing of that subdirectory, and the “empty” RewriteEngine On overrides any previous matching conditions, thus opening it up for the world to hotlink.

You can comment your .htaccess with the “#” symbol. If you do use comments in your .htaccess files, only start them in the very first column, and keep them on one line – otherwise you’ll fill your server logs with useless error messages.

Since it’s hard to write out this info and keep the .htaccess file neat and clean, I’ll forgo comments, and just list the whole thing again at the end.

All lines in your .htaccess need to be one line – don’t let individual lines break. That may happen as you read this; every entry shoul;d be on it’s on line for real-world use.

This turns off index browsing for all subdirectories from here and below:

Options All -Indexes

this sets the order of handling requests. I allow all, and deny specific
IP addresses as needed:

order allow,deny
deny from 84.24.157.106
deny from 24.30.156.64
deny from 213.224.83.
deny from 62.194.58.230
deny from 65.32.247.133
deny from 24.110.101.138
deny from 130.161.13.62
allow from all


You can also reverse the order and have something like:

order deny, allow
84.24.157.106
24.30.156.64
213.224.83.
62.194.58.230
65.32.247.133
24.110.101.138
130.161.13.62
allow from all


That denies by default, so you don’t need the “deny from”. I leave it in so I know exactly what’s happening.

Tell Apache to follow symbolic links:

Options +FollowSymLinks

This is where the fun begins!

RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.example\.com
RewriteRule (.*) http://www.example.com/$1 [R=permanent,L]


Turn on the Rewrite engine. This will allow Apache to manipulate the urls, allowing you to block referers, redirect bad guys, and a lot of other fun things.

RewriteEngine on

RewriteCondition. Here, we tell Apache what to look for. In this example, I’m saying “if the host name being requested does not include exactly www.example.com” do something. The something is in the next line.

RewriteCond %{HTTP_HOST} !^www\.example\.com

Assume for the moment you hit my site with just http://example.com. This triggers that condition so the next line, starting with rewriterule, is activated:

RewriteRule (.*) http://www.example.com/$1 [R=permanent,L]

The above line tells the RewriteEngine to literally rewrite the URL for that browser request, which conducts a redirect to the full URL, including “www”. The [R=permanent, L] tells Apache this is a Redirect, and to make it permanent, and the L tells Apache this is the Last rule in this block.

The $1 says “keep whatever was after the matched string in the initial request”. Since we matched on “primanude.com”, everything after that is retained and tacked back onto the end of the new URL. That way, someone going to http://example.com/text.html won’t lose the “text.html” part when the redirect happens.

Why is this helpful? Google treats http://example.com and http://www.example.com as different addresses from purposes of page rank, as will many utilities and server functions. With google, this has the effect of making sure all page rank “funnels” to the correct URL.

That’s it for rewriting a partial URL to a full URL. You can do a lot more than this, but I’ll leave that up to you to research and learn about.

This next block disallows hotlinking for specific refering sites. EG; if the browser hit a link from one of the matching URLs, they don’t get what they expect.

First, we turn the RewriteEngine on again, see if the requests match a couple different patterns, then see if the request contains any images. If it matches, and is a request for any of the listed images, drop the request completely.

RewriteEngine on
RewriteCond %{HTTP_REFERER} .*fusker*.*$ [NC]
RewriteCond %{HTTP_REFERER} .*usefulidiot*.*$ [NC]
RewriteRule .*\.(gif|jpg|jpeg|bmp|png)$ http://www.example.com/hotlink/hotlink.jpg [F,NC]


Remember, that last line should not be split across two lines.

The conditions (RewriteCond) tell Apache to check the referer string sent by the browser. In both of these cases, a referer containing “fusker” or “usefulidiot” will match. You can put any part of a URL in here and it will be blocked. Be careful with this!! Imagine what would happen if you put “.com” here – Apache would drop any request with a referer containing “.com” – that’s bad for business!

Let’s looks at the lines more closely.

RewriteCond %{HTTP_REFERER} .*fusker*.*$ [NC]

This simply says match any referer with the word “fusker” in it. The “.*” at the start says “any number of any charachters in followed by ‘fuscker’”. The *.* says “any pattern of “.” and any other text as well”. The “$” says “end of line”. The [NC, OR] tells Apache to not consider the case of the letters in the string. Fusker, FUSker, and FUSKER will be treated the same.

RewriteRule .*\.(gif|jpg|jpeg|bmp|png)$ http://www.example.com/hotlink/hotlink.jpg [F,NC]

The RewriteRule used here says if the request is for an image, return an image of my choice! My “hotlink.jpg” could be tubgirl, goatse, or anything equally nasty. I prefer to use a watermarked image with my domain name. It’s semi-free advertising.

Again, the [F,NC] tells Apache this is the final rule, and, to not consider case in the request. JPG = jpg, etc. The part in the “()”s is pretty cool. The pipe symbol “|” means “OR”, to Apache looks for jpg OR gif OR jpeg, etc.

This next block turns off the ability for others to hotlink your images throughout the entire site (assuming you place an .htaccess file with this code in the root directory of your site).

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://google.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.google.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://example.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://example.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://example.com/images/banners/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://example.com/images/banners$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/friends/banners/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/friends/banners/$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/banners/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/banners/$ [NC]
RewriteRule .*\.(gif|jpg|jpeg|bmp|png)$ - [F,NC]


In the above example, I’ve told Apache to let a number of refering URLS grab images. Any URL that isn’t matched will not be able to grab the images. Remember, the “^” says NOT, so if the refering URL is NOT http://www.google.com, etc, it’s a match. I’ve listed each url in a couple different ways. There ae better ways to write this that will combine the multiple versions into one single matching condition – I’m just lazy tonight.

Here’s an example:

RewriteCond %{HTTP_REFERER} !^http://(www\.|kinzie-kenner\.|tiffany-paris\.|ronni-tuscadero\.|ftv-girls\.)?example.com(/)?.*/.*$ [NC]


That says www. OR kinzie-kenner. OR tiffany-paris. OR ronni-tuscadero. OR ftv-girls. followed by example.com, followed by anything. The www. is self-explanatory. The others are possible sub-domains. You can also simply say:

RewriteCond %{HTTP_REFERER} !^.*example.com.* [NC]

I like writing them out so I know exactly what is covered and what is not.

To continue:

RewriteRule .*\.(gif|jpg|jpeg|bmp|png)$ - [F,NC]

If anything matched in the RewriteConditions, enfocre the RewriteRule. This time, I’m just dropping the request rather than returning my hotlink.jpg image. The “-” where the URL was in the previous ReWriteUrl means “don’t do anything here”.

Why do I leave Google in here? Well, this allows Google to fully spider your images. You can get creative with your directory structures and only let google spider thumbs, maybe, or have a special directory of images you want to get listed in Google – that’s up to you.

You can add entries for any URL you like. If you want to display banners on www.thisothersite.com, where the images are on www.myfirstsite.com, just add an entry explicitly allowing www.thisothersite.com.

Finally, ErrorDocument tells Apache what to do when certain response codes are encountered. I just redirect the browser to my custom 404.html page (a 404 error is generated when a file can not be found).

If you wanted to sell your 404 traffic, just change the URL to something appropriate.

ErrorDocument 400 http://www.example.com/404.html
ErrorDocument 403 http://www.example.com/404.html
ErrorDocument 404 http://www.example.com/404.html
ErrorDocument 500 http://www.example.com/404.html
ErrorDocument 501 http://www.example.com/404.html
ErrorDocument 503 http://www.example.com/404.html


A special note on redirecting 404 errors. If you want to send people back to your home directory when a file can’t be found, and put a line in your .htaccess that is similar to:

ErrorDocument 404 http://www.example.com/index.html

make DAMN SURE you have an index.html file in the root directory. If you don’t, the request will gert looped infinately. eg; apaches says it can’t find the default doc, index.html, so it will redirect the request to, you guessed it, index.html. Over and over. Until the browser crashes.

Here’s the promised complete .htaccess file:

#start
Options All -Indexes

order allow,deny
deny from 84.24.157.106
deny from 24.30.156.64
deny from 213.224.83.
deny from 62.194.58.230
deny from 65.32.247.133
deny from 24.110.101.138
deny from 130.161.13.62
allow from all

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.example\.com
RewriteRule (.*) http://www.example.com/$1 [R=permanent,L]

RewriteEngine on
RewriteCond %{HTTP_REFERER} .*fusker*.*$ [NC]
RewriteCond %{HTTP_REFERER} .*usefulidiot*.*$ [NC]
RewriteRule .*\.(gif|jpg|jpeg|bmp|png)$ - [F,NC]

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://google.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.google.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://example.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://example.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://example.com/images/banners/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://example.com/images/banners$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/friends/banners/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/friends/banners/$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/banners/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.example.com/banners/$ [NC]
RewriteRule .*\.(gif|jpg|jpeg|bmp|png)$ - [F,NC]

ErrorDocument 400 http://www.example.com/404.html
ErrorDocument 403 http://www.example.com/404.html
ErrorDocument 404 http://www.example.com/404.html
ErrorDocument 500 http://www.example.com/404.html
ErrorDocument 501 http://www.example.com/404.html
ErrorDocument 503 http://www.example.com/404.html
#end

There’s a lot more you can do with .htaccess files and mod_rewrite, but this is a just a guide on how to get started.