Load balancing with Nginx

Load balancing, Nginx No Comments »

Are you looking forward to balance HTTP or HTTPS traffic to multiple back-end servers? Nginx is one of the best choices – it’s very lightweight, requires less resources and it’s completely free.

You need to compile the nginx (there are many resources and howto’s) and add the following lines to your nginx.conf file:

upstream webcluster {
server 10.10.10.1;
server 10.10.10.2
server 10.10.10.3;
}

location / {
proxy_pass http://webcluster;
break;
}

Where 10.10.10.1, 10.10.10.2, 10.10.10.3 are the back-end web servers located in internal network defined in upstream settings above. The location / specifies that all URLs should be balanced to back-end servers.

Many folks prefer to serve HTTPS traffic with nginx and then forward all requests via HTTP protocol to the back-end servers. The back-end servers should be located on a safe VLAN or internal network.

Nginx two recent patches for null pointer and gx_http_parse_complex_uri buffer underflow

Nginx No Comments »

Nginx developers have released two patches – one that fixes null pointer reference, gx_http_parse_complex_uri() buffer underflow. Update immediate or your site can be attacked and nginx stopped remotely (so I heard).
Download the latest stable version and recompile the binary. Then run

kill -USR2 nginx_master_pid

Make sure you see new master nginx process running (you will see two master processes), kill the old one:

kill nginx_master_pid

And your site will be safe and screamingly fast once again!

Stream flash video using nginx

Nginx, Streaming No Comments »

One of the best solutions for streaming Flash video nowadays is Nginx. It’s super fast, lightweight and does easily support thousands of concurrent connections without taking up much resources.

By default flash streaming is not activated and you need to turn it on in compilation process. You enable flash streaming by:

./configure –with-http_flv_module …(add other options here)

and then add to nginx.conf the following settings:

location ~ \.flv$ {
flv;
}

Nginx can easily push hundreds of megabits per second so make sure your upstream can support it and you have deep pockets for bandwidth expenses :)

Apache settings for prefork.c

Apache No Comments »

We use a medium busy server and do prefer to run prefork.c module with the following settings:

StartServers 20
MinSpareServers 5
MaxSpareServers 20
ServerLimit 200
MaxClients 200
MaxRequestsPerChild 25000

The server is Dual Core Pentium with 2GB RAM and it’s not very busy server but does perfect.

Boost Wordpress with nginx front end

Nginx No Comments »

Prepare for real performance boost. Run Nginx in a front of Apache and serve all static content directly from Nginx. All php script requests are transparently forwarded to Apache daemon and output forwarded back to Nginx that servers it to a web visitor.

Web visitor -> Nginx -> static content -> Nginx -> Web visitor

if dynamic content then,

Web visitor -> Nginx -> dynamic content -> Apache -> Nginx -> Web visitor

For static content serving you should add the following nginx config:

location ~* ^.+\.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|rtf|js)$ {
root /usr/www;
expires 10d;
break;
}

With the following config you will serve all those files directly via Nginx and it will be very fast. Don’t forget to set-up Nginx listening to public IP and Apache configure to a different port or internal IP.

Nginx web server tips and tricks

Nginx No Comments »

Alright fellows. Many folks are using Nginx nowadays and I thought we need to open a new category dedicated to Nginx. It’s fast, light-weight and reliable web server that’s becoming very popular.

We will soon have some fresh content and howto’s about Nginx. Stay tunned!

Best web server for static content hosting on Linux

Web servers No Comments »

There are quite a few web server software daemons that can be used for almost any content hosting, but for high performance and fast static content hosting the choice is not that big.

Zeus (commercial)
Lighttpd (free)
Nginx (free)
Thttpd (free)
Cherokee (free)


We have tested quite a few http server daemons listed above (except Zeus). If you are looking for basic hosting, here is some great Linux web hosts.

My top server software would be:

Nginx and Cherokee. Easy to use, easy configuration, lightweight and very reliable. I do not pay for the software I can get for free, sorry folks that develop Zeus ;)

Redirecting users to different page or site using mod_rewrite

Apache No Comments »

OK, if you want to redirect user to a different page or even a site, you can easily do it using mod_rewrite rule set:

RewriteCond %{HTTP_USER_AGENT} “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)” [NC]
RewriteRule ^(.*) http://yoursite.com/page.html [R=301,L]

Will redirect all users with Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) to http://yoursite.com/page.html

slammed with multiple downloads from one IP

Apache No Comments »

If your web site has been slammed with multiple downloads from one IP causing the web site to slow load due to CPU, memory or network stack, it’s time for an action.

For apache users there is an easy workaround – you need mod_limitipconn module and you are set.

Apache 1.3 and version 2 has a bit different code, thus you will need to download the correct mod_limitipconn module depending on the apache version you are running.

Install this module as usual (DSO or static compile) and setup a Location definition in httpd.conf file.

MaxConnPerIP 3

Will limit 3 connections per IP. All other connections above the previous definition (3) will simply be dropped….

Enjoy !

throttling bandwidth with thttpd using URL patterns

Thttpd, Web servers No Comments »

thttpd providers excellent and easy to set-up bandwidth throttling based on files, directories, file types or default web server bandwidth speed.

If you want to cap the default bandwidth of your thttpd server to 512Kbps you will need to add the following configuration line to your thttpd config file:

** 51200If exe download files are consuming too much of your upstream, consider limiting download speed for all executable files via:

**.exe 102400

The above configuration line limits all .exe file downloads to max of 1Mbps per second at any time.

You can also limit bandwidth speed on some specific directory, for example, if you are providing a trial file downloads and want to offer better speed for your paid members on the same server. Create a folder named /trial/ and set-up thttpd config file to include

trial/**  25600

The above setting will limit all files located under /trial/  account to have a maximum download speed of 256Kbps.

limiting apache bandwidth speed and traffic leechers

Apache No Comments »

At this time, there are quite a few modules available that will stop traffic leech and limit simultaneous connections, bandwidth based on files, virtual host and so on.

mod_bw
Apache 2 (also available for Windows platform):

http://www.ivn.cl/apache/

mod_cband (Apache2):

http://cband.linux.pl/

mod_bandwidth

http://www.cohprog.com/v3/bandwidth/doc-en.html#

mod_security and suggested rules

Apache 1 Comment »

mod_security is a powerful attack shield that can block http requests based on pre-defined rules. It’s a nice shield against known software exploits (e.g. non-patched forums, scripts etc)…

A non-aggresive rule set:
# Turn the filtering engine On or Off
SecFilterEngine On

# Change Server: string
SecServerSignature “Apache”
# This setting should be set to On only if the Web site is
# using the Unicode encoding. Otherwise it may interfere with
# the normal Web site operation.
SecFilterCheckUnicodeEncoding Off

# The audit engine works independently and
# can be turned On of Off on the per-server or
# on the per-directory basis. “On” will log everything,
# “DynamicOrRelevant” will log dynamic requests or violations,
# and “RelevantOnly” will only log policy violations
SecAuditEngine RelevantOnly

# The name of the audit log file
SecAuditLog logs/audit_log

# Should mod_security inspect POST payloads
SecFilterScanPOST On

# Action to take by default
SecFilterDefaultAction “deny,log,status:403″

## ## ## ## ## ## ## ## ## ##
## ## ## ## ## ## ## ## ## ##

# Require HTTP_USER_AGENT and HTTP_HOST in all requests
# SecFilterSelective “HTTP_USER_AGENT|HTTP_HOST” “^$”

# Require Content-Length to be provided with
# every POST request
SecFilterSelective REQUEST_METHOD “^POST$” chain
SecFilterSelective HTTP_Content-Length “^$”

# Don’t accept transfer encodings we know we don’t handle
# (and you don’t need it anyway)
SecFilterSelective HTTP_Transfer-Encoding “!^$”

# Protecting from XSS attacks through the PHP session cookie
SecFilterSelective ARG_PHPSESSID “!^[0-9a-z]*$”
SecFilterSelective COOKIE_PHPSESSID “!^[0-9a-z]*$”

SecFilter “viewtopic\.php\?” chain
SecFilter “chr\(([0-9]{1,3})\)” “deny,log”

# Block various methods of downloading files to a server
SecFilterSelective THE_REQUEST “wget ”
SecFilterSelective THE_REQUEST “lynx ”
SecFilterSelective THE_REQUEST “scp ”
SecFilterSelective THE_REQUEST “ftp ”
SecFilterSelective THE_REQUEST “cvs ”
SecFilterSelective THE_REQUEST “rcp ”
SecFilterSelective THE_REQUEST “curl ”
SecFilterSelective THE_REQUEST “telnet ”
SecFilterSelective THE_REQUEST “ssh ”
SecFilterSelective THE_REQUEST “echo ”
SecFilterSelective THE_REQUEST “links -dump ”
SecFilterSelective THE_REQUEST “links -dump-charset ”
SecFilterSelective THE_REQUEST “links -dump-width ”
SecFilterSelective THE_REQUEST “links http:// ”
SecFilterSelective THE_REQUEST “links ftp:// ”
SecFilterSelective THE_REQUEST “links -source ”
SecFilterSelective THE_REQUEST “mkdir ”
SecFilterSelective THE_REQUEST “cd /tmp ”
SecFilterSelective THE_REQUEST “cd /var/tmp ”
SecFilterSelective THE_REQUEST “cd /etc/httpd/proxy ”
SecFilterSelective THE_REQUEST “/config.php?v=1&DIR ”
SecFilterSelective THE_REQUEST “&highlight=%2527%252E ”
SecFilterSelective THE_REQUEST “changedir=%2Ftmp%2F.php ”
SecFilterSelective THE_REQUEST “arta\.zip ”
SecFilterSelective THE_REQUEST “cmd=cd\x20/var ”
SecFilterSelective THE_REQUEST “HCL_path=http ”
SecFilterSelective THE_REQUEST “clamav-partial ”
SecFilterSelective THE_REQUEST “vi\.recover ”
SecFilterSelective THE_REQUEST “netenberg ”
SecFilterSelective THE_REQUEST “psybnc ”
SecFilterSelective THE_REQUEST “fantastico_de_luxe ”

SecFilter “bcc:”
SecFilter “bcc\x3a”
SecFilter “cc:”
SecFilter “cc\x3a”
SecFilter “bcc:|Bcc:|BCC:” chain
SecFilter “[A-Z0-9._%-]+@[A-Z0-9._%-]+\.[A-Z]{2,4}\,\x20[A-Z0-9._%-]+@[A-Z0-9._%-]+\.[A-Z]{2,4}”
SecFilterSelective POST_PAYLOAD “Bcc:”
SecFilterSelective POST_PAYLOAD “Bcc:\x20″
SecFilterSelective POST_PAYLOAD “cc:”
SecFilterSelective POST_PAYLOAD “cc:\x20″
SecFilterSelective POST_PAYLOAD “bcc:”
SecFilterSelective POST_PAYLOAD “bcc:\x20″
SecFilterSelective POST_PAYLOAD “bcc: ”
SecFilterSelective THE_REQUEST “Bcc:”
SecFilterSelective THE_REQUEST “Bcc:\x20″
SecFilterSelective THE_REQUEST “cc:”
SecFilterSelective THE_REQUEST “cc:\x20″
SecFilterSelective THE_REQUEST “bcc:”
SecFilterSelective THE_REQUEST “bcc:\x20″
SecFilterSelective THE_REQUEST “bcc: ”
# WEB-PHP phpbb quick-reply.php arbitrary command attempt
SecFilterSelective THE_REQUEST “/quick-reply\.php” chain
SecFilter “phpbb_root_path=”

Measuring your server bandwidth usage

Web servers No Comments »

If you are getting attacked or simply burning too much traffic (your upstream provider says so)… it’s perhaps the best time to check out  how much of traffic are you really bursting?

There is a software (freeware of course) that perfectly counts/measures your server traffic usage so go ahead and use it.

http://dast.nlanr.net/Projects/Iperf/

Perfect http daemon for large file serving

Thttpd 1 Comment »

We will soon perform some benchmarking with thttpd and large file serving, but I am confident that it will be one of the best.

Why? Because it’s FAST and I mean it…:

  • it doesn’t fork
  • perfect memory management
  • small runtime
  • implements http 1.1 protocol with minimum requirements
  • secure and robust

Checking web server header

Http protocol, Web servers No Comments »

You can easily check a web server header using a simple telnet utility.

telnet www.yahoo.com 80
HEAD / HTTP/1.0

and hit Enter two times.

You will get a header response with the HTTP status code….

HTTP/1.1 200 OK
Date: Mon, 31 Jul 2006 05:47:22 GMT
P3P: policyref=”http://p3p.yahoo.com/w3c/p3p.xml”, CP=”CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV”
Cache-Control: private
Vary: User-Agent
Set-Cookie: FPB=7cum1i2o812cr6ba; expires=Thu, 01 Jun 2006 19:00:00 GMT; path=/; domain=www.yahoo.com
Connection: close
Content-Type: text/html

Connection closed by foreign host.

WP Theme
Entries RSS Comments RSS Log in