Apache reverse proxy
could do with filling out more detail
A reverse proxy allows you to front multiple websites from a single public IP address, act as a load balancer and potentially defuse otherwise dangerous cyber attacks. There are different solutions, the instructions here are for an Apache server-based solution. Basic requirements are Apache, mod_ssl and mod_proxy installed and enabled.
Installation
Install apache2
apt-get install apache2
File /etc/apache2/mods-available/proxy.conf should look like this:
<IfModule mod_proxy.c>
# If you want to use apache2 as a forward proxy, uncomment the
# 'ProxyRequests On' line and the <Proxy *> block below.
# WARNING: Be careful to restrict access inside the <Proxy *> block.
# Open proxy servers are dangerous both to your network and to the
# Internet at large.
#
# If you only want to use apache2 as a reverse proxy/gateway in
# front of some web application server, you DON'T need
# 'ProxyRequests On'.
ProxyRequests Off
<Proxy *>
AddDefaultCharset off
Order deny,allow
Deny from all
</Proxy>
# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
#ProxyVia Off
ProxyVia On
ProxyPreserveHost On
ProxyRequests Off
ProxyTimeout 600
</IfModule>
Create file /etc/apache2/mods-available/proxy_http.conf and put this inside:
ProxyVia On
ProxyPreserveHost On
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
Add these lines in /etc/apache2/apache2.conf to block paths that should not be visible
ProxyPass /.env ! ProxyPass /.git !
Enable these modules
a2enmod proxy a2enmod proxy_http a2enmod rewrite a2enmod headers a2enmod ssl a2enmod remoteip
Optional: By default apache only looks at .conf files from /etc/apache2/sites-enabled for sites. If you want it to look at all files edit this line in /etc/apache2/apache2.conf
IncludeOptional sites-enabled/*
On busier proxies you can modify /etc/apache2/mods-enabled/mpm_event.conf for a better performance. Here is an example:
<IfModule mpm_event_module>
StartServers 5
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxRequestWorkers 250
MaxConnectionsPerChild 0
</IfModule>
And restart apache2
systemctl restart apache2
Failover proxy
Create a user on failover-proxy with a private key and one on proxy with the public key that would sync the data from proxy to failover-proxy.
On failover-proxy create script /opt/bin/proxy_sync:
#!/bin/bash #apache sync /usr/bin/rsync -rl --safe-links --delete --rsync-path="/usr/bin/sudo /usr/bin/rsync" <user>@<proxy-ip>:/etc/apache2/ /etc/apache2/ 2>&1 >> /var/log/proxy_sync.log #for letsencrypt certs /usr/bin/rsync -rl --safe-links --delete --rsync-path="/usr/bin/sudo /usr/bin/rsync" <user>@<proxy-ip>:/etc/letsencrypt/ /etc/letsencrypt/ 2>&1 >> /var/log/proxy_sync.log /usr/sbin/apache2ctl graceful
Make it executable
chmod +x /opt/bin/proxy_sync
Create a cron job for the script /etc/cron.d/proxy_sync:
*/15 * * * * <user> /opt/bin/proxy_sync
Keepalived
Keepalived is used to manage a floating IP.
To setup keepalived, install it on both servers:
sudo apt-get install keepalived
Copy the nagios check "check_http" to /usr/local/bin, from the /usr/lib/nagios/plugins of a server that has nagios-plugins installed (please don't install nagios-plugins on the reverse proxy, that package would install many dependencies).
Finally create the following /etc/keepalived/keepalived.conf on the master:
global_defs {
notification_email {
<email>
}
notification_email_from <email>
smtp_server 127.0.0.1
}
vrrp_script chk_apache {
script "/usr/local/bin/check_http -H 127.0.0.1 -e 200"
interval 3 # check every 2 seconds
weight 2 # add 2 points of prio if OK
}
vrrp_instance floating_ip {
interface ens3
state MASTER
virtual_router_id 51
priority 101
authentication {
auth_type PASS
auth_pass SHAREDPASSWORD
}
virtual_ipaddress {
<select-an-ip>
}
track_script {
chk_apache
}
}
Create exactly the same file on the failover proxy, just change priority from 101 to 100.
Explanations of the apache check
The track_script "chk_apache" uses nagios' script check_http to ask a webpage to the local apache every three seconds - and checks for a 200 "OK" error code. If the test succeeds, it adds 2 points to the priority of the server.
The server with the highest priority gets the IP address. In effect, we would have the master with a priority of 101+2, and the failover with a priority of 100+2.
If the check fails on the master, its priority goes down to 100, and the IP migrates to the failover, which still has a priority of 100+2.
Adding a vhost
In Apache we use vhost declarations to define each reverse proxy FQDN. In Ubuntu/Debian systems these are found in /etc/apache2/sites-available, typically one per vhost using a suitably descriptive name. They can also be wrapped into a single file, or of course into the main apache conf file. As they are effectively (includes) of the Apache conf, every change requires an Apache restart to apply:
apache2ctl restart
In the Debian/Ubuntu model you also need to enable a site one it's been defined, which is done with a link to the /etc/apache2/sites-available/ file newly created:
cd /etc/apache2/sites-enabled ln -s ../sites-available/yournewvhost
This approach allows you to quickly and easily take a specific site offline if there's a problem, just by deleting the link in /etc/apache2/sites-enabled and restarting Apache.
Assuming your sites will be https from the proxy outwards, start with a 301 to force https:
<VirtualHost *:80>
ServerName my.domain.name
ServerAlias my.alias.domain
Redirect 301 / https://my.domain.name
ProxyPass / http://my.realserver.nameorIP/
ProxyPassReverse / http://my.realserve.nameorIP/
CustomLog /var/log/apache2/my.domain.name.access.log combined
ErrorLog /var/log/apache2/my.domain.name.error.log
</VirtualHost>
And then add an SSL entry
<VirtualHost *:443>
ServerName my.domain.name
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/mycertificate.crt
SSLCertificateKeyFile /etc/apache2/ssl/mykey.key
SSLCertificateChainFile /etc/apache2/ssl/myintermediatecertificateifneeded.crt
ProxyPass / http://myrealserver.nameorIP/
ProxyPassReverse / http://my.domain.name/
ProxyPassReverse / http://myrealserver.nameorIP/
CustomLog /var/log/apache2/my.domain.name.access.log combined
ErrorLog /var/log/apache2/my.domain.name.error.log
</VirtualHost>
If you want the SSL proxy to also connect to the target as SSL, change the ProxyPass URLs appropriately and add to the vhost SSL (*:443) part
SSLProxyEngine On
If you run into issues with SSL connects to the target service with Apache log entries like
Error during SSL Handshake with remote server
then you may need additional settings in the vhost statement; try
SSLProxyVerify none SSLProxyCheckPeerCN off SSLProxyCheckPeerName off
and if that works selectively comment out to optimise (least settings for desired functionality).
Enabling SSL on a VM hosting a CMS behind a proxy
The CMS tries to detect if it's running on SSL automatically by checking some SERVER environment variables. When SSL is served from the proxy, the CMS can't detect it by normal means, so it needs a bit of help. In our case the solution is to edit the .htaccess file and add this directive:
<IfModule mod_env.c> SetEnv HTTPS on </IfModule>
This was the fix for both Joomla and Wordpress VMs
Maintenance Page
If you are working on a website and want to show a maintenance page for everyone except you, this can be done from the Apache vhost on the proxy.
<VirtualHost *:443>
ServerName my.website.com
ProxyPass / http://ip.add.re.ss/
ProxyPassReverse / http://ip.add.er.ss/
RewriteEngine On
RewriteCond %{REMOTE_ADDR} !^my\.ip\.addr\.ess
RewriteCond %{REQUEST_FILENAME} !/index.html
RewriteCond %{REQUEST_FILENAME} !/css/app.css
RewriteCond %{REQUEST_FILENAME} !/logo.png
RewriteRule ^.*$ /var/www/maintenance/index.html [L]
Header Set Cache-Control "max-age=0, no-store"
CustomLog /var/log/apache2/my.website.com.access.log combined
ErrorLog /var/log/apache2/my.website.com.error.log
</VirtualHost>
Redirect source IP to VMs
On the vm vhost add/edit the following lines:
SetEnvIf X-Forwarded-For "^.*\..*\..*\..*" forwarded
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" forwarded
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined env=!forwarded
CustomLog ${APACHE_LOG_DIR}/access.log forwarded env=forwarded
And enable 2 modules:
a2enmod remoteip && a2enmod headers
Redirect except one IP
Note: This works only for apache 2.4 onwards. For older versions of apache find something around RewriteEngine.
<If "%{REMOTE_ADDR} !='my.ip.add.re.ss'">
Redirect 301 / https://my.website.com
</If>
Get client's real IP address
It is a well known problem that when you run an apache web server behind a proxy, you cannot get client real IP address. To solve this issue we just have to run next commands on each backend server:
sudo a2enmod remoteip sudo echo "RemoteIPHeader X-Forwarded-For" >> /etc/apache2/apache2.conf sudo sed -i "s/LogFormat \"%v:%p %h/LogFormat \"%v:%p %a/g" /etc/apache2/apache2.conf sudo sed -i "s/LogFormat \"%h/LogFormat \"%a/g" /etc/apache2/apache2.conf sudo apache2ctl graceful
How to reverse proxy WebSockets
Enable the websockets module if not already
a2enmod proxy_wstunnel
Add these lines before the ProxyPass lines
RewriteEngine On
RewriteCond %{HTTP:Upgrade} =websocket [NC]
RewriteRule /(.*) ws://172.17.16.17/$1 [P,L]
RewriteCond %{HTTP:Upgrade} !=websocket [NC]
RewriteRule /(.*) http://172.17.16.17/$1 [P,L]
