Page 1 of 1

[Howto] site and mysql failover using VRRP

Posted: Sat Dec 09, 2017 1:24 am
by vanderheyde
Hi,

I found the need for some redundancy for the websites hosted on vesta. The idea is a failover for only the web content and the underlying databases. On the backup host, the vesta panel will be disabled to prevent any changes being done to DNS when in failover mode.

This is what I thought to be the easiest way. I'm by no means a linux guru. This guide is for a completely clean install, although it is possible to add a backup to an existing installation (just a little more work).

for this guide, i'm using the following ip addressed:

Host A (primary):
eth0 123.123.123.100
eth1 192.168.1.100

Host B (backup):
eth0 123.123.123.101
eth1 192.168.1.101

VRRP ip:
123.123.123.102

1. Install your 2 servers, preferrably with 2 interfaces. 1 public, and one on a private network for data and mysql replication.

2. Install keepalived on both servers
SpoilerShow
- make sure your 2 servers are on the same public subnet
- add the following line in /etc/sysctl.conf to make linux able to respond on an IP that is not configured in /etc/network/interfaces
net.ipv4.ip_nonlocal_bind=1
- activate it by running sysctl -p
- install keepalived: apt-get install keepalived
- sample config:

Code: Select all

vrrp_instance vrrp_vesta {
        interface eth0
        state MASTER #change to BACKUP for backup host
        virtual_router_id 20 # make sure both hosts have the same ID
        priority 100 # lower this for the backup host

        authentication {
                auth_type AH #encrypts the password
                auth_pass rN6nabz6udURPfdncM3ouk6VKDL
        }

        virtual_ipaddress {
               123.123.123.102 # the shared IP address
        }
}
- start the service: service keepalived start
3. Install VestaCP on both servers
SpoilerShow
# Connect to your server as root via SSH
ssh [email protected]
# Download installation script
curl -O http://vestacp.com/pub/vst-install.sh
# Run it
bash vst-install.sh
I used 2 different hostnames during the installation
4. Allow communication between hosts in iptables
SpoilerShow
- on both servers, edit /etc/iptables.rules and add the following right under the ":vesta - [0:0]" line:

Code: Select all

on server A, add: -A INPUT -s 192.168.1.101 -j ACCEPT
on server B, add: -A INPUT -s 192.168.1.100 -j ACCEPT
- activate it: iptables-restore /etc/iptables.rules
5. Install Rsync on the backup server
SpoilerShow
- apt-get install rsync
6. Create SSH key
SpoilerShow
- create the RSA key pair: ssh-keygen -t rsa
- copy the public key to the backup server: ssh-copy-id [email protected]
7. Install lsyncd on the primary server
SpoilerShow
- I'm using lsyncd to synchronize the data between the primary and the backup server. It uses Rsync to transfer the files.
- apt-get install lsyncd
- it does not create its log and config dir & file automatically, so we need to create it:
- mkdir /var/log/lsyncd
- mkdir /etc/lsyncd
- nano /etc/lsyncd/lsyncd.conf.lua
- sample config:

Code: Select all

settings {
        logfile = "/var/log/lsyncd/lsyncd.log",
        statusFile = "/var/log/lsyncd/lsyncd.status"
}

sync {
        default.rsyncssh,
        delete = true,
        source = "/home/",
        host = "192.168.1.101",
        targetdir = "/home",
        rsync = {
        perms = true,
        owner = true,
        group = true
        }
}
8. Configure mysql replication:
SpoilerShow
both hosts need to both master and slave in this replication. This way, when for example, a new user registers on your website while in failover mode, the user still exists when you revert to the primary server. I've done my configuration using the 'replication' tab in phpmyadmin. Make sure the make use of the private IP addresses.
host A needs to be Master of B
host A needs to be Slave of B
host B needs to be Master of A
host B needs to be Slave of A
9. add the shared IP in both vestaCP interfaces
SpoilerShow
We need the shared IP to be available in the control panel (new domains need to be added under that IP), and the apache config files need to be updated. Adding the IP in vesta makes it do all the heavy lifting. It will, however, also add the IP in the linux networking config. We'll delete that.
- in your control panel, go to 'IP' and click the '+' to add one.
- enter the shared IP and its subnet and the interface, and click add. Do this for both servers.
- Vesta has now updated the necessary config files with the extra IP. We now have to edit /etc/network/interfaces and delete (or comment) the following block:

Code: Select all

# Added by vesta
auto eth0:0
iface eth0:0 inet static
address 123.123.123.102
netmask 255.255.255.0
- restart the networking service: service networking restart
10. Disable the VestaCP interface on the backup server
SpoilerShow
To prevent any unreplicated changes being done via the control panel while in backup mode, we're going to replace this with a generic error page saying this function is not available in failover mode:

Code: Select all

	-mv /usr/local/vesta/web/ /usr/local/vesta/web.old (keep the data in case we ever need it)
	-mkdir /usr/local/vesta/web/error
	-nano /usr/local/vesta/web/error/index.html >>>> create a nice error page, I'll leave the html up to you
	-mkdir /usr/local/vesta/web/error/404
	-ln -s /usr/local/vesta/web/error/404/index.html /usr/local/vesta/web/error/index.html
11. Set crontab to copy users and groups
SpoilerShow
To make sure a newly added user in VestaCP is present on the backup server, we copy /etc/passwd & /etc/group every hour (you can change the frequency to match your needs). To include the apache & nginx config files, we need to copy /etc/apache2/conf.d/vesta.conf and /etc/nginx/conf.d/vesta.conf as well.
- edit the crontab on the primary server: crontab -e
- add the following lines:

Code: Select all

0 * * * * /usr/bin/rsync /etc/passwd [email protected]:/etc/ > /var/log/crontab.log
0 * * * * /usr/bin/rsync /etc/group [email protected]:/etc/ > /var/log/crontab.log
0 * * * * /usr/bin/rsync /etc/apache2/conf.d/vesta.conf [email protected]:/etc/apache2/conf.d/ > /var/log/crontab.log
0 * * * * /usr/bin/rsync /etc/nginx/conf.d/vesta.conf [email protected]:/etc/nginx/conf.d/ > /var/log/crontab.log
12. Set crontab to restart services to include the latest config files
SpoilerShow
- edit the crontab on the backup server: crontab -e
- add the following lines:

Code: Select all

0 */2 * * * /etc/init.d/apache2 restart > /var/log/crontab.log
0 */2 * * * /etc/init.d/nginx restart > /var/log/crontab.log

Important details when creating a new domain
- new domains must be added on the shared IP to make use of the failover
- make sure the set the MX record to the direct IP of the primary server (NOT the shared IP)


Feel free to give me tips or observations for improvement.

vanderheyde