Vesta Control Panel - Forum

Hi Guy's,

I am about to try and scale out Vesta horizontally across several nodes.
DB and /home should not be an issue, my concern however is with /usr/local/vesta.

All the nodes need access to the same user config files and I am thinking about a centralized data store like ceph/gluster to hold both /usr/local/vesta and /home.

Do you see any issue's ?

I think it is possible but not easy.

Thinks to think about:
- Different IPs
- Backups

Thanks for your input,

ip's will not be an issue due to the way I am going to externally load balance across nodes.
Backups are already set to use /home/backups with /home being mounted to gluster on a separate volume.

My concern is with /usr/local/vesta and any other user data that needs to be replicated across nodes.

I'm pretty sure you can't just sync /home or /usr/local/vesta or /etc - simply it will not work - for many reasons.

Instead of it - you can sync (via gluster) only public_html/ folders (each of them), and mysql databases separately (via mysql replication) - and simply add the same domain on both servers via VestaCP, and also create the same databases on both servers.

Basicaly, sync sites like you will do it without VestaCP.
You can not sync parts of the VestaCP because it contains different data for each server - for example - conf files binds different IP address.

I'm also in agreement with @dpeca. I'm not sure if you realize that the problem is not about share drive or rsync of /usr/local/vesta but about /home and /etc. I don't believe that there is anything in /usr/local/vesta that is machine specific. Just think of VESTA as being just like any other app. Your problem will be sync of user/service permission and configuration across machines. Vesta make changes to the system when you interface with the UI. You need to, somehow, sync either the config or commands across machines.

On the other hand, if you're looking to scale some simple Application, then you maybe looking at it the wrong way. It really just has to do with your APP requirement. If your APP is very simple, then you can horizontally scale anywhere. You easiest (not the cheapest) solution is Amazon Web Services (AWS). You're next solution would be with Dockerized containers.

Finally, it's not that you can't do with VESTA. You have to keep your installation to very small and make very little changes. That way, all you have to do is rsync /home folder to keep your app data in sync. This is assuming you already hosted your database somewhere else. You don't even need the /usr/local/vesta sync at all. But then you're using it wrong and it's probably better to go with the Docker solution.

/usr/local/vesta/data/users contains some data for domains, dns, mail, etc...
but again, you will make a mess if you try to sync it, because it contains server IPs and stuff related to /etc files...

Generally, there is no way to sync Vesta itself.
But, as I said, you can add the same domains and the same databases to both servers (or many servers), and then you can sync their public_html/ folders (via GlusterFS or via rsync) and sync mysql databases via mysql replication.

https://www.digitalocean.com/community/ ... tu-servers

https://www.digitalocean.com/community/ ... eplication

Here is how i was able to do a scale out and keep the website data in sync.

I was able to do this with the following servers and will label them accordingly.

1. HAProxy

2. 1 dedicated Vesta server designed for remote sql

3. 4 Vesta Web Servers

4. Windows with ftp program providing sync abilities between the servers using FTP.

HA Proxy
Was configured not as round robin, but a static round robin. This provided a secure and constant connection to that specific vesta webserver if users were accessing a website that required a login, example: A CRM Platform or social media site. If not configured as a Static Round Robin, each time a user clicks another link he or she will be forced to login, and login and login again and may not be able to login.

Static Round Robin directs that user to 1 specific server until the session is closed or until cache time set with HAProxy has expired.

All servers have the same content on them under /public_html/ and your code should be written with the proper SQL authentication and database information to connect to the Remote SQL vesta server.

Windows server,
I cant remember off the top of my head what program i used before that did this, though i had multiple tabs with multiple connections between the servers and set to a "sync in round robin" formation. If user uploads an image to server 1, the windows server ftp program recognized this after 5 seconds, and would copy that image to server 2, then from server 2 to server 3, from server 3 to server 4, and so on. (Yes this was made possible using a windows pc.)

If your considering Using SSL encryption, it is ideal to NOT INSTALL SSL with letsencrypt "ON THE VESTACP WEBSERVERS". But to install letsencrypt on HAProxy.

As HAProxy is a transparent proxy server and relays the data back and forth between the VestaCP WebServers, All WebServers can ideally run under the standard port 80, and HAProxy can run port 8080 or 443 with SSL. The end user accessing example .com will see the SSL Certificate and have no idea that HAProxy is the host nore accessing the site on WebServer 3.

Hopefully this will help you out in your journey to scaling out your system!
This was a major challenge for me to break and make happen with little to no documentation. While this is not the exact target you are looking for as far as scaling out servers, maybe this super long comment can give you some ideas that you may not have had prior to your work and making what you got going on to happen.

Do keep in mind all of this did NOT involve using the vesta cluster tools that came with VestaCP. They were all installed as stand alone servers, then made as a cluster using HA Proxy.