Got 10 VestaCP servers exploited

[pipoy]

Im trying to update my vesta but for some of my instances but it's failing.

I've found out that yum update is failing also

Code: Select all

Loaded plugins: fastestmirror
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=stock error was
14: curl#6 - "Could not resolve host: mirrorlist.centos.org; Unknown error"


 One of the configured repositories failed (Unknown),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=<repoid> ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable <repoid>
        or
            subscription-manager repos --disable=<repoid>

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

Cannot find a valid baseurl for repo: base/7/x86_64

Ive done changing DNS to google dns. Did not work
I started DHCLIENT. Did not work.


UPDATE: I stopped all vesta applications and stopped vesta itself. run the update and it's working.

Started all applications and vesta, yum update has worked as expected again

[aldoanizio]

For those wich use Vultr 16.04 based distros and will upgrade using apt-get I do suggest stop PHP-FMP and Nginx services before run the processes.

I have 3 droplets and in the first one I've updated the upgrade process failed at restart PHP-FPM service.

So in second and third droplet I've disabled Nginx and PHP-FPM before run upgrade command.

[really]

The fix has been released just now!
As usually there are 3 ways to update your server:

1. Via web interface
- Login as admin
- Go to updates tab
- Click un update button under vesta package

2. Via package manager
- SSH as root to your server
- yum update / apt-get update && apt-get upgrade

3. Via GitHub
- SSH as root
- Install git / yum install git /apt-get install git
- Then run following commands

Code: Select all

cd $(mktemp -d)
git clone git://github.com/serghey-rodin/vesta.git
/bin/cp -rf vesta/* /usr/local/vesta/

Some information about this indecent. We still don't have working exploit for previous version. But we know for sure that the vector of attack was through a potentially unsecure password check method. Therefore we have completely rewrite password auth function. It's bullet proof now!

Please upgrade your servers as soon as possible.

Did you forget to replace packages in repos? Debian 8.1 can't find anything outside of -nginx.

[JimKarvo]

We need to format the servers and install again the vests cp from scratch, or the security update solves the problem that exists?

[snakom23]

i can see the new update in "update section" in controll panel, but when click update, not happen nothing

[StudioMaX]

We need to format the servers and install again the vests cp from scratch, or the security update solves the problem that exists?

Update only solves the security problem with authentication. If the server has already been infected, then it must either be reinstalled, or you need to manually cure it (if you are not afraid that the virus may be hiding somewhere else).
https://superuser.com/questions/877896/ ... 24#1004724

[mehargags]

Just an observation,
I upgraded 2 Debian 8 servers and it upgraded Vesta fine but when I tried to upgrade a Debian 9 server ... it shows v19 as outdated but when I update it doesn't fetch anything. Tried with v-update-sys-vesta-all as well as apt upgrade, but it says no updates found. It looks like Debian 9 repos are still left out.

[Messiah]

Dear Community,

does somebody noticed any hacks since ?
Either this was a one-time mass exploiting or they never return to the same servers. Honeypots are already installed for all servers, but my own code that was technically the same as provided here.

Also: I have only one server hacked and it was the server installed 2 weeks ago with fresh VestaCP. I have many other servers (use Vesta everywhere), sometimes with enough old panel versions and none of them was hacked. No IPs were hidden e.g. behind CloudFlare. Does it means only latest Vesta panel build was vulnerable?

P.S. could somebody please add fix for MySql Controluser bug into the official repo just to work from the box and not using third party scripts.

[wildwolf]

Judging by audit.log's I have from several infected servers, it seems to me that it is not VestaCP that was compromised but its repository / repositories.

For example,

Code: Select all

# ausearch  -m USER_CMD -i | grep -v -- '----' | awk '{print $10}' | sort -u
cmd=-bash
cmd=/usr/local/vesta/bin/v-add-firewall-rule
cmd=/usr/local/vesta/bin/v-add-letsencrypt-domain
cmd=/usr/local/vesta/bin/v-add-user
cmd=/usr/local/vesta/bin/v-add-web-domain
cmd=/usr/local/vesta/bin/v-backup-users
cmd=/usr/local/vesta/bin/v-change-user-language
cmd=/usr/local/vesta/bin/v-change-web-domain-tpl
cmd=/usr/local/vesta/bin/v-check-user-password
cmd=/usr/local/vesta/bin/v-delete-domain
cmd=/usr/local/vesta/bin/v-delete-mail-domain
cmd=/usr/local/vesta/bin/v-list-databases
cmd=/usr/local/vesta/bin/v-list-firewall
cmd=/usr/local/vesta/bin/v-list-mail-account
cmd=/usr/local/vesta/bin/v-list-mail-accounts
cmd=/usr/local/vesta/bin/v-list-mail-domain
cmd=/usr/local/vesta/bin/v-list-mail-domains
cmd=/usr/local/vesta/bin/v-list-sys-config
cmd=/usr/local/vesta/bin/v-list-sys-ips
cmd=/usr/local/vesta/bin/v-list-sys-languages
cmd=/usr/local/vesta/bin/v-list-sys-users
cmd=/usr/local/vesta/bin/v-list-user
cmd=/usr/local/vesta/bin/v-list-user-backups
cmd=/usr/local/vesta/bin/v-list-user-favourites
cmd=/usr/local/vesta/bin/v-list-user-ips
cmd=/usr/local/vesta/bin/v-list-user-notifications
cmd=/usr/local/vesta/bin/v-list-user-packages
cmd=/usr/local/vesta/bin/v-list-users
cmd=/usr/local/vesta/bin/v-list-users-stats
cmd=/usr/local/vesta/bin/v-list-web-domain
cmd=/usr/local/vesta/bin/v-list-web-domains
cmd=/usr/local/vesta/bin/v-list-web-domain-ssl
cmd=/usr/local/vesta/bin/v-list-web-stats
cmd=/usr/local/vesta/bin/v-list-web-templates
cmd=/usr/local/vesta/bin/v-list-web-templates-backend
cmd=/usr/local/vesta/bin/v-restart-web
cmd=/usr/local/vesta/bin/v-update-letsencrypt-ssl
cmd=/usr/local/vesta/bin/v-update-sys-queue
cmd=/usr/local/vesta/bin/v-update-sys-rrd
cmd=/usr/local/vesta/bin/v-update-sys-vesta-all
cmd=/usr/local/vesta/bin/v-update-user-stats

(bash - my login).

I checked arguments to every command, they were all OK, no traces of injection attempts etc.

This makes me think that infection took place from root account (by default, auditd does not log commands run by root): if VestaCP were vulnerable, I think I would see something suspicious either in audit.log or in secure / auth.log. I didn't, therefore either the infection took place long ago (I have logs back to Mar 13) -- but in this case someone should have noticed something strange -- libudev.so is detected even by ClamAV, or the infection was performed with root privileges. If so, need to find what was in common between the affected servers. So far, I know that they all run VestaCP (I heard of infected Debian servers). For infection to happen via root account, it either needs to be downloaded and run by the user itself (does anybody verify the checksum of the Vesta installer?), or come from the repository. This can explain the fact that there are no new infections — old (compromised?) files were overwritten with new ones.

I would recommend the VestaCP team to take a look at their infrastructure and see if there are any signs of compromise / unauthorized access.

[fedekrum]

I have just tried to make a new vesta server on Digital Ocean, Ubuntu 16 and got these errors during install.

Hit:1 http://apt.vestacp.com/xenial xenial InRelease
Hit:2 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:3 https://repos.sonar.digitalocean.com/apt main InRelease
Hit:4 http://nginx.org/packages/mainline/ubuntu xenial InRelease
Hit:5 http://nyc2.mirrors.digitalocean.com/ubuntu xenial InRelease
Hit:6 http://nyc2.mirrors.digitalocean.com/ubuntu xenial-updates InRelease
Hit:7 http://nyc2.mirrors.digitalocean.com/ubuntu xenial-backports InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package vesta-php
E: Unable to locate package vesta-ioncube
E: Unable to locate package vesta-softaculous
Error: apt-get install failed

Do you think it has to do with this hack or the patch released?