Page 31 of 55

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 10:16 am
by imperio
Will be checked

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 10:40 am
by fedekrum
wildwolf wrote:
Mon Apr 09, 2018 9:59 am
Judging by audit.log's I have from several infected servers, it seems to me that it is not VestaCP that was compromised but its repository / repositories.

For example,

Code: Select all

# ausearch  -m USER_CMD -i | grep -v -- '----' | awk '{print $10}' | sort -u
cmd=-bash
cmd=/usr/local/vesta/bin/v-add-firewall-rule
cmd=/usr/local/vesta/bin/v-add-letsencrypt-domain
cmd=/usr/local/vesta/bin/v-add-user
cmd=/usr/local/vesta/bin/v-add-web-domain
cmd=/usr/local/vesta/bin/v-backup-users
cmd=/usr/local/vesta/bin/v-change-user-language
cmd=/usr/local/vesta/bin/v-change-web-domain-tpl
cmd=/usr/local/vesta/bin/v-check-user-password
cmd=/usr/local/vesta/bin/v-delete-domain
cmd=/usr/local/vesta/bin/v-delete-mail-domain
cmd=/usr/local/vesta/bin/v-list-databases
cmd=/usr/local/vesta/bin/v-list-firewall
cmd=/usr/local/vesta/bin/v-list-mail-account
cmd=/usr/local/vesta/bin/v-list-mail-accounts
cmd=/usr/local/vesta/bin/v-list-mail-domain
cmd=/usr/local/vesta/bin/v-list-mail-domains
cmd=/usr/local/vesta/bin/v-list-sys-config
cmd=/usr/local/vesta/bin/v-list-sys-ips
cmd=/usr/local/vesta/bin/v-list-sys-languages
cmd=/usr/local/vesta/bin/v-list-sys-users
cmd=/usr/local/vesta/bin/v-list-user
cmd=/usr/local/vesta/bin/v-list-user-backups
cmd=/usr/local/vesta/bin/v-list-user-favourites
cmd=/usr/local/vesta/bin/v-list-user-ips
cmd=/usr/local/vesta/bin/v-list-user-notifications
cmd=/usr/local/vesta/bin/v-list-user-packages
cmd=/usr/local/vesta/bin/v-list-users
cmd=/usr/local/vesta/bin/v-list-users-stats
cmd=/usr/local/vesta/bin/v-list-web-domain
cmd=/usr/local/vesta/bin/v-list-web-domains
cmd=/usr/local/vesta/bin/v-list-web-domain-ssl
cmd=/usr/local/vesta/bin/v-list-web-stats
cmd=/usr/local/vesta/bin/v-list-web-templates
cmd=/usr/local/vesta/bin/v-list-web-templates-backend
cmd=/usr/local/vesta/bin/v-restart-web
cmd=/usr/local/vesta/bin/v-update-letsencrypt-ssl
cmd=/usr/local/vesta/bin/v-update-sys-queue
cmd=/usr/local/vesta/bin/v-update-sys-rrd
cmd=/usr/local/vesta/bin/v-update-sys-vesta-all
cmd=/usr/local/vesta/bin/v-update-user-stats
(bash - my login).

I checked arguments to every command, they were all OK, no traces of injection attempts etc.

This makes me think that infection took place from root account (by default, auditd does not log commands run by root): if VestaCP were vulnerable, I think I would see something suspicious either in audit.log or in secure / auth.log. I didn't, therefore either the infection took place long ago (I have logs back to Mar 13) -- but in this case someone should have noticed something strange -- libudev.so is detected even by ClamAV, or the infection was performed with root privileges. If so, need to find what was in common between the affected servers. So far, I know that they all run VestaCP (I heard of infected Debian servers). For infection to happen via root account, it either needs to be downloaded and run by the user itself (does anybody verify the checksum of the Vesta installer?), or come from the repository. This can explain the fact that there are no new infections — old (compromised?) files were overwritten with new ones.

I would recommend the VestaCP team to take a look at their infrastructure and see if there are any signs of compromise / unauthorized access.
Someone said in this post that was hacked even not having vesta service in default port 8083. It makes sense that may be its done within the root account.

Check this, there are many people not able to install fron scratch.
https://www.digitalocean.com/community/ ... l-8th-2018

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 11:15 am
by kobo1d
haha and i flamed and blamed my server provider for getting hacked again..... (something happend last year to them)
but no, its vesta that caused it. i got my server infected to by XORDDos Trojan. infecttion happend as root.
no logs shows anything. my ssh is secured with pubkeys and my vestacp web-panel is secured by iptables to my ip only.
i cleaned the trojan 3 days ago and updated vestacp and did not get re-infected the past 12 hours....

edit:

fresh debian 9 install, apache2 frontend
chkrootkit, clamav and rkhunter all 3 saied my system is clear and couldnt find the trojan
a backdoor connection was opend from my servers port 39472 to 209.141.61.140:smtp (25)
parent id of the main virus body was 1 (systemd)

if any of you on debain need assistance with removing the trojan, you can pm me.

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 11:30 am
by vesta_mtl
skid wrote:
Sun Apr 08, 2018 7:05 am
Here is what we know so far:
1. The first wave happened on April 4. Servers were infected with /etc/cron.hourly/gcc.sh
2. It was an automated hack
3. CentOS, Debian, Ubuntu all distros are affected it's platform independent
4. We didn't find any traces in vesta and system logs yet
5. On April 7 infected servers started to DDoS remote hosts using /usr/lib/libudev.so.

What you can do:
The best way to stay safe is to temporary disable vesta web service

Code: Select all

service vesta stop

Code: Select all

systemctl disable vesta
or limit access to port 8083 using firewall

What we are doing:
Few users provided us with root access to their servers. We are investigating what happened. We also launched a couple honeypots in order to get full picture of the hack.
Thanks for all the hard work. It seems my Vesta was already updated. I am on CentOS DigitalOcean and when I try doing yum update, there's no Vesta update. When I try using Vesta GUI, there is no update (see image https://puu.sh/zZsbV/3509057e9c.png).

Is this already at the latest version? If so, I didn't do this, so perhaps DigitalOcean did?

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 11:42 am
by snevsky
vesta_mtl wrote:
Mon Apr 09, 2018 11:30 am


Is this already at the latest version? If so, I didn't do this, so perhaps DigitalOcean did?
Yes, you have 20 release

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 11:54 am
by pipoy
Messiah wrote:
Mon Apr 09, 2018 8:21 am
Dear Community,

does somebody noticed any hacks since ?
Either this was a one-time mass exploiting or they never return to the same servers. Honeypots are already installed for all servers, but my own code that was technically the same as provided here.

My observation, once I removed the viruses, it stopped replicating itself. I even forgot to update 2 out of 3 of my servers and it never recurred.
Defaults port as well.

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 11:55 am
by baoang
fedekrum wrote:
Mon Apr 09, 2018 10:14 am
I have just tried to make a new vesta server on Digital Ocean, Ubuntu 16 and got these errors during install.

Hit:1 http://apt.vestacp.com/xenial xenial InRelease
Hit:2 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:3 https://repos.sonar.digitalocean.com/apt main InRelease
Hit:4 http://nginx.org/packages/mainline/ubuntu xenial InRelease
Hit:5 http://nyc2.mirrors.digitalocean.com/ubuntu xenial InRelease
Hit:6 http://nyc2.mirrors.digitalocean.com/ubuntu xenial-updates InRelease
Hit:7 http://nyc2.mirrors.digitalocean.com/ubuntu xenial-backports InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package vesta-php
E: Unable to locate package vesta-ioncube
E: Unable to locate package vesta-softaculous
Error: apt-get install failed

Do you think it has to do with this hack or the patch released?
it happened to me twice when I made install at Bandwagon that ioncube was not properly installed. Seems sort of unable to locate files or sth. But the whole installation went to the final stage. Guess it might be someone hacked in?

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 11:57 am
by pipoy
fedekrum wrote:
Mon Apr 09, 2018 10:40 am
wildwolf wrote:
Mon Apr 09, 2018 9:59 am
Judging by audit.log's I have from several infected servers, it seems to me that it is not VestaCP that was compromised but its repository / repositories.

For example,

Code: Select all

# ausearch  -m USER_CMD -i | grep -v -- '----' | awk '{print $10}' | sort -u
cmd=-bash
cmd=/usr/local/vesta/bin/v-add-firewall-rule
cmd=/usr/local/vesta/bin/v-add-letsencrypt-domain
cmd=/usr/local/vesta/bin/v-add-user
cmd=/usr/local/vesta/bin/v-add-web-domain
cmd=/usr/local/vesta/bin/v-backup-users
cmd=/usr/local/vesta/bin/v-change-user-language
cmd=/usr/local/vesta/bin/v-change-web-domain-tpl
cmd=/usr/local/vesta/bin/v-check-user-password
cmd=/usr/local/vesta/bin/v-delete-domain
cmd=/usr/local/vesta/bin/v-delete-mail-domain
cmd=/usr/local/vesta/bin/v-list-databases
cmd=/usr/local/vesta/bin/v-list-firewall
cmd=/usr/local/vesta/bin/v-list-mail-account
cmd=/usr/local/vesta/bin/v-list-mail-accounts
cmd=/usr/local/vesta/bin/v-list-mail-domain
cmd=/usr/local/vesta/bin/v-list-mail-domains
cmd=/usr/local/vesta/bin/v-list-sys-config
cmd=/usr/local/vesta/bin/v-list-sys-ips
cmd=/usr/local/vesta/bin/v-list-sys-languages
cmd=/usr/local/vesta/bin/v-list-sys-users
cmd=/usr/local/vesta/bin/v-list-user
cmd=/usr/local/vesta/bin/v-list-user-backups
cmd=/usr/local/vesta/bin/v-list-user-favourites
cmd=/usr/local/vesta/bin/v-list-user-ips
cmd=/usr/local/vesta/bin/v-list-user-notifications
cmd=/usr/local/vesta/bin/v-list-user-packages
cmd=/usr/local/vesta/bin/v-list-users
cmd=/usr/local/vesta/bin/v-list-users-stats
cmd=/usr/local/vesta/bin/v-list-web-domain
cmd=/usr/local/vesta/bin/v-list-web-domains
cmd=/usr/local/vesta/bin/v-list-web-domain-ssl
cmd=/usr/local/vesta/bin/v-list-web-stats
cmd=/usr/local/vesta/bin/v-list-web-templates
cmd=/usr/local/vesta/bin/v-list-web-templates-backend
cmd=/usr/local/vesta/bin/v-restart-web
cmd=/usr/local/vesta/bin/v-update-letsencrypt-ssl
cmd=/usr/local/vesta/bin/v-update-sys-queue
cmd=/usr/local/vesta/bin/v-update-sys-rrd
cmd=/usr/local/vesta/bin/v-update-sys-vesta-all
cmd=/usr/local/vesta/bin/v-update-user-stats
(bash - my login).

I checked arguments to every command, they were all OK, no traces of injection attempts etc.

This makes me think that infection took place from root account (by default, auditd does not log commands run by root): if VestaCP were vulnerable, I think I would see something suspicious either in audit.log or in secure / auth.log. I didn't, therefore either the infection took place long ago (I have logs back to Mar 13) -- but in this case someone should have noticed something strange -- libudev.so is detected even by ClamAV, or the infection was performed with root privileges. If so, need to find what was in common between the affected servers. So far, I know that they all run VestaCP (I heard of infected Debian servers). For infection to happen via root account, it either needs to be downloaded and run by the user itself (does anybody verify the checksum of the Vesta installer?), or come from the repository. This can explain the fact that there are no new infections — old (compromised?) files were overwritten with new ones.

I would recommend the VestaCP team to take a look at their infrastructure and see if there are any signs of compromise / unauthorized access.
Someone said in this post that was hacked even not having vesta service in default port 8083. It makes sense that may be its done within the root account.

Check this, there are many people not able to install fron scratch.
https://www.digitalocean.com/community/ ... l-8th-2018

I confirm that one of my servers has a different port and it was even hacked

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 11:58 am
by AnimusAstralis
It seems that my CP autoupdated and now I can't access web UI. All services are active. What should I do?

Re: Got 10 VestaCP servers exploited

Posted: Mon Apr 09, 2018 12:01 pm
by snakom23
i did new update correctly on debian 8, but in debian 9 update not went well. Problem with debian 9.