We are happy to announce that Vesta is back under active development as of 25 February 2024. We are working on v1 candidate and expect to engage more with the community over the coming months. We are committed to open source, and we encourage contributors to help us build the future of Vesta.
Got 10 VestaCP servers exploited
Re: Got 10 VestaCP servers exploited
sending email from my webserver (as always) but nothing , only can receive
Re: Got 10 VestaCP servers exploited
Ok, anyway thanks for your help
Re: Got 10 VestaCP servers exploited
n0x, if you havent so yet, you can check out this poll and fill in your infos there:
nextgi wrote: ↑Tue Apr 10, 2018 5:11 amHi Everyone,
We have put together a survey to help us better understand the general configuration in relation to some of the working theories. If you have suggestions to broaden the survey, please let us know.
https://goo.gl/forms/qXtzd6nZFrKNw7DN2
We greatly appreciate any input.
Re: Got 10 VestaCP servers exploited
Didn't see that - Have completed it with info from the first time and today.kobo1d wrote: ↑Tue Apr 10, 2018 7:47 pmn0x, if you havent so yet, you can check out this poll and fill in your infos there:
nextgi wrote: ↑Tue Apr 10, 2018 5:11 amHi Everyone,
We have put together a survey to help us better understand the general configuration in relation to some of the working theories. If you have suggestions to broaden the survey, please let us know.
https://goo.gl/forms/qXtzd6nZFrKNw7DN2
We greatly appreciate any input.
-
- Posts: 33
- Joined: Sat Jan 20, 2018 3:45 am
- Os: Debian 8x
- Web: apache + nginx
Re: Got 10 VestaCP servers exploited
For whom that not yet resolved this problem in DIGITAL OCEAN, here are my steps to resolved my own way without any reply from the support team. We do understand in this kind of situation they have a huge backlog in their ticketing system.
1. When your droplet got taken down, you only can access their console which is very slow and with limitation. No copy paste or keyboard shortcut, only typing allowed.
2. The first thing to do is try to clean the virus from the console. Follow carefully steps in this page:
https://admin-ahead.com/forum/server-se ... ts-trojan/
If you follow it correctly, It will stop virus reinfected. Note that the virus process name in top command will be vary and never be the same in each attack, so you have to perform you own search to any suspected files.
3. After try to clean the virus, make a snapshot. It will take approximately 30 minutes to finished due to the suspended status. Then clone the droplet.
4. Immediately create a DO firewall for the clone to stop all outbound traffic. With no outbound the droplet will not be taken down again if your first cleaning is not completed. This also give you full access via ssh and more time to perform cleaning and deep inspection.
5. After make sure the virus removed, open the outbound traffic and wait at least 1 hour. In my case I got taken down 2 times previously. Then make a backup
6. Create new droplet with same specification and install fresh vestacp with clamav. Apply the new security patch.
In my case I use debian 9.
7. SSH back to your clone droplet and copy the backup to the new droplet.
Then SSH to your new installed droplet and restore the backup
8. Tightening your new vestacp such as change ssh port, vesta default port, additional authentication for login, close remote mysql port, etc. Then resetting all your sites with new IP dns record.
9. Done! Your server should goes live again with no data losses.
1. When your droplet got taken down, you only can access their console which is very slow and with limitation. No copy paste or keyboard shortcut, only typing allowed.
2. The first thing to do is try to clean the virus from the console. Follow carefully steps in this page:
https://admin-ahead.com/forum/server-se ... ts-trojan/
If you follow it correctly, It will stop virus reinfected. Note that the virus process name in top command will be vary and never be the same in each attack, so you have to perform you own search to any suspected files.
3. After try to clean the virus, make a snapshot. It will take approximately 30 minutes to finished due to the suspended status. Then clone the droplet.
4. Immediately create a DO firewall for the clone to stop all outbound traffic. With no outbound the droplet will not be taken down again if your first cleaning is not completed. This also give you full access via ssh and more time to perform cleaning and deep inspection.
5. After make sure the virus removed, open the outbound traffic and wait at least 1 hour. In my case I got taken down 2 times previously. Then make a backup
Code: Select all
v-backup-user admin
In my case I use debian 9.
Code: Select all
apt-get update && apt-get upgrade
v-update-sys-vesta-all
Code: Select all
scp /backup/admin.date.tar new-server:/backup/
Code: Select all
v-restore-user admin admin.date.tar
9. Done! Your server should goes live again with no data losses.
Re: Got 10 VestaCP servers exploited
Yes it wasmxroute wrote: ↑Tue Apr 10, 2018 5:33 pmCan you confirm that this was a root level infection and did it appear just the same as the majority of the reports here?Nipsey wrote: ↑Tue Apr 10, 2018 4:57 pm
Yesterday I installed 0.9.8-20 on a fresh vultr centos 7.4 everything was fine the whole day until I ran yum update today, centos and vesta had updates so I accepted them and then the server was infected. Now I am afraid to run yum update after installing vesta. So I guess 0.9.8-20 is affected as well.
You can try this yourself, I don't know why yum update had vesta updates when I was already running version 0.9.8-20
Re: Got 10 VestaCP servers exploited
How?. There was no /etc/cron.hourly/gcc.shRevengeFNF wrote: ↑Tue Apr 10, 2018 5:56 pmYou did have Vesta at version 0.9.8-20. Now you received the update to vesta-nginx and vesta-php to those versions.Nipsey wrote: ↑Tue Apr 10, 2018 4:57 pmYesterday I installed 0.9.8-20 on a fresh vultr centos 7.4 everything was fine the whole day until I ran yum update today, centos and vesta had updates so I accepted them and then the server was infected. Now I am afraid to run yum update after installing vesta. So I guess 0.9.8-20 is affected as well.skid wrote: ↑Tue Apr 10, 2018 3:42 pmFirst of all, there was no reports about hacks on 0.9.8-20.
Please update your servers as soon as possible.
For those who are interested in technical details here is how authentication model looked like in previous releases:
- PHP script /api/index.php receives user password via POST request
- then this script writes user password to a tmp file (for example /tmp/tmp.cWdkwNbBrR)
This operation was needed to protect password from being hijacked via "ps auxf" command.
- Path to the file was then passed to the shell script v-check-user-password:
(v-check-user-password admin /tmp/tmp.cWdkwNbBrR)
- The script reads the content of /tmp/tmp.cWdkwNbBrR and calls sub process in order to generate hash based on the file content
hash=$($BIN/v-generate-password-hash $method $salt <<< $password)
We think that this part could allow for arbitrary code execution. Theoretically you could send something like
"password; cat /etc/passwd" to get the content of /etc/passwd. However we weren't able to bypass auth protection ourselves.
Here is what we did in the new release.
- The PHP process still receives unescaped password via POST
- Then instead of transmitting this password to the script it is now creates hash
- Then this hash is written into the tmp
This way code injected string like "password; cat /etc/passwd" converts to a harmless "7v8FlZefN7aQ9OoxGkR8lFHKejCxH9g64TQVVoRUuAObszO2hJy.CAs8ZG3JUtDKYQZNIZS61" sequence of characters which makes it impossible to inject anything.
You can try this yourself, I don't know why yum update had vesta updates when I was already running version 0.9.8-20
You probably already did have your server compromised before installing the updates.
Re: Got 10 VestaCP servers exploited
imperio, skid and other VESTA Stuff,
Please consider offering an option for certificate-based access to VESTA Panel!
When enabled, only the persons with the correct certificate installed in their browsers can be presented with VESTA Panel at https://[FQDN]:8083 Then they'll need to login as usual.
Here's why I think this is a good idea
Please consider offering an option for certificate-based access to VESTA Panel!
When enabled, only the persons with the correct certificate installed in their browsers can be presented with VESTA Panel at https://[FQDN]:8083 Then they'll need to login as usual.
Here's why I think this is a good idea
- Because as far as I understand, the attack vector isn't 100% known/reproducible
- Because it will add another strong layer of security
- Because a hacker won't login even if they've already stolen admin's credentials
- Because it bullet-proofs VESTA even if another such vulnerability is ever discovered
Last edited by Felix on Wed Apr 11, 2018 7:48 am, edited 1 time in total.
Re: Got 10 VestaCP servers exploited
Felix,
I'll send your suggestion to Serghey Rodin
I'll send your suggestion to Serghey Rodin