In my case, I was wondering why the virus was stored in this folder:
/tmp/systemd-private-72defd09721c49848b659469def3d414-httpd.service-4ZhWF5/tmp/
Following this explanation,
http://0pointer.de/blog/projects/security.html, so it looks like it was uploaded using Apache service (httpd.service) and not nginx + php-fpm (Vestacp service).
My Vestacp installation parameters were:
========
bash vst-install.sh --nginx no --apache yes --phpfpm no --named yes --remi yes --vsftpd yes --proftpd no --iptables yes --fail2ban no --quota yes --exim yes --dovecot yes --spamassassin yes --clamav yes --softaculous no --mysql yes --postgresql no --hostname xxxx.xxxxxxxx.com --email
[email protected]
========
This is the warning email that I received from my firewall (CSF Firewall):
========
Subject: lfd on xxxx.xxxxxx.com: Suspicious File Alert
Time: Wed Apr 4 06:30:07 2018 -0400
File: /tmp/systemd-private-72defd09721c49848b659469def3d414-httpd.service-4ZhWF5/tmp/update
Reason: Linux Binary
Owner: admin:admin (1002:1002)
Action: No action taken
========
Also I was wondering why the user and group were "admin:admin". And I think this only happens when:
1) The malicious code is executed via VestaCP service. I doubt it. Furthermore, in my case I had vestacp port service disabled in the firewall by default (8083). Only my fixed IP was allowed to reach it.
2) The malicious code is executed via hostname/IP service, like
http://xxx.xxx.xxx.xxx
So my conclusion is that one of the possibles vector attacks was using
http://xxx.xxx.xxx.xxx and NOT vestacp service.
The pity things are:
1) I have not records in apache logs of it. They are missing or deleted or never created. I see only a /webmail/ HEAD access (from a japanese IP) 3 seconds before that "update" file was created in that "tmp" folder:
========
119.82.29.17 - - [04/Apr/2018:06:25:38 -0400] "HEAD /webmail/ HTTP/1.0" 200 - "
http://www.google.com" "Googlebot/2.1 (+
http://www.google.com/bot.html)
========
2) I am not sure if this is vestacp API related or not. Not sure if there is a way to access to API calls from Apache (for websites) instead ngnix from port 8083 (for vesta).
I hope this information helps to find the vector attack.
In order to remove it, I followed these steps:
======= DO NOT PASTE/COPY. INVESTIGATE YOUR CASE FIRST
chmod 0000 /lib/libudev.so
rm -f /lib/libudev.so
chattr +i /lib/
chmod 0000 /etc/cron.hourly/gcc.sh
rm -f /etc/cron.hourly/gcc.sh
========
Then I have looked for strange names in /usr/bin (because I had 4 virus entries there) and then I have removed them in this way:
======= DO NOT PASTE/COPY. INVESTIGATE YOUR CASE FIRST
find /etc -name '*uwdvmfnywd*'| xargs rm -f
find /etc -name '*eijaxxglfs*'| xargs rm -f
find /etc -name '*ffoljgwzyq*'| xargs rm -f
find /etc -name '*update*' | grep "rc.d" | xargs rm -f
rm -f /usr/bin/uwdvmfnywd
rm -f /usr/bin/eijaxxglfs
rm -f /usr/bin/ffoljgwzyq
rm -f /usr/bin/update
========
Then I have killed the virus process (if any active at that moment):
======== DO NOT PASTE/COPY. INVESTIGATE YOUR CASE FIRST
pkill uwdvmfnywd
========
Take in consideration that while you are doing this, the virus is creating new entries in /usr/bin and everything, but it won't be able to run again because the "chattr +i /lib/" previous command you issued. So be calm, you should remove them again later. Now continue reading...
Then you should edit the crontab:
======== DO NOT PASTE/COPY. INVESTIGATE YOUR CASE FIRST
vi /etc/crontab
========
and remove or comment the gcc.sh entry.
Then, in my case, I had to kill "sleep" pid process that was causing to reproduce the virus every 5 seconds.
So I did something like:
======= DO NOT PASTE/COPY. INVESTIGATE YOUR CASE FIRST
ps -auxf
=======
and I identified the "sleep" process at the end of the list (could be upper, but not too much) and the I killed it:
======= DO NOT PASTE/COPY. INVESTIGATE YOUR CASE FIRST
kill -9 <pid>
=======
Once killed I have repeated once this process from the beginning skipping the "libudev" part because it was not created again.
When you are done remember to change attributes to /lib/ folder:
======= DO NOT PASTE/COPY. INVESTIGATE YOUR CASE FIRST
chattr -i /lib/
=======
And check again the /etc/cron-hourly folder just in case there are still remaining virus running or not.
Also I had to reload my systemd daemon because the virus was creating services entries there that automatically will be removed when reloading them in this way:
======= DO NOT PASTE/COPY. INVESTIGATE YOUR CASE FIRST
systemctl daemon-reload
======
I had not to reboot the machine using this procedure and I am not noticing further problems since them (two days ago).
Hope it helps.