We are happy to announce that Vesta is back under active development as of 25 February 2024. We are working on v1 candidate and expect to engage more with the community over the coming months. We are committed to open source, and we encourage contributors to help us build the future of Vesta.
Strange memory consumption
Strange memory consumption
Hi guys.
I have been experiencing an unusually high memory consumption since the end of last week. That's making for example MariaDB to block continuously. I don't know if it is just me.
The 10 most consuming processes are the following using the command
Is anyone else having the same problem or do you see anything suspicious in these processes?
Thanks in advance.
EDIT:
After inspecting /var/log/mariadb/mariadb.log I can see the following messages:
The list continues, there are hundred of messages like those. Am I infected?
I have been experiencing an unusually high memory consumption since the end of last week. That's making for example MariaDB to block continuously. I don't know if it is just me.
The 10 most consuming processes are the following using the command
Code: Select all
ps aux --sort=-%mem | awk 'NR<=10{print $0}'
Code: Select all
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
mysql 31775 0.3 5.1 1076200 193960 ? Sl Feb23 7:04 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mariadb/mariadb.log --pid-file=/var/run/mariadb/mariadb.pid --socket=/var/lib/mysql/mysql.sock
clam 9053 106 3.5 233424 135408 ? Rs 21:08 0:02 /usr/sbin/clamd -c /etc/clamd.conf
root 7470 0.0 1.8 288196 69004 ? S 04:48 0:00 spamd child
root 7469 0.0 1.7 285364 66624 ? Ss 04:48 0:35 /usr/bin/spamd --pidfile /var/run/spamd.pid -d -c -m5 -H
root 7471 0.0 1.7 285364 66160 ? S 04:48 0:00 spamd child
named 2991 0.0 1.4 167800 54020 ? Ssl Feb23 0:22 /usr/sbin/named -u named -c /etc/named.conf
apache 15029 0.0 1.3 707172 51056 ? S 03:39 0:18 /usr/sbin/httpd -DFOREGROUND
apache 22241 0.0 1.3 706052 50172 ? S 18:03 0:03 /usr/sbin/httpd -DFOREGROUND
admin 7530 0.0 1.3 409292 50140 ? S 10:29 0:03 /usr/bin/php73-cgi
Thanks in advance.
EDIT:
After inspecting /var/log/mariadb/mariadb.log I can see the following messages:
Code: Select all
190130 9:17:06 [Warning] IP address '61.164.242.180' has been resolved to the host name '180.242.164.61.broad.ls.zj.dynamic.163data.com.cn', which resembles IPv4-address itself.
190130 12:01:05 [Warning] IP address '120.36.215.3' has been resolved to the host name '3.215.36.120.broad.xm.fj.dynamic.163data.com.cn', which resembles IPv4-address itself.
190130 15:18:40 [Warning] IP address '193.188.22.37' could not be resolved: Name or service not known
190130 15:47:34 [Warning] IP address '162.209.246.94' could not be resolved: Name or service not known
190130 17:52:39 [Warning] IP address '31.184.193.113' could not be resolved: Name or service not known
190130 18:34:29 [Warning] IP address '196.52.43.123' could not be resolved: Name or service not known
190130 19:30:04 [Warning] IP address '132.232.79.192' could not be resolved: Name or service not known
190130 19:48:38 [Warning] IP address '125.64.94.200' could not be resolved: Name or service not known
190131 5:36:55 [Warning] IP address '58.218.213.189' could not be resolved: Name or service not known
190131 8:48:26 [Warning] IP address '111.76.130.79' could not be resolved: Name or service not known
190131 16:47:20 [Warning] IP address '193.106.29.50' could not be resolved: Name or service not known
190131 17:48:00 [Warning] IP address '196.52.43.111' could not be resolved: Name or service not known
190131 17:56:19 [Warning] IP address '185.156.177.2' could not be resolved: Name or service not known
190131 19:58:02 [Warning] IP address '115.151.137.153' could not be resolved: Name or service not known
190131 20:12:32 [Warning] IP address '115.151.137.106' could not be resolved: Name or service not known
190131 20:40:53 [Warning] IP address '115.151.137.40' could not be resolved: Name or service not known
190131 21:52:03 [Warning] IP address '115.151.137.245' could not be resolved: Name or service not known
190131 21:57:04 [Warning] IP address '142.252.248.73' could not be resolved: Name or service not known
190131 22:06:22 [Warning] IP address '58.211.147.182' could not be resolved: Name or service not known
190201 0:54:39 [Warning] IP address '115.151.137.185' could not be resolved: Name or service not known
190201 3:37:40 [Warning] IP address '115.151.137.178' could not be resolved: Name or service not known
190201 6:42:04 [Warning] IP address '115.151.137.101' could not be resolved: Name or service not known
190201 7:28:01 [Warning] IP address '115.151.137.243' could not be resolved: Name or service not known
190201 10:24:01 [Warning] IP address '202.168.160.20' could not be resolved: Name or service not known
190201 11:11:55 [Warning] IP address '58.218.66.173' could not be resolved: Name or service not known
190201 14:25:12 [Warning] IP address '5.101.40.78' could not be resolved: Name or service not known
190201 15:46:57 [Warning] IP address '59.63.116.190' could not be resolved: Name or service not known
190201 16:27:16 [Warning] IP address '196.52.43.57' could not be resolved: Name or service not known
190201 18:09:51 [Warning] IP address '111.175.62.151' could not be resolved: Name or service not known
190201 20:17:10 [Warning] IP address '103.214.170.44' could not be resolved: Name or service not known
190202 5:36:27 [Warning] IP address '222.35.8.61' could not be resolved: Name or service not known
190202 6:39:29 [Note] Hostname 'internettl.org' does not resolve to '104.152.52.36'.
190202 6:39:29 [Note] Hostname 'internettl.org' has the following IP addresses:
190202 6:39:29 [Note] - 162.255.119.131
190202 7:18:51 [Warning] IP address '115.151.137.166' could not be resolved: Name or service not known
190202 7:40:47 [Warning] IP address '115.151.137.39' could not be resolved: Name or service not known
190202 8:15:14 [Warning] IP address '209.141.51.57' could not be resolved: Name or service not known
190202 8:24:34 [Warning] IP address '115.151.137.152' could not be resolved: Name or service not known
190202 8:46:41 [Warning] IP address '115.151.137.228' could not be resolved: Name or service not known
190202 9:30:17 [Warning] IP address '182.106.81.175' could not be resolved: Name or service not known
190202 10:36:07 [Warning] IP address '115.151.137.254' could not be resolved: Name or service not known
190202 12:04:51 [Warning] IP address '185.156.177.20' could not be resolved: Name or service not known
190202 12:26:37 [Warning] IP address '115.151.137.173' could not be resolved: Name or service not known
190202 12:47:04 [Warning] IP address '59.36.132.222' could not be resolved: Name or service not known
190202 13:32:52 [Warning] IP address '115.151.137.68' could not be resolved: Name or service not known
190202 14:45:21 [Warning] IP address '71.6.232.5' could not be resolved: Name or service not known
190202 15:01:55 [Warning] IP address '115.151.137.179' could not be resolved: Name or service not known
190202 15:24:47 [Warning] IP address '115.151.137.90' could not be resolved: Name or service not known
190202 16:32:06 [Warning] IP address '115.151.137.61' could not be resolved: Name or service not known
190202 17:48:41 [Warning] IP address '196.52.43.121' could not be resolved: Name or service not known
190202 19:51:01 [Warning] IP address '115.151.137.92' could not be resolved: Name or service not known
190202 20:13:42 [Warning] IP address '115.151.137.18' could not be resolved: Name or service not known
190202 20:56:34 [Warning] IP address '115.151.137.57' could not be resolved: Name or service not known
190202 22:25:05 [Warning] IP address '111.175.60.97' could not be resolved: Name or service not known
190203 3:58:02 [Warning] IP address '182.106.81.237' could not be resolved: Name or service not known
190203 17:48:53 [Warning] IP address '82.202.249.225' could not be resolved: Name or service not known
190203 18:36:51 [Warning] IP address '196.52.43.63' could not be resolved: Name or service not known
190204 10:26:44 [Warning] IP address '101.254.230.63' could not be resolved: Name or service not known
190204 18:15:27 [Warning] IP address '191.96.214.33' has been resolved to the host name '191.96.214.33.netsystemsresearch.com', which resembles IPv4-address itself.
190204 19:09:34 [Warning] IP address '88.214.26.17' could not be resolved: Name or service not known
190204 19:09:35 [Warning] IP address '88.214.26.18' could not be resolved: Name or service not known
190204 19:09:35 [Warning] IP address '88.214.26.19' could not be resolved: Name or service not known
190204 19:09:35 [Warning] IP address '88.214.26.20' could not be resolved: Name or service not known
190204 19:09:35 [Warning] IP address '88.214.26.39' could not be resolved: Name or service not known
190204 19:09:36 [Warning] IP address '88.214.26.40' could not be resolved: Name or service not known
190205 5:19:16 [Warning] IP address '150.109.48.100' could not be resolved: Name or service not known
190205 6:50:54 [Warning] IP address '115.151.137.246' could not be resolved: Name or service not known
190205 7:09:57 [Warning] IP address '115.151.137.79' could not be resolved: Name or service not known
190205 14:16:23 [Warning] IP address '102.165.51.177' could not be resolved: Name or service not known
190205 20:59:46 [Warning] IP address '125.65.112.202' could not be resolved: Name or service not known
190206 8:45:33 [Warning] IP address '5.101.40.34' could not be resolved: Name or service not known
190206 14:51:39 [Warning] IP address '111.73.46.33' could not be resolved: Name or service not known
190206 16:04:59 [Warning] IP address '196.52.43.58' could not be resolved: Name or service not known
190206 23:26:48 [Warning] IP address '185.156.177.16' could not be resolved: Name or service not known
190207 1:06:24 [Warning] IP address '125.212.217.214' could not be resolved: Name or service not known
190207 1:18:55 [Warning] IP address '101.254.149.83' could not be resolved: Name or service not known
190207 18:38:13 [Warning] IP address '191.96.214.45' could not be resolved: Name or service not known
190208 18:23:47 [Warning] IP address '196.52.43.101' could not be resolved: Name or service not known
190209 11:25:26 [Warning] IP address '138.68.18.223' could not be resolved: Name or service not known
190209 17:14:33 [Warning] IP address '191.101.128.13' has been resolved to the host name '191.101.128.13.netsystemsresearch.com', which resembles IPv4-address itself.
(...)
190223 8:57:17 [ERROR] mysqld: Table './admin_wp/wp_options' is marked as crashed and should be repaired
190223 8:57:17 [Warning] Checking table: './admin_wp/wp_options'
190223 9:28:08 [Warning] IP address '85.93.20.38' could not be resolved: Name or service not known
190223 11:18:08 [Warning] IP address '138.68.18.223' could not be resolved: Name or service not known
190223 11:18:08 [Warning] IP address '138.68.18.223' could not be resolved: Name or service not known
190223 14:30:09 [Warning] IP address '222.35.8.61' could not be resolved: Name or service not known
190223 15:22:31 [Warning] IP address '58.218.66.195' could not be resolved: Name or service not known
190223 16:08:59 [Warning] IP address '196.52.43.121' could not be resolved: Name or service not known
190223 19:18:26 [Warning] IP address '71.6.232.5' could not be resolved: Name or service not known
190223 20:27:12 [Warning] IP address '138.68.29.165' could not be resolved: Name or service not known
190224 0:01:25 [Warning] IP address '58.211.147.182' could not be resolved: Name or service not known
190224 8:22:41 [Warning] IP address '118.221.122.8' could not be resolved: Name or service not known
190224 11:38:48 [Warning] IP address '222.186.138.60' could not be resolved: Name or service not known
190224 13:43:18 [Warning] IP address '111.73.46.33' could not be resolved: Name or service not known
190224 18:20:24 [Warning] IP address '191.101.128.13' has been resolved to the host name '191.101.128.13.netsystemsresearch.com', which resembles IPv4-address itself.
Re: Strange memory consumption
When you create a MySQL user [email protected] MySQL has to do a reverse lookup on every IP address connecting to it to determine whether they are part of example.com.
Of course, there's no restriction on creating reverse lookups, so I can quite happily ask my provider to set the reverse lookup for my IP address to be google.com if I want... or example.com if I happen to know that's what the users in your database have. This won't let me in, as MySQL then does a forward lookup on the returned domain to make sure it matches the same IP address that's connecting.
You can switch this off with skip_name_resolve in your my.cnf. There are many good reasons for doing this.
The reason you are getting this error is that the IP address in question has no reverse lookup at all.
You also have malicious attackers from China trying to brute force their way into your database. That should be your top priority.
Of course, there's no restriction on creating reverse lookups, so I can quite happily ask my provider to set the reverse lookup for my IP address to be google.com if I want... or example.com if I happen to know that's what the users in your database have. This won't let me in, as MySQL then does a forward lookup on the returned domain to make sure it matches the same IP address that's connecting.
You can switch this off with skip_name_resolve in your my.cnf. There are many good reasons for doing this.
The reason you are getting this error is that the IP address in question has no reverse lookup at all.
You also have malicious attackers from China trying to brute force their way into your database. That should be your top priority.
Re: Strange memory consumption
I saw that explanation in Stack Overflow. The matter is that my server is stopping the MariaDB every certain amount of time because suddenly, it has not enough memory. I'm having messages such asEmohlyni wrote: ↑Wed Feb 27, 2019 1:40 amWhen you create a MySQL user [email protected] MySQL has to do a reverse lookup on every IP address connecting to it to determine whether they are part of example.com.
Of course, there's no restriction on creating reverse lookups, so I can quite happily ask my provider to set the reverse lookup for my IP address to be google.com if I want... or example.com if I happen to know that's what the users in your database have. This won't let me in, as MySQL then does a forward lookup on the returned domain to make sure it matches the same IP address that's connecting.
You can switch this off with skip_name_resolve in your my.cnf. There are many good reasons for doing this.
The reason you are getting this error is that the IP address in question has no reverse lookup at all.
You also have malicious attackers from China trying to brute force their way into your database. That should be your top priority.
Code: Select all
/usr/local/vesta/bin/v-update-sys-vesta: line 54: 6771 Segmentation fault yum -q clean all
I'm considering many options, but all of them will require reinstalling the entire system...
Re: Strange memory consumption
Unless you need to access the database remotely, I'd suggest deactivating or removing the firewall rule that allows external access to it.
From the gui, that's the one that opens port 3306.
If you know what you're doing with mysql, I'd also recommend removing all entries in the user table that allow access from host="%"
Several times I've recommended that these defaults be changed for VestaCP, but the developers don't seem to like the idea, so its up to you to turn them off. I'm fairly sure most people running vestaCP don't even realise their database port is open to the internet.
From the gui, that's the one that opens port 3306.
If you know what you're doing with mysql, I'd also recommend removing all entries in the user table that allow access from host="%"
Several times I've recommended that these defaults be changed for VestaCP, but the developers don't seem to like the idea, so its up to you to turn them off. I'm fairly sure most people running vestaCP don't even realise their database port is open to the internet.
Re: Strange memory consumption
Thanks @plutocrat for the suggestion. I agree, it's better to block external access unless strictly necessary.plutocrat wrote: ↑Fri Mar 01, 2019 9:56 amUnless you need to access the database remotely, I'd suggest deactivating or removing the firewall rule that allows external access to it.
From the gui, that's the one that opens port 3306.
If you know what you're doing with mysql, I'd also recommend removing all entries in the user table that allow access from host="%"
Several times I've recommended that these defaults be changed for VestaCP, but the developers don't seem to like the idea, so its up to you to turn them off. I'm fairly sure most people running vestaCP don't even realise their database port is open to the internet.
Re: Strange memory consumption
Except if you have to get to the database remotely, I'd recommend deactivating or expelling the firewall decide that enables outer access to it.
From the gui, that is the one that opens port 3306.
On the off chance that you realize what you're doing with Database MySQL, I'd likewise prescribe evacuating all sections in the client table that permit access from host="%"
A few times I've suggested that these defaults be changed for VestaCP, yet the designers don't appear to like the thought, so it's up to you to turn them off. I'm genuinely certain a great many people running vestaCP don't understand their database port is available to the web.
From the gui, that is the one that opens port 3306.
On the off chance that you realize what you're doing with Database MySQL, I'd likewise prescribe evacuating all sections in the client table that permit access from host="%"
A few times I've suggested that these defaults be changed for VestaCP, yet the designers don't appear to like the thought, so it's up to you to turn them off. I'm genuinely certain a great many people running vestaCP don't understand their database port is available to the web.