mysql crash every nights

[Teo]

Hello, I can't understand why the mysql (MariaDB) on my centos7 crash every night. The only thing i know is that every day i have to restart it. I know the information provided are so less, but maybe you can help me to understand where to check some issue....

thank you in advance....

Teo

[Teo]

anyone can help me? just where I could check... I only see this:

[grayfolk]

See mysql logs.

[Teo]

I don't understand, memory usage keep growing until it crashes. this is the log thank you for any help

Code: Select all

190605  3:04:48 [Warning] IP address '124.90.49.195' could not be resolved: Name or service not known
190605  3:04:49 [Warning] IP address '123.160.235.68' could not be resolved: Name or service not known
190605  4:35:54 [Warning] IP address '218.60.67.16' could not be resolved: Name or service not known
190605  5:01:04 [Warning] IP address '221.229.162.169' could not be resolved: Name or service not known
190605  5:36:16 [Warning] IP address '58.218.67.77' could not be resolved: Name or service not known
190605  5:50:42 [Warning] IP address '106.75.79.172' could not be resolved: Name or service not known
190605  6:25:10 [Note] Plugin 'FEDERATED' is disabled.
190605  6:25:11 InnoDB: The InnoDB memory heap is disabled
190605  6:25:11 InnoDB: Mutexes and rw_locks use GCC atomic builtins
190605  6:25:11 InnoDB: Compressed tables use zlib 1.2.3
190605  6:25:11 InnoDB: Using Linux native AIO
190605  6:25:11 InnoDB: Initializing buffer pool, size = 128.0M
190605  6:25:11 InnoDB: Completed initialization of buffer pool
190605  6:25:11 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
190605  6:25:11  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
190605  6:25:17  InnoDB: Waiting for the background threads to start
190605  6:25:18 InnoDB: 5.5.62 started; log sequence number 316787432
190605  6:25:18 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
190605  6:25:18 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
190605  6:25:18 [Note] Server socket created on IP: '0.0.0.0'.
190605  6:25:18 [Note] Event Scheduler: Loaded 0 events
190605  6:25:18 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.62'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
190605  6:25:54 [Warning] IP address '85.93.20.38' could not be resolved: Name or service not known
190605  6:35:18 [Warning] IP address '89.248.174.4' could not be resolved: Name or service not known
190605  6:48:24 [Warning] IP address '221.229.166.101' could not be resolved: Name or service not known
190605  6:50:23 [Note] Plugin 'FEDERATED' is disabled.
190605  6:50:23 InnoDB: The InnoDB memory heap is disabled
190605  6:50:23 InnoDB: Mutexes and rw_locks use GCC atomic builtins
190605  6:50:23 InnoDB: Compressed tables use zlib 1.2.3
190605  6:50:23 InnoDB: Using Linux native AIO
190605  6:50:23 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
190605  6:50:23 InnoDB: Completed initialization of buffer pool
190605  6:50:23 InnoDB: Fatal error: cannot allocate memory for the buffer pool
190605  6:50:23 [ERROR] Plugin 'InnoDB' init function returned error.
190605  6:50:23 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
190605  6:50:23 [ERROR] Unknown/unsupported storage engine: InnoDB
190605  6:50:23 [ERROR] Aborting

190605  6:50:23 [Note] /usr/libexec/mysqld: Shutdown complete

190605  7:31:56 [Note] Plugin 'FEDERATED' is disabled.
190605  7:31:56 InnoDB: The InnoDB memory heap is disabled
190605  7:31:56 InnoDB: Mutexes and rw_locks use GCC atomic builtins
190605  7:31:56 InnoDB: Compressed tables use zlib 1.2.3
190605  7:31:56 InnoDB: Using Linux native AIO
190605  7:31:56 InnoDB: Initializing buffer pool, size = 128.0M
190605  7:31:56 InnoDB: Completed initialization of buffer pool
190605  7:31:56 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
190605  7:31:56  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
190605  7:31:57  InnoDB: Waiting for the background threads to start
190605  7:31:58 InnoDB: 5.5.62 started; log sequence number 321169352
190605  7:31:58 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
190605  7:31:58 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
190605  7:31:58 [Note] Server socket created on IP: '0.0.0.0'.
190605  7:31:58 [Note] Event Scheduler: Loaded 0 events
190605  7:31:58 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.62'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
190605  7:33:42 [Note] /usr/libexec/mysqld: Normal shutdown

190605  7:33:42 [Note] Event Scheduler: Purging the queue. 0 events
190605  7:33:44 [Warning] /usr/libexec/mysqld: Forcing close of thread 20  user: 'admin_b2k'

190605  7:33:44 [Warning] /usr/libexec/mysqld: Forcing close of thread 14  user: 'admin_b2k'

190605  7:33:44 [Warning] /usr/libexec/mysqld: Forcing close of thread 6  user: 'admin_b2k'

190605  7:33:44  InnoDB: Starting shutdown...
190605  7:33:46  InnoDB: Shutdown completed; log sequence number 321204114
190605  7:33:46 [Note] /usr/libexec/mysqld: Shutdown complete

[adamjedgar]

What websites are you running on your server?
If WordPress, check it's logs and security as the problem may actually be coming from there.

If you have cms websites(like wordpress), what security plugins are you running on them?

[skamasle]

Maybe a memory issue

[adamjedgar]

I too have begun experiencing this issue. The Vesta installation doing it has been running fine for about a month...Then out of the blue, it has also started doing this.

I have a 1cpu 1gb ram system running a single small WordPress site.

Almost daily now mariadb is suddenly crashing.

Looking at the server graphs just prior to failure...nothing seems out of the ordinary. Mariadb just suddenly fails.

My logs read almost the same as O.P at the time of Mariadb crash.

[skamasle]

1GB of RAM is to low

You have swap setup ?

You have enabled mariadb logs ?

And what version of mariadb you have ?

Maybe mariadb 5.5 ?

May it fail during backup with some semaphore wait -> https://jira.mariadb.org/browse/MDEV-15707

Can fail in centos 7, mariadb 5.5, but cant say not much more, in my opinion, 1 gb of RAM investa is to low, if you run mail and clamd runing ( clamav use much RAM in some instalations )