Page 1 of 4

Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 4:55 am
by tomer628
Hi..

First of all my details vps is:
4 cores + 8gb Ram + 100gb SSD hard drive via OVH host
I have a website with 25,000-30,000 visit for a day..

The problem :
Mysql + Clamav crash after some hours..
Why is that? My server should be strong..
How can I fix it?
Image
Tomer.

Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 10:29 am
by tjebbeke
Can you show us your MySQL logs?

Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 10:40 am
by tomer628
tjebbeke wrote:Can you show us your MySQL logs?
Yes, this the log:

Code: Select all

151106 10:39:44 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
151106 10:39:44 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 2722 ...
151106 10:39:44 [Note] Plugin 'FEDERATED' is disabled.
151106 10:39:44 InnoDB: The InnoDB memory heap is disabled
151106 10:39:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151106 10:39:44 InnoDB: Compressed tables use zlib 1.2.3
151106 10:39:44 InnoDB: Using Linux native AIO
151106 10:39:44 InnoDB: Initializing buffer pool, size = 128.0M
151106 10:39:44 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
151106 10:39:44  InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
151106 10:39:44  InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
151106 10:39:44  InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
151106 10:39:44  InnoDB: Waiting for the background threads to start
151106 10:39:45 InnoDB: 5.5.46 started; log sequence number 0
151106 10:39:45 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
151106 10:39:45 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
151106 10:39:45 [Note] Server socket created on IP: '0.0.0.0'.
151106 10:39:45 [Note] Event Scheduler: Loaded 0 events
151106 10:39:45 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.46'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi

Status information:

Current dir: /var/lib/mysql/
Running threads: 0  Stack size: 262144
Current locks:
lock: 0x7f50000433f0:

lock: 0x7f500000dda0:

lock: 0x1d56750:

lock: 0x1d1f010:

lock: 0x1d14400:

lock: 0x1d07b60:

lock: 0x1cf6b80:

lock: 0x1cf0e00:

lock: 0x1cebb50:

lock: 0x1ce5620:

lock: 0x1508b00:

lock: 0xfa4840:

lock: 0xfa4f00:

lock: 0xfa4e40:

lock: 0xfa4d80:

lock: 0xfa49c0:

lock: 0xfa4900:

lock: 0xfa4780:

lock: 0xfa46c0:

lock: 0xfa4600:

lock: 0xfa4fc0:

lock: 0xfa4a80:

lock: 0xfa4cc0:

lock: 0xfa4c00:

lock: 0xfa4b40:

lock: 0xfa4540:

lock: 0xfa4480:

lock: 0xfa43c0:


Key caches:
default
Buffer_size:       8388608
Block_size:           1024
Division_limit:        100
Age_limit:             300
blocks used:             4
not flushed:             0
w_requests:             57
writes:                 33
r_requests:             60
reads:                   4


handler status:
read_key:           21
read_next:           0
read_rnd            24
read_first:          6
write:            1073
delete              11
update:              1

Table status:
Opened tables:         35
Open tables:           28
Open files:            20
Open streams:           0

Alarm status:
Active alarms:   0
Max used alarms: 0
Next alarm time: 0

Memory status:
Non-mmapped space allocated from system: 9547776
Number of free chunks:			 33
Number of fastbin blocks:		 0
Number of mmapped regions:		 14
Space in mmapped regions:		 23969792
Maximum total allocated space:		 0
Space available in freed fastbin blocks: 0
Total allocated space:			 8893760
Total free space:			 654016
Top-most, releasable space:		 589424
Estimated memory (with thread stack):    33517568



Events status:
LLA = Last Locked At  LUA = Last Unlocked At
WOC = Waiting On Condition  DL = Data Locked

Event scheduler status:
State      : INITIALIZED
Thread id  : 0
LLA        : n/a:0
LUA        : n/a:0
WOC        : NO
Workers    : 0
Executed   : 0
Data locked: NO

Event queue status:
Element count   : 0
Data locked     : NO
Attempting lock : NO
LLA             : drop_schema_events:382
LUA             : drop_schema_events:384
WOC             : NO
Next activation : never
151106 10:54:54 [Note] /usr/libexec/mysqld: Normal shutdown

151106 10:54:54 [Note] Event Scheduler: Purging the queue. 0 events
151106 10:54:54  InnoDB: Starting shutdown...
151106 10:54:56  InnoDB: Shutdown completed; log sequence number 1683048
151106 10:54:56 [Note] /usr/libexec/mysqld: Shutdown complete

151106 10:54:56 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
151106 10:55:41 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
151106 10:55:42 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 1664 ...
151106 10:55:42 [Note] Plugin 'FEDERATED' is disabled.
151106 10:55:42 InnoDB: The InnoDB memory heap is disabled
151106 10:55:42 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151106 10:55:42 InnoDB: Compressed tables use zlib 1.2.3
151106 10:55:42 InnoDB: Using Linux native AIO
151106 10:55:42 InnoDB: Initializing buffer pool, size = 128.0M
151106 10:55:42 InnoDB: Completed initialization of buffer pool
151106 10:55:42 InnoDB: highest supported file format is Barracuda.
151106 10:55:42  InnoDB: Waiting for the background threads to start
151106 10:55:43 InnoDB: 5.5.46 started; log sequence number 1683048
151106 10:55:43 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
151106 10:55:43 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
151106 10:55:43 [Note] Server socket created on IP: '0.0.0.0'.
151106 10:55:43 [Note] Event Scheduler: Loaded 0 events
151106 10:55:43 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.46'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
151107 14:09:26 [Warning] IP address '222.186.129.85' could not be resolved: Name or service not known
151107 19:10:56 [Note] /usr/libexec/mysqld: Normal shutdown

151107 19:10:56 [Note] Event Scheduler: Purging the queue. 0 events
151107 19:10:56  InnoDB: Starting shutdown...
151107 19:10:58  InnoDB: Shutdown completed; log sequence number 846551798
151107 19:10:58 [Note] /usr/libexec/mysqld: Shutdown complete

151107 19:10:58 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
151107 19:11:36 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
151107 19:11:37 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 1663 ...
151107 19:11:37 [Note] Plugin 'FEDERATED' is disabled.
151107 19:11:37 InnoDB: The InnoDB memory heap is disabled
151107 19:11:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151107 19:11:37 InnoDB: Compressed tables use zlib 1.2.3
151107 19:11:37 InnoDB: Using Linux native AIO
151107 19:11:37 InnoDB: Initializing buffer pool, size = 128.0M
151107 19:11:37 InnoDB: Completed initialization of buffer pool
151107 19:11:37 InnoDB: highest supported file format is Barracuda.
151107 19:11:37  InnoDB: Waiting for the background threads to start
151107 19:11:38 InnoDB: 5.5.46 started; log sequence number 846551798
151107 19:11:38 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
151107 19:11:38 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
151107 19:11:38 [Note] Server socket created on IP: '0.0.0.0'.
151107 19:11:38 [Note] Event Scheduler: Loaded 0 events
151107 19:11:38 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.46'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
151107 22:53:31 [Warning] IP address '222.186.34.82' could not be resolved: Name or service not known
151108 01:08:10 mysqld_safe Number of processes running now: 0
151108 01:08:10 mysqld_safe mysqld restarted
151108  1:08:10 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 9497 ...
151108  1:08:10 [Note] Plugin 'FEDERATED' is disabled.
151108  1:08:10 InnoDB: The InnoDB memory heap is disabled
151108  1:08:10 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151108  1:08:10 InnoDB: Compressed tables use zlib 1.2.3
151108  1:08:10 InnoDB: Using Linux native AIO
151108  1:08:10 InnoDB: Initializing buffer pool, size = 128.0M
151108  1:08:10 InnoDB: Completed initialization of buffer pool
151108  1:08:10 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 1702914621
151108  1:08:10  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 1702948669
151108  1:08:11  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
151108  1:08:11  InnoDB: Waiting for the background threads to start
151108  1:08:12 InnoDB: 5.5.46 started; log sequence number 1702948669
151108  1:08:12 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
151108  1:08:12 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
151108  1:08:12 [Note] Server socket created on IP: '0.0.0.0'.
151108  1:08:12 [Note] Event Scheduler: Loaded 0 events
151108  1:08:12 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.46'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
151108 01:09:04 mysqld_safe Number of processes running now: 0
151108 01:09:04 mysqld_safe mysqld restarted
151108  1:09:05 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 10983 ...
151108  1:09:05 [Note] Plugin 'FEDERATED' is disabled.
151108  1:09:05 InnoDB: The InnoDB memory heap is disabled
151108  1:09:05 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151108  1:09:05 InnoDB: Compressed tables use zlib 1.2.3
151108  1:09:05 InnoDB: Using Linux native AIO
151108  1:09:05 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
151108  1:09:05 InnoDB: Completed initialization of buffer pool
151108  1:09:05 InnoDB: Fatal error: cannot allocate memory for the buffer pool
151108  1:09:05 [ERROR] Plugin 'InnoDB' init function returned error.
151108  1:09:05 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
151108  1:09:05 [ERROR] Unknown/unsupported storage engine: InnoDB
151108  1:09:05 [ERROR] Aborting

151108  1:09:05 [Note] /usr/libexec/mysqld: Shutdown complete

151108 01:09:05 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
151108 05:09:55 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
151108  5:09:55 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 25857 ...
151108  5:09:55 [Note] Plugin 'FEDERATED' is disabled.
151108  5:09:55 InnoDB: The InnoDB memory heap is disabled
151108  5:09:55 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151108  5:09:55 InnoDB: Compressed tables use zlib 1.2.3
151108  5:09:55 InnoDB: Using Linux native AIO
151108  5:09:55 InnoDB: Initializing buffer pool, size = 128.0M
151108  5:09:55 InnoDB: Completed initialization of buffer pool
151108  5:09:55 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 1703054625
151108  5:09:55  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 1703057773
151108  5:09:56  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
151108  5:09:56  InnoDB: Waiting for the background threads to start
151108  5:09:57 InnoDB: 5.5.46 started; log sequence number 1703057773
151108  5:09:57 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
151108  5:09:57 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
151108  5:09:57 [Note] Server socket created on IP: '0.0.0.0'.
151108  5:09:57 [Note] Event Scheduler: Loaded 0 events
151108  5:09:57 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.46'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi

Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 12:13 pm
by tjebbeke

Code: Select all

151108  1:09:05 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
151108  1:09:05 InnoDB: Completed initialization of buffer pool
151108  1:09:05 InnoDB: Fatal error: cannot allocate memory for the buffer pool
151108  1:09:05 [ERROR] Plugin 'InnoDB' init function returned error.
151108  1:09:05 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
151108  1:09:05 [ERROR] Unknown/unsupported storage engine: InnoDB
151108  1:09:05 [ERROR] Aborting

151108  1:09:05 [Note] /usr/libexec/mysqld: Shutdown complete
In the chart, I see your server going out of memory. In the MySQL log, you see this too. Can you tune your MySQL config? Or find out why your server is using that much memory?

Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 12:24 pm
by tomer628
This my config of "/etc/my.cnf"

Code: Select all

GNU nano 2.0.9            File: /etc/my.cnf                              
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
symbolic-links=0
max_connections=200
max_user_connections=30
wait_timeout=30
interactive_timeout=50
long_query_time=5
#log-queries-not-using-indexes
#log-slow-queries=/var/log/mysql/log-slow-queries.log

#innodb_use_native_aio = 0
innodb_file_per_table

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
And for some reason Clamav also stop when mysql crashed..

I have this problem almost 3 weeks..
Hope you will know how to fix that...

Regards,
Tomer

Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 2:09 pm
by tjebbeke
I do not think it's because of MySQL. You normally have enough memory. Can you inspect /var/log/syslog or /var/log/messages at the time it goes out of memory.

Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 2:29 pm
by tomer628
tjebbeke wrote:I do not think it's because of MySQL. You normally have enough memory. Can you inspect /var/log/syslog or /var/log/messages at the time it goes out of memory.
OK..
before 2 hours ago this crashed again...
and this what i got in var/log/messages :
thanks for your help...

Code: Select all

Nov  8 17:28:28 host kernel: __ratelimit: 1184 callbacks suppressed
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: __ratelimit: 1291 callbacks suppressed
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: __ratelimit: 1180 callbacks suppressed
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:38 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: __ratelimit: 1066 callbacks suppressed
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:43 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: __ratelimit: 949 callbacks suppressed
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:48 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: __ratelimit: 842 callbacks suppressed
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:53 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: __ratelimit: 633 callbacks suppressed
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:58 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:29:03 host kernel: __ratelimit: 375 callbacks suppressed
Nov  8 17:29:03 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:29:03 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:29:03 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:29:55 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:29:58 host kernel: httpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Nov  8 17:29:58 host kernel: httpd cpuset=/ mems_allowed=0
Nov  8 17:29:58 host kernel: Pid: 29179, comm: httpd Not tainted 2.6.32-573.7.1.el6.x86_64 #1
Nov  8 17:29:58 host kernel: Call Trace:
Nov  8 17:29:58 host kernel: [<ffffffff810d6d71>] ? cpuset_print_task_mems_allowed+0x91/0xb0
Nov  8 17:29:58 host kernel: [<ffffffff8112a570>] ? dump_header+0x90/0x1b0
Nov  8 17:29:58 host kernel: [<ffffffff81232c5c>] ? security_real_capable_noaudit+0x3c/0x70
Nov  8 17:29:58 host kernel: [<ffffffff8112a9f2>] ? oom_kill_process+0x82/0x2a0
Nov  8 17:29:58 host kernel: [<ffffffff8112a931>] ? select_bad_process+0xe1/0x120
Nov  8 17:29:58 host kernel: [<ffffffff8112ae30>] ? out_of_memory+0x220/0x3c0
Nov  8 17:29:58 host kernel: [<ffffffff8113780c>] ? __alloc_pages_nodemask+0x93c/0x950
Nov  8 17:29:58 host kernel: [<ffffffff8117021a>] ? alloc_pages_current+0xaa/0x110
Nov  8 17:29:58 host kernel: [<ffffffff81127967>] ? __page_cache_alloc+0x87/0x90
Nov  8 17:29:58 host kernel: [<ffffffff8112734e>] ? find_get_page+0x1e/0xa0
Nov  8 17:29:58 host kernel: [<ffffffff81128907>] ? filemap_fault+0x1a7/0x500
Nov  8 17:29:58 host kernel: [<ffffffff81151ec4>] ? __do_fault+0x54/0x530
Nov  8 17:29:58 host kernel: [<ffffffff81152497>] ? handle_pte_fault+0xf7/0xb20
Nov  8 17:29:58 host kernel: [<ffffffff811a24ca>] ? __link_path_walk+0x7da/0x1000
Nov  8 17:29:58 host kernel: [<ffffffff811290a0>] ? generic_file_aio_read+0x380/0x700
Nov  8 17:29:58 host kernel: [<ffffffff81153159>] ? handle_mm_fault+0x299/0x3d0
Nov  8 17:29:58 host kernel: [<ffffffff811b4560>] ? mntput_no_expire+0x30/0x110
Nov  8 17:29:58 host kernel: [<ffffffff8104f156>] ? __do_page_fault+0x146/0x500
Nov  8 17:29:58 host kernel: [<ffffffff810f31f5>] ? call_rcu_sched+0x15/0x20
Nov  8 17:29:58 host kernel: [<ffffffff810f320e>] ? call_rcu+0xe/0x10
Nov  8 17:29:58 host kernel: [<ffffffff8153eb4e>] ? do_page_fault+0x3e/0xa0
Nov  8 17:29:58 host kernel: [<ffffffff8153bef5>] ? page_fault+0x25/0x30
Nov  8 17:29:58 host kernel: Mem-Info:
Nov  8 17:29:58 host kernel: Node 0 DMA per-cpu:
Nov  8 17:29:58 host kernel: CPU    0: hi:    0, btch:   1 usd:   0
Nov  8 17:29:58 host kernel: CPU    1: hi:    0, btch:   1 usd:   0
Nov  8 17:29:58 host kernel: CPU    2: hi:    0, btch:   1 usd:   0
Nov  8 17:29:58 host kernel: CPU    3: hi:    0, btch:   1 usd:   0
Nov  8 17:29:58 host kernel: Node 0 DMA32 per-cpu:
Nov  8 17:29:58 host kernel: CPU    0: hi:  186, btch:  31 usd:   0
Nov  8 17:29:58 host kernel: CPU    1: hi:  186, btch:  31 usd:   0
Nov  8 17:29:58 host kernel: CPU    2: hi:  186, btch:  31 usd:   0
Nov  8 17:29:58 host kernel: CPU    3: hi:  186, btch:  31 usd:   0
Nov  8 17:29:58 host kernel: Node 0 Normal per-cpu:
Nov  8 17:29:58 host kernel: CPU    0: hi:  186, btch:  31 usd:   0
Nov  8 17:29:58 host kernel: CPU    1: hi:  186, btch:  31 usd:   0
Nov  8 17:29:58 host kernel: CPU    2: hi:  186, btch:  31 usd:   0
Nov  8 17:29:58 host kernel: CPU    3: hi:  186, btch:  31 usd:   0
Nov  8 17:29:58 host kernel: active_anon:1847147 inactive_anon:565 isolated_anon:0
Nov  8 17:29:58 host kernel: active_file:1859 inactive_file:2074 isolated_file:63
Nov  8 17:29:58 host kernel: unevictable:0 dirty:1789 writeback:17 unstable:0
Nov  8 17:29:58 host kernel: free:25609 slab_reclaimable:3891 slab_unreclaimable:28443
Nov  8 17:29:58 host kernel: mapped:1656 shmem:623 pagetables:37747 bounce:0
Nov  8 17:29:58 host kernel: Node 0 DMA free:15748kB min:128kB low:160kB high:192kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Nov  8 17:29:58 host kernel: lowmem_reserve[]: 0 3000 7860 7860
Nov  8 17:29:58 host kernel: Node 0 DMA32 free:44972kB min:25744kB low:32180kB high:38616kB active_anon:2734724kB inactive_anon:344kB active_file:1972kB inactive_file:2448kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:3072216kB mlocked:0kB dirty:2192kB writeback:64kB mapped:1624kB shmem:344kB slab_reclaimable:1228kB slab_unreclaimable:9372kB kernel_stack:176kB pagetables:5396kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:5280 all_unreclaimable? no
Nov  8 17:29:58 host kernel: lowmem_reserve[]: 0 0 4860 4860
Nov  8 17:29:58 host kernel: Node 0 Normal free:41624kB min:41708kB low:52132kB high:62560kB active_anon:4653864kB inactive_anon:1916kB active_file:5464kB inactive_file:5656kB unevictable:0kB isolated(anon):0kB isolated(file):124kB present:4977280kB mlocked:0kB dirty:4964kB writeback:4kB mapped:5000kB shmem:2148kB slab_reclaimable:14336kB slab_unreclaimable:104400kB kernel_stack:6240kB pagetables:145592kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4704 all_unreclaimable? no
Nov  8 17:29:58 host kernel: lowmem_reserve[]: 0 0 0 0
Nov  8 17:29:58 host kernel: Node 0 DMA: 3*4kB 1*8kB 1*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15748kB
Nov  8 17:29:58 host kernel: Node 0 DMA32: 697*4kB 291*8kB 105*16kB 44*32kB 17*64kB 45*128kB 28*256kB 5*512kB 4*1024kB 2*2048kB 3*4096kB = 45260kB
Nov  8 17:29:58 host kernel: Node 0 Normal: 1220*4kB 798*8kB 569*16kB 259*32kB 99*64kB 28*128kB 4*256kB 2*512kB 1*1024kB 0*2048kB 0*4096kB = 41648kB
Nov  8 17:29:58 host kernel: 4793 total pagecache pages
Nov  8 17:29:58 host kernel: 0 pages in swap cache
Nov  8 17:29:58 host kernel: Swap cache stats: add 0, delete 0, find 0/0
Nov  8 17:29:58 host kernel: Free swap  = 0kB
Nov  8 17:29:58 host kernel: Total swap = 0kB
Nov  8 17:29:58 host kernel: 2047999 pages RAM
Nov  8 17:29:58 host kernel: 81813 pages reserved
Nov  8 17:29:58 host kernel: 419625 pages shared
Nov  8 17:29:58 host kernel: 1915678 pages non-shared
Nov  8 17:29:58 host kernel: [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
Nov  8 17:29:58 host kernel: [  433]     0   433     2711      158   0     -17         -1000 udevd
Nov  8 17:29:58 host kernel: [  730]     0   730     2709      158   0     -17         -1000 udevd
Nov  8 17:29:58 host kernel: [ 1035]     0  1035     2280      125   1       0             0 dhclient
Nov  8 17:29:58 host kernel: [ 1089]     0  1089     6899      102   1     -17         -1000 auditd
Nov  8 17:29:58 host kernel: [ 1123]     0  1123    63855      189   0       0             0 rsyslogd
Nov  8 17:29:58 host kernel: [ 1145]    25  1145    97241     2832   0       0             0 named
Nov  8 17:29:58 host kernel: [ 1172]    81  1172     5358       66   3       0             0 dbus-daemon
Nov  8 17:29:58 host kernel: [ 1203]     0  1203     1020       31   0       0             0 acpid
Nov  8 17:29:58 host kernel: [ 1318]     0  1318    16556      195   2     -17         -1000 sshd
Nov  8 17:29:58 host kernel: [ 1332]     0  1332    13028      137   1       0             0 vsftpd
Nov  8 17:29:58 host kernel: [ 1703]     0  1703     4816       70   2       0             0 dovecot
Nov  8 17:29:58 host kernel: [ 1705]    97  1705     3246       54   0       0             0 anvil
Nov  8 17:29:58 host kernel: [ 1706]     0  1706     3278       81   2       0             0 log
Nov  8 17:29:58 host kernel: [ 1711]     0  1711     3589      373   0       0             0 config
Nov  8 17:29:58 host kernel: [ 1720]     0  1720    61117    12910   2       0             0 spamd
Nov  8 17:29:58 host kernel: [ 1721]     0  1721    61117    12915   0       0             0 spamd
Nov  8 17:29:58 host kernel: [ 1724]     0  1724    61117    12915   0       0             0 spamd
Nov  8 17:29:58 host kernel: [ 1738]    93  1738    23834      275   0       0             0 exim
Nov  8 17:29:58 host kernel: [ 1755]     0  1755   113185     2268   2       0             0 httpd
Nov  8 17:29:58 host kernel: [ 1763]     0  1763    42375      755   0       0             0 httpd
Nov  8 17:29:58 host kernel: [ 1771]     0  1771    16500      275   1       0             0 nginx
Nov  8 17:29:58 host kernel: [ 1772]   496  1772    17829     2070   3       0             0 nginx
Nov  8 17:29:58 host kernel: [ 1773]   496  1773    17828     2068   3       0             0 nginx
Nov  8 17:29:58 host kernel: [ 1774]   496  1774    16501      527   0       0             0 nginx
Nov  8 17:29:58 host kernel: [ 1786]     0  1786    13698      222   1       0             0 vesta-nginx
Nov  8 17:29:58 host kernel: [ 1787]   501  1787    13698      233   2       0             0 vesta-nginx
Nov  8 17:29:58 host kernel: [ 1791]     0  1791    34135      520   3       0             0 vesta-php
Nov  8 17:29:58 host kernel: [ 1792]   501  1792    34135      515   2       0             0 vesta-php
Nov  8 17:29:58 host kernel: [ 1793]   501  1793    34135      516   3       0             0 vesta-php
Nov  8 17:29:58 host kernel: [ 1804]     0  1804    29216      190   0       0             0 crond
Nov  8 17:29:58 host kernel: [ 1833]     0  1833   208581     3125   0       0             0 fail2ban-server
Nov  8 17:29:58 host kernel: [ 1885]     0  1885     1016       24   0       0             0 mingetty
Nov  8 17:29:58 host kernel: [ 1887]     0  1887     1020       26   0       0             0 agetty
Nov  8 17:29:58 host kernel: [ 1822]    48  1822   127559    14459   2       0             0 httpd
Nov  8 17:29:58 host kernel: [ 2989]    48  2989   127554    14504   2       0             0 httpd
Nov  8 17:29:58 host kernel: [ 3029]    48  3029   127511    14395   3       0             0 httpd
Nov  8 17:29:58 host kernel: [ 3030]   501  3030   128370    15203   1       0             0 httpd
Nov  8 17:29:58 host kernel: [16739]   501 16739   125088    14610   2       0             0 httpd
Nov  8 17:29:58 host kernel: [16744]    48 16744   128230    15104   0       0             0 httpd
Nov  8 17:29:58 host kernel: [16769]   501 16769   127902    14781   1       0             0 httpd
Nov  8 17:29:58 host kernel: [16771]    48 16771   122908    12559   2       0             0 httpd
Nov  8 17:29:58 host kernel: [16773]   501 16773   127928    14810   2       0             0 httpd
Nov  8 17:29:58 host kernel: [16894]    48 16894   122748    12427   3       0             0 httpd
Nov  8 17:29:58 host kernel: [16943]    48 16943   121785    11463   2       0             0 httpd
Nov  8 17:29:58 host kernel: [16958]   501 16958   124986    14480   3       0             0 httpd
Nov  8 17:29:58 host kernel: [17003]   501 17003   122455    12135   3       0             0 httpd
Nov  8 17:29:58 host kernel: [17033]    48 17033   121790    11468   3       0             0 httpd
Nov  8 17:29:58 host kernel: [17077]    48 17077   123865    13542   2       0             0 httpd
Nov  8 17:29:58 host kernel: [17161]   501 17161   121785    11520   3       0             0 httpd
Nov  8 17:29:58 host kernel: [17162]    48 17162   122174    11867   1       0             0 httpd
Nov  8 17:29:58 host kernel: [17195]    48 17195   127911    14789   3       0             0 httpd
Nov  8 17:29:58 host kernel: [17233]    48 17233   121769    11451   3       0             0 httpd
Nov  8 17:29:58 host kernel: [17343]    48 17343   121779    11457   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28111]     0 28111    27042      135   2       0             0 mysqld_safe
Nov  8 17:29:58 host kernel: [28390]    27 28390   607586    16678   2       0             0 mysqld
Nov  8 17:29:58 host kernel: [28759]    48 28759   123486    13163   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28761]   501 28761   118623     8303   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28762]   501 28762   118623     8300   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28765]   501 28765   123485    13134   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28766]    48 28766   123439    13101   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28767]    48 28767   118635     8312   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28768]    48 28768   124023    13705   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28775]    48 28775   123480    13130   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28776]   501 28776   123421    13095   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28777]   501 28777   121771    11405   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28778]    48 28778   120802    10479   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28779]    48 28779   123480    13132   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28780]    48 28780   121766    11443   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28781]   501 28781   118623     8309   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28782]    48 28782   123470    13121   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28798]    48 28798   121865    11547   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28799]    48 28799   121849    11419   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28800]    48 28800   121783    11465   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28801]    48 28801   121851    11528   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28802]    48 28802   121759    11436   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28803]    48 28803   121759    11436   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28804]    48 28804   121783    11460   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28805]    48 28805   121780    11457   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28806]    48 28806   114101     3538   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28807]    48 28807   118623     8252   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28808]    48 28808   121782    11460   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28809]    48 28809   123437    13091   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28811]   501 28811   124726    14244   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28812]   501 28812   121787    11467   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28815]    48 28815   121759    11441   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28816]    48 28816   121759    11389   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28847]    48 28847   121764    11446   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28848]    48 28848   121786    11416   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28849]    48 28849   121787    11418   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28850]    48 28850   120503    10180   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28851]    48 28851   114112     3549   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28852]    48 28852   121771    11448   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28853]   501 28853   124846    14304   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28854]    48 28854   121768    11445   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28856]    48 28856   121784    11416   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28857]   501 28857   125046    14540   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28858]    48 28858   121791    11422   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28859]    48 28859   121780    11457   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28860]   501 28860   124726    14243   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28862]    48 28862   121787    11464   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28864]    48 28864   121784    11461   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28867]    48 28867   121770    11447   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28868]   501 28868   124789    14244   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28869]    48 28869   114101     3538   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28874]    48 28874   121790    11422   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28875]    48 28875   121783    11465   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28876]    48 28876   114100     3537   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28877]    48 28877   121785    11418   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28878]    48 28878   121777    11454   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28879]    48 28879   121785    11467   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28880]    48 28880   121783    11460   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28882]    48 28882   121797    11427   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28883]    48 28883   121769    11446   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28884]   501 28884   122489    11986   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28885]    48 28885   121782    11414   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28886]    48 28886   121780    11462   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28888]    48 28888   121780    11462   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28889]   501 28889   123940    13513   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28936]    48 28936   121787    11469   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28937]   501 28937   121786    11418   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28938]    48 28938   121772    11449   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28939]    48 28939   121786    11463   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28940]    48 28940   121786    11418   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28941]    48 28941   121780    11412   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28942]    48 28942   121780    11457   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28946]   501 28946   124295    13732   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28947]    48 28947   121770    11401   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28948]    48 28948   114100     3537   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28951]    48 28951   114112     3549   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28952]   501 28952   114100     3540   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28953]    48 28953   121784    11414   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28956]    48 28956   121785    11467   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28957]    48 28957   121770    11401   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28958]    48 28958   121782    11464   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28959]    48 28959   121791    11473   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28961]    48 28961   121788    11465   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28962]   501 28962   121768    11448   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28963]   501 28963   121790    11470   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28966]    48 28966   114107     3545   0       0             0 httpd
Nov  8 17:29:58 host kernel: [28967]    48 28967   121790    11421   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28973]   501 28973   121787    11421   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28974]   501 28974   114102     3542   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28975]    48 28975   121849    11418   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28978]    48 28978   121779    11461   3       0             0 httpd
Nov  8 17:29:58 host kernel: [28979]   501 28979   123315    12803   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28987]    48 28987   121780    11457   1       0             0 httpd
Nov  8 17:29:58 host kernel: [28988]    48 28988   121773    11450   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28989]   501 28989   121797    11461   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28996]    48 28996   114098     3536   2       0             0 httpd
Nov  8 17:29:58 host kernel: [28997]    48 28997   114101     3538   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29047]   501 29047   121788    11458   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29048]    48 29048   121785    11417   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29049]    48 29049   121774    11451   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29050]    48 29050   121782    11460   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29051]    48 29051   123437    13114   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29053]    48 29053   121790    11467   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29054]    48 29054   121790    11421   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29058]    48 29058   121785    11462   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29059]    48 29059   121785    11462   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29060]    48 29060   121779    11456   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29065]    48 29065   121769    11446   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29066]    48 29066   114100     3537   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29067]    48 29067   121783    11465   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29075]    48 29075   114101     3538   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29076]   501 29076   124344    13851   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29081]    48 29081   121774    11405   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29082]    48 29082   121790    11467   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29083]    48 29083   121780    11457   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29084]    48 29084   121847    11524   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29085]   501 29085   124610    14104   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29090]   501 29090   124347    13854   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29091]    48 29091   121784    11461   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29098]    48 29098   121780    11457   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29099]   501 29099   119906     9364   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29100]    48 29100   114107     3545   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29103]    48 29103   121774    11451   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29105]   501 29105   121783    11463   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29108]    48 29108   114116     3553   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29109]   501 29109   124727    14191   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29110]    48 29110   121783    11410   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29114]    48 29114   121764    11442   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29115]    48 29115   114104     3541   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29174]   501 29174   124678    14135   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29175]    48 29175   121780    11457   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29176]    48 29176   121785    11416   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29178]    48 29178   121776    11453   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29179]   501 29179   124731    14203   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29181]   501 29181   124726    14184   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29182]    48 29182   121785    11416   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29187]    48 29187   121788    11419   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29188]   501 29188   121782    11463   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29189]    48 29189   121783    11460   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29200]   501 29200   121779    11459   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29201]   501 29201   121213    10702   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29205]    48 29205   121791    11421   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29206]    48 29206   121783    11460   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29213]   501 29213   120610    10045   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29214]    48 29214   121783    11414   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29215]    48 29215   121770    11452   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29228]    48 29228   121782    11413   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29229]    48 29229   124017    13689   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29230]    48 29230   121783    11415   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29234]    48 29234   121798    11429   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29235]    48 29235   121786    11463   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29241]    48 29241   121780    11457   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29242]    48 29242   121779    11456   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29247]    48 29247   114107     3544   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29248]    48 29248   121784    11415   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29255]    48 29255   121773    11405   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29256]    48 29256   114104     3541   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29260]    48 29260   114112     3550   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29261]    48 29261   114100     3537   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29262]    48 29262   121788    11420   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29265]    48 29265   121789    11421   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29317]   501 29317   121786    11463   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29318]    48 29318   121786    11463   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29319]    48 29319   114066     3503   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29320]    48 29320   121839    11409   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29321]    48 29321   121783    11465   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29322]   501 29322   123482    13136   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29327]   501 29327   114101     3541   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29328]    48 29328   121782    11459   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29345]    48 29345   121780    11458   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29346]    48 29346   121788    11419   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29348]    48 29348   121782    11459   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29353]   501 29353   125049    14583   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29354]    48 29354   121783    11414   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29361]    48 29361   121782    11459   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29362]    48 29362   114112     3549   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29370]    48 29370   123350    13015   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29371]    48 29371   121788    11465   1       0             0 httpd
Nov  8 17:29:58 host kernel: [29372]   501 29372   121785    11465   3       0             0 httpd
Nov  8 17:29:58 host kernel: [29393]    48 29393   121786    11416   0       0             0 httpd
Nov  8 17:29:58 host kernel: [29394]    48 29394   121778    11455   2       0             0 httpd
Nov  8 17:29:58 host kernel: [29401]    48 29401   121768    11445   0       0             0 httpd
Nov  8 17:29:58 host kernel: Out of memory: Kill process 28390 (mysqld) score 8 or sacrifice child
Nov  8 17:29:58 host kernel: Killed process 28390, UID 27, (mysqld) total-vm:2430344kB, anon-rss:65408kB, file-rss:1308kB
Nov  8 17:30:00 host kernel: __ratelimit: 719 callbacks suppressed
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:00 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: __ratelimit: 1099 callbacks suppressed
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:05 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:10 host kernel: __ratelimit: 1355 callbacks suppressed
Nov  8 17:30:10 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:10 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:10 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:10 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:10 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:10 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:30:10 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:20 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:36:25 host kernel: __ratelimit: 572 callbacks suppressed
Nov  8 17:36:25 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: __ratelimit: 877 callbacks suppressed
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:44 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: __ratelimit: 1539 callbacks suppressed
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:49 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: __ratelimit: 1910 callbacks suppressed
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:54 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: __ratelimit: 1964 callbacks suppressed
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:44:59 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 113.160.248.120
Nov  8 17:45:00 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 113.160.248.120
Nov  8 17:45:03 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 113.160.248.120
Nov  8 17:45:03 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 113.160.248.120
Nov  8 17:45:04 host kernel: __ratelimit: 1815 callbacks suppressed
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:04 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:45:06 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 113.160.248.120
Nov  8 17:45:06 host fail2ban.actions[1833]: NOTICE [ssh-iptables] Ban 113.160.248.120
Nov  8 17:45:08 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 113.160.248.120
Nov  8 17:45:09 host kernel: __ratelimit: 1783 callbacks suppressed
Nov  8 17:45:09 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:47:35 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:47:35 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:47:35 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:55:07 host fail2ban.actions[1833]: NOTICE [ssh-iptables] Unban 113.160.248.120
Nov  8 18:04:25 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 195.154.71.4
Nov  8 18:04:25 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 195.154.71.4
Nov  8 18:04:26 host fail2ban.filter[1833]: INFO [ssh-iptables] Found 195.154.71.4
and this from mysqld.log:

Code: Select all

151108 14:29:43 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
151108 14:29:43 [Note] Server socket created on IP: '0.0.0.0'.
151108 14:29:43 [Note] Event Scheduler: Loaded 0 events
151108 14:29:43 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.46'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
151108 16:23:03 mysqld_safe Number of processes running now: 0
151108 16:23:03 mysqld_safe mysqld restarted
151108 16:23:04 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 24978 ...
151108 16:23:04 [Note] Plugin 'FEDERATED' is disabled.
151108 16:23:04 InnoDB: The InnoDB memory heap is disabled
151108 16:23:04 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151108 16:23:04 InnoDB: Compressed tables use zlib 1.2.3
151108 16:23:04 InnoDB: Using Linux native AIO
151108 16:23:04 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
151108 16:23:04 InnoDB: Completed initialization of buffer pool
151108 16:23:04 InnoDB: Fatal error: cannot allocate memory for the buffer pool
151108 16:23:04 [ERROR] Plugin 'InnoDB' init function returned error.
151108 16:23:04 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
151108 16:23:04 [ERROR] Unknown/unsupported storage engine: InnoDB
151108 16:23:04 [ERROR] Aborting

151108 16:23:04 [Note] /usr/libexec/mysqld: Shutdown complete

151108 16:23:04 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
151108 17:23:56 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
151108 17:23:56 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 28390 ...
151108 17:23:56 [Note] Plugin 'FEDERATED' is disabled.
151108 17:23:56 InnoDB: The InnoDB memory heap is disabled
151108 17:23:56 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151108 17:23:56 InnoDB: Compressed tables use zlib 1.2.3
151108 17:23:56 InnoDB: Using Linux native AIO
151108 17:23:56 InnoDB: Initializing buffer pool, size = 128.0M
151108 17:23:56 InnoDB: Completed initialization of buffer pool
151108 17:23:56 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 1728288280
151108 17:23:56  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 1728295247
151108 17:23:57  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
151108 17:23:57 InnoDB: 5.5.46 started; log sequence number 1728295247
151108 17:23:57 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
151108 17:23:57 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
151108 17:23:57 [Note] Server socket created on IP: '0.0.0.0'.
151108 17:23:57 [Note] Event Scheduler: Loaded 0 events
151108 17:23:57 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.46'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL) by Remi
151108 17:29:59 mysqld_safe Number of processes running now: 0
151108 17:29:59 mysqld_safe mysqld restarted
151108 17:29:59 [Note] /usr/libexec/mysqld (mysqld 5.5.46) starting as process 5816 ...
151108 17:29:59 [Note] Plugin 'FEDERATED' is disabled.
151108 17:29:59 InnoDB: The InnoDB memory heap is disabled
151108 17:29:59 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151108 17:29:59 InnoDB: Compressed tables use zlib 1.2.3
151108 17:29:59 InnoDB: Using Linux native AIO
151108 17:30:00 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
151108 17:30:00 InnoDB: Completed initialization of buffer pool
151108 17:30:00 InnoDB: Fatal error: cannot allocate memory for the buffer pool
151108 17:30:00 [ERROR] Plugin 'InnoDB' init function returned error.
151108 17:30:00 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
151108 17:30:00 [ERROR] Unknown/unsupported storage engine: InnoDB
151108 17:30:00 [ERROR] Aborting

151108 17:30:00 [Note] /usr/libexec/mysqld: Shutdown complete

151108 17:30:00 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
and finally the Apache log - /var/log/httpd/error_log:

Code: Select all

[Sun Nov 08 03:20:14 2015] [notice] Digest: generating secret for digest authentication ...
[Sun Nov 08 03:20:14 2015] [notice] Digest: done
[Sun Nov 08 03:20:14 2015] [notice] mod_ruid2/0.9.8 enabled
[Sun Nov 08 03:20:14 2015] [notice] Apache/2.2.15 (Unix) DAV/2 mod_fcgid/2.3.9 PHP/5.4.45 mod_ssl/2.2.15 OpenSSL/1.0.0-fips configured -- resuming normal o$
[Sun Nov 08 13:10:39 2015] [notice] caught SIGTERM, shutting down
[Sun Nov 08 13:11:22 2015] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Nov 08 13:11:22 2015] [notice] Digest: generating secret for digest authentication ...
[Sun Nov 08 13:11:22 2015] [notice] Digest: done
[Sun Nov 08 13:11:22 2015] [notice] mod_ruid2/0.9.8 enabled
[Sun Nov 08 13:11:23 2015] [notice] Apache/2.2.15 (Unix) DAV/2 mod_fcgid/2.3.9 PHP/5.4.45 mod_ssl/2.2.15 OpenSSL/1.0.0-fips configured -- resuming normal o$
[Sun Nov 08 13:11:41 2015] [error] server reached MaxClients setting, consider raising the MaxClients setting
[Sun Nov 08 18:43:28 2015] [notice] caught SIGTERM, shutting down
[Sun Nov 08 18:44:21 2015] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Nov 08 18:44:21 2015] [notice] Digest: generating secret for digest authentication ...
[Sun Nov 08 18:44:21 2015] [notice] Digest: done
[Sun Nov 08 18:44:21 2015] [notice] mod_ruid2/0.9.8 enabled
[Sun Nov 08 18:44:22 2015] [notice] Apache/2.2.15 (Unix) DAV/2 mod_fcgid/2.3.9 PHP/5.4.45 mod_ssl/2.2.15 OpenSSL/1.0.0-fips configured -- resuming normal o$
[Sun Nov 08 18:55:46 2015] [error] server reached MaxClients setting, consider raising the MaxClients setting
[Sun Nov 08 18:58:43 2015] [notice] caught SIGTERM, shutting down
[Sun Nov 08 18:58:44 2015] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Nov 08 18:58:44 2015] [notice] Digest: generating secret for digest authentication ...
[Sun Nov 08 18:58:44 2015] [notice] Digest: done
[Sun Nov 08 18:58:44 2015] [notice] mod_ruid2/0.9.8 enabled
[Sun Nov 08 18:58:45 2015] [notice] Apache/2.2.15 (Unix) DAV/2 mod_fcgid/2.3.9 PHP/5.4.45 mod_ssl/2.2.15 OpenSSL/1.0.0-fips configured -- resuming normal o$
[Sun Nov 08 18:59:00 2015] [error] server reached MaxClients setting, consider raising the MaxClients setting


Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 6:12 pm
by tjebbeke

Code: Select all

Nov  8 17:28:28 host kernel: nf_conntrack: table full, dropping packet.
Nov  8 17:28:33 host kernel: __ratelimit: 1291 callbacks suppressed
http://security.stackexchange.com/quest ... ing-packet

Code: Select all

Nov  8 17:29:58 host kernel: httpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Nov  8 17:29:58 host kernel: httpd cpuset=/ mems_allowed=0[
Nov  8 17:29:58 host kernel: Pid: 29179, comm: httpd Not tainted 2.6.32-573.7.1.el6.x86_64 #1/code]

If I search on google they usually said to create a swap.

Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 7:08 pm
by mehargags
Because usually only such blog sites have that visitor load...I presume this is a Wordpress blog... and can definitely say its because of some BAD plugin(s), especially any SEO related plugins that loop out too much traffic and payload choking your MySQL and other processes. I have fixed many such blogs so I know a clean WP install with plugin restore one by one is the solution.

Again... presuming its a WP site.. :)

Re: Mysql crash after some hours even i have 8gb Ram

Posted: Sun Nov 08, 2015 7:30 pm
by tomer628
mehargags wrote:Because usually only such blog sites have that visitor load...I presume this is a Wordpress blog... and can definitely say its because of some BAD plugin(s), especially any SEO related plugins that loop out too much traffic and payload choking your MySQL and other processes. I have fixed many such blogs so I know a clean WP install with plugin restore one by one is the solution.

Again... presuming its a WP site.. :)
ok if you say so...

this my plugins installed.. :
Add Meta Tags
AddThis Smart Layers
AddToAny Share Buttons
All In One SEO Pack
Dynamic To Top
Facebook Auto Publish
Facebook Comments Plugin
Google Analytics Dashboard for WP
Google Language Translator
Google XML Sitemaps
iThemes Security
Jetpack of Wordpress.com
WP-Sticky
Google analytics from Yoast
Duplicate Post