We are happy to announce that Vesta is back under active development as of 25 February 2024. We are working on v1 candidate and expect to engage more with the community over the coming months. We are committed to open source, and we encourage contributors to help us build the future of Vesta.
what does "upstream timed out" error mean?
-
- Posts: 301
- Joined: Tue Dec 22, 2015 2:06 pm
what does "upstream timed out" error mean?
My Joomla website is running on VestaCP on CentOS 7. The PHP version is 5.6.21, which was installed with "PHP Selector" from skamasle.
Today I checked the log file /var/log/httpd/domains/joomlagate.com.error.log, there are following error records:
Thank you.
Today I checked the log file /var/log/httpd/domains/joomlagate.com.error.log, there are following error records:
What does "upstream timed out" error mean? How to fix this?2016/05/15 06:08:51 [error] 30728#30728: *1920053 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 42.236.48.92, server: joomlagate.com, request: "GET /index.php HTTP/1.1", upstream: "http://120.27.137.71:8080/index.php", host: "www.joomlagate.com", referrer: "http://www.joomlagate.com/index.php"
2016/05/15 06:08:51 [error] 30728#30728: *1920063 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 220.181.108.162, server: joomlagate.com, request: "GET /index.php?option=com_kunena&view=category&defaultmenu=155&Itemid=154&catid=46&limitstart=20 HTTP/1.1", upstream: "http://120.27.137.71:8080/index.php?opt ... itstart=20", host: "www.joomlagate.com"
2016/05/15 06:09:08 [error] 30729#30729: *1919841 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 59.127.53.104, server: joomlagate.com, request: "POST /?option=com_komento HTTP/2.0", upstream: "https://120.27.137.71:8443/?option=com_komento", host: "www.joomlagate.com", referrer: "https://www.joomlagate.com/"
2016/05/15 06:09:08 [error] 30729#30729: *1919841 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 59.127.53.104, server: joomlagate.com, request: "POST /?option=com_komento HTTP/2.0", upstream: "https://120.27.137.71:8443/?option=com_komento", host: "www.joomlagate.com", referrer: "https://www.joomlagate.com/"
2016/05/15 06:09:08 [error] 30729#30729: *1919844 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 117.136.0.91, server: joomlagate.com, request: "POST /?option=com_komento HTTP/2.0", upstream: "https://120.27.137.71:8443/?option=com_komento", host: "www.joomlagate.com", referrer: "https://www.joomlagate.com/"
2016/05/15 06:09:08 [error] 30729#30729: *1920045 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 106.120.85.34, server: joomlagate.com, request: "POST /?option=com_komento HTTP/2.0", upstream: "https://120.27.137.71:8443/?option=com_komento", host: "www.joomlagate.com", referrer: "https://www.joomlagate.com/"
2016/05/15 06:09:08 [error] 30729#30729: *1920045 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 106.120.85.34, server: joomlagate.com, request: "POST /?option=com_komento HTTP/2.0", upstream: "https://120.27.137.71:8443/?option=com_komento", host: "www.joomlagate.com", referrer: "https://www.joomlagate.com/"
2016/05/15 06:09:08 [error] 30729#30729: *1920056 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 203.208.60.233, server: joomlagate.com, request: "GET /index.php?option=com_kunena&view=topic&catid=59&id=61436&layout=unread&Itemid=154 HTTP/1.1", upstream: "http://120.27.137.71:8080/index.php?opt ... Itemid=154", host: "joomlagate.com"
2016/05/15 06:50:53 [crit] 30728#30728: *1921989 SSL_do_handshake() failed (SSL: error:14094085:SSL routines:SSL3_READ_BYTES:ccs received early) while SSL handshaking, client: 64.41.200.103, server: 120.27.137.71:443
2016/05/15 06:50:53 [crit] 30728#30728: *1921990 SSL_do_handshake() failed (SSL: error:14094085:SSL routines:SSL3_READ_BYTES:ccs received early) while SSL handshaking, client: 64.41.200.103, server: 120.27.137.71:443
Thank you.
Re: what does "upstream timed out" error mean?
That means it takes your web server more than 60 seconds to respond. To solve this problem in nginx.conf change the value of proxy_read_timeout directive. This directive determines how long nginx will wait to get the response to a request. By default it's 60 seconds. Change it to 300 seconds:
ref: http://howtounix.info/howto/110-connect ... r-in-nginx
Code: Select all
proxy_read_timeout 300;
-
- Posts: 301
- Joined: Tue Dec 22, 2015 2:06 pm
Re: what does "upstream timed out" error mean?
hi,@SS88, your instruction worked. After I insert that line to the "server" section of my snginx.conf (I use HTTPS) , there is no more "upstream timed out" errors recorded in the log file.
However, now I got following new errors, could you please give me further help to solve this?
However, now I got following new errors, could you please give me further help to solve this?
Thank you.[Sun May 15 08:26:54.946756 2016] [fcgid:warn] [pid 25306] [client 124.88.120.128:44564] mod_fcgid: read data timeout in 120 seconds, referer: https://www.joomlagate.com/index.php?op ... &Itemid=19
[Sun May 15 08:26:54.946818 2016] [core:error] [pid 25306] [client 124.88.120.128:44564] End of script output before headers: index.php, referer: https://www.joomlagate.com/index.php?op ... &Itemid=19
[Sun May 15 08:33:11.776640 2016] [fcgid:warn] [pid 25308] [client 101.226.166.218:44846] mod_fcgid: read data timeout in 120 seconds, referer: https://120.27.137.71/index.php?Itemid= ... w=category
[Sun May 15 08:33:11.776699 2016] [core:error] [pid 25308] [client 101.226.166.218:44846] End of script output before headers: index.php, referer: https://120.27.137.71/index.php?Itemid= ... w=category
[Sun May 15 08:42:05.375294 2016] [fcgid:warn] [pid 25009] [client 113.240.234.205:45175] mod_fcgid: read data timeout in 120 seconds, referer: https://www.joomlagate.com/
[Sun May 15 08:42:05.375362 2016] [core:error] [pid 25009] [client 113.240.234.205:45175] End of script output before headers: index.php, referer: https://www.joomlagate.com/
Re: what does "upstream timed out" error mean?
Change these to all 300.
300 = 5 minutes
If your pages are not loading within 5 minutes I would look into why. I had a browse and pages were loading for me quite quickly so you may want to take a look as to why these pages are not loading within 120 seconds (as that's a very long time to wait)
You also may need to increase FcgidIOTimeout to 300 (this setting is a Fast CGI setting)
Code: Select all
keepalive_timeout 300;
proxy_read_timeout 300;
proxy_connect_timeout 300;
fastcgi_read_timeout 300;
If your pages are not loading within 5 minutes I would look into why. I had a browse and pages were loading for me quite quickly so you may want to take a look as to why these pages are not loading within 120 seconds (as that's a very long time to wait)
You also may need to increase FcgidIOTimeout to 300 (this setting is a Fast CGI setting)
-
- Posts: 301
- Joined: Tue Dec 22, 2015 2:06 pm
Re: what does "upstream timed out" error mean?
Today I met "upstream timed out" error again on my re-installed VPS server. So I followed your above post to change those timing values.
Well, I found all those 4 parameters in this file: /etc/nginx/nginx.conf.
However, the format of "keepalive_timeout" directive is not a single value, but a pair :
This is the default value of that line.
What kind of new value should I change to for this line?
Thank you.
Well, I found all those 4 parameters in this file: /etc/nginx/nginx.conf.
However, the format of "keepalive_timeout" directive is not a single value, but a pair :
Code: Select all
keepalive_timeout 60 60;
What kind of new value should I change to for this line?
Thank you.
Re: what does "upstream timed out" error mean?
baijianpeng wrote:Today I met "upstream timed out" error again on my re-installed VPS server. So I followed your above post to change those timing values.
Well, I found all those 4 parameters in this file: /etc/nginx/nginx.conf.
However, the format of "keepalive_timeout" directive is not a single value, but a pair :
This is the default value of that line.Code: Select all
keepalive_timeout 60 60;
What kind of new value should I change to for this line?
Thank you.
The first parameter sets a timeout during which a keep-alive client connection will stay open on the server side. The zero value disables keep-alive client connections. The optional second parameter sets a value in the “Keep-Alive: timeout=time” response header field. Two parameters may differ.Syntax: keepalive_timeout timeout [header_timeout];
Default: keepalive_timeout 75s;
Context: http, server, location
The “Keep-Alive: timeout=time” header field is recognized by Mozilla and Konqueror. MSIE closes keep-alive connections by itself in about 60 seconds.
http://nginx.org/en/docs/http/ngx_http_ ... ve_timeout
-
- Posts: 2
- Joined: Tue Jun 28, 2016 9:23 pm
Re: what does "upstream timed out" error mean?
May i ask why you would set a tmeout limit of 5 minutes? asking for DDos troubles here?
Re: what does "upstream timed out" error mean?
If you're under a real DDoS you have no chance anyway.ashleycalebhart wrote:May i ask why you would set a tmeout limit of 5 minutes? asking for DDos troubles here?
-
- Posts: 301
- Joined: Tue Dec 22, 2015 2:06 pm
Re: what does "upstream timed out" error mean?
Well, I tried to change both number to "300" for the "keepalive_timeout" parameter, then I saved that conf file and tried to restart Nginx, then I got following error:
At the same time, I checked the Nginx log files and found following records:
But, my website pages can still be opened. This is weird!
Code: Select all
root@mail:/etc/nginx# systemctl restart nginx
Job for nginx.service failed because the control process exited with error code. See "systemctl status nginx.service" and "journalctl -xe" for details.
root@mail:/etc/nginx# systemctl status nginx
● nginx.service - LSB: Stop/start nginx
Loaded: loaded (/etc/init.d/nginx; bad; vendor preset: enabled)
Active: failed (Result: exit-code) since Sat 2017-01-28 08:28:44 CST; 9s ago
Docs: man:systemd-sysv-generator(8)
Process: 6286 ExecStop=/etc/init.d/nginx stop (code=exited, status=0/SUCCESS)
Process: 6325 ExecStart=/etc/init.d/nginx start (code=exited, status=1/FAILURE)
CGroup: /system.slice/nginx.service
├─25307 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.con
├─25308 nginx: worker process
└─25309 nginx: cache manager process
Jan 28 08:28:43 mail.joomlagate.com nginx[6325]: nginx: [emerg] bind() to 120.27.137.71:443 failed (98: Address a
Jan 28 08:28:43 mail.joomlagate.com nginx[6325]: nginx: [emerg] bind() to 10.47.93.1:80 failed (98: Address alrea
Jan 28 08:28:43 mail.joomlagate.com nginx[6325]: nginx: [emerg] bind() to 120.27.137.71:80 failed (98: Address al
Jan 28 08:28:43 mail.joomlagate.com nginx[6325]: nginx: [emerg] bind() to 127.0.0.1:8084 failed (98: Address alre
Jan 28 08:28:43 mail.joomlagate.com nginx[6325]: nginx: [emerg] bind() to 120.27.137.71:443 failed (98: Address a
Jan 28 08:28:44 mail.joomlagate.com nginx[6325]: nginx: [emerg] still could not bind()
Jan 28 08:28:44 mail.joomlagate.com systemd[1]: nginx.service: Control process exited, code=exited status=1
Jan 28 08:28:44 mail.joomlagate.com systemd[1]: Failed to start LSB: Stop/start nginx.
Jan 28 08:28:44 mail.joomlagate.com systemd[1]: nginx.service: Unit entered failed state.
Jan 28 08:28:44 mail.joomlagate.com systemd[1]: nginx.service: Failed with result 'exit-code'.
lines 1-21/21 (END)
Now my Nginx can NOT be restarted, even after I change those numbers back to "60 60".root@mail:/var/log/nginx# tail error.log
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 120.27.137.71:443 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 10.47.93.1:80 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 120.27.137.71:80 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 127.0.0.1:8084 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 120.27.137.71:443 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 10.47.93.1:80 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 120.27.137.71:80 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 127.0.0.1:8084 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: bind() to 120.27.137.71:443 failed (98: Address already in use)
2017/01/28 08:28:41 [emerg] 6331#6331: still could not bind()
But, my website pages can still be opened. This is weird!
-
- Posts: 301
- Joined: Tue Dec 22, 2015 2:06 pm
Re: what does "upstream timed out" error mean?
Ok, since I am not a pro , so I choose to "reboot" my server. After reboot, Nginx seems OK now.
Code: Select all
root@mail:~# systemctl status nginx
● nginx.service - LSB: Stop/start nginx
Loaded: loaded (/etc/init.d/nginx; bad; vendor preset: enabled)
Active: active (running) since Sat 2017-01-28 08:37:55 CST; 1min 35s ago
Docs: man:systemd-sysv-generator(8)
Process: 783 ExecStart=/etc/init.d/nginx start (code=exited, status=0/SUCCESS)
CGroup: /system.slice/nginx.service
├─1037 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.con
├─1038 nginx: worker process
└─1039 nginx: cache manager process
Jan 28 08:37:54 mail.joomlagate.com systemd[1]: Starting LSB: Stop/start nginx...
Jan 28 08:37:55 mail.joomlagate.com systemd[1]: Started LSB: Stop/start nginx.