Facebook Debugger gets Error 500 On Rewritten URLs

[Galaxian]

/var/log/apache2 or httpd
this logs

error_log?

Here's an excerpt:

Code: Select all

[Wed Mar 04 15:42:03 2015] [notice] child pid 22198 exit signal Segmentation fault (11)
[Wed Mar 04 15:42:09 2015] [notice] child pid 22094 exit signal Segmentation fault (11)
[Wed Mar 04 15:42:09 2015] [notice] child pid 22197 exit signal Segmentation fault (11)
[Wed Mar 04 15:43:50 2015] [notice] child pid 22230 exit signal Segmentation fault (11)
[Wed Mar 04 15:44:23 2015] [notice] child pid 22229 exit signal Segmentation fault (11)
[Wed Mar 04 15:44:56 2015] [notice] child pid 22347 exit signal Segmentation fault (11)
[Wed Mar 04 15:45:44 2015] [notice] child pid 22139 exit signal Segmentation fault (11)
[Wed Mar 04 15:45:44 2015] [notice] child pid 22195 exit signal Segmentation fault (11)
[Wed Mar 04 15:46:40 2015] [notice] child pid 22742 exit signal Segmentation fault (11)
[Wed Mar 04 15:48:30 2015] [notice] child pid 22819 exit signal Segmentation fault (11)
[Wed Mar 04 15:49:12 2015] [notice] child pid 21320 exit signal Segmentation fault (11)
[Wed Mar 04 15:49:12 2015] [notice] child pid 22337 exit signal Segmentation fault (11)
[Wed Mar 04 15:49:13 2015] [notice] child pid 22370 exit signal Segmentation fault (11)
[Wed Mar 04 15:49:16 2015] [notice] child pid 22231 exit signal Segmentation fault (11)
[Wed Mar 04 15:49:16 2015] [notice] child pid 22924 exit signal Segmentation fault (11)
[Wed Mar 04 15:49:46 2015] [notice] child pid 22928 exit signal Segmentation fault (11)
[Wed Mar 04 15:50:12 2015] [notice] child pid 22923 exit signal Segmentation fault (11)
[Wed Mar 04 15:50:19 2015] [notice] child pid 22929 exit signal Segmentation fault (11)
[Wed Mar 04 15:50:26 2015] [notice] child pid 22966 exit signal Segmentation fault (11)
[Wed Mar 04 15:50:35 2015] [notice] child pid 22771 exit signal Segmentation fault (11)
[Wed Mar 04 15:50:46 2015] [notice] child pid 22925 exit signal Segmentation fault (11)
[Wed Mar 04 15:50:54 2015] [notice] child pid 23230 exit signal Segmentation fault (11)
[Wed Mar 04 15:51:02 2015] [notice] child pid 22930 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:07 2015] [notice] child pid 23226 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:07 2015] [notice] child pid 23277 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:07 2015] [notice] child pid 23288 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:09 2015] [notice] child pid 23232 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:19 2015] [notice] child pid 23249 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:22 2015] [notice] child pid 23286 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:24 2015] [notice] child pid 23447 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:25 2015] [notice] child pid 23287 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:25 2015] [notice] child pid 23435 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:25 2015] [notice] child pid 23449 exit signal Segmentation fault (11)
[Wed Mar 04 15:53:51 2015] [notice] child pid 23457 exit signal Segmentation fault (11)
[Wed Mar 04 15:54:47 2015] [notice] child pid 23452 exit signal Segmentation fault (11)
[Wed Mar 04 15:55:58 2015] [notice] child pid 23436 exit signal Segmentation fault (11)
[Wed Mar 04 15:56:14 2015] [notice] child pid 23456 exit signal Segmentation fault (11)
[Wed Mar 04 15:56:19 2015] [notice] caught SIGTERM, shutting down
[Wed Mar 04 15:56:20 2015] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Wed Mar 04 15:56:20 2015] [notice] Digest: generating secret for digest authentication ...
[Wed Mar 04 15:56:20 2015] [notice] Digest: done
[Wed Mar 04 15:56:20 2015] [notice] mod_ruid2/0.9.4 enabled
[Wed Mar 04 15:56:20 2015] [notice] Apache/2.2.15 (Unix) DAV/2 mod_fcgid/2.3.9 PHP/5.4.37 mod_ssl/2.2.15 OpenSSL/1.0.0-fips configured -- resuming normal operations
[Wed Mar 04 15:57:44 2015] [notice] child pid 23908 exit signal Segmentation fault (11)
[Wed Mar 04 15:58:08 2015] [notice] child pid 23910 exit signal Segmentation fault (11)
[Wed Mar 04 15:58:54 2015] [notice] child pid 23911 exit signal Segmentation fault (11)
[Wed Mar 04 15:58:55 2015] [notice] child pid 23913 exit signal Segmentation fault (11)
[Wed Mar 04 15:59:42 2015] [notice] child pid 24084 exit signal Segmentation fault (11)
[Wed Mar 04 16:00:29 2015] [notice] child pid 23909 exit signal Segmentation fault (11)
[Wed Mar 04 16:01:43 2015] [notice] child pid 23914 exit signal Segmentation fault (11)
[Wed Mar 04 16:02:37 2015] [notice] child pid 24655 exit signal Segmentation fault (11)

[Galaxian]

Here's another test with weird results (IMO):

http://www.funfactz.com/test.php
http://www.funfactz.com/test2/

Both load the same file, but test2 is rewritten. /test2/, I'm assuming would throw a 500 for the Facebook scraper. Also, test2 shows a Google IP even though I have remote_ip installed as required by PageSpeed Service? What's going on?

[skurudo]

[Wed Mar 04 15:45:44 2015] [notice] child pid 22139 exit signal Segmentation fault (11)

Now we cleary see, apache2 die sometimes and somehow.

It's time for you to debug script and webserver dump:
http://httpd.apache.org/dev/debugging.html
http://zecrazytux.net/troubleshooting/a ... g-tutorial
http://sysadmin.carlusgg.com/?p=197

[skurudo]

Both load the same file, but test2 is rewritten. /test2/, I'm assuming would throw a 500 for the Facebook scraper. Also, test2 shows a Google IP even though I have remote_ip installed as required by PageSpeed Service? What's going on?

Somewhere error, need to find..

[Galaxian]

It appears that for those conditions stated in the original post, the server is trying to load the URLs directly instead of firing the rewrites.

Also, for the base directory, it tries to load public_html/public_html/ ?

I've had a friend try to help me. He ran strace and came back with this:

Code: Select all

gettimeofday({1425576206, 783685}, NULL) = 0
gettimeofday({1425576206, 783723}, NULL) = 0
stat("/home/admin/web/funny-insults.com/public_html/comebacks/", 0x7fff73783150) = -1 ENOENT (No such file or directory)
lstat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/admin", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/admin/web", {st_mode=S_IFDIR|0751, st_size=4096, ...}) = 0
lstat("/home/admin/web/funny-insults.com", {st_mode=S_IFDIR|0551, st_size=4096, ...}) = 0
lstat("/home/admin/web/funny-insults.com/public_html", {st_mode=S_IFDIR|0751, st_size=4096, ...}) = 0
open("/home/admin/web/funny-insults.com/public_html/.htaccess", O_RDONLY|O_CLOEXEC) = 43
fcntl(43, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
fcntl(43, F_SETFD, FD_CLOEXEC)          = 0
fstat(43, {st_mode=S_IFREG|0664, st_size=754, ...}) = 0
read(43, "Options +FollowSymLinks\r\nRewriteEngine on\r\n\r\nRewriteCond %{HTTP_USER_AGENT} libwww-perl.* \r\nRewriteRule .* \342\200\223 [F,L]\r\n\r\nRewriteCond %{HTTP_HOST} ^funny-insults.com [NC]\r\nRewriteRule ^(.*)$ http://www.funny-insults.com/$1 [L,R=301]\r\n\r\nRewriteRule ^index\\.php$ - [L]\r\nRewriteCond %{REQUEST_FILENAME} !-f\r\nRewriteCond %{REQUEST_FILENAME} !-d\r\nRewriteRule . /index.php [L]\r\n\r\nRewriteCond %{HTTP:Accept-Encoding} gzip\r\nRewriteCond %{REQUEST_FILENAME}.jgz -f\r\nRewriteRule (.*)\\.js$ $1\\.js.jgz [L]\r\nAddType \"text/javascript\" .js.jgz\r\nAddEncoding gzip .jgz\r\n\r\n<FilesMatch \"\\.(ico|pdf|flv|jpg|jpeg|png|gif|js|css|swf)(\\.gz)?$\">\r\nHeader set Expires \"Thu, 15 Apr 2015 20:00:00 GMT\"\r\nHeader set Vary \"Accept-Encoding\"\r\nHeader unset ETag\r\nFileETag None\r\n</FilesMatch>", 4096) = 754
read(43, "", 4096)                      = 0
read(43, "", 4096)                      = 0
close(43)                               = 0
lstat("/home/admin/web/funny-insults.com/public_html/comebacks", 0x7fff73783150) = -1 ENOENT (No such file or directory)
capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
capget(0x20080522, 0, {0, CAP_SETGID|CAP_SETUID, 0}) = 0
capset(0x20080522, 0, {CAP_SETGID|CAP_SETUID, CAP_SETGID|CAP_SETUID, 0}) = 0
setgroups(1, [48])                      = 0
setgid(501)                             = 0
setuid(501)                             = 0
capget(0x20080522, 0, NULL)             = -1 EFAULT (Bad address)
capget(0x20080522, 0, {CAP_SETGID|CAP_SETUID, CAP_SETGID|CAP_SETUID, 0}) = 0
capset(0x20080522, 0, {0, CAP_SETGID|CAP_SETUID, 0}) = 0
stat("/home/admin/web/funny-insults.com/public_html/comebacks", 0x7fff73783090) = -1 ENOENT (No such file or directory)
stat("/home/admin/web/funny-insults.com/public_html/comebacks", 0x7fff73783090) = -1 ENOENT (No such file or directory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
chdir("/etc/httpd")                     = 0
rt_sigaction(SIGSEGV, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x7fdad7335710}, {SIG_DFL, [], SA_RESTORER|SA_RESETHAND, 0x7fdad7335710}, 8) = 0
kill(4964, SIGSEGV)                     = 0
rt_sigreturn(0x1364)                    = 254639434
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
Process 4964 detached

But he is quite stumped by the problem still.

I am considering migrating the sites that use Google PageSpeed Service away from VestaCP. However much I like this panel, it seems that this problem is dragging on and too complex for me.

[skurudo]

Try to use less redirects or use smart url / routing in php.

[joem]

I've done this, but I have no idea what I've just done or how it pertains to my question regarding error logs. Could you provide an explanation?

Sorry I often quote the last post, Anyways It has nothing to do with the error logs I was trying to provide you with a solution to fix your issue. I am guessing this did not work?

[Galaxian]

Try to use less redirects or use smart url / routing in php.

I don't know how I can use "less redirects" than it already is:

Code: Select all

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /index.php [L,QSA]

I've done this, but I have no idea what I've just done or how it pertains to my question regarding error logs. Could you provide an explanation?

Sorry I often quote the last post, Anyways It has nothing to do with the error logs I was trying to provide you with a solution to fix your issue. I am guessing this did not work?

Unfortunately not, so I had to undo those changes. :(

[skurudo]

stat("/home/admin/web/funny-insults.com/public_html/comebacks/", 0x7fff73783150) = -1 ENOENT (No such file or directory)
stat("/home/admin/web/funny-insults.com/public_html/comebacks", 0x7fff73783090) = -1 ENOENT (No such file or directory)
stat("/home/admin/web/funny-insults.com/public_html/comebacks", 0x7fff73783090) = -1 ENOENT (No such file or directory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---

I think, you need to see your scripts
and this may be need to check 1) memory 2) disk 3) database (if you're use one)

[Galaxian]

stat("/home/admin/web/funny-insults.com/public_html/comebacks/", 0x7fff73783150) = -1 ENOENT (No such file or directory)
stat("/home/admin/web/funny-insults.com/public_html/comebacks", 0x7fff73783090) = -1 ENOENT (No such file or directory)
stat("/home/admin/web/funny-insults.com/public_html/comebacks", 0x7fff73783090) = -1 ENOENT (No such file or directory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---

I think, you need to see your scripts
and this may be need to check 1) memory 2) disk 3) database (if you're use one)

This has nothing to do with my scripts.... It's an under-the-bonnet problem under the conditions that I specified in the first post.

Again, I'll break it down:

- AFAIK, only affects Facebook scraper. You know, when you share something on Facebook, it'll show 500 server error.
- Affects URL rewrites through apache and base URL, but not when using direct files
- Switching to the caching template seems to alleviate the problem somewhat, but I can't use this because of some dynamic features (is this not a big clue at all?)
- It is only on the websites that use Google PageSpeed Service. Yes, I've tried disabling all rewriters, etc, to no avail.

Edit:

My friend has done further diagnosis and found that the crash is occurring in: mod_extract_forwarded

This would explain why it only happens with sites using Google PageSpeed Service.

We don't know what to do from here.