WordPress.org

Ready to get started?Download WordPress

Forums

Jetpack by WordPress.com
Jetpack returning bad 302 HTTP format to varnish (15 posts)

  1. Jasonheffner
    Member
    Posted 1 year ago #

    I'm currently trying to resolve an issue specific to varnish and jetpack. When clicking on Authorize jetpack after enabling the plugin and being sent back to our site, varnish will present a 503 Guru mediation error. An example string looks like this..

    http://somesite.psu.edu/someblog/wp-admin/admin.php?page=jetpack&action=authorize&_wpnonce=cec3b377a1&code=vjBHdZkLIi&state=2

    If the page is reloaded with the same url the site will return as it should, and display "Your JetPack is already connected." The initial 302 redirect is malformed and varnish won't display it.

    Some info from varnish.. note varnish is simply passing the request. If I take varnish out it simply works since the browser will display the malformed 302 and redirect.

    12 RxRequest    c GET
       12 RxURL        c /someblog/wp-admin/admin.php?page=jetpack&action=authorize&_wpnonce=cec3b377a1&code=83VksrsrRa&state=2
       12 RxProtocol   c HTTP/1.1
       12 RxHeader     c Host: somesite.psu.edu
       12 RxHeader     c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:18.0) Gecko/20100101 Firefox/18.0
       12 RxHeader     c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
       12 RxHeader     c Accept-Language: en-US,en;q=0.5
       12 RxHeader     c Accept-Encoding: gzip, deflate
       12 RxHeader     c Cookie: __utma=165035810.2146825052.1359667423.1359667423.1359667423.1; __utmb=165035810.13.10.1359667423; __utmc=165035810; __utmz=165035810.1359667423.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); bp-activity-oldestpage=1; __qca=P0-1744595654-13596
       12 RxHeader     c Connection: keep-alive
       12 RxHeader     c X-Forwarded-For: 128.118.1.123
       12 VCL_call     c recv pass
       12 VCL_call     c hash
       12 Hash         c /someblog/wp-admin/admin.php?page=jetpack&action=authorize&_wpnonce=cec3b377a1&code=83VksrsrRa&state=2
       12 Hash         c somesite.psu.edu
       12 VCL_return   c hash
       12 VCL_call     c pass pass
       12 Backend      c 20 default_director web1
       12 FetchError   c http format error
       12 VCL_call     c error deliver
       12 VCL_call     c deliver deliver
       12 TxProtocol   c HTTP/1.1
       12 TxStatus     c 503
       12 TxResponse   c Service Unavailable
       12 TxHeader     c Server: Varnish
       12 TxHeader     c Content-Type: text/html; charset=utf-8
       12 TxHeader     c Retry-After: 5
       12 TxHeader     c Content-Length: 418
       12 TxHeader     c Accept-Ranges: bytes
       12 TxHeader     c Date: Thu, 31 Jan 2013 21:26:07 GMT
       12 TxHeader     c Age: 3
       12 TxHeader     c Connection: close
       12 TxHeader     c X-Cache: MISS
       12 Length       c 418
       12 ReqEnd       c 812364436 1359667565.384453297 1359667567.905995607 0.000034332 2.521497965 0.000044346

    I have ruled out the varnish config, timeouts, load, ssl, and other factors. I have recreated the problem on two separate multisite installs.

    http://wordpress.org/extend/plugins/jetpack/

  2. Jeremy Herve
    Happiness Engineer
    Plugin Author

    Posted 1 year ago #

    While I am not really familiar with Varnish environments, other Jetpack users have reported issues in the past, and the following fix seems to solve the issue:
    http://usefulmix.com/wordpress-nginx-varnish-jetpack-xml_rpc-32700-glitch-fix/

    Could you have a look at it, and let me know if this helps?

    If it doesn't, could you let me know your site URL? If you want it to remain private, you can also contact us via this form:
    http://en.support.wordpress.com/contact/?jetpack=needs-service

  3. Jasonheffner
    Member
    Posted 1 year ago #

    That fix just bypasses varnish for your entire site. Unfortunately that is not going to work in our case as we have multiple users all connecting Jetpack from within our multisite installation. Yes, it does work for displaying the malformed response from the plugin as I have tried it, however, once varnish is proxying it spits it out.

    The call does actually go through and enables the plugin, and a refresh will create the correct response. I believe the initial response to the call is a malformed return and rejected by Varnish. I've seen some folks using a squid proxy to reconstruct the malformed response. I'm going to look more in depth as to why the response is malformed, unless you can help?

    fyi.. this started happening with the latest release and is the only page having problems on the site.

  4. Jeremy Herve
    Happiness Engineer
    Plugin Author

    Posted 1 year ago #

    I'll forward that issue to someone who has a better understanding of how Varnish works, if you don't mind. :)

    We'll get back to you as soon as we can address the issue.

  5. Jasonheffner
    Member
    Posted 1 year ago #

    Thanks, We really appreciate it. I sent the site name and info to your support form citing this page.

    I also contacted a few with varnish to try and determine why it sees the response as malformed. I may have to do a capture in order to see the full details.

  6. Jasonheffner
    Member
    Posted 1 year ago #

    I've managed to increase varnish settings to compensate for the large header that this particular request sends back to varnish. I still believe there is a coding issue with the session cookies jetpack creates in the header return.

    In varnish, I increased http_max_hdr and http_resp_size above the defaults to compensate.

  7. George Stephanis
    Automattic Lead Code Monkey
    Plugin Author

    Posted 1 year ago #

    Howdy, Jason!

    Just trying to confirm one thing -- do you have Varnish Caching turned on or off for your admin directories and xmlrpc.php?

    If it's on, I'd recommend turning it off via something like ...

    # Exclude the following
    if (req.url ~ "^/([^/]*/?)wp-admin/(.*)" || req.url ~ "^/xmlrpc\.php" )  {
        return(pass);
    }

    (I'm not a varnish or regex expert, but I believe that's right)

    If it's already off, then yeah, the headers are probably the issue.

  8. Jasonheffner
    Member
    Posted 1 year ago #

    It's been turned off.. the tcpdump I sent them confirms it's the headers. I can configure varnish to compensate for the large headers, but question how it was coded.

    if (req.url ~ "wp-(login|admin)|login" || req.url ~ "preview=true" || req.url ~ "xmlrpc.php") {
    return (pass);
    }

  9. Jasonheffner
    Member
    Posted 1 year ago #

    George,

    Here was the header that call initiates and the dump I sent them a few months back. I think you can see the issue then and how they are checking states.

    Date: Fri, 01 Feb 2013 21:43:08 GMT
    Server: Apache
    Expires: Wed, 11 Jan 1984 05:00:00 GMT
    Cache-Control: no-cache, must-revalidate, max-age=0
    Pragma: no-cache
    X-Frame-Options: SAMEORIGIN
    Set-Cookie: jetpackState[message]=authorized; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[message]=authorized; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=widgets; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=latex; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=shortcodes; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=custom-css; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=stats; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=enhanced-distribution; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=after-the-deadline; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=sharedaddy; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=shortlinks; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy%2Cshortlinks; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=gravatar-hovercards; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy%2Cshortlinks%2Cgravatar-hovercards; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=subscriptions; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy%2Cshortlinks%2Cgravatar-hovercards%2Csubscriptions; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=contact-form; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy%2Cshortlinks%2Cgravatar-hovercards%2Csubscriptions%2Ccontact-form; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=post-by-email; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy%2Cshortlinks%2Cgravatar-hovercards%2Csubscriptions%2Ccontact-form%2Cpost-by-email; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=mobile-push; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy%2Cshortlinks%2Cgravatar-hovercards%2Csubscriptions%2Ccontact-form%2Cpost-by-email%2Cmobile-push; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=publicize; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy%2Cshortlinks%2Cgravatar-hovercards%2Csubscriptions%2Ccontact-form%2Cpost-by-email%2Cmobile-push%2Cpublicize; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=module_activation_failed; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=notes; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[reactivated_modules]=vaultpress%2Cwidgets%2Clatex%2Cshortcodes%2Ccustom-css%2Cstats%2Cenhanced-distribution%2Cafter-the-deadline%2Csharedaddy%2Cshortlinks%2Cgravatar-hovercards%2Csubscriptions%2Ccontact-form%2Cpost-by-email%2Cmobile-push%2Cpublicize%2Cnotes; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[error]=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[module]=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/wp-admin; domain=somesite.psu.edu
    Set-Cookie: jetpackState[php_errors]=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/wp-admin; domain=somesite.psu.edu
    Location: https://somesite.psu.edu/wp-admin/admin.php?page=jetpack
    Connection: close
    Transfer-Encoding: chunked
    Content-Type: text/html; charset=UTF-8
  10. George Stephanis
    Automattic Lead Code Monkey
    Plugin Author

    Posted 1 year ago #

    Oh dear me, yes, that's a lot of cookies.

  11. George Stephanis
    Automattic Lead Code Monkey
    Plugin Author

    Posted 1 year ago #

    Well, if we can get the errors causing those resolved, the problem should go away.

    Are you getting anything in your PHP error log indicating why module activation could have failed?

  12. Jasonheffner
    Member
    Posted 1 year ago #

    Module activation doesn't fail. The call from the backend fails to be displayed to the client because varnish chokes on the header, and delivers a 503. Refreshing the page on the client side shows the module already enabled.

    I suspect a looping issue on module activation because of all the cookies created. jetpackState[error] and jetpackState[module] are set numerous times in the same header.

    I'm able to alleviate the error by increasing the http_max_hdr and http_resp_size in Varnish as mentioned above, but the initial cause is still there.

  13. Jasonheffner
    Member
    Posted 1 year ago #

    George, there was nothing in the logs about any of the individual module_activation_failed errors from the cookies. The modules in jetpack do get enabled properly upon activation. As you can see some kind of loop happens when enabling them all in jetpack activation the first time on a site.

  14. Tim Moore
    Moderator
    Plugin Author

    Posted 1 year ago #

    @Jasonheffner: is your Varnish config set up to save cookies or no? It looks like attempts are being made to set the same cookie over and over and failing.

  15. Jasonheffner
    Member
    Posted 1 year ago #

    @Tim I don't save cookies in varnish, but if a user is logged in the session is passed to apache and not cached. The above was a tcpdump between varnish and apache and was the only traffic I saw. Does jetpack make additional calls in process to set the cookies and hence generate those failed returns?

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic

Tags

No tags yet.