Figured this out.
I was using nginx as a reverse proxy for Apache.
Turns out that when it was getting an https hit and passing it to apache, it wasn’t setting a certain header “X-Forwarded-Proto” so wordpress’s is_ssl() function wasn’t recognizing the ssl.
I added proxy_set_header X-Forwarded-Proto $scheme; to my nginx config, which solved all the problems.
