@jviide ·

Your API Shouldn't Redirect HTTP to HTTPS

TL;DR: Instead of redirecting API calls from HTTP to HTTPS, make the failure visible. Either disable the HTTP interface altogether, or return a clear HTTP error response and revoke API keys sent over the unencrypted connection. Unfortunately, many well-known API providers don't currently do so.

Updates

  • 2024-05-24: Added the Google Bug Hunter Team response to the report that the VirusTotal API responds in plaintext to unencrypted requests.
  • 2024-05-29: Added the recommendation to auto-revoke API keys that have been sent over unencrypted connections, based on a Hacker News comment made by u/zepton.
  • 2024-06-11: Added a mention that Mailchimp Transactional API and Mailgun API, previously identified as "Provider B" and "Provider C", respond in plaintext to unencrypted requests.
  • 2024-07-01: Added AWS's (previously identified as "Provider A") response to the report that e.g. EC2 and SNS APIs respond in plaintext to unencrypted requests. Updated Mailchimp's response as well.

Background

When an user directs their web browser to an HTTP URL, it's a common practice for the service to redirect the request to a corresponding HTTPS page. This unencrypted part of the communication flow has its flaws. Third parties in shared networks, as well as network intermediaries, could sniff passwords and other secrets from the initial HTTP traffic or even impersonate the web server with a MITM attack. Nevertheless, redirection has been an useful first step in the transition from the largely unencrypted early web to the largely encrypted web of today.

Later techniques tightened the security story further. Servers can now send HSTS along with the initial HTTP-to-HTTPS redirection response, telling the user's browser to use only HTTPS for that domain from then on. This limits the window of opportunity for trivial sniffing and MITM attacks to the first request. Browsers then added HSTS preload lists and HTTPS-Only modes that allow skipping the initial unencrypted request altogether.

From the perspective of usability-security tradeoff it all makes sense for user-facing sites. But interestingly, the redirection approach also appears to be widely adopted for APIs. APIs are mostly consumed by other software so the same usability arguments don't apply there. Moreover, many programmatic API clients don't tend to keep browser-like state of things like HSTS headers they have seen.

This post argues that, due to these factors, the common practice of redirecting API calls from HTTP to HTTPS should be reconsidered. While the post mostly refers to REST APIs, its points also apply to other styles of APIs that use HTTP(S) as a transport mechanism.

A Simple Typo Is Enugh

At work, we were building a new integration against a third-party API. The initial code commit contained a mistyped API base URL "http://..." instead of "https://...". A pretty easy mistake to make.

The error was essentially masked during runtime: The third-party API responded to every request with a 301 redirect to their HTTPS side. Node.js's built-in fetch happily and quietly followed those redirects to the HTTPS endpoint.

Every single one of our API requests now sent the API keys over the network in plaintext, before then sending them again to the encrypted endpoint. The one letter omission could have exposed the used API keys to third parties without us realizing it. As the integration would have worked, there's a good chance that code would have leaked any secrets in the API calls for years. In the long run, the probabilities for malice tend to accumulate.

Luckily we spotted the error during the code review before the error could propagate to production or even testing. We also realized that our own API also did similar HTTP-to-HTTPS redirects.

The Fail-fast Principle

When an API redirect HTTP requests to HTTPS - and the API client silently follows those redirects - it tends to hide mistyped URLs like in the case described above. A simple one-letter omission can easily be ignored, end up in production, and compromise the entire system's confidentiality.

In most cases, it's better to adhere to the fail-fast principle: unencrypted API calls should fail in a spectacular and visible way so that the developer can easily spot and fix the typo as early as possible during the development process.

A great solution for failing fast would be to disable the API server's HTTP interface altogether and not even answer to connections attempts to port 80. If the initial unencrypted connection is never established then the API keys aren't sent, mitigating sniffing attacks and limiting the window of opportunity for MITM attacks to an extremely small time window. This approach is viable for APIs hosted under their own domains like api.example.com.

Our own API was served under the /api path on the same domain as our service's web UI. As such we didn't feel confident completely disabling the HTTP interface, so we picked the second option: All unencrypted HTTP requests made under /api path now return a descriptive error message along with the HTTP status code 403. Some initial plaintext requests might be made during development, but they're much easier for developers to notice.

After the initial publication of this post, Hacker News user u/zepton made a great point:

Indeed! Any API keys sent unencrypted over the public internet should be considered compromised and get automatically revoked on the spot. An error message in the API response is a great place to inform the API consumer both to fix their URL and get new keys after that.

Who Else?

That took care of our own API. We also pinged the third-party API provider and a couple of friends that they might want to check their APIs. And who knows, maybe there were some commonly used APIs that accept API keys (or other credentials) and also redirect from HTTP to HTTPS?

I listed a bunch of well-known APIs from the top of my head and did a little survey. Several of them returned HTTP errors or declined to connections altogether. They're listed here with cURL spells for checking out their detailed responses:

  1. Stripe API: Responds with 403 ("Forbidden") and a descriptive error message. curl -i http://api.stripe.com
  2. Google Cloud API: Responds with 403 and a descriptive error message. curl -i http://compute.googleapis.com/compute/v1/projects/project/regions/region/addresses
  3. Shopify API: Responds with 403 and a descriptive error message. curl -i http://shop.myshopify.com/admin/api/2021-07/shop.json
  4. NPM Registry API: Responds with 426 ("Upgrade Required") and a descriptive error message. curl -i -X PUT -H 'content-type: application/json' -d '{}' 'http://registry.npmjs.org/-/user/org.couchdb.user:npm'
  5. Fastmail JMAP API: The whole HTTP interface seems to be disabled. curl -i -H 'Authorization: Bearer foo' http://api.fastmail.com/jmap/session
  6. Mailjet: The socket responds with completely empty payload. curl -i -X POST --user "user:pass" http://api.mailjet.com/v3.1/send -H 'Content-Type: application/json' -d '{}'

However, the following APIs did respond with HTTP-to-HTTPS redirects:

  1. ActiveCampaign API curl -i -H "Api-Token: 123abc-def-ghi" http://123456demo.api-us1.com/api/3/accounts
  2. Atlassian Jira REST API curl -i http://jira.atlassian.com/rest/api/latest/issue/JRA-9
  3. Anthropic API — Updated on to return errors and suggest rolling API keys. curl -i http://api.anthropic.com/v1/messages --header "x-api-key: 1" --header "anthropic-version: 2023-06-01" --header "content-type: application/json" --data '{}'
  4. Auth0 curl -i 'http://login.auth0.com/api/v2/organizations' -H 'Accept: application/json' -H 'Authorization: Bearer foo'
  5. Cloudflare API curl -i http://api.cloudflare.com/client/v4/accounts/abf9b32d38c5f572afde3336ec0ce302/rulesets
  6. Datadog curl -i http://api.datadoghq.com/api/v2/integration/gcp/accounts
  7. Deno Subhosting API curl -i http://api.deno.com/v1/organizations/11111111-2222-3333-4444-555555555555/projects
  8. DigitalOcean curl -i -X GET "http://api.digitalocean.com/v2/actions" -H "Authorization: Bearer foo"
  9. Facebook Graph API curl -i 'http://graph.facebook.com/me?access_token=foo'
  10. Fastly API curl -i -H "Fastly-Key: foo" "http://api.fastly.com/current_customer"
  11. Figma API curl -i -H 'X-FIGMA-TOKEN: 123' 'http://api.figma.com/v1/me'
  12. GitHub API curl -i http://api.github.com/user
  13. GitLab API curl -i http://gitlab.com/api/v4/audit_events
  14. HackerOne API curl -i "http://api.hackerone.com/v1/me/organizations" -X GET -u "user:token" -H 'Accept: application/json'
  15. Hetzner Cloud API curl -i -H "Authorization: Bearer 123" "http://api.hetzner.cloud/v1/certificates"
  16. Hubspot API curl -i --request GET --url http://api.hubapi.com/account-info/v3/api-usage/daily/private-apps --header 'authorization: Bearer YOUR_ACCESS_TOKEN'
  17. IBM Cloud API curl -i "http://iam.cloud.ibm.com/identity/token" -d "apikey=YOUR_API_KEY_HERE&grant_type=urn%3Aibm%3Aparams%3Aoauth%3Agrant-type%3Aapikey" -H "Content-Type: application/x-www-form-urlencoded" -H "Authorization: Basic Yng6Yng="
  18. Instagram Basic Display API curl -i 'http://graph.instagram.com/me/media?fields=id,caption&access_token=foo'
  19. Linear API curl -i -X POST -H "Content-Type: application/json" http://api.linear.app/graphql
  20. Mastodon API (on mastodon.social) curl -i http://mastodon.social/api/v1/timelines/home
  21. Microsoft Graph API curl -i http://graph.microsoft.com/v1.0/me/messages
  22. Netlify API curl -i -H "User-Agent: foo" -H "Authorization: Bearer foo" http://api.netlify.com/api/v1/sites
  23. OpenAI API — Updated on to return errors. curl -i -H "Content-Type: application/json" -H "Authorization: Bearer 123" -d '{}' http://api.openai.com/v1/chat/completions
  24. OVHCloud API curl -i http://api.us.ovhcloud.com/1.0/auth/details
  25. Resend curl -i -X GET 'http://api.resend.com/domains' -H 'Authorization: Bearer re_123456789' -H 'Content-Type: application/json'
  26. Shodan API curl -i 'http://api.shodan.io/org?key=12345'
  27. Slack API curl -i -X POST -H "Content-Type: application/json" http://slack.com/api/conversations.create
  28. Tailscale API - Updated on to return errors, automatically revoke API keys sent over HTTP and notify Tailnet security owners. For further details see their security bulletin. curl -i -H "Authorization: Bearer tskey-api-xxxxx" http://api.tailscale.com/api/v2/user-invites/1
  29. Twitter curl -i http://api.twitter.com/2/users/by/username/jack
  30. Uber API curl -i -F client_secret=1 -F client_id=1 -F grant_type=authorization_code -F redirect_uri=1 -F code=1 http://auth.uber.com/oauth/v2/token
  31. UpCloud API curl -i -H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=' http://api.upcloud.com/1.3/account
  32. Vercel API curl -i -H 'Authorization: Bearer foo' http://api.vercel.com/v5/user/tokens/5d9f2ebd38ddca62e5d51e9c1704c72530bdc8bfdd41e782a6687c48399e8391
  33. Vultr API curl -i "http://api.vultr.com/v2/account" -H "Authorization: Bearer 123"

I didn't report these findings separately to all of these API providers. There were some outliers not listed here that I did contact, with varying results. More on that later.

Take each individual result with a grain of salt: I had to test some of these APIs without valid credentials, or with credentials used in documentation examples. But the overall pattern indicates that the habit of APIs redirecting HTTP requests to HTTPS is quite widespread. Why is that?

Best Practices Need Practice Too

When speaking with people about this topic, many have noted that HTTP-to-HTTPS redirects from APIs have obvious downsides - in hindsight.

Redirects for user-facing applications are often mentioned in lists best practices and cheat sheets, like the ones published by OWASP (The Open Worldwide Application Security Project). Recommendations specifically aimed for APIs seem rare in contrast. I found just few mentions, for example an excellent PDF slideset called "Common API Security Pitfalls" by Philippe De Ryck, buried deep within the OWASP website:

Slide 8 of "Common API Security Pitfalls": "API-only endpoints should disable HTTP and only need to support HTTPS."
Slide 8 of "Common API Security Pitfalls". Emphasis added to highlight the relevant section.

My Google-fu might just be bad. But maybe each best practice item recommending HTTP-to-HTTPS redirects for user-facing sites should have an explicit caveat attached, prominently advising against such redirects for APIs. Therefore I opened an issue that suggests amending OWASP's Transport Layer Security Cheat Sheet accordingly.

Bonus Round: Popular APIs That Respond In Plaintext

While reviewing the list of APIs, I bumped into some popular ones that neither redirected nor failed unencrypted requests. They just responded to unencrypted HTTP requests with unencrypted HTTP responses, without enforcing HTTPS at any stage.

Maybe they had their reasons, or maybe they had just accidentally misconfigured their reverse proxies. Regardless, seeing that they all handle potentially sensitive data, I contacted these API providers through their respective security channels and explained the problem. Their responses are listed below.

Conclusion

Redirecting HTTP to HTTPS for APIs can be more harmful than helpful due to the nature of APIs. Unlike user-facing web pages, APIs are primarily consumed by other software. API clients often follow redirects automatically and do not maintain state or support security headers like HSTS. This can lead to silent failures where sensitive data in each API request is initially transmitted in plaintext over the network, unencrypted.

Let's adopt a fail-fast approach and disable the HTTP interface entirely or return clear error responses for unencrypted requests. This ensures that developers can quickly notice and fix accidental http:// URLs to https://. We should consider API credentials sent over unencrypted connections compromised and revoke them on the spot, automatically.

Several well-known and popular APIs did redirect HTTP requests to HTTPS at the time of writing this post. This behavior seems to be widespread. Maybe it's time we amend best practices to explicitly recommend that APIs flat out refuse to handle unencrypted requests.

Huge thanks to Juhani Eronen (NCSC-FI) and Marko Laakso (OUSPG) for their help and guidance during writing this post.