Cloudflare's forbidden Steam

I’m moving more code into Cloudflare workers everyday as they are cheap, fast, and powerful. But it hasn’t been without struggle.

This is the story of one such struggle with a troublesome endpoint.

The endpoint in question verifies the OpenID authentication signature directly with Steam. In short, this code runs after the user authenticates with Steam, and when we receive a callback, verify the data within is recognized by Steam.

When I deployed the site, the Steam login flow kept failing with a 403 forbidden error.

I spent several hours slowly debugging the issue. I had a fetch request that would work locally and on other servers, but not on Cloudflare.

An online search yielded others struggling with the same issue: Steam blocking requests from Cloudflare. The answer suggested Steam blocks requests from Cloudflare IPs.

Trust but verify. I set up traefik on a VPS to proxy requests to our intended Steam endpoint. The proxy worked for local node.js requests but failed for Cloudflare. The proxy should be masking the client’s IP so Steam isn’t blocking Cloudflare IPs and the culprit lies elsewhere.

HTTP headers

Could there be discrepancies in fetch implementations? Using tools like httpbin.org/headers I noticed the headers from a node.js fetch were different from Cloudflare’s fetch. Every fetch request that leaves a Cloudflare worker has several Cf- headers added:

{
    "Cdn-Loop": "cloudflare; loops=1; subreqs=2", 
    "Cf-Connecting-Ip": "2a06:98c0:3600::103", 
    "Cf-Ew-Via": "13", 
    "Cf-Ray": "8de1598e90bb1080-ORD", 
    "Cf-Visitor": "{\"scheme\":\"https\"}", 
    "Cf-Worker": "workers-playground-users.workers.dev", 
}

I added these headers to my node.js test environment, and sure enough, Steam requests started to fail.

After 20 seconds of trial and error, stripping the Cf-Worker header allowed the node.js requests to succeed.

The good news is that this problem is solvable, the bad news is that it will still require the use of a proxy server as the Cf- headers are tacked onto the request after it has left the worker as part of Cloudflare’s infrastructure.

I updated the traefik config to strip this header (the entire traefik config is posted below):

# A traefik proxy for steamcommunity requests (ie: login) blocked on cloudflare.
http:
  routers:
    steamcommunity:
      rule: "Host(`yourdomain.com`) && Path(`/openid/login`) && Method(`POST`)"
      entryPoints: ["websecure"]
      service: "steamcommunity"
      middlewares: 
        - "steam-header"
      tls:
        certResolver: "myresolver"

  middlewares:
    steam-header:
      headers:
        customRequestHeaders:
          Host: "https://steamcommunity.com"
          Cf-Worker: ""

  services:
    steamcommunity:
      loadBalancer:
        servers:
          - url: "https://steamcommunity.com"

Still, requests from Cloudflare would receive 403 forbidden.

Cue an hour of racking my brain. What was I missing? Could Steam actually be blocking based on IPs and traefik was leaking it somehow? I didn’t think I had the PROXY protocol enabled.

DIY Proxy

In frustration, I had AI write a zero dependency node.js proxy server that would proxy the url, method, body, but not headers:

import http from "node:http";

const EXTERNAL_URL = "https://steamcommunity.com";
const server = http.createServer(async (req, res) => {
  let body = "";
  for await (const chunk of req) {
    body += chunk.toString();
  }

  try {
    const externalResponse = await fetch(EXTERNAL_URL + req.url, {
      method: req.method,
      body,
    });

    const data = await externalResponse.text();

    res.writeHead(externalResponse.status, {
      ...externalResponse.headers,
      "Content-Length": Buffer.byteLength(data),
    });

    res.end(data);
  } catch (error) {
    console.error("Error:", error);
    res.writeHead(500, { "Content-Type": "text/plain" });
    res.end("An error occurred while processing your request.");
  }
});

const PORT = process.env.PORT ?? 3333;
server.listen(PORT, () => {
  console.log(`Server is running on http://localhost:${PORT}`);
});

I threw this proxy into a distroless container on google cloud run. The container image weighed 125MB, which is amusingly heavy-weight, but it did the job. Cloudflare and node.js started seeing the same result – mind you, the result was an HTML page and not the expected verification response, but it proved I wasn’t going crazy when it comes to proxying setups.

Unfortunately, the HTML returned triggered a phishing report from Cloudflare’s Identity Digital Safety Team, which gave me 48 hours to “remedy DNS abuse” or risk suspension. Good thing this blog provides all the context needed to see this isn’t abuse, and that the traefik config is locked down to a specific path and method!

The fetch discrepancy

Then it dawned on me. httpbin.org/headers would only show GET request headers. The Steam verification request involves a POST with a body URLSearchParams, so I should have been using httpbin.org/anything.

const resp = await fetch("https://httpbin.org/anything", {
  method: "POST",
  body: new URLSearchParams({ hello: 'world' }),
}).then((x) => x.json());

And that’s when I saw it. There is a discrepancy in fetch implementations. According to the spec, when the body is of type URLSearchParams, the content type should be set to:

application/x-www-form-urlencoded;charset=UTF-8

Browsers and Node.js follow the spec, but Cloudflare doesn’t. Cloudflare sets the content-type to:

text/plain;charset=UTF-8

When faced with this content type, Steam responded with redirect – and the 403 forbidden I was seeing was the follow of the redirect.

I reported the bug:

The workaround

I wish a workaround wasn’t necessary, but at least it’s simple: specify the expected content type header manually.

fetch("https://httpbin.org/anything", {
  method: "POST",
  body: new URLSearchParams({ hello: 'world' }),
  headers: {
    "Content-Type": "application/x-www-form-urlencoded;charset=UTF-8"
  }
});

Conclusion

Takeaways:

• Steam blocks OpenID signature verification requests from Cloudflare based on the presence of “Cf-Worker” HTTP header and requires requests to go through a proxy that strips the header
• Cloudflare has a non-spec compliant fetch implementation for URLSearchParams bodies that requires a workaround

Porting this code to Cloudflare Workers was a test of debugging skills and grit.

I don’t understand why Steam would block Cloudflare requests and why their preferred method of detection is via the Cf-Worker header. What’s equally confusing is that Steam hosts their API on a separate domain:

api.steampowered.com

And this domain is reachable from Cloudflare and works perfectly fine. If I had to bet, I’d bet it is because Steam’s OpenID routes are hosted under:

https://steamcommunity.com/openid

And Steam has their blanket denial of Cloudflare at the domain root.

Throughout this debugging adventure, Cloudflare Workers playground came in handy to test out ideas.

In the future, I need to be more wary of 3rd party services. Prior to this escapade, I hadn’t considered that 3rd party services would start blocking my requests. Let me beef up my monitoring and make sure e2e tests are executed in an environment as close to production as possible!