Name: System Design Masterclass
Availability: InStock

Ever worked somewhere that blocked social media? Used a VPN to appear to be in another country? Set http_proxy so a build could reach the internet? All three are the same thing.

A forward sits between a set of clients and the rest of the internet, and makes requests on their behalf. Your machine talks to the proxy, and the proxy talks to the site.

The defining property is not the technology. It is who deployed it and whose side it is on.

A hand-drawn diagram in two rows. The top row, labelled a forward proxy is deployed by the client's side, shows you, then your proxy which knows it is you, then any server on the internet. The bottom row, labelled a reverse proxy is deployed by the server's side, shows any client on the internet, then their proxy which knows the servers, then a server. A handwritten note reads: same box, opposite purpose, one hides the clients and the other hides the servers. — SketchFigure 1. The same machine facing opposite directions
With a forward proxy the client knows about it and deliberately routes through it, and the destination server usually has no idea a proxy was involved.
With a the client has no idea, and the servers behind it are hidden. Nothing in the software has to differ, and the same product can do either job.

The protocol difference between talking to a and talking to a server is one line.

Two request blocks. Straight to the server, origin form: GET /index.html HTTP/1.1 with a Host header of example.com. Through a proxy, absolute form: GET http://example.com/index.html HTTP/1.1 with the same Host header. A note explains that the proxy needs the full URL because it has to decide where to open a connection and has not been told by DNS. — FigureFigure 2. Talking to a proxy differs from talking to a server by one line
Normally a client resolves the host, connects to it, and asks for a path. With a proxy it connects to the proxy and asks for a complete URL, and the proxy resolves the host itself.
That last point matters more than it looks: your machine may never look the site up at all.

Because it is such a small change, an proxy needs no cooperation from the destination and no special client. Setting one environment variable is enough for most tools, which is why it became the universal corporate mechanism.

Then HTTPS arrived and broke most of it

A sequence diagram with three lifelines: your browser, the proxy and example.com. The browser sends CONNECT example.com:443, noted as a hostname and a port and nothing else. The proxy opens a plain TCP connection to the site and replies 200 Connection established. The browser then does the TLS handshake and everything else directly, noted as the proxy copying bytes it cannot read. — SequenceFigure 3. For HTTPS the proxy is told a hostname and then goes deaf
CONNECT asks the proxy to open a raw tunnel. The TLS handshake happens end to end through it, between the client and the real server.
So the proxy knows which site you visited and nothing whatsoever about what you did there.

None of that is theory. A forward proxy is about seventy lines of code, and it can print exactly what it was handed.

A recorded terminal session. A 70-line forward proxy runs on 127.0.0.1 port 8899. A curl to http://example.com through it returns HTTP 200, and the proxy logged GET followed by the complete URL http://example.com/, annotated as the proxy being able to read and change all of this. A curl to https://example.com also returns HTTP 200, and the proxy logged CONNECT example.com:443, annotated as everything after this being encrypted. — RecordedFigure 4. A real proxy, printing what each request handed it
Plain HTTP arrives as a complete URL, so the path, the headers, the cookies and the body are all equally readable and equally changeable.
arrives as a hostname and a port, and then the proxy is relaying bytes it cannot interpret.

A table of six things a proxy might see, under plain HTTP and under HTTPS via CONNECT. The hostname and the port are visible under both. The path and query string, request headers and cookies, the request body, and the response with the ability to change it are visible under plain HTTP and marked no, not without TLS interception, under HTTPS. — FigureFigure 5. Two rows are visible either way, and the rest depend on the scheme
This is not a policy setting. It is what the protocol hands the proxy.
Read the top two rows again before assuming HTTPS hides your browsing from a network operator. It does not hide **where** you went, only what you did there.

That distinction decides a lot of real design: filtering by domain works on both, and filtering by URL only works on one.

Forward Proxy

The rest of this lesson

Continue this lesson

The Box That Speaks for You

What Changes on the Wire

Then HTTPS arrived and broke most of it