Fixing HTTP 502 Bad Gateway in Docker Reverse Proxy Setups — Upstream Connection Refused Due to Loopback Binding

Proxy Logs Show connect() failed (111: Connection refused)

We started by hitting the public endpoint (GET /api/health). The client received an HTTP 502 Bad Gateway. However, directly curling the backend container on its published port (curl http://host-ip:3000/health) returned a perfect 200 OK.

Our first action was to check the proxy container logs. They showed the upstream failure explicitly:

[error] 30#30: *1 connect() failed (111: Connection refused) while connecting to upstream, 
client: 10.88.0.1, server: api.internal-corp.net, request: "GET /api/health HTTP/1.1", 
upstream: "http://backend_svc:3000/health"

For the Traefik stack, the debug-level logs gave an even clearer picture:
DBG 502 Bad Gateway error="dial tcp 172.20.0.3:3000: connect: connection refused".

The proxy resolved the service name (backend_svc) to a container IP (172.20.0.3) without issue, but the TCP handshake was rejected.

Backend Container Showed No Crash

We immediately checked the backend container’s running state and logs to rule out an application crash-loop:

docker ps | grep backend
# Output: Up 3 hours (healthy)

docker logs backend-container --tail 30 --since 5m
# Output: [YYYY-MM-DD] Server listening on http://localhost:3000
#          [YYYY-MM-DD] GET /health 200 2ms (from direct host access only)

Notice the log line: listening on http://localhost:3000. The backend was up, handling health checks, but only for requests originating from within its own network namespace.

Ruling Out the Obvious (What We Tried First)

We spent about 20 minutes chasing false leads before narrowing in on the real culprit:

1. Verified Network Connectivity (The ping Test)

We exec’d into the proxy container and checked if it could resolve and ping the backend hostname:

docker exec -it nginx-proxy ping backend_svc
# Output: 64 bytes from 172.20.0.3: icmp_seq=1 ttl=64 time=0.1 ms

ICMP worked. The container was reachable. This ruled out DNS resolution and network segmentation.

2. Checked Docker Compose networks and expose

We reviewed the docker-compose.yml and confirmed both services were explicitly attached to the same app_net network. We also noticed the backend had expose: - "3000". Adding or removing expose had zero effect because it merely documents port intent; it does not alter the listening socket inside the container.

3. Attempted restart: unless-stopped on the Proxy

We assumed the proxy container itself might be unhealthy. Restarting it did nothing—the 502 persisted immediately upon restart because the underlying network handshake was still failing.

4. Toggled proxy_pass Trailing Slashes

We adjusted Nginx’s proxy_pass http://backend_svc:3000/ to remove the trailing slash, and vice versa. This changed the URI path construction but did not affect the connection refusal on port 3000.

At this point, the persistent connection refused on port 3000, coupled with the backend log saying localhost, triggered the next investigative step.

netstat Revealed the Loopback Bind

We exec’d into the backend container and inspected the active listening ports:

docker exec -it backend-container netstat -tulpn
# Active Internet connections (only servers)
# Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
# tcp        0      0 127.0.0.1:3000          0.0.0.0:*               LISTEN      1/node

The process was bound exclusively to 127.0.0.1:3000. In Linux container networking, 127.0.0.1 is the loopback interface inside that container’s network namespace. Connections from the proxy container arrive via the container’s virtual Ethernet interface (eth0), which has the IP 172.20.0.3. Since the Node.js process was not listening on 0.0.0.0 (all interfaces), the kernel rejected the SYN packet from the proxy with an RST, translating to connection refused.

We confirmed this by testing a curl from inside the backend container to itself on 127.0.0.1 (which worked), and then to its own eth0 IP (which failed):

docker exec -it backend-container curl -v http://127.0.0.1:3000/health # 200 OK
docker exec -it backend-container curl -v http://172.20.0.3:3000/health # Failed: Connection refused

This proved the issue was purely at the application binding layer, unrelated to the reverse proxy configuration.

Rebinding the Application Listener to 0.0.0.0

We located the application’s server initialization (server.js) and modified the listen call:

// Before (causes the 502 in containerized environments)
const server = app.listen(3000, '127.0.0.1', () => {
  console.log(`Server listening on http://localhost:3000`);
});

// After (accepts traffic from any interface, including the Docker bridge)
const server = app.listen(3000, '0.0.0.0', () => {
  console.log(`Server listening on http://0.0.0.0:3000`);
});

We rebuilt the backend image and redeployed the stack:

docker-compose build --no-cache backend
docker-compose up -d backend

Important: We did not alter the Nginx or Traefik proxy configurations. The upstream URL remained http://backend_svc:3000. This confirms the proxy side was correct all along.

Confirming the Fix with In-Container Diagnostics

Post-deployment, we repeated our diagnostic checks:

1. Checked listening ports inside the backend container:bashdocker exec -it backend-container netstat -tulpn | grep 3000 # Output: tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 1/nodeThe 0.0.0.0 confirmed the process now accepts connections on any incoming interface.
2. Exec’d into the proxy container and performed a direct curl to the backend using its service name:bashdocker exec -it nginx-proxy curl -v http://backend_svc:3000/health # Output: < HTTP/1.1 200 OKThe proxy now successfully established the TCP handshake.
3. Checked the public endpoint:bashcurl -v https://api.internal-corp.net/api/health # Output: HTTP/2 200 The 502 error was resolved permanently.

Instrumenting Network Binding Checks in CI

To prevent this issue from recurring across team deployments, we added two lightweight checks to our CI pipeline:

1. Automated netstat Assertion in Container Tests

We added a containerized integration test that spins up the backend and uses netstat -tulpn to assert that the listening address includes 0.0.0.0 or ::, failing the build if only 127.0.0.1 is present.

2. Proxy-to-Backend Health Check

We configured the reverse proxy’s health check to target the backend container’s service name rather than localhost. In Docker Compose:

healthcheck:
  test: ["CMD", "curl", "-f", "http://backend_svc:3000/health"]
  interval: 10s

This ensures the health check validates cross-container reachability, not just local process liveness.

3. Templating the Bind Address via Environment Variables

We updated the application code to read the bind host from an environment variable (BIND_HOST), defaulting to 0.0.0.0 for containerized deployments and 127.0.0.1 for local developer workstations, allowing the same image to function safely in both contexts.

References

• Official Docker Container Networking Guide — Understanding bridge networks and inter-container communication
• Official Nginx Reverse Proxy Documentation — Troubleshooting upstream connection failures
• Official Traefik User Guide — Debugging 502 Bad Gateway errors
• Mozilla Developer Network — Node.js server.listen() address binding

(Last verified: 2 July 2026 — Docker Engine 27.x, Node.js 20.x, Linux kernel 6.x)