A production-oriented concurrent HTTP reverse proxy and load balancer implemented in pure Go using low-level concurrency, networking, and standard library primitives with zero third-party dependencies.
The project demonstrates low-level systems and backend engineering concepts, including:
- Concurrent request routing
- Reverse proxy internals
- Active + passive health checking
- Retry and failover coordination
- Thread-safe shared state management
- Per-IP token bucket rate limiting
- Structured observability
- Graceful shutdown
- Containerized deployment workflows
The proxy leverages Go concurrency primitives such as:
goroutinessync.RWMutexsync/atomiccontextcancellation- channel-based coordination patterns
to implement thread-safe backend management, runtime failure isolation, concurrent health monitoring, and coordinated retry under load.
Supports multiple routing algorithms, selected via config/config.json:
- Round Robin
- Weighted Round Robin
Weighted routing enables heterogeneous backend clusters where stronger nodes receive proportionally more traffic. If a backend becomes unhealthy, traffic is redistributed across currently healthy backends automatically.
Retries are coordinated independently for each request. On failure, the proxy picks a different healthy backend that hasn't already been tried in that request cycle.
Automatically retries requests on:
- Connection refused (
ECONNREFUSED) - TCP reset (
ECONNRESET) - EOF / broken upstream connection
- Any
net.Error(includes backend timeout)
When a transport error occurs, the backend is immediately marked unhealthy (passive health check).
Retries on these upstream response codes:
502 Bad Gateway503 Service Unavailable504 Gateway Timeout
Retries are transparently redirected to a different healthy backend. Clients are transparently retried against alternate healthy backends whenever possible.
The proxy combines both active and passive health checking. The active and passive checks serve different failure-detection roles and work together to improve recovery responsiveness.
A background health checker goroutine probes every backend (including currently dead ones) on a configurable interval using:
GET <backend-url>/health
Only an HTTP 200 response marks a backend healthy. Because dead backends are also probed, this is what enables automatic recovery - a backend that was quarantined by a passive check will come back online once it passes an active check.
Backends are immediately marked unhealthy during live traffic when:
- A transport failure occurs mid-request
- A retry attempt exhausts all available healthy backends
This provides fast failure isolation without waiting for the next active check cycle
Implements concurrent token-bucket rate limiting.
Features:
- Per-client-IP buckets
- Burst handling
- Automatic token refill
- Thread-safe bucket management
Protects upstream services from:
- Abusive clients
- Request floods
- Accidental overload
How it works:
- Each IP starts with a full bucket (
bursttokens) - Tokens refill at
requests_per_secondper second - Each request consumes one token
- When the bucket is empty, the request is rejected with
429 Too Many Requests - Stale buckets are periodically cleaned up to prevent unbounded memory growth
Rate limiting can be disabled entirely by setting rate_limit.enabled: false in config.
Each proxied request emits a log entry. With logging.format: "json" (default):
{
"time": "2025-01-15T10:30:00Z",
"client": "127.0.0.1",
"method": "GET",
"path": "/api/data",
"backend": "http://localhost:8081",
"status": 200,
"duration_ms": 12,
"algorithm": "weighted-round-robin"
}Set logging.format to any other value (e.g. "text") for plain-text output.
The rate_limited field appears only when a request is rejected by the rate limiter.
Live runtime statistics:
curl http://localhost:8080/admin/statsFull example response shape:
{
"algorithm": "weighted-round-robin",
"total_requests": 125,
"retry_count": 18,
"rate_limited_requests": 3,
"backend_stats": [
{ "url": "http://localhost:8081", "alive": true, "weight": 5 },
{ "url": "http://localhost:8082", "alive": true, "weight": 3 },
{ "url": "http://localhost:8083", "alive": false, "weight": 2 }
],
"healthy_backends": 2,
"unhealthy_backends": 1,
"uptime": "4m32s",
"timestamp": "2025-01-15T10:30:00Z"
}Send Ctrl+C (or SIGTERM) to the proxy. It will:
The proxy supports graceful termination using SIGINT / SIGTERM.
- Stop accepting new incoming connections
- Wait up to 15 seconds for active in-flight requests to complete
- Stop the health checker goroutine via
contextcancellation - Exit cleanly
[shutdown] received signal: interrupt - initiating graceful shutdown
[health] checker stopped
[shutdown] proxy has been shut down gracefully
┌─────────────────────┐
│ Client │
└──────────┬──────────┘
│
▼
┌──────────────────────────────────┐
│ Reverse Proxy (:8080) │
│----------------------------------│
│ • Rate Limiting │
│ • Retry Orchestration │
│ • Header Rewriting │
│ • Load Balancing │
│ • Passive Health Checks │
│ • Structured Logging │
│ • Graceful Shutdown │
└────────────────┬─────────────────┘
│
▼
┌────────────────────────┐
│ Backend Pool │
│------------------------│
│ Active Health Checker │
│ probes all backends │
│ including dead nodes │
└──────────┬─────────────┘
│
┌────────────────────────┼────────────────────────┐
▼ ▼ ▼
┌────────────────┐ ┌────────────────┐ ┌────────────────┐
│ Backend 1 │ │ Backend 2 │ │ Backend 3 │
│ localhost │ │ localhost │ │ localhost │
│ :8081 │ │ :8082 │ │ :8083 │
│ Weight: 5 │ │ Weight: 3 │ │ Weight: 2 │
└────────────────┘ └────────────────┘ └────────────────┘
| Component | Responsibility |
|---|---|
| Proxy | Request lifecycle management |
| Balancer | Backend selection |
| Retry Engine | Classifies transport errors and HTTP status codes |
| Health Checker | Active backend probing |
| Passive Recovery | Immediate quarantine on transport failure |
| Rate Limiter | Per-IP traffic protection |
| Stats Endpoint | Runtime observability |
| Logger | Structured per-request logging |
.
├── Dockerfile
├── LICENSE
├── README.md
├── build
├── cmd
│ ├── proxy
│ │ └── main.go
│ └── testserver
│ └── main.go
├── config
│ └── config.json
├── go.mod
└── internal
├── backend
│ ├── backend.go
│ └── pool.go
├── balancer
│ ├── balancer.go
│ ├── roundrobin.go
│ └── weightedrr.go
├── config
│ └── config.go
├── health
│ └── checker.go
├── logger
│ └── logger.go
├── proxy
│ ├── proxy.go
│ └── retry.go
├── ratelimit
│ └── limiter.go
└── stats
└── stats.go
Configuration is loaded from config/config.json at startup.
| Field | Type | Default | Description |
|---|---|---|---|
server.port |
int | 8080 |
Port the proxy listens on |
server.read_timeout |
duration | "10s" |
HTTP read timeout |
server.write_timeout |
duration | "30s" |
HTTP write timeout |
algorithm |
string | "round-robin" |
Load balancing algorithm |
backends[].url |
string | - | Upstream backend address |
backends[].weight |
int | 1 |
Backend traffic weight |
health_check.enabled |
bool | true |
Enable/disable active health checks |
health_check.interval |
duration | "10s" |
Time between active probe cycles |
health_check.timeout |
duration | "3s" |
Per-probe HTTP timeout |
health_check.path |
string | "/health" |
Endpoint path probed on each backend |
rate_limit.enabled |
bool | true |
Enable/disable per-IP rate limiting |
rate_limit.requests_per_second |
float | 5 |
Token refill rate per IP |
rate_limit.burst |
int | 10 |
Bucket burst size (initial token count) |
logging.format |
string | "json" |
Log format |
logging.level |
string | "info" |
Log Verbosity |
retry.enabled |
bool | true |
Enable/disable retry & failover handling |
retry.max_attempts |
int | 3 |
Maximum retry attempts |
Duration values use Go duration syntax: "10s", "500ms", "1m30s".
- Go 1.26+ (see
go.mod) - Linux/macOS recommended
- Docker (optional)
Verify Go version:
go versiongit clone https://github.com/sohamm3/resilient-reverse-proxy.git
cd resilient-reverse-proxyCreate the build directory:
mkdir -p buildBuild the reverse proxy:
go build -o build/proxyServer ./cmd/proxyBuild the test backend servers:
go build -o build/testServer ./cmd/testserverOption A - Start all three backends (recommended for first run):
Open terminal 1:
./build/testbackendsThis launches:
| Backend | Port |
|---|---|
| Backend 1 | 8081 |
| Backend 2 | 8082 |
| Backend 3 | 8083 |
Option B - Start specific backends only:
# Single backend
./build/testbackends 8081
# Two specific backends
./build/testbackends 8081 8082Open terminal 2:
./build/proxyServerThe proxy reads config/config.json, runs an initial health check on all backends, then starts listening:
localhost:8080
The proxy rewrites outbound upstream requests and injects forwarding metadata for observability and upstream awareness.
curl -i http://localhost:8080/These headers are forwarded internally to backend servers.
| Header | Value |
|---|---|
X-Forwarded-For |
Original client IP address chain |
X-Forwarded-Host |
Original Host header from client request |
X-Forwarded-Proto |
Original request protocol (http or https) |
These headers are returned back to the client.
| Header | Value |
|---|---|
X-Backend-Server |
URL of the backend selected for handling the request |
X-Response-Time |
Total proxy round-trip latency in milliseconds |
curl http://localhost:8080/for i in {1..10}; do
curl -s http://localhost:8080/
done- Backend 1 appears more frequently
- Backend 2 appears moderately
- Backend 3 appears less frequently
Traffic distribution follows configured backend weights.
curl http://localhost:8080/admin/stats./build/testbackends 8081./build/proxyServercurl http://localhost:8080/admin/statsExpected:
- Backend
8081appears healthy - Backends
8082and8083appear unhealthy
./build/testbackends 8082 8083The active health checker continuously probes unhealthy nodes and automatically restores recovered backends back into rotation.
Generate concurrent requests:
for i in {1..50}; do
curl -s http://localhost:8080/ &
done
wait- Some requests succeed normally
- Some requests return:
429 Too Many Requests
This exercises:
- Concurrency safety (goroutines, atomics, RWMutex)
- Retry handling under partial failure
- Rate limiting (the default config allows 5 req/s burst 10 - tune
rate_limitin config for high-throughput tests) - Weighted backend distribution
Check the stats endpoint after the test to see live counters:
curl http://localhost:8080/admin/statsThe test backend server has built-in failure injection via query parameters.
Abruptly closes the TCP connection mid-request using HTTP connection hijacking:
curl "http://localhost:8080/?crash=true"Triggers:
- Passive health check => the backend is immediately marked unhealthy
- Retry failover => the request is retried on a different backend
- Backend quarantine => excluded from routing until active health checks restore it
Returns HTTP 502 from the upstream backend:
curl "http://localhost:8080/?badgateway=true"Triggers:
- HTTP status retry classification
- Retry on an alternate healthy backend
The proxy was stress-tested under concurrent workloads using hey with structured JSON logging, active/passive health checks, retry orchestration, and simulated upstream latency enabled.
hey -n 100000 -c 500 http://localhost:8080/| Metric | Value |
|---|---|
| Total Requests | 100,000 |
| Concurrent Clients | 500 |
| Throughput | ~10k req/sec |
| Success Rate | 100% |
| P95 Latency | ~94ms |
Validated concurrent request routing, backend selection, observability overhead, and runtime stability under sustained load.
hey -n 50000 -c 200 "http://localhost:8080/?badgateway=true"Simulated upstream 502 Bad Gateway failures under concurrent load.
The proxy automatically:
- retried requests on alternate healthy backends
- prevented retry storms using per-request backend exclusion
- maintained successful responses during failure injection
hey -n 50000 -c 200 "http://localhost:8080/?crash=true"Validated passive health isolation, transport failure detection, and retry coordination during abrupt upstream TCP connection drops.
docker build -t resilient-reverse-proxy .docker run --network host resilient-reverse-proxyThe Docker image contains only the proxy binary. Backend servers must run separately and be reachable from the container
--network hostgives the container direct access to the host's network interfaces, allowing the proxy to reach backends running onlocalhost:8081–8083. This flag is Linux-only; on macOS and Windows, replacelocalhostinconfig/config.jsonwithhost.docker.internaland update theCOPY configstep accordingly.
Future improvements may include:
- docker-compose orchestration
- containerized backend clusters
- Kubernetes deployment manifests
- service discovery integration
- Go concurrency primitives (
goroutines,sync.RWMutex,sync/atomic) - Reverse proxy internals (
httputil.ReverseProxy) - Active + passive health recovery with automatic backend restoration
- Coordinated retry and failover handling
- Token-bucket rate limiting with double-checked locking for bucket creation
- Context-based graceful shutdown propagated to all background goroutines
- Multi-stage Docker builds producing a minimal, non-root runtime image
- Non-root container execution
- Structured request logging with a thread-safe bounded in-memory request history for recent-request inspection and stats access.
- Zero third-party dependencies