diff --git a/backend/docs/limits.md b/backend/docs/limits.md index cd77f597e..71ad2add5 100644 --- a/backend/docs/limits.md +++ b/backend/docs/limits.md @@ -65,6 +65,7 @@ remaining points, and reset timers is not exposed. | Limit Type | Value | HTTP Status When Exceeded | |------------|-------|----------------------------| | JSON Body | 1MB | 413 Payload Too Large | +| JSON Body on `POST /api/v1/events` | 256KB | 413 Payload Too Large | | Query per param | 2KB | 400 Bad Request | | Query total | 8KB | 400 Bad Request | | Header per key | 16KB | 431 Request Header Fields Too Large | @@ -73,6 +74,7 @@ remaining points, and reset timers is not exposed. ### Rationale - **Body (1MB)**: Invoice metadata can be detailed JSON, but unbounded bodies cause memory pressure. 1MB accommodates complex invoices while preventing abuse. +- **Body on the event ingest route (256KB)**: A batch is capped at 100 Soroban events, so 256KB is generous while keeping the indexer ingress far below the general budget. See [security.md](./security.md#event-ingest-endpoint-hardening) for the full framing policy, which also covers `Content-Length` and chunked-encoding handling. - **Query per param (2KB)**: 64-char hex invoice IDs are ~128 bytes. 2KB provides ample headroom for legitimate values. - **Query total (8KB)**: Allows multiple filter params (invoice_id, status, business, pagination) without hitting limits. - **Header per key (16KB)**: Large enough for JWT tokens (~1-4KB) with room for metadata. diff --git a/backend/docs/security.md b/backend/docs/security.md index 053b8a671..4c7e5754f 100644 --- a/backend/docs/security.md +++ b/backend/docs/security.md @@ -2,13 +2,40 @@ ## Event Ingest Endpoint Hardening -The `POST /api/v1/events` endpoint (used by indexers to ingest Soroban events has additional security hardening: +`POST /api/v1/events` is the ingress the indexer uses to submit Soroban events. A request that reaches it is buffered and parsed before any business validation runs, so its framing is validated first, by `src/middleware/event-ingest-limits.ts`. The policy applies exclusively to this route so the rest of the API keeps the default 1 MB budget. -- **Content-Type Enforcement**: Requests must use `application/json` content type; unsupported content types are rejected with `415 Invalid Content Type`. -- **Content-Length Requirement**: A valid `Content-Length` header must be present, with maximum of 256 KB; exceeding this limit returns `413 Payload Too Large`. -- **Chunked Encoding Rejection**: Chunked transfer encoding is rejected unless the allowlisted header `X-Allow-Chunked-Encoding` is present (to prevent smuggling via intermediate proxies). +### Framing policy -These protections are applied exclusively to the `/api/v1/events` route to preserve flexibility for other endpoints. +| Condition | Status | Error code | +| --- | --- | --- | +| Content type is not exactly `application/json` (parameters such as `charset` are allowed) | `415` | `INVALID_CONTENT_TYPE` | +| `Content-Length` absent on a non-chunked request | `411` | `CONTENT_LENGTH_REQUIRED` | +| `Content-Length` is not a single non-negative integer (including duplicated headers) | `400` | `INVALID_CONTENT_LENGTH` | +| Declared or actual body above 256 KB | `413` | `BODY_LIMIT_EXCEEDED` | +| `Transfer-Encoding: chunked` without an allowlisted upstream proxy | `400` | `CHUNKED_ENCODING_NOT_ALLOWED` | +| `Transfer-Encoding: chunked` combined with `Content-Length` | `400` | `AMBIGUOUS_REQUEST_FRAMING` | +| Body bytes do not match the declared `Content-Length` | `400` | `CONTENT_LENGTH_MISMATCH` | +| Body is not parseable JSON | `400` | `INVALID_JSON_BODY` | + +### Why the checks are ordered this way + +The header guard runs before any body parser, and the application-wide 1 MB `express.json` parser explicitly skips this route (see `isEventIngestRequest` in `src/app.ts`). An oversized or ambiguously framed request is therefore refused without buffering its payload, and the 256 KB budget is bound to the route instead of being inherited from the global parser. + +Chunked encoding is evaluated before `Content-Length` because HTTP strips `Content-Length` from chunked requests: checking length first would mask a smuggled request behind a `411`. Seeing both headers at once is the canonical request-smuggling signature, since intermediaries disagree on which one delimits the message, so that combination is rejected outright even for allowlisted proxies. + +### Chunked-encoding allowlist + +Chunked bodies are refused by default. An upstream proxy that must forward them declares itself with the `X-Allow-Chunked-Encoding` header, and the value has to match an entry in `EVENT_INGEST_CHUNKED_PROXY_ALLOWLIST`, a comma-separated list of proxy identifiers. Presence of the header alone is not sufficient; with the variable unset (the default) every chunked request is rejected. + +```bash +EVENT_INGEST_CHUNKED_PROXY_ALLOWLIST=edge-proxy-1,edge-proxy-2 +``` + +Only terminate chunked ingest at a proxy you control, and make sure that proxy strips any client-supplied `X-Allow-Chunked-Encoding` header before forwarding. + +### Rejection messages + +Every rejection message is a constant. Body-parser failures are re-mapped rather than surfaced, because its native messages quote the offending payload bytes (for example `Unexpected token c ... is not valid JSON`). No response from this endpoint echoes request payload content back to the caller. ## CORS Policy diff --git a/backend/jest.config.js b/backend/jest.config.js index e67863f53..2baebf0dc 100644 --- a/backend/jest.config.js +++ b/backend/jest.config.js @@ -30,6 +30,12 @@ module.exports = { lines: 95, statements: 95, }, + "src/middleware/event-ingest-limits.ts": { + branches: 95, + functions: 95, + lines: 95, + statements: 95, + }, }, collectCoverageFrom: [ "scripts/lib/secret-scan-utils.js", @@ -42,6 +48,7 @@ module.exports = { "src/middleware/access-log.ts", "src/services/eventProcessor.ts", "src/middleware/cache-headers.ts", + "src/middleware/event-ingest-limits.ts", "src/controllers/v1/bids.ts", "src/lib/entityId.ts", ], diff --git a/backend/src/app.ts b/backend/src/app.ts index 5ab51af85..72635329b 100644 --- a/backend/src/app.ts +++ b/backend/src/app.ts @@ -11,6 +11,7 @@ import v1Routes from "./routes/v1"; import webhookRoutes from "./routes/webhooks"; import healthRoutes from "./routes/health"; import { requestLogger } from "./middleware/request-logger"; +import { isEventIngestRequest } from "./middleware/event-ingest-limits"; import { lagMonitor } from "./services/lagMonitor"; import { alertRouter, Severity } from "./services/alertRouter"; @@ -49,14 +50,22 @@ declare global { // Security Middleware app.use(helmet()); app.use(cors(corsOptionsDelegate)); -app.use( - express.json({ - limit: "1mb", - verify: (req: express.Request, res: express.Response, buf: Buffer) => { - req.rawBody = buf; - }, - }) -); +const globalJsonParser = express.json({ + limit: "1mb", + verify: (req: express.Request, res: express.Response, buf: Buffer) => { + req.rawBody = buf; + }, +}); + +// The event ingest route enforces a stricter 256 KB budget with its own parser, +// so the 1 MB parser must not buffer that payload first. +app.use((req, res, next) => { + if (isEventIngestRequest(req)) { + next(); + return; + } + globalJsonParser(req, res, next); +}); app.set("trust proxy", true); // Test middleware to simulate no IP for coverage diff --git a/backend/src/middleware/event-ingest-limits.ts b/backend/src/middleware/event-ingest-limits.ts index 8cfdf9572..70740c76c 100644 --- a/backend/src/middleware/event-ingest-limits.ts +++ b/backend/src/middleware/event-ingest-limits.ts @@ -1,52 +1,231 @@ -import { Request, Response, NextFunction } from "express"; - -const MAX_BODY_SIZE = 256 * 1024; // 256 KB -const ALLOWLISTED_PROXY_HEADER = "X-Allow-Chunked-Encoding"; // Example header, adjust as needed - -export function eventIngestLimitsMiddleware(req: Request, res: Response, next: NextFunction) { - // Check Content-Type - const contentType = req.headers["content-type"]; - if (!contentType || !contentType.includes("application/json")) { - return res.status(415).json({ - error: { - message: "Unsupported content type, must be application/json", - code: "INVALID_CONTENT_TYPE", - }, - }); - } - - // Check Content-Length - const contentLength = req.headers["content-length"]; - if (!contentLength) { - return res.status(411).json({ - error: { - message: "Content-Length header is required", - code: "CONTENT_LENGTH_REQUIRED", - }, - }); - } - - const contentLengthNum = parseInt(contentLength, 10); - if (isNaN(contentLengthNum) || contentLengthNum > MAX_BODY_SIZE) { - return res.status(413).json({ - error: { - message: `Request body too large, maximum is ${MAX_BODY_SIZE} bytes`, - code: "BODY_LIMIT_EXCEEDED", - }, - }); - } - - // Check Transfer-Encoding - const transferEncoding = req.headers["transfer-encoding"]; - const allowChunked = req.headers[ALLOWLISTED_PROXY_HEADER.toLowerCase()] !== undefined; - if (transferEncoding && transferEncoding.includes("chunked") && !allowChunked) { - return res.status(400).json({ - error: { - message: "Chunked transfer encoding is not allowed", - code: "CHUNKED_ENCODING_NOT_ALLOWED", - }, - }); - } +import express, { NextFunction, Request, RequestHandler, Response } from "express"; + +/** + * Hardened ingestion limits for `POST /api/v1/events`. + * + * The endpoint is the only unauthenticated-ish ingress the indexer uses, so the + * framing of the request is validated before a single byte of the payload is + * buffered: + * + * 1. `eventIngestLimitsMiddleware` inspects headers only and rejects requests + * whose framing cannot be trusted (wrong content type, missing/ambiguous + * `Content-Length`, `Transfer-Encoding: chunked` from a non-allowlisted + * proxy). + * 2. `eventIngestBodyParser` parses the body with a 256 KB budget that is + * independent from the 1 MB application-wide budget. + * + * No rejection ever echoes payload bytes: every message below is a constant. + */ - next(); +export const EVENT_INGEST_PATH = "/api/v1/events"; +export const EVENT_INGEST_MAX_BODY_BYTES = 256 * 1024; + +/** Header an allowlisted upstream proxy uses to declare itself. */ +export const CHUNKED_PROXY_HEADER = "x-allow-chunked-encoding"; + +/** Comma-separated list of proxy identifiers permitted to forward chunked bodies. */ +export const CHUNKED_PROXY_ALLOWLIST_ENV = "EVENT_INGEST_CHUNKED_PROXY_ALLOWLIST"; + +const JSON_MEDIA_TYPE = "application/json"; +const CONTENT_LENGTH_PATTERN = /^\d+$/; + +export interface EventIngestLimitsOptions { + maxBodyBytes?: number; + allowedChunkedProxies?: string[]; } + +const readHeader = (raw: string | string[] | undefined): string | undefined => { + if (Array.isArray(raw)) return raw.join(","); + if (typeof raw !== "string") return undefined; + return raw; +}; + +const parseAllowlist = (raw: string | undefined): string[] => + (raw ?? "") + .split(",") + .map((entry) => entry.trim()) + .filter((entry) => entry.length > 0); + +const isJsonMediaType = (contentType: string | undefined): boolean => { + if (!contentType) return false; + const [mediaType] = contentType.split(";"); + return mediaType.trim().toLowerCase() === JSON_MEDIA_TYPE; +}; + +const reject = ( + res: Response, + status: number, + code: string, + message: string +): void => { + res.status(status).json({ error: { message, code } }); +}; + +/** + * Header-only guard. Must run before any body parser so oversized or ambiguous + * requests are refused without buffering the payload. + */ +export const createEventIngestLimitsMiddleware = ( + options: EventIngestLimitsOptions = {} +): RequestHandler => { + const maxBodyBytes = options.maxBodyBytes ?? EVENT_INGEST_MAX_BODY_BYTES; + + return (req: Request, res: Response, next: NextFunction): void => { + const allowedProxies = + options.allowedChunkedProxies ?? + parseAllowlist(process.env[CHUNKED_PROXY_ALLOWLIST_ENV]); + + if (!isJsonMediaType(readHeader(req.headers["content-type"]))) { + reject( + res, + 415, + "INVALID_CONTENT_TYPE", + "Unsupported media type. Use application/json." + ); + return; + } + + const contentLength = readHeader(req.headers["content-length"]); + const transferEncoding = readHeader(req.headers["transfer-encoding"]); + const isChunked = (transferEncoding ?? "") + .toLowerCase() + .split(",") + .some((encoding) => encoding.trim() === "chunked"); + + if (isChunked) { + const proxyId = readHeader(req.headers[CHUNKED_PROXY_HEADER])?.trim(); + + if (!proxyId || !allowedProxies.includes(proxyId)) { + reject( + res, + 400, + "CHUNKED_ENCODING_NOT_ALLOWED", + "Chunked transfer encoding is not accepted on this endpoint." + ); + return; + } + + // Both framing headers at once is the canonical request-smuggling + // signature: intermediaries disagree on which one wins. + if (contentLength !== undefined) { + reject( + res, + 400, + "AMBIGUOUS_REQUEST_FRAMING", + "Content-Length must not be combined with Transfer-Encoding: chunked." + ); + return; + } + + next(); + return; + } + + if (contentLength === undefined) { + reject( + res, + 411, + "CONTENT_LENGTH_REQUIRED", + "Content-Length header is required." + ); + return; + } + + if (!CONTENT_LENGTH_PATTERN.test(contentLength.trim())) { + reject( + res, + 400, + "INVALID_CONTENT_LENGTH", + "Content-Length must be a single non-negative integer." + ); + return; + } + + if (Number(contentLength) > maxBodyBytes) { + reject( + res, + 413, + "BODY_LIMIT_EXCEEDED", + `Request body exceeds the ${maxBodyBytes} byte limit for this endpoint.` + ); + return; + } + + next(); + }; +}; + +export const eventIngestLimitsMiddleware = createEventIngestLimitsMiddleware(); + +const defaultJsonParser = express.json({ + limit: EVENT_INGEST_MAX_BODY_BYTES, + type: JSON_MEDIA_TYPE, + verify: (req: Request, _res: Response, buf: Buffer) => { + req.rawBody = buf; + }, +}); + +/** + * Parses the ingest body against the per-route budget and normalises + * body-parser failures into stable, payload-free error responses. + */ +export const createEventIngestBodyParser = ( + parser: RequestHandler = defaultJsonParser +): RequestHandler => { + return (req: Request, res: Response, next: NextFunction): void => { + parser(req, res, (err?: unknown) => { + if (!err) { + next(); + return; + } + + const type = (err as { type?: string }).type; + + if (type === "entity.too.large") { + reject( + res, + 413, + "BODY_LIMIT_EXCEEDED", + `Request body exceeds the ${EVENT_INGEST_MAX_BODY_BYTES} byte limit for this endpoint.` + ); + return; + } + + if (type === "request.size.invalid") { + reject( + res, + 400, + "CONTENT_LENGTH_MISMATCH", + "Request body size did not match the declared Content-Length." + ); + return; + } + + if (type === "entity.parse.failed" || type === "encoding.unsupported") { + // Deliberately drops the parser message, which quotes payload bytes. + reject(res, 400, "INVALID_JSON_BODY", "Request body is not valid JSON."); + return; + } + + next(err); + }); + }; +}; + +export const eventIngestBodyParser = createEventIngestBodyParser(); + +/** Guard chain to mount on the ingest route, in order. */ +export const eventIngestLimits: RequestHandler[] = [ + eventIngestLimitsMiddleware, + eventIngestBodyParser, +]; + +/** + * True when the request targets the ingest route, which owns its own parser and + * must therefore be skipped by the application-wide JSON body parser. + */ +export const isEventIngestRequest = (req: Request): boolean => { + if (req.method !== "POST") return false; + const path = (req.path ?? "").replace(/\/+$/, "").toLowerCase(); + return path === EVENT_INGEST_PATH; +}; diff --git a/backend/src/routes/v1/index.ts b/backend/src/routes/v1/index.ts index 5859d9399..0d693c64f 100644 --- a/backend/src/routes/v1/index.ts +++ b/backend/src/routes/v1/index.ts @@ -22,7 +22,7 @@ import { } from "../../services/eventValidator"; import { FileRawEventStore } from "../../services/rawEventStore"; import { getRateLimitPolicies } from "../../middleware/rate-limit"; -import { eventIngestLimitsMiddleware } from "../../middleware/event-ingest-limits"; +import { eventIngestLimits } from "../../middleware/event-ingest-limits"; const router = Router(); const eventIdStore = new FileRawEventStore(new DefaultEventValidator()); @@ -82,7 +82,7 @@ router.post( ); // Event processing endpoint (for indexer to post events) -router.post("/events", eventIngestLimitsMiddleware, async (req, res) => { +router.post("/events", ...eventIngestLimits, async (req, res) => { try { const events = Array.isArray(req.body) ? req.body : [req.body]; const validation = validateEventBatch(events); diff --git a/backend/src/tests/events-ingest-limits.test.ts b/backend/src/tests/events-ingest-limits.test.ts index 372baa9ce..2423b0494 100644 --- a/backend/src/tests/events-ingest-limits.test.ts +++ b/backend/src/tests/events-ingest-limits.test.ts @@ -15,13 +15,95 @@ jest.mock("pg", () => { }; }, { virtual: true }); +import express, { NextFunction, Request, RequestHandler, Response } from "express"; import supertest from "supertest"; import app from "../app"; import { statusService } from "../services/statusService"; +import { + CHUNKED_PROXY_ALLOWLIST_ENV, + CHUNKED_PROXY_HEADER, + EVENT_INGEST_MAX_BODY_BYTES, + EventIngestLimitsOptions, + createEventIngestBodyParser, + createEventIngestLimitsMiddleware, + eventIngestBodyParser, + eventIngestLimits, + eventIngestLimitsMiddleware, + isEventIngestRequest, +} from "../middleware/event-ingest-limits"; + +// Any string that appears in a rejection response would mean payload bytes are +// being echoed back to the caller. +const PAYLOAD_CANARY = "canary-payload-marker"; + +// The ingest endpoint is machine-to-machine; an API key exempts it from CSRF. +const INDEXER_API_KEY = "qlx_test_indexer_key"; + +interface GuardOutcome { + status?: number; + body?: { error?: { code?: string; message?: string } }; + nextCalled: boolean; +} + +const runGuard = ( + headers: Record, + options?: EventIngestLimitsOptions +): GuardOutcome => { + const middleware = options + ? createEventIngestLimitsMiddleware(options) + : eventIngestLimitsMiddleware; + + const outcome: GuardOutcome = { nextCalled: false }; + const res = { + status(code: number) { + outcome.status = code; + return this; + }, + json(body: unknown) { + outcome.body = body as GuardOutcome["body"]; + return this; + }, + }; + + middleware({ method: "POST", headers } as unknown as Request, res as unknown as Response, () => { + outcome.nextCalled = true; + }); + + return outcome; +}; + +const buildParserApp = (parser: RequestHandler) => { + const testApp = express(); + testApp.post("/events", parser, (req: Request, res: Response) => { + res.status(200).json({ parsed: true, rawBodyBytes: req.rawBody?.length ?? 0 }); + }); + testApp.use((err: Error, _req: Request, res: Response, _next: NextFunction) => { + res.status(500).json({ error: { code: "FORWARDED_TO_ERROR_HANDLER" } }); + }); + return testApp; +}; + +const validEvent = (id: string) => ({ + id, + ledger: 42, + txHash: `tx-${id}`, + timestamp: 1700000000, + complianceHold: false, + indexedAt: "2026-01-01T00:00:00.000Z", + type: "InvoiceSettled", + payload: { + invoice_id: `inv-${id}`, + business: "business-1", + investor: "investor-1", + amount: "1000", + }, +}); + +describe("Event ingest limits", () => { + const originalAllowlist = process.env[CHUNKED_PROXY_ALLOWLIST_ENV]; -describe("Event Ingest Limits Middleware Tests", () => { beforeEach(() => { - // Reset status service mock ledger to avoid degraded mode + delete process.env[CHUNKED_PROXY_ALLOWLIST_ENV]; statusService.setMockCurrentLedger(100000); statusService.updateLastIndexedLedger(100000); }); @@ -30,125 +112,371 @@ describe("Event Ingest Limits Middleware Tests", () => { statusService.setMockCurrentLedger(null); }); - describe("Content-Type Validation", () => { - it("rejects requests with missing Content-Type header", async () => { - const res = await supertest(app) - .post("/api/v1/events") - .set("Content-Length", "20") - .send('{"test":"data"}'); + afterAll(() => { + if (originalAllowlist === undefined) { + delete process.env[CHUNKED_PROXY_ALLOWLIST_ENV]; + return; + } + process.env[CHUNKED_PROXY_ALLOWLIST_ENV] = originalAllowlist; + }); - expect(res.status).toBe(415); - expect(res.body.error.code).toBe("INVALID_CONTENT_TYPE"); + describe("content type", () => { + it("rejects a missing Content-Type with 415", () => { + const outcome = runGuard({ "content-length": "20" }); + + expect(outcome.status).toBe(415); + expect(outcome.body?.error?.code).toBe("INVALID_CONTENT_TYPE"); + expect(outcome.nextCalled).toBe(false); + }); + + it("rejects a non-JSON Content-Type with 415", () => { + const outcome = runGuard({ + "content-type": "text/plain", + "content-length": "20", + }); + + expect(outcome.status).toBe(415); + expect(outcome.body?.error?.code).toBe("INVALID_CONTENT_TYPE"); + }); + + it("rejects media types that merely start with application/json", () => { + const outcome = runGuard({ + "content-type": "application/jsonrequest", + "content-length": "20", + }); + + expect(outcome.status).toBe(415); + expect(outcome.body?.error?.code).toBe("INVALID_CONTENT_TYPE"); + }); + + it("accepts application/json with charset parameters", () => { + const outcome = runGuard({ + "content-type": "Application/JSON; charset=utf-8", + "content-length": "20", + }); + + expect(outcome.nextCalled).toBe(true); + expect(outcome.status).toBeUndefined(); + }); + }); + + describe("content length", () => { + it("rejects a missing Content-Length with 411", () => { + const outcome = runGuard({ "content-type": "application/json" }); + + expect(outcome.status).toBe(411); + expect(outcome.body?.error?.code).toBe("CONTENT_LENGTH_REQUIRED"); + expect(outcome.nextCalled).toBe(false); + }); + + it("rejects a non-numeric Content-Length with 400", () => { + const outcome = runGuard({ + "content-type": "application/json", + "content-length": "not-a-number", + }); + + expect(outcome.status).toBe(400); + expect(outcome.body?.error?.code).toBe("INVALID_CONTENT_LENGTH"); + }); + + it("rejects duplicated Content-Length headers with 400", () => { + const outcome = runGuard({ + "content-type": "application/json", + "content-length": ["12", "999"], + }); + + expect(outcome.status).toBe(400); + expect(outcome.body?.error?.code).toBe("INVALID_CONTENT_LENGTH"); + }); + + it("rejects a Content-Length above the 256KB budget with 413", () => { + const outcome = runGuard({ + "content-type": "application/json", + "content-length": String(EVENT_INGEST_MAX_BODY_BYTES + 1), + }); + + expect(outcome.status).toBe(413); + expect(outcome.body?.error?.code).toBe("BODY_LIMIT_EXCEEDED"); + }); + + it("accepts a Content-Length exactly at the budget", () => { + const outcome = runGuard({ + "content-type": "application/json", + "content-length": String(EVENT_INGEST_MAX_BODY_BYTES), + }); + + expect(outcome.nextCalled).toBe(true); + }); + + it("honours a caller-supplied budget", () => { + const outcome = runGuard( + { "content-type": "application/json", "content-length": "2048" }, + { maxBodyBytes: 1024 } + ); + + expect(outcome.status).toBe(413); + expect(outcome.body?.error?.code).toBe("BODY_LIMIT_EXCEEDED"); + }); + }); + + describe("transfer encoding", () => { + it("rejects chunked encoding when no proxy header is present", () => { + const outcome = runGuard({ + "content-type": "application/json", + "transfer-encoding": "chunked", + }); + + expect(outcome.status).toBe(400); + expect(outcome.body?.error?.code).toBe("CHUNKED_ENCODING_NOT_ALLOWED"); + expect(outcome.nextCalled).toBe(false); + }); + + it("rejects chunked encoding from a proxy that is not on the allowlist", () => { + const outcome = runGuard( + { + "content-type": "application/json", + "transfer-encoding": "chunked", + [CHUNKED_PROXY_HEADER]: "rogue-proxy", + }, + { allowedChunkedProxies: ["edge-proxy-1"] } + ); + + expect(outcome.status).toBe(400); + expect(outcome.body?.error?.code).toBe("CHUNKED_ENCODING_NOT_ALLOWED"); + }); + + it("accepts chunked encoding from an allowlisted proxy", () => { + const outcome = runGuard( + { + "content-type": "application/json", + "transfer-encoding": "gzip, chunked", + [CHUNKED_PROXY_HEADER]: " edge-proxy-1 ", + }, + { allowedChunkedProxies: ["edge-proxy-1"] } + ); + + expect(outcome.nextCalled).toBe(true); + expect(outcome.status).toBeUndefined(); + }); + + it("reads the allowlist from the environment when no option is given", () => { + process.env[CHUNKED_PROXY_ALLOWLIST_ENV] = " edge-proxy-1 , edge-proxy-2 "; + + const outcome = runGuard({ + "content-type": "application/json", + "transfer-encoding": ["chunked"], + [CHUNKED_PROXY_HEADER]: "edge-proxy-2", + }); + + expect(outcome.nextCalled).toBe(true); + }); + + it("rejects chunked encoding combined with Content-Length as ambiguous framing", () => { + const outcome = runGuard( + { + "content-type": "application/json", + "transfer-encoding": "chunked", + "content-length": "20", + [CHUNKED_PROXY_HEADER]: "edge-proxy-1", + }, + { allowedChunkedProxies: ["edge-proxy-1"] } + ); + + expect(outcome.status).toBe(400); + expect(outcome.body?.error?.code).toBe("AMBIGUOUS_REQUEST_FRAMING"); + expect(outcome.nextCalled).toBe(false); + }); + + it("still requires Content-Length for non-chunked transfer encodings", () => { + const outcome = runGuard({ + "content-type": "application/json", + "transfer-encoding": "gzip", + }); + + expect(outcome.status).toBe(411); + expect(outcome.body?.error?.code).toBe("CONTENT_LENGTH_REQUIRED"); + }); + }); + + describe("body parser budget", () => { + it("rejects a body above the budget with 413", async () => { + const oversized = JSON.stringify([{ pad: "x".repeat(EVENT_INGEST_MAX_BODY_BYTES) }]); + + const res = await supertest(buildParserApp(eventIngestBodyParser)) + .post("/events") + .set("Content-Type", "application/json") + .send(oversized); + + expect(res.status).toBe(413); + expect(res.body.error.code).toBe("BODY_LIMIT_EXCEEDED"); + }); + + it("rejects malformed JSON without echoing payload bytes", async () => { + const res = await supertest(buildParserApp(eventIngestBodyParser)) + .post("/events") + .set("Content-Type", "application/json") + .send(PAYLOAD_CANARY); + + expect(res.status).toBe(400); + expect(res.body.error.code).toBe("INVALID_JSON_BODY"); + expect(JSON.stringify(res.body)).not.toContain(PAYLOAD_CANARY); + }); + + it("parses a valid body and exposes the raw bytes", async () => { + const body = JSON.stringify([validEvent("parser-ok")]); + + const res = await supertest(buildParserApp(eventIngestBodyParser)) + .post("/events") + .set("Content-Type", "application/json") + .send(body); + + expect(res.status).toBe(200); + expect(res.body.rawBodyBytes).toBe(Buffer.byteLength(body)); + }); + + it("maps a Content-Length mismatch to 400", async () => { + const failingParser = createEventIngestBodyParser((_req, _res, next) => { + next(Object.assign(new Error("size mismatch"), { type: "request.size.invalid" })); + }); + + const res = await supertest(buildParserApp(failingParser)) + .post("/events") + .set("Content-Type", "application/json") + .send("{}"); + + expect(res.status).toBe(400); + expect(res.body.error.code).toBe("CONTENT_LENGTH_MISMATCH"); }); - it("rejects requests with non-JSON Content-Type", async () => { + it("forwards unrecognised parser errors to the error handler", async () => { + const failingParser = createEventIngestBodyParser((_req, _res, next) => { + next(new Error("stream aborted")); + }); + + const res = await supertest(buildParserApp(failingParser)) + .post("/events") + .set("Content-Type", "application/json") + .send("{}"); + + expect(res.status).toBe(500); + expect(res.body.error.code).toBe("FORWARDED_TO_ERROR_HANDLER"); + }); + }); + + describe("guard chain", () => { + const buildGuardedApp = () => { + const testApp = express(); + testApp.post("/events", ...eventIngestLimits, (req: Request, res: Response) => { + res.status(200).json({ accepted: true, events: req.body }); + }); + return testApp; + }; + + it("passes a valid, correctly framed request through to the handler", async () => { + const events = [validEvent("chain-ok")]; + + const res = await supertest(buildGuardedApp()) + .post("/events") + .set("Content-Type", "application/json") + .send(JSON.stringify(events)); + + expect(res.status).toBe(200); + expect(res.body).toEqual({ accepted: true, events }); + }); + + it("rejects before parsing when the declared size exceeds the budget", async () => { + const res = await supertest(buildGuardedApp()) + .post("/events") + .set("Content-Type", "application/json") + .send(JSON.stringify([{ pad: `${PAYLOAD_CANARY}${"x".repeat(EVENT_INGEST_MAX_BODY_BYTES)}` }])); + + expect(res.status).toBe(413); + expect(res.body.error.code).toBe("BODY_LIMIT_EXCEEDED"); + expect(JSON.stringify(res.body)).not.toContain(PAYLOAD_CANARY); + }); + }); + + describe("isEventIngestRequest", () => { + it.each([ + [{ method: "POST", path: "/api/v1/events" }, true], + [{ method: "POST", path: "/api/v1/events/" }, true], + [{ method: "POST", path: "/API/V1/EVENTS" }, true], + [{ method: "GET", path: "/api/v1/events" }, false], + [{ method: "POST", path: "/api/v1/invoices" }, false], + [{ method: "POST", path: undefined }, false], + ])("matches %p as %p", (candidate, expected) => { + expect(isEventIngestRequest(candidate as unknown as Request)).toBe(expected); + }); + }); + + describe("POST /api/v1/events", () => { + it("rejects a non-JSON content type with 415", async () => { const res = await supertest(app) .post("/api/v1/events") + .set("x-api-key", INDEXER_API_KEY) .set("Content-Type", "text/plain") - .set("Content-Length", "20") - .send("plain text"); + .send(PAYLOAD_CANARY); expect(res.status).toBe(415); expect(res.body.error.code).toBe("INVALID_CONTENT_TYPE"); + expect(JSON.stringify(res.body)).not.toContain(PAYLOAD_CANARY); }); - it("accepts requests with application/json Content-Type", async () => { - // This will fail later validation but should pass middleware + it("rejects an oversized body with 413 before it is parsed", async () => { + const oversized = JSON.stringify([ + { ...validEvent("too-big"), pad: `${PAYLOAD_CANARY}${"x".repeat(EVENT_INGEST_MAX_BODY_BYTES)}` }, + ]); + const res = await supertest(app) .post("/api/v1/events") + .set("x-api-key", INDEXER_API_KEY) .set("Content-Type", "application/json") - .set("Content-Length", "20") - .send('{"test":"data"}'); + .send(oversized); - expect(res.status).not.toBe(415); + expect(res.status).toBe(413); + expect(res.body.error.code).toBe("BODY_LIMIT_EXCEEDED"); + expect(JSON.stringify(res.body)).not.toContain(PAYLOAD_CANARY); }); - }); - describe("Content-Length Validation", () => { - it("rejects requests with missing Content-Length header", async () => { + it("rejects a body between the route budget and the global 1MB budget", async () => { + const body = JSON.stringify([{ pad: "x".repeat(400 * 1024) }]); + expect(Buffer.byteLength(body)).toBeGreaterThan(EVENT_INGEST_MAX_BODY_BYTES); + expect(Buffer.byteLength(body)).toBeLessThan(1024 * 1024); + const res = await supertest(app) .post("/api/v1/events") + .set("x-api-key", INDEXER_API_KEY) .set("Content-Type", "application/json") - .send('{"test":"data"}'); + .send(body); - expect(res.status).toBe(411); - expect(res.body.error.code).toBe("CONTENT_LENGTH_REQUIRED"); + expect(res.status).toBe(413); + expect(res.body.error.code).toBe("BODY_LIMIT_EXCEEDED"); }); - it("rejects requests with Content-Length exceeding 256KB", async () => { - const largeBody = "x".repeat(256 * 1024 + 1); + it("rejects malformed JSON without echoing payload bytes", async () => { const res = await supertest(app) .post("/api/v1/events") + .set("x-api-key", INDEXER_API_KEY) .set("Content-Type", "application/json") - .set("Content-Length", String(largeBody.length)) - .send(largeBody); + .send(`{"broken": ${PAYLOAD_CANARY}`); - expect(res.status).toBe(413); - expect(res.body.error.code).toBe("BODY_LIMIT_EXCEEDED"); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe("INVALID_JSON_BODY"); + expect(JSON.stringify(res.body)).not.toContain(PAYLOAD_CANARY); }); - it("accepts requests with Content-Length within limit", async () => { + it("lets a valid, correctly framed event batch reach the route handler", async () => { const res = await supertest(app) .post("/api/v1/events") + .set("x-api-key", INDEXER_API_KEY) .set("Content-Type", "application/json") - .set("Content-Length", "20") - .send('{"test":"data"}'); - - expect(res.status).not.toBe(411); - expect(res.status).not.toBe(413); - }); - }); + .send(JSON.stringify([validEvent(`ingest-ok-${Date.now()}`)])); - describe("Transfer-Encoding Validation", () => { - it("rejects requests with chunked Transfer-Encoding without allowlist header", async () => { - // Note: supertest doesn't easily send chunked encoding, so we'll test via direct middleware call - const { eventIngestLimitsMiddleware } = require("../middleware/event-ingest-limits"); - const mockReq = { - headers: { - "content-type": "application/json", - "content-length": "20", - "transfer-encoding": "chunked", - }, - }; - let mockRes = { - status: jest.fn().mockReturnThis(), - json: jest.fn(), - }; - let nextCalled = false; - const mockNext = () => { nextCalled = true; }; - - eventIngestLimitsMiddleware(mockReq as any, mockRes as any, mockNext); - - expect(mockRes.status).toHaveBeenCalledWith(400); - expect(mockRes.json).toHaveBeenCalledWith(expect.objectContaining({ - error: expect.objectContaining({ - code: "CHUNKED_ENCODING_NOT_ALLOWED", - }), - })); - expect(nextCalled).toBe(false); - }); - - it("accepts requests with chunked Transfer-Encoding when allowlist header is present", async () => { - const { eventIngestLimitsMiddleware } = require("../middleware/event-ingest-limits"); - const mockReq = { - headers: { - "content-type": "application/json", - "content-length": "20", - "transfer-encoding": "chunked", - "x-allow-chunked-encoding": "true", - }, - }; - let mockRes = { - status: jest.fn().mockReturnThis(), - json: jest.fn(), - }; - let nextCalled = false; - const mockNext = () => { nextCalled = true; }; - - eventIngestLimitsMiddleware(mockReq as any, mockRes as any, mockNext); - - expect(mockRes.status).not.toHaveBeenCalled(); - expect(nextCalled).toBe(true); + // The handler envelope proves the guard chain forwarded the request; the + // outcome of event processing itself is covered by the ingestion suites. + expect([411, 413, 415]).not.toContain(res.status); + expect(res.body).toHaveProperty("results"); + expect(res.body.results).toHaveLength(1); }); }); });