test: add AWS XKS CI test against remote server

[p0wline]

Context

Before this PR, XKS tests only ran locally against a mock server. There was no way to catch regressions in the full AWS integration path.

What this PR does

1. scratchstack 0.10 → 0.11 migration

The SigV4 verification library rejected secret keys longer than 43 characters. Version 0.11 removes that restriction, which was blocking the real AWS infrastructure (key was 52 chars).

2. CI workflow .github/workflows/test_xks_remote.yml

On every PR and push:

• Builds a non-fips release binary
• Deploys it to the persistent EC2 test server (aws-xks-kms.cosmian.dev) via SSH
• Step 1 — Proxy-direct tests: provisions three test keys (aws_xks_kek, encrypt_only_key, decrypt_only_key), enforces usage restrictions (encrypt-only / decrypt-only), then runs the vendored AWS XKS curl test suite directly against the server using SigV4 — no AWS KMS involved
• Step 2 — End-to-end test: calls aws kms encrypt / aws kms decrypt via the real AWS KMS API, which routes through the configured external key store to our proxy — verifies the full chain from AWS to Cosmian KMS and back

3. Cherry-pick of a14031598 (fix: create a dedicated AWS-XKS user)

Pulled in from your fix/xks_permissions branch. The server now uses the reserved identity [aws-xks-service] for all XKS operations instead of awsPrincipalArn from the request body. This is required for the end-to-end test to work: when AWS KMS calls the proxy, the injected ARN varies per caller, but [aws-xks-service] gets a single permanent grant at startup on every XKS key.

4. test_data submodule update

Points to fix/xks-config-env-override which fixes config.sh to respect env vars (see test_data PR #13). Without this, the vendored test suite ignored the SigV4 credentials injected by the CI workflow.

5. lychee.toml

Added kms\.example\.com and kms\.different\.com to the link-checker exclusions — these are fictional hostnames used in unit tests inside start_kms_server.rs and were causing lychee to fail on every commit.

Required GitHub secrets (repo Cosmian/kms)

Secret	Purpose
AWS_XKS_ACCESS_KEY_ID	IAM key ID for aws kms encrypt/decrypt (xks-test user)
AWS_XKS_SECRET_ACCESS_KEY	Matching IAM secret
AWS_XKS_KEY_ARN	ARN of the KMS key backed by the external key store
KMS_XKS_SIGV4_ACCESS_KEY_ID	SigV4 access key configured on the proxy server
KMS_XKS_SIGV4_SECRET_ACCESS_KEY	Matching SigV4 secret key
KMS_XKS_SSH_PRIVATE_KEY	SSH private key to deploy the binary to the EC2 instance
KMS_XKS_SSH_KNOWN_HOSTS	SSH known hosts entry for aws-xks-kms.cosmian.dev
KMS_XKS_SSH_USER	SSH user on the EC2 instance

Note

Add remote AWS XKS CI test and harden Sigv4Middleware with scratchstack 0.11

• Adds a GitHub Actions workflow (test_xks_remote.yml) that builds the release KMS binary, deploys it to a remote XKS server via deploy_xks_remote.sh, and runs the vendored XKS test suite plus an AWS KMS encrypt/decrypt end-to-end test
• Migrates aws_xks.Sigv4Middleware.call from the synchronous scratchstack 0.10 API to the 0.11 validator with a local SigningKeyService; request verification now runs in Tokio's blocking pool with a CPU-sized semaphore
• Adds enforce_sigv4_request_freshness to reject requests outside a five-minute replay window, with timestamp extraction from X-Amz-Date header, Date header, or X-Amz-Date query parameter
• Adds remote test runner (test_xks_remote.sh) that provisions test keys, grants permissions to both AWS identities, enforces encrypt-only and decrypt-only cases, and runs the vendored curl suite over HTTPS
• Risk: XKS requests that previously passed without timestamp validation will now be rejected if the SigV4 timestamp is missing, invalid, or outside the five-minute window; the deployment script replaces the remote binary and rolls back on health-check failure, but a failed rollback could leave the service down

^{Macroscope summarized 65b72e5.}

[macroscopeapp]

Macroscope has since reviewed this pull request. An earlier review was skipped by a cost limit; a review has now completed, so that notice no longer applies.

[macroscopeapp]

🟠 High aws_xks/sigv4_middleware.rs:282

A request queued on SIGV4_VERIFICATION_PERMITS can pass the five-minute freshness check after that window has expired, weakening the anti-replay control. server_timestamp is captured before permit acquisition and reused by both enforce_sigv4_request_freshness and sigv4_validate_request; capture the current time after acquiring the permit or immediately before validation.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @crate/server/src/routes/aws_xks/sigv4_middleware.rs around line 282:

A request queued on `SIGV4_VERIFICATION_PERMITS` can pass the five-minute freshness check after that window has expired, weakening the anti-replay control. `server_timestamp` is captured before permit acquisition and reused by both `enforce_sigv4_request_freshness` and `sigv4_validate_request`; capture the current time after acquiring the permit or immediately before validation.