refactor: rewrite staging API tester in Go

[michaelkedar]

This PR migrates the staging_api_test service from Python (gcp/workers/staging_api_test/) to Go (go/cmd/staging_api_test/), aligning it with the ongoing Python-to-Go migration across the repository.

Key Changes

1. Go Implementation (go/cmd/staging_api_test/):

   • loader.go: Downloads and directly processes all.zip (from GCS or local path) in memory/stream, extracting lightweight records (SimpleVuln) and partitioning query pools (VulnQueryIDs, PackageQueryIDs, and LargeBatchQueryIDs)
     without creating temporary JSON shards on disk.
   • generator.go: High-performance, concurrent traffic generator with:
     • Micro-ticker scheduling for smooth rate pacing across /v1/vulns/{id}, /v1/query (package, version, PURL), and standard/large /v1/querybatch requests.
     • Thread-safe independent RNG instances per worker goroutine.
     • Configurable request rates and batch size limits.
     • Custom high-throughput http.Transport with HTTP/2, connection pooling, and response body draining.
     • Atomic counters and periodic stats reporting (GeneratorStats).
     • Diagnostic response snippet logging for HTTP errors ($\ge 400$).
   • main.go: CLI flags with environment variable fallbacks, signal-based graceful shutdown (SIGINT/SIGTERM), and structured logging.
   • Unit Tests: Full unit test coverage in loader_test.go and generator_test.go covering payload builders, zip extraction, edge cases, and traffic generator lifecycles.

2. Build & Deployment:

   • go/Dockerfile: Added staging-api-test multi-stage build target using distroless Debian 12.
   • deployment/build-and-stage.yaml: Updated Cloud Build steps to build staging-api-test from go/Dockerfile using BuildKit with --build-context bindings=../bindings.
   • Cleanup: Removed deprecated Python scripts and configs under gcp/workers/staging_api_test/.

Verification

• Passed unit tests and race detection: go test -race -v ./cmd/staging_api_test/...
• Passed full test suite: make go-tests
• Passed formatting and lint checks: poetry run tools/lint_and_format.sh

agy

---

_{gemini sure is wordy...}