Skip to content

ci: govulncheck gate for the backend, and patch what it found - #310

Merged
dusanstanojeviccs merged 4 commits into
mainfrom
claude/traceway-issue-129df7
Aug 26, 2026
Merged

ci: govulncheck gate for the backend, and patch what it found#310
dusanstanojeviccs merged 4 commits into
mainfrom
claude/traceway-issue-129df7

Conversation

@FrameAutomata

Copy link
Copy Markdown
Collaborator

Closes #308.

Step one of that issue was to actually run the scan rather than assume the CLI's result carried over. It doesn't — the gap is materially larger than the 4 advisories #308 projected, and part of it is not a toolchain problem at all.

What the scan found

./cmd/traceway-runner — the one binary we build and hand to users — had 9 reachable stdlib vulnerabilities, not 4. The extra five come from its own call graph: crypto/x509 certificate verification, net/textproto header parsing, the http2 transport. The CLI never exercises those, which is why its result under-predicted this.

Whole module, default tags: 21 reachable, and critically 5 of them are module-level, which no toolchain bump fixes:

Advisory Module Bump
GO-2026-6061 google.golang.org/grpc v1.81.1 → v1.82.1
GO-2026-5970 golang.org/x/text v0.37.0 → v0.39.0
GO-2026-5764 aws-sdk-go-v2/service/s3 v1.96.0 → v1.97.3
GO-2026-5764 aws-sdk-go-v2/.../eventstream v1.7.4 → v1.7.8
GO-2026-5676, GO-2025-4233 github.com/quic-go/quic-go v0.54.0 → v0.59.1

Without those the new gate would have been red the moment it merged, which is the failure mode that makes a security job get ignored.

The remaining 16 are stdlib, and go1.26.6 is the highest fixed-in version across all of them — confirmed from the OSV records, not assumed.

Toolchain

toolchain go1.26.6, not a raise of the go line, for the reason #307 gives: since Go 1.21 the go line is a hard floor propagated to consumers, and this module is imported as tracewaybackend by cli/test/contract and three examples/ modules. Forcing every importer up over a CVE in the build toolchain is the wrong lever.

Verified this actually reaches the release path: setup-go@v5 parses only the go line (parseGoVersionFile matches /^go (\d+(\.\d+)*)/m) and never sets GOTOOLCHAIN, so it installs the 1.26.2 floor and the go command then switches to 1.26.6 on first use in the module. release-traceway.yml:167 builds the runner binaries through exactly that path.

CI gate

One job per shipped storage backend rather than a single default scan. The tags are not cosmetic — go list -deps ./cmd/traceway returns 0 ClickHouse packages on the default build and 11 under transactional_pg,telemetry_ch, so a default-only scan would leave the driver stacks the published images actually run entirely unscanned. Rows map 1:1 to Dockerfile.sqlite / Dockerfile.duckdb+.browser / Dockerfile+.minimal.

Daily schedule plus workflow_dispatch, because both push and PR triggers are path-filtered and a CVE lands in the database without anyone pushing a commit. govulncheck is pinned: freshness comes from the schedule, since the advisory database is fetched at run time regardless of binary version.

Path filters include cli/**, not just backend/**backend/go.mod:207 carries replace github.com/tracewayapp/traceway/cli => ../cli, so the scan compiles working-tree CLI source and a cli-only change can move the result. cli.yml already carries the reciprocal filter.

Also concurrency (matching four existing workflows) and timeout-minutes: 15 — govulncheck's advisory-database client has no HTTP timeout and that fetch is the first thing it does, so a network hang would otherwise sit at GitHub's 360-minute default across three legs on an unattended cron.

Verification

All three supported combinations — default dual-SQLite, telemetry_duckdb with CGO, transactional_pg telemetry_chbuild, test, and scan clean: 0 reachable vulnerabilities, exit 0.

Two caveats, stated rather than buried:

  • The scan ran against a local mirror of the advisory database, not vuln.go.dev. That host is blocked by my sandbox's egress policy. Rather than route around it, I built a v1 database from golang.org/x/vulndb (fetched through the allowed Go module proxy, snapshot dated 2026-08-25) using that project's own cmd/indexdb, and scanned against it with -db file://…. The workflow itself uses the real default. CI will be the first run against the live database.
  • One pre-existing test failure under telemetry_duckdb: app/controllers fails on a telemetry-migration dialect error in setup_test.go. I confirmed it fails identically on a clean origin/main worktree, so it is not from this change. Root-caused and filed as app/controllers tests fail under -tags telemetry_duckdb: setup_test.go is missing the !telemetry_duckdb build constraint #309 with a verified one-line fix; deliberately not included here, since it is unrelated test-harness churn on a security PR.

On #308's point 4 — single source of truth for the toolchain

I'd argue against building one, and one premise in the issue turns out not to hold.

benchmark-hardware.yml's hardcoded go-version: '1.25' is not a stale backend pin — it matches benchmarks/loadgen/go.mod (go 1.25), the module it actually builds. So it is correct as written.

That leaves three real pins with no Go mechanism to unify short of a go.work. Checked and absent repo-wide: no go.work, no .go-version, no dependabot.yml, no renovate.json. A .go-version file would govern only setup-go's installed Go while the toolchain directive still selects the compiler — a sixth hand-edited file, not a single source of truth. The honest framing is that the available mechanism is a dependency bot (Renovate's gomod manager does update toolchain directives), which is new infrastructure rather than a config change.

Meanwhile the daily per-module schedule is the right detector for #307's actual failure mode — bump two of three modules, miss one. That is what this adds for backend/.

Known gaps, deliberately not here

  • cli/go.mod still has no toolchain pin, and release-cli.yml builds published binaries from it at 1.26.2. ci: enforce the CLI's documented pre-commit gate #307 already makes exactly that edit, so duplicating it here would only conflict. Worth noting the CLI's govulncheck is still local-only (just vulncheck) until ci: enforce the CLI's documented pre-commit gate #307 lands.
  • The oxc build tag is not scanned. It is orthogonal to the storage axis, compiles (//go:build oxc && cgo), and is documented via SYMBOLICATOR_PARSER, but needs scripts/build-oxc-shim.sh to build. Noted in the workflow.
  • linux/amd64 only. Release also cross-compiles the runner for darwin and windows. Every finding so far is either a stdlib symbol the toolchain pin resolves for all targets or a platform-independent module version, so a GOOS axis is a follow-up rather than an oversight.
  • The toolchain pin does not reach the Docker images. The official golang base images set ENV GOTOOLCHAIN=local, under which a toolchain directive is silently ignored — verified from the image's source Dockerfile. Not a risk: the floating golang:1.26-* tags resolve to ≥ what CI scans. But it means this gate certifies the runner binaries, not the images, and the comments say so.

README.md and docs/pages/learn/contributing.mdx also still said Go 1.25 — below the module's own floor, and stated as an install prerequisite. Fixed here since the PR touches the same fact in CLAUDE.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LW8APoZZwVdoHkPeUYGnwW


Generated by Claude Code

claude added 4 commits August 26, 2026 00:14
govulncheck against backend/ at the current `go 1.26.2` pin reports 21
reachable vulnerabilities: 16 in the standard library and 5 in required
modules. The shipped artifact is the worst of it -- `./cmd/traceway-runner`
alone has 9 reachable stdlib vulnerabilities, and that binary runs on user
infrastructure with an outbound HTTPS long-poll as its main loop, so
crypto/tls, crypto/x509, net/http and net/textproto are all on its hot path.

Toolchain: `toolchain go1.26.6`, not a raise of the `go` line. Since Go 1.21
the `go` line is a hard floor propagated to consumers, and the backend is
importable as tracewaybackend; forcing every importer up over a CVE in the
build toolchain is the wrong lever. Same reasoning as the CLI module.
go1.26.6 is the highest fixed-in version across all 16 stdlib findings.

setup-go@v5 reads only the `go` line and never sets GOTOOLCHAIN, so it
installs the 1.26.2 floor and the go command then switches to 1.26.6 on
first use in the module. release-traceway.yml builds the runner binaries
through exactly that path.

Modules, which no toolchain bump can fix:

  GO-2026-6061  google.golang.org/grpc              v1.81.1 -> v1.82.1
  GO-2026-5970  golang.org/x/text                   v0.37.0 -> v0.39.0
  GO-2026-5764  aws-sdk-go-v2/service/s3            v1.96.0 -> v1.97.3
                aws-sdk-go-v2/.../eventstream       v1.7.4  -> v1.7.8
  GO-2026-5676  github.com/quic-go/quic-go          v0.54.0 -> v0.59.1
  GO-2025-4233  github.com/quic-go/quic-go          (same bump)

Verified: all three supported build-tag combinations (default dual-SQLite,
telemetry_duckdb with CGO, transactional_pg telemetry_ch) build, test and
scan clean -- govulncheck reports 0 reachable vulnerabilities and exits 0
for each. The one failing package under telemetry_duckdb
(app/controllers, a telemetry-migration dialect error in setup_test.go)
fails identically before this change and is untouched by it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LW8APoZZwVdoHkPeUYGnwW
Nothing scanned backend/ -- the only govulncheck in the repo was the CLI's,
and backend/ is the module that ships a binary to users.

One job per supported build-tag combination rather than a single default
scan. The tags are not cosmetic: they select which db/ and repositories/
packages compile in, and each backend brings its own driver stack. Measured
on this tree, `go list -deps ./cmd/traceway` returns 0 ClickHouse packages
on the default build and 11 under transactional_pg,telemetry_ch, so a
default-only scan would leave the driver stacks the published Docker images
actually run completely unscanned. The three in the matrix are the only
combinations that compile; app/db/guard_*.go rejects the rest.

A daily schedule plus workflow_dispatch, because both push and pull_request
are path-filtered and a CVE lands in the database without anyone pushing a
commit. Staggered off the CLI workflow's 06:17. govulncheck is pinned:
freshness comes from the schedule, since the advisory database is fetched
at run time regardless of binary version.

No -test flag -- the gate is about what ships, and failing a release on a
test-only dependency would train people to ignore the job.

Scope is stated in the workflow rather than left implied: this scans
linux/amd64 source, while release also cross-compiles the runner for darwin
and windows. Every finding so far is either a stdlib symbol the toolchain
pin resolves for all targets or a platform-independent module version, so a
GOOS axis is a deliberate follow-up rather than an oversight.

Verified green for all three combinations on this tree.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LW8APoZZwVdoHkPeUYGnwW
Cleanup pass over the previous two commits.

Removed three comments citing `.github/workflows/cli-lint.yml`, which does
not exist on main and never has -- it is added by the still-open #307. They
claimed cli/ was already gated (it is not; the CLI's only vulncheck is the
local `just vulncheck`) and described a cron stagger against a schedule that
does not exist. This one is the worst of the set: it disguises a real gap
rather than just being noise.

Dropped the `if [ -n "$TAGS" ]` branch. Verified against this module that
`govulncheck -tags "" ./...` is identical to the untagged form -- 0 called
vulnerabilities, exit 0 -- so the conditional was dead code with two paths,
one of which two of the three matrix legs never took.

Added `cli/**` to the path filters. backend/go.mod carries
`replace github.com/tracewayapp/traceway/cli => ../cli`, so the scan compiles
working-tree CLI source; a cli-only PR could change what is reachable and
never trigger this gate. cli.yml already carries the reciprocal filter.

Added `concurrency` (the repo already uses it in four workflows) and
`timeout-minutes: 15`. govulncheck's advisory-database client has no HTTP
timeout, and its fetch is the first thing it does, so a network hang would
otherwise sit at GitHub's 360-minute default across three legs on an
unattended cron.

Corrected two overclaims. "These three are the only combinations that
compile" is true of the storage axis only -- the orthogonal `oxc` symbolicator
tag also builds and is not scanned. And the toolchain pin does not reach the
Docker images at all: the official golang bases set GOTOOLCHAIN=local, so a
`toolchain` directive is ignored there and the images build with whatever the
floating golang:1.26-* tag ships. That is still >= what CI scans, so the
images are not at risk, but the comment claimed coverage it does not have.

Comments trimmed 42/104 -> 29/93, toward the repo's convention (cli.yml 0/52,
cli-contract.yml 4/41, release-traceway.yml 9/344).

README.md and docs/pages/learn/contributing.mdx still said Go 1.25, below the
module's own floor -- contributing.mdx states it as an install prerequisite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LW8APoZZwVdoHkPeUYGnwW
CI caught this: both `contract` and `build-test` failed on
`go: updates to go.mod needed; to update it: go mod tidy`.

cli/test/contract requires `github.com/tracewayapp/traceway/backend v0.0.0`
through a local `replace`, so it records the backend's transitive
requirements in its own go.mod. Bumping grpc, x/text, quic-go and the AWS SDK
in the backend left those stale, which is a hard error rather than an
auto-fix in a build.

`go mod tidy` propagates the same versions and nothing else. The `go 1.26.2`
line is unchanged and no `toolchain` directive is added here, so this stays
off the line #307 edits in this file.

Verified: `cd cli/test/contract && go test ./...` passes (ok, 5.5s -- it
boots the backend in SQLite mode and asserts the wire contract), and
`go mod tidy -diff` is clean afterwards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LW8APoZZwVdoHkPeUYGnwW
@dusanstanojeviccs
dusanstanojeviccs merged commit 6497c72 into main Aug 26, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backend has no govulncheck gate, and ships traceway-runner binaries built with an unpatched Go toolchain

3 participants