Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,9 @@ jobs:
with:
go-version-file: go.mod
cache: true
- name: go test -tags=live (etag + retry)
- name: go test -tags=live (etag + retry + pages + polling)
env:
GITHUB_TOKEN: ${{ github.token }}
run: go test -tags=live -run 'TestETag_Live|TestRetry_Live' ./etag/... ./retry/...
# Server-to-server endpoints only; user-to-server ones 403 under
# the App installation token.
run: go test -tags=live -run 'TestETag_Live|TestRetry_Live|TestPages_Live|TestPoll_Live' ./etag/... ./retry/... ./pages/... ./polling/...
25 changes: 24 additions & 1 deletion .github/workflows/fuzz.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,30 @@ jobs:
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fuzz-corpus
name: fuzz-corpus-etag
path: etag/testdata/fuzz/
if-no-files-found: ignore
retention-days: 30

fuzz-retry-after:
name: Fuzz Retry-After parser
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: go.mod
cache: true
# -fuzz takes one target, hence a separate job.
- name: Run fuzz for 10 minutes
id: fuzz
run: go test -fuzz=FuzzParseRetryAfter -fuzztime=10m ./retry/...
- name: Upload crash corpus
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fuzz-corpus-retry
path: retry/testdata/fuzz/
if-no-files-found: ignore
retention-days: 30
123 changes: 123 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,128 @@ All notable changes to **go-github-kit** are documented in this file.
The format is based on [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.7.0] - 2026-08-19

Compatibility and correctness release. Fixes an ETag defect that could
serve an empty response body, adds `NewE` for SDK constructors that return
an error, and updates all documentation and examples for go-github v87 and
later. No API breaks: every addition is a new symbol, and no exported
signature or type shape changed. One behavioural change needs action if you
supply your own `etag.Cache`; see Changed.

### Added

- `ghkit.NewE[T](factory func(*http.Client) (T, error), opts ...Option)`,
for SDK constructors that can fail. It does not restore type inference on
`github.NewClient`, which is variadic over its own options, so the
closure is still required; what it adds is somewhere for the
constructor's error to go. Factory errors are wrapped as
`"ghkit: factory: %w"`, so `errors.Is` still separates them from ghkit's
own config sentinels. `ghkit.New` is unchanged and still binds directly
to single-argument constructors such as `githubv4.NewClient`.
- `ghkit.WithETagTransport(func(*etag.Transport))`, which hands the caller
the constructed transport so `Stats()` can be polled for a `/healthz`
endpoint or a metrics gauge. Previously the transport was built inside
`HTTPClient` and unreachable, so the `Stats()` recipe the README and
`MIGRATION.md` describe had no supported entry point. The option enables
the ETag layer on its own, so it cannot silently never fire.
- `polling.WithFullJitter(frac)`, applying uniform jitter over
`[interval - span/2, interval + span/2]` so concurrent pollers
de-correlate. `polling.WithJitter` keeps its documented deterministic
offset; whichever of the two is applied last wins.
- `ErrNilClient` in `pages`, `polling` and `search`, plus
`search.ErrEmptyQuery`. These conditions previously returned inline
`errors.New` values that no caller could match with `errors.Is`. Message
strings are unchanged.
- `etag.KindBypassEmptyBody`, the `Event` kind for the empty-body bypass
described under Fixed. It counts toward `Stats().TotalBypasses` like the
other bypass kinds.
- `ghkit.ErrETagTransportType`, returned by `HTTPClient` if the constructed
ETag transport is not an `*etag.Transport`. Unreachable today; exported so
it is matchable rather than an opaque string if it ever fires.

### Fixed

- `etag`: HEAD requests now bypass the cache layer entirely. A response to
a HEAD carries no body (`net/http` sets `Body` to `http.NoBody`), so the
layer hashed an empty body against the server's ETag over the real one.
Every such request recorded a drift mismatch, and ten inside the 60s
window degraded the whole transport to passive mode. Once degraded, a
HEAD-stored entry made a later GET on the same URL receive a synthesised
200 carrying an empty body and `Content-Length: 0`. The mirror case
needed no degradation at all: a GET populated the entry and a subsequent
HEAD was handed a 200 with a body, violating HEAD semantics. HEAD
responses also keep their `ContentLength`, which the bounded read
previously zeroed. Two consequences follow: a HEAD no longer sends
`If-None-Match`, so it can no longer come back as a free 304, and it no
longer carries the `cond` cache-status header, so `cond.StatusOf` reports
`Updated` for every HEAD.
- `etag`: responses with an empty body now bypass the cache, and a cached
entry with an empty body is evicted and treated as a miss. Storing one was
pointless (replaying it hands the caller nothing) and validating one
recorded a false drift mismatch on every request, which is the HEAD defect
above in general form. Evicting rather than merely skipping matters
because a pre-1.7.0 persistent `Cache` can hold body-less HEAD entries
under the key a GET uses, and skipping alone would leave them permanently:
nothing overwrites them when the wire response is also empty.
GitHub serves an empty body this way only for the raw representation of an
empty file or blob, so a poller watching one now spends a rate-limit unit
per poll where it previously got a free 304.
- `retry`: guard against a nil `resp.Body` when draining a response before
a retry or a `Retry-After` abort. Reachable through a custom
`WithBaseTransport` whose `RoundTrip` returns a bare `&http.Response`,
where it panicked the caller's goroutine.

### Changed

- `go.mod` and `examples/go.mod` relax the Go directive from `1.26.5` to
`1.26` and add `toolchain go1.26.6`. The directive was a patch-level floor
imposed on every consumer and hard-failed builds under
`GOTOOLCHAIN=local`; the floor is now `1.26`, which lowers a requirement
rather than raising one. The `toolchain` line pins CI and local
development to 1.26.6, which carries the `net/http`, `net/url` and
`crypto/tls` fixes govulncheck requires. A `toolchain` directive in a
dependency is ignored, so it imposes nothing downstream.
- `etag`: the cache key now includes a digest of `Accept` and
`X-GitHub-Api-Version` when either is present, so requests differing only
by those headers no longer share an entry and evict each other. Requests
carrying neither header keep exactly the key they had before. The
in-process LRU is bounded and self-evicts; a consumer-supplied
`etag.Cache` with no TTL retains old-format entries indefinitely, so set
a TTL or flush the key prefix on upgrade. During a rolling deploy both
formats coexist, which costs hit rate but is safe.

### CI

- The `pages` and `polling` live probes now run in the live job; both were
previously present but never executed. The `pages` probe moved off
`/user/repos`, a user-to-server endpoint that 403s under the App
installation token CI supplies, onto a public commits endpoint.
- `FuzzParseRetryAfter` now gets coverage-guided fuzzing in its own job.
`-fuzz` takes a single target, so it cannot share the ETag job. The crash
corpus upload previously hardcoded `etag/testdata/fuzz/`, silently
discarding any `retry` crash.

### Documentation

- Quick start, recipes and all nine go-github examples updated for v87 and
later, which changed `NewClient` to
`(opts ...ClientOptionsFunc) (*Client, error)` and turned `UserAgent`,
`BaseURL` and `UploadURL` into read-only methods with no setters.
`examples/go.mod` moves from go-github v85 to v90.
- Removed an incorrect claim that `WithEnterpriseURLs` requires a trailing
slash and errors without one. go-github normalises it, and has in every
version this project has documented.
- The Prometheus recipe now builds on `ghkit.HTTPClient` instead of
hand-rolling `etag.NewTransport` plus `ratelimit.NewTransport`, which
silently omitted retry, throttle and oauth2.
- README badges: the Go Report Card badge is removed, the service is retired
and rendered "go report: retired". Replaced with per-workflow status
badges plus release, Go version and license badges.
- `ghtest` is documented as shipping four helpers rather than two;
`ETagServer` (1.5.0) and `LinkHeader` (1.4.0) shipped undocumented.
`examples/README.md` gained the missing `graphql-v4` row.

## [1.6.2] - 2026-07-30

Maintenance release. Bumps the Go toolchain to 1.26.5 to pick up the
Expand Down Expand Up @@ -683,6 +805,7 @@ and rotating PATs alike.
- `golang.org/x/oauth2` v0.36.0
- `golang.org/x/time` v0.15.0

[1.7.0]: https://github.com/pcanilho/go-github-kit/releases/tag/v1.7.0
[1.6.2]: https://github.com/pcanilho/go-github-kit/releases/tag/v1.6.2
[1.6.1]: https://github.com/pcanilho/go-github-kit/releases/tag/v1.6.1
[1.6.0]: https://github.com/pcanilho/go-github-kit/releases/tag/v1.6.0
Expand Down
23 changes: 16 additions & 7 deletions MIGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@ If your repo already has a hand-rolled stack of `oauth2.Transport`, `go-github-r

## What the swap changes

- **Drift safety hatch is automatic, not a flag**. If your repo today exposes a runtime flag like `--enable-precomputed-etag=false` to fall back to passive mode under drift, you can retire it. ghkit detects drift on every cacheable 200 and silently switches to passive mode after 10 mismatches in a 60-second window; after a 1-hour cooldown it probes back to precompute and recovers if the algorithm is working again. Wire `etag.WithEventCallback(...)` and filter on `KindDriftDetected`/`KindDriftRecovered` for transition alerts; poll `(*etag.Transport).Stats()` for live state on `/healthz`. As of v1.6.0, `Stats` also carries `TotalHits`/`TotalMisses`/`TotalStores`/`TotalBypasses` so a polling adapter can publish per-Outcome counters without scraping DEBUG-level slog records.
- **Drift safety hatch is automatic, not a flag**. If your repo today exposes a runtime flag like `--enable-precomputed-etag=false` to fall back to passive mode under drift, you can retire it. ghkit detects drift on every cacheable 200 and silently switches to passive mode after 10 mismatches in a 60-second window; after a 1-hour cooldown it probes back to precompute and recovers if the algorithm is working again. Wire `etag.WithEventCallback(...)` and filter on `KindDriftDetected`/`KindDriftRecovered` for transition alerts; obtain the transport with `ghkit.WithETagTransport(func(tr *etag.Transport) { ... })` and poll its `Stats()` for live state on `/healthz`. As of v1.6.0, `Stats` also carries `TotalHits`/`TotalMisses`/`TotalStores`/`TotalBypasses` so a polling adapter can publish per-Outcome counters without scraping DEBUG-level slog records.
- **Stricter cache-policy gate**. ghkit's etag transport refuses to cache responses carrying `Cache-Control: no-store` or `Vary: *` (RFC 9111 alignment). GitHub does not currently emit either, so the practical effect is zero, but the behavior is stricter than a typical in-tree port.
- **Log/metric label fidelity**. ghkit's etag transport emits a small built-in slog event set with a generic path-template allowlist. Your repo's bespoke route table (`/repos/{o}/{r}/commits/{sha}/branches-where-head` and similar) collapses to a coarse fallback label. If your dashboards key on per-route labels, keep emitting metrics from your own RoundTripper above ghkit's transport; do not rely on ghkit's slog event labels for app-specific path cardinality.
- **Upstream features ghkit does not curate**. The named `ratelimit.With*` options cover the common callbacks; for upstream features ghkit does not expose (the abort callback on `WithTotalSleepLimit`, custom limit providers, before-request hooks), use `ratelimit.WithUpstreamOptions(opts ...any)` to forward raw `gofri/go-github-ratelimit/v2` options.

## Recipe 1: Kubernetes operator with rotating PAT

Shape: a long-lived process holds one `*http.Client`. Each reconcile reads a fresh token from disk and clones a `*github.Client` on top of the shared transport with `(*github.Client).WithAuthToken(tok)`. The transport keeps the ETag cache and rate-limit bucket warm across reconciles.
Shape: a long-lived process holds one `*http.Client`. Each reconcile reads a fresh token from disk and builds a `*github.Client` on top of the shared transport, passing the token as a construction option. The SDK copies the `*http.Client` it is given, so the shared transport is not mutated. The transport keeps the ETag cache and rate-limit bucket warm across reconciles.

### Before

Expand Down Expand Up @@ -60,7 +60,7 @@ import (
hc, err := ghkit.HTTPClient(
ghkit.WithETagCache(
etag.WithCache(etag.NewLRUCache(flags.Controller.ETagCacheSize)),
etag.WithKeyScope(installationID),
etag.WithKeyScope(strconv.FormatInt(installationID, 10)),
),
ghkit.WithRateLimit(
ratelimit.WithPrimaryLimitDetected(func(c *ratelimit.PrimaryEvent) {
Expand All @@ -78,12 +78,17 @@ hc, err := ghkit.HTTPClient(
if err != nil { return err }

// Per reconcile -- new client, same transport:
gh := github.NewClient(hc).WithAuthToken(readGitHubToken())
gh, err := github.NewClient(
github.WithHTTPClient(hc),
github.WithAuthToken(readGitHubToken()),
)
```

What moves: the bespoke `etag_transport.go` and the manual `github_ratelimit.NewClient(...)` call collapse into options. Compression handling is implicit. The reconcile-loop call site is unchanged. `PrimaryEvent` / `SecondaryEvent` are type aliases of the upstream `gofri` types, so callback bodies do not need to change.

If you are pinned to `go-github/v84` and ghkit is on `v85`, keep your major: ghkit does not import `go-github` from `HTTPClient()` -- it returns `*http.Client`, which you hand to your own go-github version.
Keep whichever `go-github` major you are pinned to. ghkit does not import `go-github` at all, so no ghkit release can force you to bump it or rewrite import paths. `HTTPClient()` returns an `*http.Client` you hand to your own version.

Note that the "before" snippet above uses the pre-v87 `github.NewClient(hc).WithAuthToken(...)` API. From v87 the constructor is `NewClient(opts ...ClientOptionsFunc) (*Client, error)`, as shown in the "after" snippet.

## Recipe 2: Multi-installation webhook processor

Expand Down Expand Up @@ -139,7 +144,7 @@ func (p *Processor) getGitHubClient(ctx context.Context, installationID int64) (
ghkit.WithTimeout(5 * time.Second),
)
if err != nil { return nil, err }
return github.NewClient(hc), nil
return github.NewClient(github.WithHTTPClient(hc))
}
```

Expand Down Expand Up @@ -187,12 +192,16 @@ func NewClient(token string, opts ...ETagOptions) (*Client, error) {

```go
import (
"net/http"

ghkit "github.com/pcanilho/go-github-kit"
"github.com/pcanilho/go-github-kit/etag"
)

func NewClient(token string, etagSize int) (*github.Client, error) {
return ghkit.New(github.NewClient,
return ghkit.NewE(func(hc *http.Client) (*github.Client, error) {
return github.NewClient(github.WithHTTPClient(hc))
},
ghkit.WithToken(token),
ghkit.WithETagCache(etag.WithCache(etag.NewLRUCache(etagSize))),
ghkit.WithRequestsPerSecond(1.3, 1),
Expand Down
11 changes: 6 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ help:
@echo "Common targets:"
@echo " make test -- run the full test suite with -race"
@echo " make test-unit -- run short unit tests only"
@echo " make test-live -- run the live ETag drift probe (needs GITHUB_TOKEN)"
@echo " make test-fuzz -- fuzz the ETag hash for 30 seconds"
@echo " make test-live -- run the live probes (needs GITHUB_TOKEN)"
@echo " make test-fuzz -- fuzz the ETag hash and Retry-After parser"
@echo " make bench -- write benchmarks to dist/bench-current.txt"
@echo " make bench-update -- prompt to update docs/bench-baseline.txt manually"
@echo " make bench-update -- prompt to update the benchmark baseline"
@echo " make lint -- golangci-lint run"
@echo " make vuln -- govulncheck on the module"
@echo " make tidy -- go mod tidy with a diff gate"
Expand All @@ -23,17 +23,18 @@ test-unit:

test-live:
@[ -n "$$GITHUB_TOKEN" ] || { echo "GITHUB_TOKEN required"; exit 1; }
go test -tags=live -run TestETag_Live ./etag/...
go test -tags=live -run 'TestETag_Live|TestRetry_Live|TestPages_Live|TestPoll_Live' ./etag/... ./retry/... ./pages/... ./polling/...

test-fuzz:
go test -fuzz=FuzzETag_ComputeExpectedETag -fuzztime=30s ./etag/...
go test -fuzz=FuzzParseRetryAfter -fuzztime=30s ./retry/...

bench:
@mkdir -p dist
go test -bench=. -benchmem -run=^$$ ./... | tee dist/bench-current.txt

bench-update:
@echo "Review dist/bench-current.txt and copy manually to docs/bench-baseline.txt."
@echo "Review dist/bench-current.txt and copy manually to dist/bench-baseline.txt."

lint:
golangci-lint run
Expand Down
Loading
Loading