feat(providers): add AWS Bedrock as a built-in provider with native SigV4 auth - #705
feat(providers): add AWS Bedrock as a built-in provider with native SigV4 auth#705cru-Luis-Rodriguez wants to merge 5 commits into
Conversation
|
Pushed a third commit (
Also dropped a New tests cover each: the two |
|
🔍 OpenCodeReview found 3 issue(s) in this PR.
[documentation · low] 📄
|
| func providerAcceptsAWSSettings(providerName string, entry *ProviderEntry) bool { | ||
| if preset, isPreset := llm.LookupProvider(providerName); isPreset { | ||
| return preset.AmbientAuth | ||
| } | ||
| return llm.NormalizeProtocol(entry.Protocol) == llm.ProtocolAnthropicBedrock | ||
| } |
There was a problem hiding this comment.
[bug · medium]
Bug: For preset providers, this function only checks the static preset.AmbientAuth flag and ignores any user-level protocol override in entry.Protocol. Since the resolver (see tryProviderConfig in resolver.go lines 363-369) allows entry.Protocol to override the preset's default protocol, a user could set protocol: openai on the bedrock preset and aws_region/aws_profile would still pass validation here — creating dead config that reads as applied but has no effect at runtime.
The fix should determine the effective protocol by checking entry.Protocol first (when non-empty) before falling back to the preset default, similar to how the resolver does it.
Suggestion:
| func providerAcceptsAWSSettings(providerName string, entry *ProviderEntry) bool { | |
| if preset, isPreset := llm.LookupProvider(providerName); isPreset { | |
| return preset.AmbientAuth | |
| } | |
| return llm.NormalizeProtocol(entry.Protocol) == llm.ProtocolAnthropicBedrock | |
| } | |
| func providerAcceptsAWSSettings(providerName string, entry *ProviderEntry) bool { | |
| if preset, isPreset := llm.LookupProvider(providerName); isPreset { | |
| // Honor user-level protocol override: if the entry explicitly sets a | |
| // non-Bedrock protocol, AWS settings are meaningless even though the | |
| // preset defaults to ambient auth. | |
| if entry.Protocol != "" { | |
| return llm.NormalizeProtocol(entry.Protocol) == llm.ProtocolAnthropicBedrock | |
| } | |
| return preset.AmbientAuth | |
| } | |
| return llm.NormalizeProtocol(entry.Protocol) == llm.ProtocolAnthropicBedrock | |
| } |
There was a problem hiding this comment.
Confirmed and fixed in 68d250c — thank you, this was a real hole and the diagnosis was exact.
I took it a step further than the config layer, because the same preset-only assumption existed on the resolver side and was the more serious half. tryProviderConfig skipped the api_key requirement on isPreset && preset.AmbientAuth, evaluated before the entry's protocol override was read. So providers.bedrock.protocol = openai resolved with no key and no URL (the bedrock preset has no BaseURL), producing an endpoint that spoke a token protocol while carrying nothing to authenticate with — reported as configured, failing only at request time.
Ambient auth is now derived once, after the override is applied:
ambientAuth := protocol == ProtocolAnthropicBedrock ||
(isPreset && preset.AmbientAuth && entry.Protocol == "")That value gates the api_key requirement, the resolved endpoint's AmbientAuth, and the --model allowlist relaxation, which had the same assumption. It also makes the reverse case coherent: an entry that explicitly selects anthropic-bedrock signs its requests whatever preset it sits under, so it no longer demands a key it would ignore.
providerAcceptsAWSSettings follows the shape you suggested — the entry's protocol decides whenever it sets one, with the preset's flag answering only when the entry is silent — so the two layers now agree on what "ambient" means. Tests added for both: an overridden entry that must require a key again, and aws_region rejected on that entry then accepted once the protocol is set back.
| case strings.Contains(msg, "ValidationException"), | ||
| strings.Contains(msg, "model identifier is invalid"), | ||
| strings.Contains(msg, "don't have access to the model"): |
There was a problem hiding this comment.
[bug · medium]
Matching on "ValidationException" alone is too broad. AWS Bedrock returns ValidationException for many reasons unrelated to model IDs — e.g., malformed request body, invalid parameter values, content exceeding size limits, or missing required fields. When any of those occur, the user will see guidance to run aws bedrock list-inference-profiles, which is misleading and won't fix the actual problem.
Consider narrowing this match to only catch model-related validation errors, for example by requiring both "ValidationException" and a model-related keyword, or by removing "ValidationException" from this case entirely and relying on the more specific "model identifier is invalid" and "don't have access to the model" patterns.
There was a problem hiding this comment.
Agreed, and already fixed — this review ran against fdd1ad2, one commit behind. 0da3406 narrowed exactly this, for the same reason you give:
- The bare
ValidationExceptionarm is gone. Onlymodel identifier is invalid(and an inference-profile-not-found shape) map to the list-inference-profiles guidance; a request-shape rejection such asInput is too long for requested modelnow falls through to the generic wrapper, which preserves the service's own message. - The credential arm no longer matches a bare
expired, which had been claimingx509: certificate has expiredwas an expired SSO session. - Separately,
AccessDeniedExceptionhad two meanings collapsed into one message.You don't have access to the model with the specified model IDneeds model access enabled per account and per region in the console, which no IAM policy provides, so it is now matched ahead of the genericAccessDeniedcode and says so.
Tests cover each with the verbatim service wording rather than paraphrases.
|
Addressed all three findings from the OCR run above in 68d250c, with replies in the two inline threads. The third one could not be posted inline ("Line could not be resolved"), so for the record: it was right. Worth noting on the second finding ( Suite, |
|
CI / test (pull_request) |
|
Thanks for the ping — fixed in 273357a. The failure was the
On why the core module jumps so many minors: there's no supported way to move Verified locally against the CI configuration (go1.26.5): One note for a separate PR: govulncheck still reports |
|
@cru-Luis-Rodriguez CI is failing due to missing SPDX license headers on the two new test files: The merge conflict has been resolved on our side. Could you rebase on main and run |
Bedrock serves the same Messages API as api.anthropic.com, so this reuses
AnthropicClient wholesale and lets the official SDK's bedrock middleware
handle what differs: SigV4 signing, moving the model from the body into the
URL path, injecting anthropic_version, and deriving the host from the region.
No new protocol implementation, no AWS request plumbing.
Configuration is an empty provider entry — there is no api_key to set:
{
"provider": "bedrock",
"model": "us.anthropic.claude-sonnet-4-6",
"providers": { "bedrock": { "aws_profile": "...", "aws_region": "..." } }
}
aws_profile and aws_region are optional; without them the standard AWS chain
decides, as with any other AWS tool. Setting them makes a run reproducible
without exporting AWS_PROFILE first. Model accepts a foundation model ID, an
inference profile ID, or an application inference profile ARN when usage has
to be attributed for cost allocation.
Four things this needed beyond registering a provider, each found by running
it rather than reading it:
- The resolver required a non-empty api_key, and separately required both
URL and Token to consider an endpoint complete. Bedrock has none of the
three, so a correct config fell through every strategy and reported "no
valid LLM endpoint configured" — the error for having configured nothing.
Both gates now recognise ambient authentication, via an AmbientAuth flag
on Provider and ResolvedEndpoint. Providers that do use api_key are
unaffected, which TestNonAmbientProviderStillRequiresAPIKey pins.
- bedrock.WithConfig prefers bearer auth over SigV4 whenever
cfg.BearerAuthTokenProvider is non-nil, and LoadDefaultConfig populates
that provider from the SSO token cache. An SSO-authenticated caller —
most enterprise setups — therefore sent its OIDC access token and got
403 "Invalid API Key format: Must start with pre-defined prefix". The
provider is cleared unless AWS_BEARER_TOKEN_BEDROCK was set deliberately,
which restores SigV4 while leaving an explicit bearer token working.
- The SDK would also attach an API-key header of its own, which Bedrock
rejects even when empty. Authorization and X-Api-Key are removed before
the signing middleware runs.
- bedrock.WithLoadDefaultConfig panics when AWS config cannot be loaded.
A CLI should not answer an expired session with a stack trace, so the
config is loaded directly and the failure deferred to the first request
as a sentence naming the likely fix.
The preset's Models list is taken verbatim from `aws bedrock list-inference-profiles`
on a live account rather than inferred: suffix conventions vary per family, so
us.anthropic.claude-sonnet-5 is correct while us.anthropic.claude-sonnet-5-v1:0 is
rejected with 400 "The provided model identifier is invalid." The global.* cross-region
variants are listed alongside us.* since either is a valid routing target. That list
only gates --model overrides; an application inference profile ARN still works via the
model field.
Two existing tests needed updating: the provider-order list gains "bedrock",
and TestProviders_AllProtocolsCanonical now delegates to ValidateProtocol
instead of re-listing the canonical names, so the next protocol added cannot
silently leave it behind.
Verified end-to-end against a live Bedrock account: reviews complete and
return findings using SigV4 credentials from an SSO profile, with no AWS
variables in the environment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e CLI
Registering the provider was not enough to make it usable: every config-related
path still assumed an api_key, and Bedrock's own error wording sends users after
the wrong problem.
- ProviderEntry gains aws_profile and aws_region. They were readable by the
resolver but absent from the struct the CLI marshals, and config is
unmarshalled into it and written back on every config command — so a
hand-written aws_region was silently deleted the first time the user ran
`ocr config model`, with no error and nothing to suggest why the next review
reached a different region.
- `ocr config set providers.<name>.aws_region|aws_profile` now works, for both
the providers and custom_providers paths. Values are trimmed; whitespace
inside one is rejected. Region names are deliberately not validated against
a fixed list — AWS adds regions faster than an embedded list stays correct,
and a wrong region already fails at request time. Setting either field on a
provider that authenticates by api_key is an error rather than dead config
that reads as applied.
- The provider wizard treats the model step as final for an ambient provider
instead of demanding a key. An API-key prompt that has to be left blank reads
as a step the user failed to complete, and applyOfficialProviderConfig
rejected the empty value anyway, so bedrock was unreachable through
`ocr config provider` entirely. The gate is now a named check keyed off
AmbientAuth, so key-based providers keep the requirement.
- `ocr llm test` prints the resolved region and profile in place of the URL,
which is empty for bedrock because the region decides the host. A request
that reached the wrong region otherwise fails as though the model ID were
malformed.
- Bedrock rejections are translated into the action that fixes them, since two
of them are actively misleading as the service words them: "Invalid API Key
format" names a credential no bedrock user can configure (it means a bearer
token reached the request), and a model merely absent from the region comes
back as "The provided model identifier is invalid." Expired credentials point
at `aws sso login` with the profile filled in; AccessDenied is named as an
IAM gap on bedrock:InvokeModel rather than a bad credential; a rejected model
points at `aws bedrock list-inference-profiles` and the -v1:0 suffix trap.
Every other protocol shares this client type, so the translation is gated on
the bedrock flag and returns other errors untouched.
The unknown-config-key message is pinned byte-for-byte by an existing test; it is
updated for the two new provider fields and for anthropic-bedrock as a protocol
value.
Verified against a live Bedrock account: `ocr llm test` reports region and
profile and completes over SigV4; a -v1:0 model ID and an unresolvable profile
each produce their intended message rather than a bare 400 or 403.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…l gating
Four defects found reviewing the two commits before this one. Each was verified
by execution or against SDK source, not inferred.
- AWS_BEARER_TOKEN_BEDROCK was unreachable for exactly the users it was meant
to serve. The provider was cleared only when the variable was unset, on the
strength of WithConfig's doc comment ("if the AWS_BEARER_TOKEN_BEDROCK
environment variable is set, it is used"). The code disagrees with that
comment: bedrock.go consults the variable only `if
cfg.BearerAuthTokenProvider == nil`. So an SSO profile plus a deliberately
configured Bedrock API key sent the SSO OIDC token instead of the key — the
same silent substitution this patch exists to prevent, and explainError then
blamed a token that never left the machine. Cleared unconditionally now,
which is what gives the variable the precedence it documents.
- A model that the account has not enabled was reported as an IAM problem.
Bedrock answers both authorization failures with AccessDeniedException, and
the fixes have nothing in common: "You don't have access to the model with
the specified model ID" needs model access granted in the console, per
account and per region, which no IAM policy provides. The specific wording
is now matched ahead of the generic code, and the clause for it is no longer
stranded in an unreachable branch.
- A bare ValidationException match claimed every request-shape rejection was a
model-ID problem: "Input is too long for requested model" sent the user off
to list inference profiles. Only the model-identifier wording is matched now;
everything else keeps the service's own message, which is the whole point of
the function. The credential-expiry arm likewise no longer matches a bare
"expired", which caught `x509: certificate has expired`.
- --model rejected any Bedrock identifier absent from the preset's Models list,
contradicting both the preset's own comment and this PR's description. A
preset list cannot be an allowlist here: identifiers are scoped to an account
and a region, and an application inference profile ARN — the value to use
when spend has to be attributed — can never appear in a list compiled
upstream. The list stays a picker for `ocr config model`; it no longer gates
an override for an ambient-auth provider. Key-based providers keep the
check, so a typo against a hosted API is still caught locally.
Also: dropped a cfg.URL normalization block that could not have any effect,
since WithConfig is appended last and installs its own base URL — the comment
claimed a purpose the code did not have. Pinned AWS_CONFIG_FILE in the test that
constructs a client, which was reading the developer's real ~/.aws/config. Fixed
the column alignment of the region line in `ocr llm test`.
Verified: `ocr llm test` still completes over SigV4 against a live account; an
identifier the preset does not list now reaches Bedrock and returns Bedrock's own
verdict rather than a local rejection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OCR's own review of this PR found that ambient auth was read off the preset while the protocol could be overridden per entry, which left two ways to configure something that reads as applied and cannot work. `providers.bedrock.protocol = openai` resolved with no api_key and no URL: the key requirement was skipped because the preset declares AmbientAuth, but the endpoint then spoke a protocol with no SigV4 signing and carried nothing to authenticate with. Ambient auth is now derived from the protocol actually in force, after the override is applied, so such an entry needs a token again — and conversely an entry that selects the bedrock protocol explicitly signs its requests whatever preset it sits under. The same value gates the --model allowlist, which had the same preset-only assumption. `ocr config set providers.bedrock.aws_region` accepted AWS settings on that same overridden entry. The check now lets the entry's protocol decide whenever it sets one, falling back to the preset's flag only when the entry is silent. Also corrects two stale doc comments the review flagged: ValidateProtocol accepts four protocol names, not three, and the package comment now lists anthropic-bedrock among the supported protocols. The third finding in that review — a bare ValidationException match in explainError — was already fixed in the preceding commit; the bot reviewed the commit before it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
govulncheck fails the CI test job because the pinned AWS SDK tree pulls in github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.3, which is affected by GO-2026-5764 (fixed in v1.7.8). Upgrading the direct dependency aws-sdk-go-v2/config to current resolves eventstream to v1.7.16, past the fixed version. The diff is scoped to the AWS module tree (plus smithy-go, its runtime companion); no other dependencies move. The Bedrock provider's behavior is unchanged: the newer config module still populates BearerAuthTokenProvider from the SSO token cache, so the unconditional clearing in NewAnthropicBedrockClient remains necessary and correct, and it still does not consult AWS_BEARER_TOKEN_BEDROCK itself, so the anthropic-sdk-go re-read of that variable keeps its documented precedence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
e421156 to
48b9f40
Compare
Closes #659. Adds
bedrockas a built-in provider with native SigV4 auth, taking option (1) direct dependency as you decided in that issue.Two commits: the provider itself, then the config/CLI surface it turned out to need. The second one exists because registering a provider was not sufficient to make it configurable — details under Q1–Q3 below.
Sample config
{ "provider": "bedrock", "model": "us.anthropic.claude-sonnet-4-6", "providers": { "bedrock": { "aws_region": "us-west-2", "aws_profile": "example-profile" } } }Both AWS fields are optional; without them the standard credential chain decides, as with any other AWS tool. There is no
api_keyline, and none is accepted as a substitute for a signature.Answers to the five questions from #659
1. Provider entry shape.
api_keyis optional and ignored, not forbidden. Rejecting a stray key would punish someone who pasted one out of habit, and it cannot leak into a request: the client deletesAuthorizationandX-Api-Keybefore the signing middleware runs. The resolver's gate is nowapiKey == "" && !(isPreset && preset.AmbientAuth), so every key-based provider keeps its requirement —TestNonAmbientProviderStillRequiresAPIKeypins that.aws_regionandaws_profileare real fields on the provider entry, JSON-serializable underproviders.bedrock, not read purely from the env chain. Pinning them is what makes a run reproducible without exportingAWS_PROFILEfirst, which matters most on CI runners that carry a different default.While wiring that up I found a bug worth fixing here rather than later: the fields were readable by the resolver but absent from the
ProviderEntrystruct the CLI marshals. Config is unmarshalled into that struct and written back on every config command, so a hand-writtenaws_regionwas silently deleted the first time the user ranocr config modelorocr config set— no error, and nothing to explain why the next review reached a different region.TestConfigRoundTripKeepsAWSSettingsis the regression test.2. Interactive wizard. Keyed off
AmbientAuth, never off a provider name. For an ambient provider the model step is the last one — the wizard confirms there instead of advancing to an API-key prompt that has to be left blank, which reads as a step the user failed to complete. Before this,bedrockwas unreachable throughocr config providerentirely:applyOfficialProviderConfigrejected the empty key. That gate is now a namedcheckAPIKeyRequirementso the ambient case is explicit and testable, andapiKeyStepCanConfirmalso accepts an empty value for an ambient provider, which is the state reachable when an existing config is edited.Yes, it still runs the connection test at the end — that is exactly where an expired SSO session surfaces, and the cheapest moment to catch it.
What is not in this PR: collecting region and profile as two optional inputs in that step. It needs a real design decision about the wizard's shape (placeholders showing what the AWS chain currently resolves, so blank reads as "inherit
us-west-2" rather than "unset"), and I would rather agree on it with you than guess. Until then the wizard configures bedrock and leaves both to the AWS chain, andocr config setcovers pinning them. Happy to add it as a follow-up commit here.3. Non-interactive config. Yes, both need to be settable, and they now are:
One
applyProviderFieldcase coversproviders.*andcustom_providers.*, sincesetProviderValueandsetCustomProviderFieldboth funnel there. Values are trimmed; whitespace inside one is rejected. Region names are not validated against a fixed list — AWS adds regions faster than an embedded list stays correct, and a wrong region already fails at request time with a clear message (see Q5). Setting either field on a provider that authenticates by api_key is an error rather than stored dead config, which catchesproviders.anthropic.aws_regiontypos; for a custom provider it is accepted once the entry declares the bedrock protocol.The unknown-config-key message is pinned byte-for-byte by an existing test, so it is updated for the two new fields and for
anthropic-bedrockas a protocol value.4. Model list semantics. The preset ships a
Modelslist, but it should be read as a starting point, not a closed set, and free-form entry has to keep working — the existing custom-model path in the model picker already provides it. Three reasons Bedrock cannot be validated like a hosted API:us.anthropic.claude-sonnet-5is correct whileus.anthropic.claude-sonnet-5-v1:0is rejected outright. The listed IDs are taken verbatim fromaws bedrock list-inference-profileson a live account rather than inferred, andglobal.*variants are listed besideus.*since either is a valid routing target.So an ID missing from the preset must not be rejected. That is now implemented rather than merely argued:
preset.Modelsstill gates a--modeloverride for key-based providers — a typo against a hosted API is worth catching locally — but not for an ambient-auth preset, where the list is a picker forocr config modeland nothing more. Before that change,--modelwith an application inference profile ARN failed locally with "is not available for provider", which contradicted this very section.5. Connection-test messaging. Bedrock's own wording sends people after the wrong problem, so failures are classified before falling through to a generic error. Real output from this branch:
The URL line is replaced by region and profile, because bedrock has no configured URL — the region decides the host, and a request that reached the wrong one otherwise fails as though the model ID were malformed.
aws sso login --profile NAME"ExpiredToken, refresh failure)aws sso loginsuggestionAccessDeniedException: You don't have access to the model…AccessDeniedExceptionnamingbedrock:InvokeModelaws bedrock list-inference-profiles --region <r>and calls out the-v1:0suffix trapValidationExceptiononmax_tokens, a reset, a throttle)Invalid API Key formatapi_keyapplies to bedrock and that a bearer token reached the request; ifAWS_BEARER_TOKEN_BEDROCKis set, names that variable specificallyTwo notes on that last row. It is checked first, because it occurs with otherwise-valid credentials and a later "expired"/"denied" branch would mislabel it. And the SDK reports the pre-middleware
/v1/messagespath in its error text, which makes correct path rewriting look broken — the surrounding message now says which region and profile were actually used, so the URL is not the only context available.Every other protocol shares this client type, so the translation is gated on the bedrock flag and returns other errors untouched (
TestExplainErrorLeavesNonBedrockErrorsAlone).On the SDK bug
Per your note, the workaround stays in this patch:
bedrock.WithConfigprefers bearer auth whenevercfg.BearerAuthTokenProvideris non-nil, andLoadDefaultConfigpopulates it from the SSO token cache — so an SSO-authenticated caller silently sends its OIDC access token and gets403 Invalid API Key format. The provider is therefore cleared beforeWithConfigruns.There is a second, smaller SDK problem worth knowing about, which I hit while reviewing my own patch.
WithConfig's doc comment says the environment variable takes precedence:The code does the opposite — it consults the variable only
if cfg.BearerAuthTokenProvider == nil. My first version of this patch trusted the comment and cleared the provider only when the variable was unset, which meant an SSO user who deliberately set a Bedrock API key still sent the SSO token. Clearing unconditionally is what actually delivers the documented precedence, so that is what this does. Glad to open theanthropic-sdk-goissue for both points separately; say the word if you would rather this patch shrink once they land.bedrock.WithLoadDefaultConfigalso panics when AWS config cannot be loaded, so the config is loaded directly and the failure deferred to the first request as a sentence naming the likely fix — a CLI should not answer an expired session with a stack trace.Verification
go vetpass;gofmtclean.config setaccept/reject/trim cases, the wizard skipping the key step, and each error classification.-v1:0model ID and an unresolvable profile each produce their intended message rather than a bare 400 or 403.Rebased on current
main, so it carries thefinalizeResolvedEndpointrefactor and the per-run provider/model overrides from #687 — the resolver change is down to one completeness condition.