Skip to content

Generalize model backends and default to Gemini 3.5 Flash#102

Open
eshulman2 wants to merge 3 commits into
forge-sdlc:mainfrom
eshulman2:generalize-llm-backend-default-gemini
Open

Generalize model backends and default to Gemini 3.5 Flash#102
eshulman2 wants to merge 3 commits into
forge-sdlc:mainfrom
eshulman2:generalize-llm-backend-default-gemini

Conversation

@eshulman2

@eshulman2 eshulman2 commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Make Forge provider-neutral at the product layer and use Gemini 3.5 Flash on Vertex AI as the default model backend.

Changes

  • Add explicit LLM_BACKEND selection: vertex-ai, google-genai, or anthropic
  • Default new configurations to vertex-ai with gemini-3.5-flash
  • Use provider-native credentials:
    • GOOGLE_CLOUD_PROJECT / GOOGLE_CLOUD_LOCATION plus ADC for Vertex AI
    • GOOGLE_API_KEY for the Gemini API
    • ANTHROPIC_API_KEY for the Anthropic API
  • Keep existing LLM_API_KEY, VERTEX_*, and ANTHROPIC_VERTEX_* aliases working
  • Infer legacy backends for existing Anthropic, Gemini API, and Vertex deployments that do not set LLM_BACKEND
  • Route orchestrator and container model construction through the same explicit backend/model contract
  • Reject incompatible backend/model combinations with actionable configuration errors
  • Replace product-facing Claude-specific language with Deep Agents/LangChain terminology
  • Update setup and configuration documentation

Default configuration

LLM_BACKEND=vertex-ai
GOOGLE_CLOUD_PROJECT=your-gcp-project
GOOGLE_CLOUD_LOCATION=global
LLM_MODEL=gemini-3.5-flash

Vertex AI uses Application Default Credentials.

Validation

  • uv run ruff check src/forge containers/entrypoint.py
  • uv run ruff format --check src/ containers/entrypoint.py
  • Backend/configuration/model-factory/container-environment matrix: 23 tests passed
  • Python compilation check for configuration, agent factory, and container entrypoint

Compatibility

Existing deployments using only ANTHROPIC_API_KEY continue on the Anthropic backend with the prior direct-compatible model ID. Existing legacy Vertex project variables continue to select Vertex AI. GOOGLE_API_KEY without an explicit backend selects the Gemini API. New deployments default to Gemini 3.5 Flash on Vertex AI.

@eshulman2 eshulman2 force-pushed the generalize-llm-backend-default-gemini branch from 71c15ba to 1f70e48 Compare June 25, 2026 12:19
@eshulman2 eshulman2 force-pushed the generalize-llm-backend-default-gemini branch from faffad0 to 16f5912 Compare July 13, 2026 08:09
@eshulman2 eshulman2 changed the title Generalize LLM backend configuration Generalize model backends and default to Gemini 3.5 Flash Jul 13, 2026
@eshulman2 eshulman2 force-pushed the generalize-llm-backend-default-gemini branch 3 times, most recently from e9624a4 to 2fa773b Compare July 13, 2026 11:24
@eshulman2 eshulman2 force-pushed the generalize-llm-backend-default-gemini branch from 2fa773b to ab578b4 Compare July 13, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant