Skip to content

Add Google Gemini as an API provider (via Anthropic-compatible gateway) - #329

Open
zwanski-start wants to merge 1 commit into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-94-ken
Open

zwanski-start wants to merge 1 commit into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-94-ken

Conversation

@zwanski-start

Copy link
Copy Markdown

Closes #94

What changed

AutoForge's agents run on the Claude Agent SDK / Claude CLI, which speaks the Anthropic Messages API. Google doesn't offer an Anthropic-compatible endpoint for Gemini, so this PR adds Gemini through the existing API provider abstraction (API_PROVIDERS in registry.py → get_effective_sdk_env()). It targets an Anthropic-compatible gateway (LiteLLM) that translates /v1/messages calls to the Gemini API. GLM, Kimi, Ollama and Azure already plug in the same way.

  • registry.py: adds a new gemini provider called "Google Gemini (via LiteLLM)"
    • Default base URL is http://localhost:4000 (LiteLLM's default port)
    • Auth goes through ANTHROPIC_AUTH_TOKEN (the gateway master key, if one is set)
    • Models: gemini-2.5-pro (default) and gemini-2.5-flash
    • The existing logic clears ambient ANTHROPIC_API_KEY and Vertex vars, so Claude credentials don't leak to the gateway.
  • Settings UI (SettingsModal.tsx, views/SettingsView.tsx): Gemini shows an editable base URL with a http://localhost:4000 placeholder, a custom model name input, and a hint explaining the gateway and API key.
  • Docs: Gemini setup is documented in README.md, CLAUDE.md and .env.example (one-line litellm --model gemini/gemini-2.5-pro --alias gemini-2.5-pro --port 4000).
  • test_api_providers.py (new tests):
    • every provider preset is well-formed
    • the Gemini defaults map to the right SDK env vars
    • user overrides (base URL, token, model) take precedence

Verification

  • ruff check . passes
  • pytest test_api_providers.py test_client.py: 30 passed
  • python test_security.py: all tests pass
  • npm run lint: 0 errors (6 warnings that were already there)
  • npm run build succeeds
  • Ran the server on port 5345 with an isolated HOME, so real settings weren't touched:
    • /api/settings/providers lists gemini
    • In the Settings page, choosing "Google Gemini" shows the hint, the API key field, base URL http://localhost:4000, the two Gemini models and the custom model input
    • The selection saves as api_provider=gemini, api_model=gemini-2.5-pro

Not tested: a real agent session against Gemini (no Gemini key or LiteLLM gateway in this environment).

Assumptions / notes

  • I went with the gateway approach rather than a native Gemini client. Swapping out the Claude Agent SDK (tools, hooks, MCP, sandboxing, compaction) for a provider-neutral agent loop would be a much larger architectural change. This approach fits the existing provider design and needs no new dependencies.
  • The model IDs only have to match what the gateway serves. Any other Gemini model (or a LiteLLM model_list alias) can be typed into the custom model input.
  • Tool-use quality with Gemini depends on the model and the gateway's translation. The docs say this.
  • I couldn't push to AutoForgeAI/autoforge (403, read-only access), so this PR comes from the zwanski-start fork.

🤖 Generated with Claude Code

…AutoForgeAI#94)

Adds a 'gemini' preset to API_PROVIDERS that points the Claude Agent SDK
at an Anthropic-compatible gateway (LiteLLM, default localhost:4000)
which translates requests to the Gemini API. The settings UI exposes an
editable base URL and custom model name for it. Documents setup in
README, CLAUDE.md and .env.example, and adds provider env tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zwanski-start

Copy link
Copy Markdown
Author

🔍 QA report: ✅ Passed

Tester: Barbara (cubefarm agent, in a fresh session) · Round: 1 · Author: Ken

Adds a 'gemini' provider entry (LiteLLM gateway at http://localhost:4000, gemini-2.5-pro/flash) to the existing provider setup, plus Settings UI fields, docs and tests. On a live server with no API key, selecting Gemini saves the right settings and the agent would receive the gateway URL and Gemini model with Claude/Vertex credentials blanked. Custom base URL and model overrides saved from the UI, and the page shows no console errors. Lint, build and the new tests pass, and the remaining pytest and mypy failures also occur on master.

Check Details
✅ Issue requirement: Gemini selectable as a provider GET /api/settings/providers lists gemini with base_url http://localhost:4000 and models gemini-2.5-pro and gemini-2.5-flash. PATCH api_provider=gemini (after a custom provider with base URL https://x.example) returned 200 and saved api_base_url=http://localhost:4000, api_model=gemini-2.5-pro.
✅ Environment passed to the agent for Gemini (get_effective_sdk_env) With an isolated HOME: ANTHROPIC_BASE_URL=http://localhost:4000, all three model slots=gemini-2.5-pro, ANTHROPIC_API_KEY and the Vertex variables set to empty. With no gateway key, ANTHROPIC_AUTH_TOKEN is left out (same as Ollama).
✅ Settings UI for Gemini Clicked 'Google Gemini' in the Settings page: the gateway hint, API key field, base URL http://localhost:4000 with a Change button, both Gemini models and the custom model input all appear. Setting custom model gemini-2.5-flash-lite and base URL http://127.0.0.1:4001 saved both (checked via /api/settings). No console errors or warnings.
✅ Phone layout (390px) The Settings page is squeezed by the fixed sidebar, which is the same on master. The new hint text wraps narrowly and its link text runs slightly over the edge.
✅ Input validation PATCH api_provider=123 returns 422.
✅ Code review 7 files: provider entry in registry.py, UI show/hide conditions and placeholder in SettingsModal.tsx and SettingsView.tsx, docs, new tests. Follows the existing provider pattern. No debug code, no new dependencies, no change to security.py or the localhost-only middleware, no database schema change.
⏭️ UI e2e tests All 15 tests fail before starting because the Playwright browser isn't installed, and npx playwright install chromium timed out after 300 s. This is an environment problem, not the PR. The changed UI was checked by hand in the browser instead.
⏭️ Live agent run against Gemini No Gemini key or LiteLLM gateway available. Covered by code review and unit tests, as the QA rules require.
🧪 Commands run
Command Result
ruff check . All checks passed
mypy . 25 errors on the PR vs 24 on master. The only extra one is in venv/bin/activate_this.py (local venv folder, not PR code)
python -m pytest -q 153 passed, 21 failed. Master has the same 21 failures (test_devserver_security.py)
python -m pytest -q test_api_providers.py test_client.py 30 passed
python test_security.py ALL TESTS PASSED
python test_security_integration.py ALL INTEGRATION TESTS PASSED
npm --prefix ui run build success (tsc -b && vite build)
npm --prefix ui run lint 0 errors, 6 warnings (same as master)
npm --prefix ui run test:e2e 15 failed: Playwright browser not installed, environment problem
npx playwright install chromium timed out after 300s
HOME=<scratch> uvicorn server.main:app --port 5443 + curl /api/settings/providers, PATCH /api/settings 200 for valid updates, 422 for invalid api_provider; server stopped cleanly, port freed

No browser screenshots were taken in this round.

Posted by cubefarm · merges automatically once GitHub's checks pass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Support Google Gemini as an LLM backend

1 participant