diff --git a/AGENTS.md b/AGENTS.md index c810f20..f4dad59 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,35 +1,51 @@ # Repository Guidelines -Python SDK for Inference Gateway — a thin, hand-written synchronous client over auto-generated Pydantic models for the gateway's OpenAI- and Anthropic-compatible APIs. Distributed on PyPI as `inference-gateway`. Requires Python 3.12+. +Python SDK for Inference Gateway - a thin, hand-written synchronous client over auto-generated Pydantic models for the gateway's OpenAI- and Anthropic-compatible APIs. Distributed on PyPI as `inference-gateway`. Requires Python 3.12+ (CI runs 3.13). ## Commands All development runs through `task` (see `Taskfile.yml`): -- `task install` — `pip install -e ".[dev]"` -- `task format` — Black + isort on `inference_gateway/`, `tests/`, `examples/` -- `task lint` — Black `--check`, isort `--check-only`, mypy. Stricter than CI, which only runs Black and pytest. -- `task test` — pytest suite; `task test:coverage` adds terminal + HTML coverage -- `task generate` — fetches/validates the OpenAPI spec, regenerates `inference_gateway/models.py` -- `task build` — clean + lint + test, then `python -m build` -- `task precommit:install` — point git at `.githooks/` (a plain shell script, not the pre-commit framework) +- `task install` - `pip install -e ".[dev]"` +- `task format` - Black + isort on `inference_gateway/`, `tests/`, `examples/` +- `task lint` - Black `--check`, isort `--check-only`, mypy on `inference_gateway/` and `examples/`. Stricter than CI (`.github/workflows/ci.yml`), which only runs `black --check .` and `pytest tests/` - no mypy or isort. +- `task test` - pytest suite; `task test:coverage` adds terminal + HTML coverage +- `task generate` - downloads (`oas-download`) and validates the OpenAPI spec, then regenerates `inference_gateway/models.py` +- `task build` - clean + lint + test, then `python -m build` +- `task dev:setup` - full bootstrap: install, download spec, generate, install the pre-commit hook, test +- `task precommit:install` - point git at `.githooks/` (`git config core.hooksPath .githooks`); `task precommit:run` runs the hook directly. The hook is a plain shell script, not the pre-commit framework: it runs `task format` on staged Python files and blocks the commit if anything was reformatted. Focused test: `pytest tests/test_client.py::test_list_models -v` ## Layout -- `inference_gateway/client.py` — hand-written `InferenceGatewayClient` with two synchronous backends: `requests.Session` (default) and `httpx.Client` (`use_httpx=True`). Keep both paths behaviorally aligned in `_make_request` and `_process_stream_response`. -- `inference_gateway/models.py` — **auto-generated, do not edit by hand**. Update the spec upstream in `inference-gateway/schemas` (or local `openapi.yaml`/`templates/`), then `task generate`; pin the spec with `SCHEMAS_REF=refs/tags/vX.Y.Z`. New public models also need `inference_gateway/__init__.py` imports and `__all__` entries. -- `tests/` — client behavior; `examples/` — runnable usage examples (chat, list, mcp, messages, tools). +- `inference_gateway/client.py` - hand-written `InferenceGatewayClient` with two synchronous backends: `requests.Session` (default) and `httpx.Client` (`use_httpx=True`). Keep both paths behaviorally aligned in `_make_request` and `_process_stream_response`. Streaming has separate code paths per backend: httpx needs a `with self.client.stream(...)` context manager, requests uses the `stream=True` kwarg. +- Public errors: `InferenceGatewayError` (base) -> `InferenceGatewayAPIError` (carries `status_code` + `response_data`) and `InferenceGatewayValidationError` (Pydantic validation failures). +- `inference_gateway/models.py` - **auto-generated by `datamodel-codegen`, do not edit by hand**. Update the spec upstream in `inference-gateway/schemas` (or local `openapi.yaml`, or the Jinja templates `templates/header.jinja2` / `templates/pydantic/BaseModel.jinja2`), then `task generate`; pin the spec with `SCHEMAS_REF=refs/tags/vX.Y.Z`. New public models also need `inference_gateway/__init__.py` imports and `__all__` entries; new endpoints need a `client.py` method. +- `tests/` - client behavior; `examples/` - runnable usage examples (chat, list, mcp, messages, tools). + +## Provider-specific metadata + +Some providers (notably Google Gemini reasoning models) attach opaque per-call metadata like `thought_signature` that must round-trip verbatim on follow-up requests. It is preserved when the caller appends the returned `message` object to the conversation; when rebuilding from dicts, populate `ToolCallExtraContent` / `Google` explicitly. See README "Provider-Specific Tool-Call Metadata". ## Style Black + isort at 100 columns; 4-space indents, LF line endings; YAML/JSON at 2 spaces. mypy with `disallow_untyped_defs = true` and the pydantic plugin. `snake_case` for functions/tests, `PascalCase` for classes and models. +### Code Readability + +- Write self-explanatory code: clear names and small, single-purpose functions carry the intent. + If a block needs a comment to be understood, extract it into a well-named function or variable. +- No inline comments inside function bodies. +- Doc comments on functions and types are at most 5 lines: what it does and why, not how. +- No comments above modules, packages, or files. +- Tool directives are not comments and stay where the tool needs them (lint suppressions, build + tags, compiler pragmas, code generation markers). + ## Testing -pytest + `unittest.mock` only — never make live network calls. Existing tests patch `requests.Session.request` and assert exact URL, params, JSON body, and timeout; mirror that for new client methods. Streaming yields raw `SSEvent` objects — callers parse `chunk.data` themselves. Never commit real API tokens; examples take configuration from env vars. +pytest + `unittest.mock` only - never make live network calls or spin up servers. Existing tests patch `requests.Session.request` and assert exact URL, params, JSON body, and timeout; mirror that for new client methods. Streaming (`create_chat_completion_stream`) yields raw `SSEvent` objects with `data: [DONE]` sentinels filtered out - callers `json.loads(chunk.data)` and validate against `CreateChatCompletionStreamResponse` themselves. Never commit real API tokens; examples take configuration from env vars. ## Commits & Releases -Conventional Commits drive semantic-release versioning (`.releaserc.yaml`): `feat:` → minor; `fix:`/`perf:`/`refactor:`/`docs:`/`ci:`/`chore:`/`test:`/`build:` → patch. PRs should note tests run and flag generated-code updates or API compatibility concerns. +Conventional Commits drive semantic-release versioning (`.releaserc.yaml`): `feat:` -> minor; `fix:`/`perf:`/`refactor:`/`impr:`/`docs:`/`ci:`/`chore:`/`style:`/`test:`/`build:` -> patch. Releases are triggered manually via the `Release` GitHub Action: it bumps `pyproject.toml`, updates `CHANGELOG.md`, tags `v${version}`, creates a GitHub release, and publishes to PyPI. PRs should note tests run and flag generated-code updates or API compatibility concerns. diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 09e7363..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,73 +0,0 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Overview - -Python SDK for the Inference Gateway — a unified client over multiple LLM providers (OpenAI, Anthropic, Google, Groq, Ollama, etc.). Distributed on PyPI as `inference-gateway`. Requires Python 3.12+ (target 3.13). - -The SDK is intentionally thin: a hand-written client (`inference_gateway/client.py`) wrapping auto-generated Pydantic models (`inference_gateway/models.py`). - -## Common commands - -Tasks are orchestrated with [`task`](https://taskfile.dev) (`Taskfile.yml`): - -- `task install` — install dev dependencies (`pip install -e ".[dev]"`) -- `task test` — run pytest suite -- `task test:coverage` — pytest with coverage report (term + html) -- `task lint` — black --check, isort --check-only, and mypy on `inference_gateway/` and `examples/` -- `task format` — apply black + isort -- `task generate` — regenerate `inference_gateway/models.py` from `openapi.yaml` (depends on `oas-download` → fetches the latest spec from `inference-gateway/schemas`) -- `task build` — runs `clean` + `lint` + `test`, then `python -m build` -- `task dev:setup` — full environment bootstrap (install, download OAS, generate, install pre-commit hook, test) -- `task precommit:install` — point git at the `.githooks/` pre-commit hook (`git config core.hooksPath .githooks`) -- `task precommit:run` — run `.githooks/pre-commit` directly - -Run a single test: `pytest tests/test_client.py::test_list_models -v` - -Note: CI (`.github/workflows/ci.yml`) only runs `black --check .` and `pytest tests/` — it does NOT run mypy or isort. `task lint` is stricter than CI. The `.githooks/pre-commit` hook (a plain shell script, no pip `pre-commit` framework) runs `task format` on staged Python files at commit time and blocks the commit if anything was reformatted — install it once with `task precommit:install`. - -## Architecture - -### Code generation pipeline - -`inference_gateway/models.py` is **auto-generated** by `datamodel-codegen` from `openapi.yaml` (synced from the [`inference-gateway/schemas`](https://github.com/inference-gateway/schemas) repo). Never edit it by hand — changes will be overwritten on the next `task generate`. - -The generation uses custom Jinja templates in `templates/` (notably `templates/header.jinja2` for the file header and `templates/pydantic/BaseModel.jinja2` for model rendering). When a new model is needed, the workflow is: - -1. Update the spec upstream in `inference-gateway/schemas` -2. Run `task generate` here (downloads latest spec, validates, regenerates) -3. If the new model needs to be public, add it to `inference_gateway/__init__.py`'s imports and `__all__` -4. Update `client.py` if a new endpoint is needed - -### Client layer - -`InferenceGatewayClient` (`inference_gateway/client.py`) supports two HTTP backends selected via `use_httpx`: - -- `requests.Session` (default) — synchronous -- `httpx.Client` — also synchronous but configurable - -Both code paths must be kept in sync; `_make_request` and `_process_stream_response` handle the dispatch. Streaming has separate code paths per backend because httpx requires a `with self.client.stream(...)` context manager while requests uses `stream=True` kwarg. - -Three exception types form the public error hierarchy: `InferenceGatewayError` (base) → `InferenceGatewayAPIError` (carries `status_code` + `response_data`) and `InferenceGatewayValidationError` (Pydantic validation failures). - -### Streaming protocol - -`create_chat_completion_stream` yields `SSEvent` objects (not parsed completion chunks). Callers are expected to `json.loads(chunk.data)` and validate against `CreateChatCompletionStreamResponse` themselves — see the README's streaming example. `data: [DONE]` sentinels are filtered out by `_process_stream_response`. - -### Provider-specific metadata - -Some providers (notably Google Gemini reasoning models) attach opaque per-call metadata like `thought_signature` that must round-trip verbatim on follow-up requests. This is preserved automatically when the caller appends the returned `message` object directly to the conversation. If reconstructing from dicts, the `ToolCallExtraContent` / `Google` models must be populated explicitly. See README "Provider-Specific Tool-Call Metadata". - -## Testing - -Tests in `tests/test_client.py` use `unittest.mock.patch` on `requests.Session.request` — no live HTTP, no fixtures spinning up servers. When adding client methods, mirror this style: mock the response, assert the exact URL/params/json passed to `request`. - -## Release process - -Releases are driven by [semantic-release](https://semantic-release.gitbook.io) (`.releaserc.yaml`) and triggered manually via the `Release` GitHub Action workflow. Commit messages **must follow Conventional Commits** because they drive version bumps and changelog sections: - -- `feat:` → minor bump -- `fix:`, `perf:`, `refactor:`, `impr:`, `ci:`, `docs:`, `chore:`, `style:`, `test:`, `build:` → patch bump - -The release flow bumps `pyproject.toml` version, updates `CHANGELOG.md`, tags as `v${version}`, creates a GitHub release, and publishes to PyPI via twine.