Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 29 additions & 13 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,51 @@
# Repository Guidelines

Python SDK for Inference Gateway — a thin, hand-written synchronous client over auto-generated Pydantic models for the gateway's OpenAI- and Anthropic-compatible APIs. Distributed on PyPI as `inference-gateway`. Requires Python 3.12+.
Python SDK for Inference Gateway - a thin, hand-written synchronous client over auto-generated Pydantic models for the gateway's OpenAI- and Anthropic-compatible APIs. Distributed on PyPI as `inference-gateway`. Requires Python 3.12+ (CI runs 3.13).

## Commands

All development runs through `task` (see `Taskfile.yml`):

- `task install` — `pip install -e ".[dev]"`
- `task format` — Black + isort on `inference_gateway/`, `tests/`, `examples/`
- `task lint` — Black `--check`, isort `--check-only`, mypy. Stricter than CI, which only runs Black and pytest.
- `task test` — pytest suite; `task test:coverage` adds terminal + HTML coverage
- `task generate` — fetches/validates the OpenAPI spec, regenerates `inference_gateway/models.py`
- `task build` — clean + lint + test, then `python -m build`
- `task precommit:install` — point git at `.githooks/` (a plain shell script, not the pre-commit framework)
- `task install` - `pip install -e ".[dev]"`
- `task format` - Black + isort on `inference_gateway/`, `tests/`, `examples/`
- `task lint` - Black `--check`, isort `--check-only`, mypy on `inference_gateway/` and `examples/`. Stricter than CI (`.github/workflows/ci.yml`), which only runs `black --check .` and `pytest tests/` - no mypy or isort.
- `task test` - pytest suite; `task test:coverage` adds terminal + HTML coverage
- `task generate` - downloads (`oas-download`) and validates the OpenAPI spec, then regenerates `inference_gateway/models.py`
- `task build` - clean + lint + test, then `python -m build`
- `task dev:setup` - full bootstrap: install, download spec, generate, install the pre-commit hook, test
- `task precommit:install` - point git at `.githooks/` (`git config core.hooksPath .githooks`); `task precommit:run` runs the hook directly. The hook is a plain shell script, not the pre-commit framework: it runs `task format` on staged Python files and blocks the commit if anything was reformatted.

Focused test: `pytest tests/test_client.py::test_list_models -v`

## Layout

- `inference_gateway/client.py` — hand-written `InferenceGatewayClient` with two synchronous backends: `requests.Session` (default) and `httpx.Client` (`use_httpx=True`). Keep both paths behaviorally aligned in `_make_request` and `_process_stream_response`.
- `inference_gateway/models.py` — **auto-generated, do not edit by hand**. Update the spec upstream in `inference-gateway/schemas` (or local `openapi.yaml`/`templates/`), then `task generate`; pin the spec with `SCHEMAS_REF=refs/tags/vX.Y.Z`. New public models also need `inference_gateway/__init__.py` imports and `__all__` entries.
- `tests/` — client behavior; `examples/` — runnable usage examples (chat, list, mcp, messages, tools).
- `inference_gateway/client.py` - hand-written `InferenceGatewayClient` with two synchronous backends: `requests.Session` (default) and `httpx.Client` (`use_httpx=True`). Keep both paths behaviorally aligned in `_make_request` and `_process_stream_response`. Streaming has separate code paths per backend: httpx needs a `with self.client.stream(...)` context manager, requests uses the `stream=True` kwarg.
- Public errors: `InferenceGatewayError` (base) -> `InferenceGatewayAPIError` (carries `status_code` + `response_data`) and `InferenceGatewayValidationError` (Pydantic validation failures).
- `inference_gateway/models.py` - **auto-generated by `datamodel-codegen`, do not edit by hand**. Update the spec upstream in `inference-gateway/schemas` (or local `openapi.yaml`, or the Jinja templates `templates/header.jinja2` / `templates/pydantic/BaseModel.jinja2`), then `task generate`; pin the spec with `SCHEMAS_REF=refs/tags/vX.Y.Z`. New public models also need `inference_gateway/__init__.py` imports and `__all__` entries; new endpoints need a `client.py` method.
- `tests/` - client behavior; `examples/` - runnable usage examples (chat, list, mcp, messages, tools).

## Provider-specific metadata

Some providers (notably Google Gemini reasoning models) attach opaque per-call metadata like `thought_signature` that must round-trip verbatim on follow-up requests. It is preserved when the caller appends the returned `message` object to the conversation; when rebuilding from dicts, populate `ToolCallExtraContent` / `Google` explicitly. See README "Provider-Specific Tool-Call Metadata".

## Style

Black + isort at 100 columns; 4-space indents, LF line endings; YAML/JSON at 2 spaces. mypy with `disallow_untyped_defs = true` and the pydantic plugin. `snake_case` for functions/tests, `PascalCase` for classes and models.

### Code Readability

- Write self-explanatory code: clear names and small, single-purpose functions carry the intent.
If a block needs a comment to be understood, extract it into a well-named function or variable.
- No inline comments inside function bodies.
- Doc comments on functions and types are at most 5 lines: what it does and why, not how.
- No comments above modules, packages, or files.
- Tool directives are not comments and stay where the tool needs them (lint suppressions, build
tags, compiler pragmas, code generation markers).

## Testing

pytest + `unittest.mock` only — never make live network calls. Existing tests patch `requests.Session.request` and assert exact URL, params, JSON body, and timeout; mirror that for new client methods. Streaming yields raw `SSEvent` objects — callers parse `chunk.data` themselves. Never commit real API tokens; examples take configuration from env vars.
pytest + `unittest.mock` only - never make live network calls or spin up servers. Existing tests patch `requests.Session.request` and assert exact URL, params, JSON body, and timeout; mirror that for new client methods. Streaming (`create_chat_completion_stream`) yields raw `SSEvent` objects with `data: [DONE]` sentinels filtered out - callers `json.loads(chunk.data)` and validate against `CreateChatCompletionStreamResponse` themselves. Never commit real API tokens; examples take configuration from env vars.

## Commits & Releases

Conventional Commits drive semantic-release versioning (`.releaserc.yaml`): `feat:` → minor; `fix:`/`perf:`/`refactor:`/`docs:`/`ci:`/`chore:`/`test:`/`build:` → patch. PRs should note tests run and flag generated-code updates or API compatibility concerns.
Conventional Commits drive semantic-release versioning (`.releaserc.yaml`): `feat:` -> minor; `fix:`/`perf:`/`refactor:`/`impr:`/`docs:`/`ci:`/`chore:`/`style:`/`test:`/`build:` -> patch. Releases are triggered manually via the `Release` GitHub Action: it bumps `pyproject.toml`, updates `CHANGELOG.md`, tags `v${version}`, creates a GitHub release, and publishes to PyPI. PRs should note tests run and flag generated-code updates or API compatibility concerns.
73 changes: 0 additions & 73 deletions CLAUDE.md

This file was deleted.