Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,28 +9,28 @@ All development runs through `task` (see `Taskfile.yml`):
- `task install` - `pip install -e ".[dev]"`
- `task format` - Black + isort on `inference_gateway/`, `tests/`, `examples/`
- `task lint` - Black `--check`, isort `--check-only`, mypy on `inference_gateway/` and `examples/`. Stricter than CI (`.github/workflows/ci.yml`), which only runs `black --check .` and `pytest tests/` - no mypy or isort.
- `task test` - pytest suite; `task test:coverage` adds terminal + HTML coverage
- `task test` - pytest suite, `task test:coverage` adds terminal + HTML coverage
- `task generate` - downloads (`oas-download`) and validates the OpenAPI spec, then regenerates `inference_gateway/models.py`
- `task build` - clean + lint + test, then `python -m build`
- `task dev:setup` - full bootstrap: install, download spec, generate, install the pre-commit hook, test
- `task precommit:install` - point git at `.githooks/` (`git config core.hooksPath .githooks`); `task precommit:run` runs the hook directly. The hook is a plain shell script, not the pre-commit framework: it runs `task format` on staged Python files and blocks the commit if anything was reformatted.
- `task precommit:install` - point git at `.githooks/` (`git config core.hooksPath .githooks`). `task precommit:run` runs the hook directly. The hook is a plain shell script, not the pre-commit framework: it runs `task format` on staged Python files and blocks the commit if anything was reformatted.

Focused test: `pytest tests/test_client.py::test_list_models -v`

## Layout

- `inference_gateway/client.py` - hand-written `InferenceGatewayClient` with two synchronous backends: `requests.Session` (default) and `httpx.Client` (`use_httpx=True`). Keep both paths behaviorally aligned in `_make_request` and `_process_stream_response`. Streaming has separate code paths per backend: httpx needs a `with self.client.stream(...)` context manager, requests uses the `stream=True` kwarg.
- Public errors: `InferenceGatewayError` (base) -> `InferenceGatewayAPIError` (carries `status_code` + `response_data`) and `InferenceGatewayValidationError` (Pydantic validation failures).
- `inference_gateway/models.py` - **auto-generated by `datamodel-codegen`, do not edit by hand**. Update the spec upstream in `inference-gateway/schemas` (or local `openapi.yaml`, or the Jinja templates `templates/header.jinja2` / `templates/pydantic/BaseModel.jinja2`), then `task generate`; pin the spec with `SCHEMAS_REF=refs/tags/vX.Y.Z`. New public models also need `inference_gateway/__init__.py` imports and `__all__` entries; new endpoints need a `client.py` method.
- `tests/` - client behavior; `examples/` - runnable usage examples (chat, list, mcp, messages, tools).
- `inference_gateway/models.py` - **auto-generated by `datamodel-codegen`, do not edit by hand**. Update the spec upstream in `inference-gateway/schemas` (or local `openapi.yaml`, or the Jinja templates `templates/header.jinja2` / `templates/pydantic/BaseModel.jinja2`), then `task generate`. Pin the spec with `SCHEMAS_REF=refs/tags/vX.Y.Z`. New public models also need `inference_gateway/__init__.py` imports and `__all__` entries, and new endpoints need a `client.py` method.
- `tests/` - client behavior. `examples/` - runnable usage examples (chat, list, mcp, messages, tools).

## Provider-specific metadata

Some providers (notably Google Gemini reasoning models) attach opaque per-call metadata like `thought_signature` that must round-trip verbatim on follow-up requests. It is preserved when the caller appends the returned `message` object to the conversation; when rebuilding from dicts, populate `ToolCallExtraContent` / `Google` explicitly. See README "Provider-Specific Tool-Call Metadata".
Some providers (notably Google Gemini reasoning models) attach opaque per-call metadata like `thought_signature` that must round-trip verbatim on follow-up requests. It is preserved when the caller appends the returned `message` object to the conversation. When rebuilding from dicts, populate `ToolCallExtraContent` / `Google` explicitly. See README "Provider-Specific Tool-Call Metadata".

## Style

Black + isort at 100 columns; 4-space indents, LF line endings; YAML/JSON at 2 spaces. mypy with `disallow_untyped_defs = true` and the pydantic plugin. `snake_case` for functions/tests, `PascalCase` for classes and models.
Black + isort at 100 columns, 4-space indents, LF line endings, YAML/JSON at 2 spaces. mypy with `disallow_untyped_defs = true` and the pydantic plugin. `snake_case` for functions/tests, `PascalCase` for classes and models.

### Code Readability

Expand All @@ -41,11 +41,13 @@ Black + isort at 100 columns; 4-space indents, LF line endings; YAML/JSON at 2 s
- No comments above modules, packages, or files.
- Tool directives are not comments and stay where the tool needs them (lint suppressions, build
tags, compiler pragmas, code generation markers).
- No semicolons in documentation prose (Markdown files, doc comments): split the sentence or use
a dash instead.

## Testing

pytest + `unittest.mock` only - never make live network calls or spin up servers. Existing tests patch `requests.Session.request` and assert exact URL, params, JSON body, and timeout; mirror that for new client methods. Streaming (`create_chat_completion_stream`) yields raw `SSEvent` objects with `data: [DONE]` sentinels filtered out - callers `json.loads(chunk.data)` and validate against `CreateChatCompletionStreamResponse` themselves. Never commit real API tokens; examples take configuration from env vars.
pytest + `unittest.mock` only - never make live network calls or spin up servers. Existing tests patch `requests.Session.request` and assert exact URL, params, JSON body, and timeout - mirror that for new client methods. Streaming (`create_chat_completion_stream`) yields raw `SSEvent` objects with `data: [DONE]` sentinels filtered out - callers `json.loads(chunk.data)` and validate against `CreateChatCompletionStreamResponse` themselves. Never commit real API tokens - examples take configuration from env vars.

## Commits & Releases

Conventional Commits drive semantic-release versioning (`.releaserc.yaml`): `feat:` -> minor; `fix:`/`perf:`/`refactor:`/`impr:`/`docs:`/`ci:`/`chore:`/`style:`/`test:`/`build:` -> patch. Releases are triggered manually via the `Release` GitHub Action: it bumps `pyproject.toml`, updates `CHANGELOG.md`, tags `v${version}`, creates a GitHub release, and publishes to PyPI. PRs should note tests run and flag generated-code updates or API compatibility concerns.
Conventional Commits drive semantic-release versioning (`.releaserc.yaml`): `feat:` -> minor, `fix:`/`perf:`/`refactor:`/`impr:`/`docs:`/`ci:`/`chore:`/`style:`/`test:`/`build:` -> patch. Releases are triggered manually via the `Release` GitHub Action: it bumps `pyproject.toml`, updates `CHANGELOG.md`, tags `v${version}`, creates a GitHub release, and publishes to PyPI. PRs should note tests run and flag generated-code updates or API compatibility concerns.