feat(websearch): add Keenable as a web search provider

[ilya-bogin-keenable]

Closes #8801. Supersedes #8802, which was closed in the 90-day cleanup; this is a fresh implementation on current master, following the AnySearch provider's shape.

Adds Keenable as an opt-in web search provider. Disclosure: I work at Keenable. The default provider and the order of existing options are unchanged.

Modifications / 改动点

Two built-in tools, registered only when provider_settings.websearch_provider == "keenable":

Tool	Endpoint	Notes
web_search_keenable	POST /v1/search	query, max_results (1-50, default 10), site (one domain). Results go through _search_result_payload, so ChatUI citations work (added to WEB_SEARCH_CITATION_TOOL_NAMES).
keenable_extract_web_page	GET /v1/fetch	url, max_chars. Returns the same URL: / Content: shape as the Tavily and Exa extract tools.

• Works without a key. Like AnySearch, an empty websearch_keenable_key list is valid: the tools call Keenable's keyless endpoints (/v1/search/public, /v1/fetch/public), which are free and rate limited. Configured keys use the authenticated endpoints, rotate through _KeyRotator, and fail over on 401/403/429 like the Tavily tools; a 5xx fails fast.
• Every request sends X-Keenable-Title: astrbot; the keyless endpoints reject requests without it.
• Snippets are requested at about 500 characters (snippet_max_length) and whitespace is collapsed, so results stay compact in the prompt.

Files:

• astrbot/core/tools/web_search_tools.py: request helper, both tools, rotator, legacy str→list key migration.

• astrbot/core/astr_main_agent.py: keenable branch in _apply_web_search_tools, citation tool name.

• astrbot/core/config/default.py: websearch_keenable_key: [], keenable appended to the provider options, key field (secret: True).

• dashboard/src/i18n/locales/{zh-CN,en-US,ja-JP,ru-RU}/features/config-metadata.json: key label and hint.

• docs/{en,zh}/use/websearch.md: provider table row, keyless note, extraction list.

• Tests in tests/unit/test_web_search_tools.py, test_astr_main_agent.py, test_func_tool_manager.py.

• This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

$ uvx --from ruff==0.15.22 ruff format --check .
516 files already formatted
$ uvx --from ruff==0.15.22 ruff check .
All checks passed!
$ uv run pytest tests/unit
1537 passed
$ node dashboard/scripts/check-i18n-keys.mjs
PASS
$ cd docs && pnpm run docs:build
build complete

New unit tests use response fixtures shaped like real Keenable responses (empty description, page text in snippet). They cover keyless vs keyed endpoints and headers, failover on 429, all keys exhausted, no failover on 500, the keyless rate-limit error, payload building (max_results clamping, site trimming), fetch params, empty content, empty URL, legacy key migration, built-in registration, and tool injection.

Run live against the Keenable API with the actual tool classes and no key:

• KeenableWebSearchTool.call(query=..., max_results=5) returned 5 results with citation indexes.
• site="github.com" returned only github.com URLs.
• KeenableExtractWebPageTool.call(url="https://docs.astrbot.app/en/use/websearch.html", max_chars=400) returned the page as Markdown.
• An invalid key surfaced Keenable web search failed: ... Invalid API key, status: 401.

---

Checklist / 检查清单

• 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
  / 如果 PR 中有新加入的功能，已经通过 Issue / 邮件等方式和作者讨论过。

  Tracked in [Feature] Add Keenable as a built-in web search provider #8801.

• 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
  / 我的更改经过了良好的测试，并已在上方提供了"验证步骤"和"运行截图"。

• 📚 I checked the affected WebUI instructions and screenshots in docs/zh and docs/en against the changed navigation, page structure, and labels, and updated them in this PR (or explained why no documentation update is needed). For renamed, moved, or merged entry points, I included an old entry → new entry mapping in the documentation and changelog.
  / 我已对照变化后的 WebUI 入口、页面结构和术语，核对并在本 PR 中更新 docs/zh 和 docs/en 的相关操作说明与截图（或说明无需更新文档的原因）。入口改名、移动或合并时，已在文档和 changelog 中补充 旧入口 → 新入口 对照。

  No entry point changed. websearch.md gains a Keenable row; the existing settings steps (AI → Capabilities → Web Search) apply unchanged.

• 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml.
  / 我确保没有引入新依赖库 (only the existing aiohttp).

• 😮 My changes do not introduce malicious code.
  / 我的更改没有引入恶意代码。

Summary by Sourcery

Add opt-in Keenable web search and webpage extraction support across the provider configuration, agent tools, citations, dashboard, and documentation.

New Features:

• Add Keenable as an opt-in web search provider with search and webpage extraction tools.
• Support anonymous Keenable access as well as configurable API keys with key rotation and retry handling.
• Include Keenable search results in citation processing and dashboard references.

Enhancements:

• Expose Keenable provider settings and localized configuration metadata.
• Document Keenable capabilities, keyless usage, and webpage extraction in English and Chinese web search guides.

Tests:

• Add unit coverage for Keenable requests, payloads, authentication modes, failure handling, tool registration, injection, and citations.

[sourcery-ai]

Hey - I've found 3 issues

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/tools/web_search_tools.py" line_range="1634-1642" />
<code_context>
+        if not url:
+            return "Error: url must be a non-empty string."
+
+        params: dict = {"url": url}
+        try:
+            max_chars = int(kwargs.get("max_chars") or 0)
+        except (TypeError, ValueError):
+            max_chars = 0
+        if max_chars > 0:
+            params["max_chars"] = max_chars
+
+        result = await _keenable_fetch(provider_settings, params)
+        content = result.get("content")
+        if not content:
</code_context>
<issue_to_address>
**Fetches attacker-chosen URLs**

When a chat user can invoke the extraction tool with a URL for a private or link-local endpoint reachable by Keenable’s fetch infrastructure, the tool forwards the URL to Keenable without scheme, host, or address checks. Keenable fetches the target and the tool returns its content to the chat, exposing internal resources reachable from Keenable’s fetch infrastructure.

Restrict fetches to approved HTTP(S) destinations and ensure Keenable blocks private, loopback, and link-local addresses after DNS resolution.
</issue_to_address>

### Comment 2
<location path="astrbot/core/tools/web_search_tools.py" line_range="1487" />
<code_context>
+        Exception: If the request fails after all configured keys are exhausted,
+            or if a non-retryable HTTP error is returned.
+    """
+    keys = provider_settings.get("websearch_keenable_key", [])
+
+    last_error = None
</code_context>
<issue_to_address>
**Keyless mode fails**

When the saved `websearch_keenable_key` value is explicitly `null`, `len(keys)` raises `TypeError` before `_keenable_request` can use the public endpoint, so both Keenable tools fail instead of using keyless mode.

Normalize a `null` key setting to an empty list before calling `len(keys)`.

Also at `astrbot/core/tools/web_search_tools.py:1490`.
</issue_to_address>

### Comment 3
<location path="astrbot/core/astr_main_agent.py" line_range="167" />
<code_context>
         "web_search_brave",
         "web_search_exa",
         "web_search_anysearch",
+        "web_search_keenable",
     }
 )
</code_context>
<issue_to_address>
**Keenable citations disappear**

When a dashboard chat uses Keenable and the assistant cites a search result, `extract_web_search_refs` in both dashboard chat services rejects `web_search_keenable` because it is absent from their supported-tool lists. The assistant can emit `<ref>` citations, but the saved chat response has no corresponding source citations.

Add `web_search_keenable` to both dashboard citation-extraction allowlists, or have them use the shared web-search tool-name list.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 3 findings to address first, and when this provider is selected, search queries and URLs supplied for extraction are sent to Keenable. If that data transfer is the wrong choice, reverting stops future requests but cannot recall information already sent.

Blocking findings: astrbot/core/tools/web_search_tools.py:1642, astrbot/core/tools/web_search_tools.py:1487, astrbot/core/astr_main_agent.py:167

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

[sourcery-ai]

🟡 Medium · Fetches attacker-chosen URLs

When a chat user can invoke the extraction tool with a URL for a private or link-local endpoint reachable by Keenable’s fetch infrastructure, the tool forwards the URL to Keenable without scheme, host, or address checks. Keenable fetches the target and the tool returns its content to the chat, exposing internal resources reachable from Keenable’s fetch infrastructure.

Restrict fetches to approved HTTP(S) destinations and ensure Keenable blocks private, loopback, and link-local addresses after DNS resolution.

Prompt for AI agents

In `astrbot/core/tools/web_search_tools.py` at lines 1634-1642:

**Fetches attacker-chosen URLs**

When a chat user can invoke the extraction tool with a URL for a private or link-local endpoint reachable by Keenable’s fetch infrastructure, the tool forwards the URL to Keenable without scheme, host, or address checks. Keenable fetches the target and the tool returns its content to the chat, exposing internal resources reachable from Keenable’s fetch infrastructure.

Restrict fetches to approved HTTP(S) destinations and ensure Keenable blocks private, loopback, and link-local addresses after DNS resolution.