Add versioned prompt profiles and event-entity generation pipeline

MANIFEST.in

@@ -1,3 +1,5 @@
 include requirements.txt
 include graphgen/configs/*.yaml
 recursive-include graphgen/configs *.yaml
+recursive-include graphgen/templates/profiles *.json
+recursive-include graphgen/templates/event_profiles *.json *.txt

README.md

@@ -374,3 +374,9 @@ This project is licensed under the [Apache License 2.0](LICENSE).
 ## 📅 Star History
 
 [![Star History Chart](https://star-history.dera.page/svg?repos=InternScience/GraphGen&type=Date)](https://star-history.dera.page/#InternScience/GraphGen&Date)
+
+## Maintained fork: multi-project SDK and event profiles
+
+This repository also maintains the `Leon-Algo/GraphGen` fork. Fork-specific reusable capabilities, project-owned event extraction profiles, installation guidance, and compatibility boundaries are documented in [docs/MULTI_PROJECT_SDK.md](docs/MULTI_PROJECT_SDK.md). The immutable version/tag and release verification policy is in [docs/RELEASING.md](docs/RELEASING.md); the fork patch ledger and SAG attribution are in [docs/PMS_PATCHES.md](docs/PMS_PATCHES.md).
+
+Fork profiles are opt-in. The default LightRAG + ECE workflow remains unchanged, and PMS-specific prompts are not a default policy for other projects.

README_zh.md

@@ -375,3 +375,9 @@ GraphGen 首先根据源文本构建细粒度的知识图谱，然后利用期
 
 [![Star History Chart](https://star-history.dera.page/svg?repos=InternScience/GraphGen&type=Date)](https://star-history.dera.page/#InternScience/GraphGen&Date)
 
+
+## 自维护 Fork：多项目 SDK 与事件 Profile
+
+本仓库同时维护 `Leon-Algo/GraphGen` fork。跨项目可复用能力、项目自有事件抽取 profile、安装与兼容边界见[多项目 SDK 指南](docs/MULTI_PROJECT_SDK.md)；不可变版本/tag 与发布验证规则见[发布规范](docs/RELEASING.md)；fork 补丁及 SAG 来源归属见[补丁台账](docs/PMS_PATCHES.md)。
+
+Fork 能力均显式 opt-in：默认 LightRAG + ECE 管线保持不变，PMS 专用提示词不会自动成为其他项目的策略。

docs/MULTI_PROJECT_SDK.md

@@ -0,0 +1,84 @@
+# GraphGen Fork: reusable Graph + SAG event capabilities
+
+This repository is the independently maintained `Leon-Algo/GraphGen` fork of GraphGen. It remains compatible with the upstream project where documented, while shipping opt-in fork capabilities. The upstream PR is not a release dependency: projects consume reviewed fork commits/tags directly.
+
+## What is reusable
+
+- **GraphGen foundation:** pipeline/operators, storage integration, generators, and the default LightRAG + ECE route.
+- **Fork capabilities:** versioned generation prompt profiles, output gates, generation concurrency controls, and the opt-in event/entity star-graph builder plus event-join partitioner.
+- **SAG-derived event mechanics:** event extraction response contract, numeric grounding, star encoding (one event node linked to participants), deterministic merging, and shared-participant joins. Adapted assets and code retain their upstream license/source attribution in `docs/PMS_PATCHES.md`.
+
+These mechanisms do not define a universal business ontology. A consuming project selects a versioned event extraction profile and its own generation prompt profile. The `pms_event.v1` and `pms_policy.v2/v3` profiles are PMS-specific examples and are never implicitly applied to unrelated projects. `generic_event.v1` and `example_event.v1` are synthetic examples, not validated domain policies.
+
+## Install a reviewed fork version
+
+For a reproducible project, pin an immutable commit in the consuming project's `pyproject.toml` and commit its lock file:
+
+```toml
+[project]
+dependencies = [
+  "graphg @ git+https://github.com/Leon-Algo/GraphGen.git@<full-commit-sha>",
+]
+```
+
+Resolve and lock with `uv lock` (or add it with `uv add`). Avoid moving branch refs in production. For short-lived development only, install `git+https://github.com/Leon-Algo/GraphGen.git@pms/main`. The distribution name is `graphg`; Python imports use `graphgen`.
+
+## Select event extraction explicitly
+
+The default `kg_method` remains `light_rag`; the normal ECE partitioner remains unchanged. To use the event route, configure the build node explicitly:
+
+```yaml
+- id: build_kg
+  op_name: build_kg
+  type: map
+  params:
+    kg_method: event_entity
+    event_profile: generic_event.v1
+
+- id: partition
+  op_name: partition
+  type: map
+  params:
+    method: event_join
+    method_params:
+      event_entity_type: EVENT
+      anchor_attribute: anchor
+      require_same_anchor: false
+      max_events_per_community: 6
+      max_tokens_per_community: 4096
+```
+
+For a project-specific profile, copy a profile directory into a project-controlled location and point `event_profile` to the absolute or explicitly configured `profile.json` path; do not rely on the process working directory. Never edit a profile already used by a released run. The loader validates the manifest and prompt, fails closed on missing/invalid inputs, and records SHA-256 fingerprints on the loaded profile object.
+
+An event profile contains:
+
+- `schema_version` and immutable `profile_id`;
+- `prompt_file` with the required `{input_text}` placeholder;
+- `entity_types` allowlist used both in prompt input and response validation;
+- `anchor` with required/optional status, optional regex, and optional anchor entity type;
+- `grounding.number_grounding`;
+- `graph.event_entity_type` and `graph.legacy_event_id`.
+
+Regex capture group 1 is the anchor when present; otherwise the complete match is used. Event nodes carry the extracted anchor as an explicit string attribute; the partitioner never parses it from the node ID. `require_same_anchor: true` rejects missing anchors and prevents joins across anchor values. Set it to false only when cross-event joining without anchor isolation is acceptable. Limits remain explicit partition parameters.
+
+The packaged `generic_event.v1` profile demonstrates an anchor-free schema; `example_event.v1` demonstrates a custom anchor and domain entity set. They are fixtures, not a claim that a particular project's extraction policy has been validated. PMS workflows should explicitly select `pms_event.v1` and the corresponding PMS generation policy; legacy configurations with `kg_method: event_entity` and no profile retain the PMS-compatible profile.
+
+## Generation profiles and gates
+
+Generation profiles use the independent `prompt_profile` parameter (built-in profile ID or explicit profile directory); do not assume it selects event extraction policy. PMS output gate `pms_project_anchor` is opt-in and is not a generic default. Each project should own its prompt policy and only enable domain gates it has validated.
+
+## Profile authoring checklist
+
+1. Define a versioned profile ID and the consuming project's allowed entity types.
+2. Write prompt instructions for that domain; include `{input_text}` exactly once or as required by the application.
+3. Set an explicit anchor regex or `required: false`; set `anchor_entity_type` only if it is in the entity allowlist.
+4. Decide numeric grounding and whether event_join must enforce same-anchor isolation.
+5. Test supported, missing, malformed, ungrounded, and unknown-entity responses with a fake LLM before real model smoke.
+6. Test extraction → star graph → partition on representative project data, then pin the profile bytes and GraphGen commit in the consuming project.
+
+## Upgrade and compatibility policy
+
+- A released tag and profile directory are immutable. Fixes and policy changes get a new commit and a new tag/profile ID; do not retarget tags or edit released profile contents in place.
+- Consumers pin a full commit SHA or immutable release tag and commit their lock file. Upgrade by changing the pin deliberately, rerunning tests/smokes, and reviewing GraphGen's release notes and `docs/PMS_PATCHES.md`.
+- `light_rag` remains the default. `event_entity` is opt-in. PMS profile defaults exist only for backward compatibility of the PMS fork integration; new projects should always specify their profile explicitly.
+- Fork release/tag format, validation gate, attribution, upstream base, and exact release evidence are maintained in `docs/RELEASING.md`.

docs/PMS_PATCHES.md

@@ -0,0 +1,95 @@
+# PMS Fork 补丁台账
+
+本仓库是 `open-sciencelab/GraphGen`（upstream，Apache-2.0）的自维护 fork，分支
+`pms/main`。上游修复可通过 rebase 同步，冲突以本 fork 语义为准。PMS 主仓库只钉
+住本 fork 的 commit SHA（receipt `_graphgen_source_identity` 自动记录）。
+
+## #1 版本化 Prompt Profile（2026-09-18）
+
+- **动机**：官方所有 prompt 是 `graphgen/templates/` 下的硬编码常量，无任何
+  config 覆盖机制；此前 PMS 侧只能 monkeypatch `generator.build_prompt`。
+- **实现**：
+  - `graphgen/templates/prompt_profiles.py`：profile 加载器。profile = 内置 id
+    （`graphgen/templates/profiles/<id>/`）或文件路径；每 method 一个
+    `<template_key>.json`；结构与官方模板逐键校验（fail-closed）。
+  - `graphgen/bases/base_generator.py`：`apply_prompt_profile()` +
+    `template()`；generators 声明 `TEMPLATE_KEY` 并经 `self.template(...)`
+    渲染；未应用 profile 时与官方常量逐字节一致。
+  - 10 个 generate method（atomic/aggregated/multi_hop/cot/multi_choice/
+    multi_answer/fill_in_blank/masked_fill_in_blank/true_false/vqa）全部接入。
+  - `graphgen/operators/generate/generate_service.py`：generate 节点新增
+    `prompt_profile` / `output_gate` / `llm_concurrency` 三个可选参数。
+- **内置 profile**：`pms_policy.v2`（atomic + multi_hop 的 PMS project-query
+  出题策略模板，正本迁移自 PMS 受控侧 v2 模板）。
+
+## #2 确定性输出门（2026-09-18）
+
+- `graphgen/operators/generate/gates.py`：注册表 + `pms_project_anchor` 门
+  （题面必须含完整项目编号 P-/I-+≥6 位且答案非空，否则丢弃该 QA，不改写）。
+- generate 节点经 `output_gate` 参数启用。
+
+## #3 生成 LLM 并发封顶（2026-09-18）
+
+- `graphgen/operators/generate/llm_concurrency.py`：事件循环级信号量，
+  把同 loop 在途 generate 压到上限（35B-A3B 高并发实测会过载挂死）。
+- generate 节点经 `llm_concurrency` 参数启用（PMS 默认 4）。
+
+## #4 空响应有界重试（复位自丢失补丁，2026-09-18）
+
+- `graphgen/models/llm/api/openai_client.py::generate_answer`：思考模型偶发
+  仅返回 `<think>` 内容、过滤后为空串；`GRAPHGEN_EMPTY_RETRY=N`（默认 0，
+  行为同官方）时对空响应做最多 N 次重试。历史版本此补丁仅存于 env 注入、
+  源码实现已丢失，本条为正确复位。
+
+## #5 空抽取 assistant prefill 重试（复位自丢失补丁，2026-09-18）
+
+- `openai_client.generate_answer` 新增 `assistant_prefill` 参数；
+  `schema_guided_extractor.extract` 在响应为空且
+  `GRAPHGEN_EMPTY_EXTRACTION_PREFILL=1` 时，以确定性 prefill `("entity"<|>`
+  重试一次。非空响应永不重试/改写。
+
+## #6 logprob 调用关思考（复位自丢失补丁，2026-09-18）
+
+- `generate_topk_per_token`：思考模型前若干 token 消耗在 `<think>` 内，导致
+  judge/loss 的 next-token yes/no logprob 退化。`GRAPHGEN_DISABLE_THINKING=1`
+  时对 logprob 调用注入 `extra_body.chat_template_kwargs.enable_thinking=false`
+  （Qwen 兼容后端）。普通生成调用不受影响。
+
+## #7 事件-实体星型图管线（Phase 2，2026-10-09）
+
+SAG 思想移植（MIT，arXiv:2606.15971；参考源码 `workspace/sag_reference/`，方案见 PMS 主仓
+`docs/2026-09-18-graphgen-phase2-sag-event-entity-plan.md` §3.2 复用地图）：
+
+- `templates/kg/event_entity_extraction.py` 保留旧 Python 常量的兼容导出；实际策略由版本化 event profile 资源提供。
+- `models/kg_builder/event_contract.py`：纯 stdlib 响应合同（禁合同外字段、is_valid 自报、
+  无锚 fail-closed 丢弃并计数）+ 数字接地校验（SAG grounding 移植）+ 确定性事件 ID。
+- `models/kg_builder/event_entity_kg_builder.py`：事件→星型编码（EVENT 节点 + 成员边，
+  边属性含角色）。实体描述/普通边继续复用官方合并；**EVENT 节点与事件成员边用确定性
+  SEP 合并、不走 LLM 摘要**（SAG 事件是事实文本，不能让 merge summarizer 把 event_id
+  扩写成“项目身份/分工总结”污染出题上下文）。事件描述限长 4,000 字符，且原文链以
+  `source_id` SEP 保持闭合。
+- `operators/build_kg/`：`kg_method: event_entity` 分发（默认 light_rag 与官方逐字节一致）。
+- `models/partitioner/event_join_partitioner.py`：共享实体 join + 同锚硬约束的事件簇分区
+  （SAG join 语义离线物化），产出官方 `Community`，metadata 携带 event_ids。
+- `models/generator/support.py` + atomic/multi_hop 解析 + GenerateService 校验：
+  `<support>{"cited": [...]}` 引用必须存在于本分区节点集合（EVENT 节点名即事件审计
+  ID，不依赖自定义 Community metadata 以保证 JSONL cache 往返一致），失败剥离保留题面。
+- `templates/profiles/pms_policy.v3/`：v2 模板 + support 输出要求。
+
+## #8 通用事件抽取 Profile 与多项目 SAG 能力
+
+- 将事件抽取策略从 builder/契约代码中抽离为 `templates/event_profiles/<id>/` 下版本化 manifest + prompt。内置 `pms_event.v1` 保留原始 PMS prompt、8 类实体、P-/I- 锚点和旧事件 ID；`generic_event.v1` 与 `example_event.v1` 是机制示例/测试 fixture，不代表外部领域已验证策略。
+- `templates/event_profile_loader.py` 校验 schema、实体类型、锚点 regex、prompt 占位符及图设置，暴露 manifest/prompt SHA-256；显式无效 profile fail-closed。profile 从包资源读取，不依赖当前工作目录。
+- `build_kg.params.event_profile` 传入事件 builder。老 PMS 配置缺省解析为 `pms_event.v1`；新项目应显式传入自己的 profile。`light_rag` 默认保持不变。
+- `event_contract.py` 仍保证严格响应 JSON、实体 allowlist 和可选数字接地；实体类型、锚点是否必需及锚点模式由 profile 驱动。无锚模式不自动合成 PROJECT/其他锚点实体。
+- 事件节点写入显式 `anchor` 属性。`event_join` 按 `event_entity_type` 和 `anchor_attribute` 配置工作，`require_same_anchor` 控制同锚硬隔离；不再从 ID 推断锚点。PMS profile 使用旧 ID 以兼容既有图缓存，新 profile 使用不编码 domain anchor 的 ID。
+- `setup.py` 与 `MANIFEST.in` 将生成 profile JSON、事件 manifest/prompt 作为 `graphgen` wheel/sdist 包资源。
+- 多项目安装、profile authoring 与升级见 `docs/MULTI_PROJECT_SDK.md`；fork 不可变 tag 与发布门禁见 `docs/RELEASING.md`。
+
+## 验收
+
+- 默认路径（无 profile/gate/并发参数）与官方行为逐字节一致；
+- PMS 主仓库 `tests/test_pms_graphgen_policy_generation.py`、
+  `tests/test_run_graphgen_defaults.py` 为契约回归（rft 环境按路径加载
+  纯 stdlib 模块校验）；
+- 事件 profile 合同和真实生成由 GraphGen venv smoke 验收。

docs/RELEASE_NOTES-1.0.0.md

@@ -0,0 +1,23 @@
+# GraphGen Fork 1.0.0 — initial multi-project SDK release
+
+Tag: `graphgen-fork-v1.0.0` (to be created only after passing release validation).
+
+Distribution version: `1.0.0` (`graphg`). Upstream base: `InternScience/GraphGen` commit `3a3eb097318d34b07f6f31cb60de5b20712d65ce`.
+
+## Highlights
+
+- Profile-driven event/entity extraction: projects supply entity types, prompt, anchor policy, numeric grounding and event graph identity without patching GraphGen source.
+- Built-in `pms_event.v1` preserves PMS extraction behavior and legacy event IDs; `generic_event.v1` and `example_event.v1` are synthetic examples only.
+- `event_join` reads explicit anchor node attributes and supports opt-in same-anchor isolation independently of event ID encoding.
+- Default LightRAG + ECE remains unchanged; event extraction is opt-in.
+- Event and generation prompt profile assets are included in wheel and sdist and load through package resources.
+- See [multi-project SDK guide](MULTI_PROJECT_SDK.md), [release policy](RELEASING.md), and [patch ledger](PMS_PATCHES.md).
+
+## Compatibility and validation record
+
+GraphGen partition integration: 12 passed after correcting tests to the synchronous storage/partitioner APIs. Profile, fake-LLM builder/partition, and package tests: 12 passed. PMS contract regression: `tests/test_pms_graphgen_policy_generation.py` — 23 passed, 1 skipped.
+A real one-chunk GraphGen run using the configured PMS launcher/model pool completed extraction → event_join → COT generation. The synthetic non-PMS event anchored on `CASE-A7`; the graph contained the event and two participants; the final QA identified the operator and completed mission, included no internal event hash, and the process exited 0. The run used a synthetic fixture and validates integration mechanics, not any production project's domain accuracy.
+
+The existing GraphGen partitioner integration suite could not collect because its tests import an unexported `NetworkXStorage` from `graphgen.models`. An unrelated GraphGen extraction e2e test failed at Ray GCS startup (`Could not read 'temp_dir' from GCS`). These environment/baseline failures are not represented as passing tests.
+
+SAG-derived code/prompt attribution and MIT licensing are tracked in `docs/PMS_PATCHES.md`.

docs/RELEASING.md

@@ -0,0 +1,43 @@
+# GraphGen fork release and tagging policy
+
+The fork is an independently versioned Python distribution (`graphg`) maintained at `Leon-Algo/GraphGen`. Upstream merge status does not control fork releases. Release tags are immutable identifiers for exact commits; never move or reuse a published tag.
+
+## Tag format
+
+Use `graphgen-fork-vMAJOR.MINOR.PATCH`:
+
+- **MAJOR**: breaking config/profile/schema or graph identity changes requiring consumer migration.
+- **MINOR**: backward-compatible capability or new profile/API.
+- **PATCH**: backward-compatible bug fix or documentation/packaging correction.
+
+PMS domain profiles are independently versioned by their profile IDs and are not implied by a package release. Profiles referenced by production configs must not be edited after release; changes get a new profile ID.
+
+## Release gate
+
+Before tagging:
+
+1. Review the fork diff and working tree; ensure no unrelated user changes are included.
+2. Run deterministic profile/contract/builder/partitioner tests, relevant upstream GraphGen tests, and PMS compatibility tests.
+3. Run a non-PMS extraction → star graph → event_join smoke and a PMS compatibility smoke; confirm default LightRAG + ECE behavior is unchanged.
+4. Build wheel and sdist in a disposable directory; inspect both for prompt/profile assets and install the wheel into a clean environment to load each built-in profile.
+5. Run `git diff --check`, record exact commit, test commands/results, profile hashes and upstream base.
+
+If any required gate fails, do not tag or push the release.
+
+## Release sequence
+
+1. Merge only reviewed release files into the intended fork branch; preserve unrelated local changes.
+2. Commit with a concise message and verify the final commit SHA and clean release diff.
+3. Confirm the desired tag is unused locally and on the fork remote.
+4. Create the tag on that exact commit, then push the intended branch and tag to the `fork` remote (`https://github.com/Leon-Algo/GraphGen.git`). Never push the fork release tag to upstream by accident.
+5. Publish release notes that include: distribution version, tag and commit SHA, upstream base commit, key behavior/config changes, profile IDs and asset hashes, compatibility/migration notes, license/source attribution, test and smoke evidence, and any known limitations.
+
+Example commands (replace placeholders only after verification):
+
+```bash
+git tag -a graphgen-fork-v1.0.0 <verified-commit> -m "GraphGen fork 1.0.0"
+git push fork pms/main
+git push fork graphgen-fork-v1.0.0
+```
+
+Consumers should pin the release tag or preferably full commit SHA and lock their environment. Tag creation/push is an externally visible release action and must occur only after the release gate passes and the maintainer authorizes publication.