Repository navigation
feat(sdk): data-processor host contract: typed create, window and instant evaluations (0.37.0) - #207
Merged
Merged
Conversation
….37.0) The host contract PJ4's on-demand plugin bridge (slice 8a-2) consumes, ported from #202 onto 0.36.0 with the agreed review changes. Data processors only: catalog v2, plot-tab slots, markers on v2 and dialog scene views wait for the release their consumers need. - Four tail slots on PJ_data_processors_host_vtable_t: create_data_processor_v2, submit_evaluation, poll_evaluation, release_evaluation; hasTypedRequests() detects them. - PJ_data_processor_request_t (read-prefix struct_size rule, a typed dataset_id, 0 = name resolution), the output element and the budget (a nonzero reserved field and a cap below the report envelope are rejected; NULL means host defaults). WINDOW and INSTANT are exclusive. - States PENDING, COMPLETED, FAILED; 3 is reserved and never emitted. FAILED carries the COMPLETED report shape with stopped "error"; parseEvaluationReport() validates types and coverage and rejects a raw NUL. - A scripted submit implies EPHEMERAL and rejects HISTORY_EXEMPT; a submit naming an installed node takes its stored recipe and rejects recipe fields in the request. - readDataProcessorRequest() / readEvaluationBudget(): the host-side read-prefix decoders in one place; null string views with a size are rejected. - INFER_OUTPUTS and the v1 "<name>:<type>" on_demand route are dropped: v1 create rejects on_demand with a hint to use v2. Every member offset is pinned on x64; the fake host enforces the contract through the raw C slots. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N9J3ZtWHDRzRsiJTi2o861
- Time flags by kind: markers accept WINDOW on create_data_processor_v2 (coverage_ns echoed in the config, start > end rejected) and reject INSTANT; transform accepts none. - Document the keys a host adds to data_processor_config for on_demand and markers, and where a create's failures are reported. - Declared outputs: single value, positional values or a table keyed by output name. - max_bytes bounds the run's whole memory, not only the interpreter's. - readDataProcessorRequest rejects a count or string size that cannot be an object before reading the array or string (red first: an ASAN abort on a UINT64_MAX count); pinned with a decoder test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E228Y7uZ8QdznAjEnrXKSz
The first development pre-release of the data-processor host contract; changelog entries stay under [Unreleased] until the final 0.37.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E228Y7uZ8QdznAjEnrXKSz
facontidavide
force-pushed
the
feat/data-processor-host-contract
branch
from
October 18, 2026 16:46
f6b84ad to
fe25418
Compare
The floor check read CHANGELOG.md, VERSION, the surface snapshot and the headers with the platform default encoding, which is cp1252 on Windows and fails on the changelog's UTF-8 text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E228Y7uZ8QdznAjEnrXKSz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Base: main (0.36.0). Replaces the draft #202 as the data-processor host contract; #202 stays open as a reference only. Shares the 0.37.0 minor with #205 (widget family): whichever merges second rebases its
VERSION,CHANGELOGand feature floors, a mechanical change.Goals
The host contract that lets a plugin create an on-demand data processor (a script evaluated at a consumer-requested time, PJ4's derived objects) and run explicit window or instant evaluations of it. Data processors only: catalog v2, plot-tab scene tabs and dialog scene views wait for a later minor.
PJ_data_processors_host_vtable_t:create_data_processor_v2,submit_evaluation,poll_evaluation,release_evaluation. Appended and guarded bystruct_size;hasTypedRequests()is true only when all four are present and non-NULL.PJ_data_processor_request_t: typed outputs,time_flags(WINDOW or INSTANT, mutually exclusive),dataset_id, window and instant times.PJ_evaluation_budget_t: time, evaluations, memory (interpreter plus native objects) and report bytes.pj_base/sdk/data_processors_host.hpp:readDataProcessorRequestandreadEvaluationBudgetread the size prefix first, then validate every count, size, pointer and flag before a host touches it. A count or size that cannot be represented is rejected before any traversal.parseEvaluationReportinpj_plugins/sdk/evaluation_report.hpp: the one report shape (coverage.{candidates, evaluated, stopped, error?, gaps}and the bundles), with int64 timestamps as JSON integers,evaluatedequal to the bundle count, no input later than itsrequested_ns, and the inferred-output type vocabulary checked.plugin_data_api.h: transform takes no time flags; markers take WINDOW oncreate_data_processor_v2(a generator clipped to the window,coverage_nsin its config) and no submit; on_demand takes INSTANT on create (a pinned finding) and WINDOW or INSTANT on submit. EPHEMERAL with INSTANT on create is rejected (an instant preview issubmit_evaluation's job). A scripted submit is ephemeral by definition; HISTORY_EXEMPT on submit is rejected. The additive config keys of on_demand (state,detail,requested_ns,lag_ns,upstream,findings) and markers (computing,error,coverage_ns) are documented.Not in this PR (no host consumer before the PJ4 slice that needs them): a
labelfield, an INFER flag (inference applies when no output is declared), typed marker scope, the non-blocking marker create flag, the marker metadata constants.How to evaluate
pj_base/include/pj_base/plugin_data_api.h(the DATA PROCESSORS section and the request/budget structs), thendata_processors_host.hppandevaluation_report.hpp.pj_plugins/tests/data_processors_api_test.cpp(decoders, wrappers, report parser, the NULL-slot and short-prefix cases) and the ABI layout sentinels;check_feature_floors.pylists the new surfaces.input_countthat overran under ASAN, now guarded.VERSIONto0.37.0-dev.1(the changelog stays under[Unreleased]), and its merge commit is taggedv0.37.0-dev.1. Pre-releasesv0.37.0-dev.N(Conan to JFrog, GitHub pre-release, no conda; tooling in ci(release): development pre-releases X.Y.Z-dev.N #206) while PJ4 builds its bridge against it;v0.37.0is cut once when PJ4's bridge has passed its walkthrough. No ABI promise between dev builds; the final tag freezes it.🤖 Generated with Claude Code
https://claude.ai/code/session_01E228Y7uZ8QdznAjEnrXKSz