feat(jev): add TypeSafe decision integration by waleedlatif1 · Pull Request #8103 · simstudioai/sim

waleedlatif1 · 2026-09-21T08:07:18Z

Summary

Add native Jev Choice, Score, Noul, and batch evaluation tools for both Agent tools and explicit workflow steps. Jev uses TypeSafe's System One decision API.
Validate requests and responses, project model-bound secrets through the shared executor, and retain probabilities, confidence, and token usage.
Add the official TypeSafe SVG and brand color, JSON-generation helpers, workflow templates, suggested skills, and generated catalog/docs metadata.

Type of Change

New feature

Testing

273 Vitest tests pass across the Jev tools, workflow handler, Agent adapter, provider utilities, and shared JSON sanitizer. Agent coverage includes all four operations, configured and model-supplied parameters, credentials, unfilled text inputs, structured runtime references, and the Score rubric schema.
App type-check, lint, all 47 audits, block registry, canvas sentence validation, and docs manifest checks pass after rebasing onto staging.
Visually checked the official SVG in light, dark, and brand-tile contexts.
13 prior live API checks passed through the shared tool executor and workflow block handler: all four operations, structured state/instructions/rubrics, object and serialized batches, pinned/latest/preview models, and invalid-key handling.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-09-21T08:07:22Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 22, 2026 6:51am UTC

waleedlatif1 · 2026-09-21T08:07:53Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-21T08:07:57Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

greptile-apps · 2026-09-21T08:11:45Z

Confidence Score: 5/5

The PR appears safe to merge; no outstanding correctness or security failure remains.

Summary

Adds TypeSafe Jev as a native decision integration for agents and workflows.

Introduces Choice, Score, Noul, and batch evaluation tools with request and response validation.
Registers the Jev block, credentials, metadata, documentation, icon, templates, and suggested skills.
Preserves model information, confidence, probability distributions, and token usage in tool outputs.
Adds coverage for direct tools, workflow execution, agent adapters, structured inputs, and invalid responses.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Agent or workflow block] --> B{Jev operation}
    B --> C[Choice]
    B --> D[Score]
    B --> E[Noul]
    B --> F[Batch Evaluate]
    C --> G[Shared Jev executor]
    D --> G
    E --> G
    F --> G
    G --> H[TypeSafe System One API]
    H --> I[Validate response]
    I --> J[Choice / score / probability results]
    I --> K[Model and token usage]

_{Reviews (8) · Last reviewed commit: "fix(jev): expose text inputs to Agent mo..."}

cubic-dev-ai

All reported issues were addressed across 25 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-21T08:22:20Z

@greptile

waleedlatif1 · 2026-09-21T08:22:21Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-21T08:22:27Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 25 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

waleedlatif1 · 2026-09-21T16:01:17Z

@greptile

waleedlatif1 · 2026-09-21T16:01:18Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-21T16:01:25Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 25 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

waleedlatif1 · 2026-09-21T16:09:54Z

@greptile

waleedlatif1 · 2026-09-21T16:09:55Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-21T16:10:03Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 25 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-21T16:17:56Z

@greptile

waleedlatif1 · 2026-09-21T16:17:57Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-21T16:18:04Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 25 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-22T06:39:45Z

@greptile

waleedlatif1 · 2026-09-22T06:39:46Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-22T06:39:51Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 26 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic}

waleedlatif1 · 2026-09-22T06:50:09Z

@greptile

waleedlatif1 · 2026-09-22T06:50:11Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-22T06:50:16Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 26 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic}

waleedlatif1 · 2026-09-22T06:57:23Z

@greptile

waleedlatif1 · 2026-09-22T06:57:24Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-22T06:57:30Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 26 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic}

vercel Bot deployed to Preview September 21, 2026 08:09 View deployment

greptile-apps Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread apps/sim/tools/jev/evaluate.ts Outdated

cubic-dev-ai Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread apps/sim/tools/jev/choice.ts

Comment thread apps/sim/tools/jev/evaluate.ts Outdated

Comment thread apps/sim/tools/jev/shared.ts

Comment thread apps/sim/tools/jev/shared.ts

vercel Bot temporarily deployed to Preview September 21, 2026 08:21 Inactive

cubic-dev-ai Bot reviewed Sep 21, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview September 21, 2026 16:01 Inactive

cubic-dev-ai Bot reviewed Sep 21, 2026

View reviewed changes

greptile-apps Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread apps/sim/blocks/blocks/jev.ts

vercel Bot temporarily deployed to Preview September 21, 2026 16:09 Inactive

cubic-dev-ai Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread apps/sim/blocks/blocks/jev.ts Outdated

vercel Bot temporarily deployed to Preview September 21, 2026 16:17 Inactive

cubic-dev-ai Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread apps/sim/blocks/blocks/jev.ts

waleedlatif1 mentioned this pull request Sep 21, 2026

feat(typesafe): add Jev decision integration #8119

Closed

10 tasks

waleedlatif1 force-pushed the codex/jev-integration branch from b3fc80d to 9cce6a8 Compare September 22, 2026 06:39

vercel Bot deployed to Preview September 22, 2026 06:41 View deployment

cubic-dev-ai Bot reviewed Sep 22, 2026

View reviewed changes

Comment thread apps/sim/tools/jev/score.ts

waleedlatif1 added 7 commits September 21, 2026 23:47

feat(jev): add TypeSafe decision integration

75cecd7

fix(jev): validate batch answers against requested questions

64864b9

improvement(jev): refine criteria helpers and suggested skills

9bea4ab

fix(jev): sanitize generated JSON inputs

35b8a66

fix(jev): clarify empty Noul canvas sentence

c730bc5

fix(jev): preserve Agent tool criteria and rubric schema

9f0e689

fix(jev): expose text inputs to Agent models

0e794b0

waleedlatif1 force-pushed the codex/jev-integration branch from 9cce6a8 to 0e794b0 Compare September 22, 2026 06:49

vercel Bot deployed to Preview September 22, 2026 06:51 View deployment

cubic-dev-ai Bot reviewed Sep 22, 2026

View reviewed changes

greptile-apps Bot reviewed Sep 22, 2026

View reviewed changes

Comment thread apps/sim/components/icons.tsx

cubic-dev-ai Bot reviewed Sep 22, 2026

View reviewed changes

Conversation

waleedlatif1 commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 21, 2026

Uh oh!

greptile-apps Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Sep 22, 2026

Uh oh!

waleedlatif1 commented Sep 22, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 22, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Sep 21, 2026 •

edited

Loading

vercel Bot commented Sep 21, 2026 •

edited

Loading

greptile-apps Bot commented Sep 21, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading