Skip to content

Commit 9283e1b

Browse files
feat(render): export UniMate constraint manifests (#139)
1 parent 0f389fb commit 9283e1b

7 files changed

Lines changed: 545 additions & 0 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"posecode-render": minor
3+
---
4+
5+
Add a deterministic UniMate constraint manifest exporter for sparse key-pose in-betweening workflows.

‎docs/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,5 +27,6 @@ The repository root retains the canonical [`LICENSE`](../LICENSE) and [`NOTICE`]
2727
## Development references
2828

2929
- [Product usage analytics](product-analytics.md)
30+
- [Posecode × UniMate sparse key-pose bridge](integrations/unimate.md)
3031
- [Vercel agent notes](development/VERCEL_AGENTS.md)
3132
- [Pose diagnostics summary](diagnostics/pose-summary.json)

‎docs/integrations/unimate.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Posecode × UniMate: sparse key-pose bridge
2+
3+
**Status:** implementable interchange boundary, with the rig-aware UniMate adapter proposed as the next joint step.
4+
5+
Posecode should remain the editable source of truth for named phases, sparse key poses, root intent, and contact constraints. UniMate should generate only the motion between those authored anchors. This avoids treating a generated 60 FPS clip as the primary artifact and directly matches UniMate's existing `x1_known` / `keep_mask` replacement path.
6+
7+
```mermaid
8+
flowchart LR
9+
A[Posecode document] --> B[constraint manifest v1]
10+
B --> C[rig mapping and FK]
11+
C --> D[UniMate pose encoder]
12+
D --> E[x1_known + keep_mask]
13+
E --> F[UniMate in-betweening]
14+
F --> G[contact and ROM diagnostics]
15+
G --> H[editable BVH / GLB / Blender Action]
16+
```
17+
18+
## Implemented boundary
19+
20+
`buildUniMateConstraintManifest(ir, { fps: 30 })` now emits a deterministic JSON-safe schedule with:
21+
22+
- one-based contiguous prompt clips compatible with Blender timelines;
23+
- the start pose and every phase endpoint as a named key-pose reference;
24+
- carried local Euler channels, root travel and facing intent;
25+
- phase-range `ground-lock`, `reach`, `pin`, and `grip` constraints;
26+
- display-only cues kept separate from model prompts;
27+
- Posecode-to-Mixamo bone bindings; and
28+
- an explicit mask contract: position and rotation are known at reference frames, velocity remains generative.
29+
30+
The manifest intentionally does **not** claim to be UniMate's normalized `(T, J, 12)` tensor. That final encoding requires the chosen rig's rest hierarchy, global FK positions, and `dataset_stats.npy` from the selected checkpoint. The adapter must compute those values next to UniMate, where that context exists.
31+
32+
## Smallest useful joint prototype
33+
34+
1. Add a manifest importer to [UniMate-B3D](https://github.com/nopeburger/UniMate-B3D).
35+
2. Map each Posecode semantic bone to the selected Blender deform rig, apply every reference pose, then reuse B3D's `encode_pose` path.
36+
3. Build `x1_known` and `keep_mask` with channels `0:9` fixed at reference frames and channels `9:12` left free, matching B3D's current pose-reference behavior.
37+
4. Generate the gaps, then retain Posecode keyframes as hard anchors during retiming.
38+
5. Return an editable Blender Action plus machine-readable diagnostics.
39+
40+
## Evaluation
41+
42+
Use the released 22-joint Mixamo checkpoint first, with 10 to 20 short movements containing two to five key poses. Compare text-only UniMate, Posecode interpolation, and the combined system on:
43+
44+
| Measure | Target |
45+
| --- | --- |
46+
| Authored key-pose rotation error | effectively zero at pinned frames |
47+
| Root endpoint error | under 2 cm |
48+
| Planted-foot drift | under 2 cm during declared contacts |
49+
| ROM violations | none after validation/post-process |
50+
| Animator correction effort | fewer key edits than either baseline |
51+
52+
The most informative failure set is also small: quadruped rigs that rotate upright, rare topologies, opposing contact constraints, and phase references placed too close for a 60-frame generation window.
53+
54+
## Contribution split and licensing
55+
56+
- **Posecode:** manifest producer, Mixamo binding profile, fixtures, browser demo, contact/ROM diagnostics, and evaluation report.
57+
- **UniMate / UniMate-B3D:** rig-specific FK and pose encoding, checkpoint normalization, generation, and Blender Action creation.
58+
- **Shared:** benchmark clips, failure taxonomy, and a short technical report if results warrant it.
59+
60+
The new Posecode adapter is part of `posecode-render` and therefore AGPL-3.0-only. UniMate's repository code is MIT; UniMate-B3D is GPL-3.0-or-later. The released UniMate weights are CC BY-NC 4.0, so model-backed commercial use is a separate question from code reuse and must not be represented as MIT-covered.
61+
62+
## Acceptance test
63+
64+
A bridge is complete when four Posecode-authored Mixamo key poses can be imported, used as fixed UniMate references, generated between, and exported as a Blender Action while preserving all four anchors and reporting contact/ROM residuals.

‎packages/posecode-render/README.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,24 @@ bounded self-collision pairs. They measure the post-solver procedural driver,
7373
before optional skinned-character or mocap surface reconciliation, and report
7474
outcomes without changing the authored motion.
7575

76+
## UniMate constraint interchange
77+
78+
Use `buildUniMateConstraintManifest()` to turn a validated IR into a JSON-safe
79+
sparse key-pose schedule for a rig-aware UniMate adapter:
80+
81+
```ts
82+
import { buildUniMateConstraintManifest } from "posecode-render";
83+
84+
const manifest = buildUniMateConstraintManifest(ir, { fps: 30 });
85+
```
86+
87+
The manifest preserves named phase endpoints, root intent, Mixamo bone
88+
bindings, and contact constraints. It deliberately stops before UniMate's
89+
normalized motion tensor because that conversion requires the destination
90+
rig's rest geometry and the selected checkpoint's normalization statistics.
91+
See the [integration note](https://github.com/posecode-dev/posecode/blob/main/docs/integrations/unimate.md)
92+
for the bridge boundary and acceptance test.
93+
7694
No GPU, no diffusion model: generation is a fraction of a cent of text, and
7795
rendering is plain forward kinematics.
7896

‎packages/posecode-render/src/index.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1684,3 +1684,14 @@ export {
16841684
export type { PhaseSegment } from "./timeline.js";
16851685
export { exportBVH, type BvhExportOptions } from "./bvh.js";
16861686
export { exportGLTF, buildAnimatedRig, type GltfExportOptions } from "./gltf.js";
1687+
export {
1688+
UNIMATE_CONSTRAINT_SCHEMA,
1689+
POSECODE_MIXAMO_BINDINGS,
1690+
buildUniMateConstraintManifest,
1691+
type UniMateBoneBinding,
1692+
type UniMateConstraintKeyframe,
1693+
type UniMateConstraintManifest,
1694+
type UniMateConstraintOptions,
1695+
type UniMateConstraintSet,
1696+
type UniMatePromptClip,
1697+
} from "./unimate.js";

0 commit comments

Comments
 (0)