[leejet]

Summary

• Add a reusable CPU PE cache that retains the latest tensor per named input and regenerates it when its key changes.
• Integrate the cache into 25 DiT implementations, including Flux, Qwen Image, Z-Image, Wan, LTX, and MiniT2I. Each graph creates inputs through the existing make_input() path.

Related Issue / Discussion

N/A

Additional Information

N/A

Checklist

• I have read and confirmed this PR follows the contribution guidelines.

Summary by CodeRabbit

• Performance

  • Positional embeddings are now reused across repeated runs when the relevant inputs and settings match, reducing redundant generation work across many image and video models.
  • Embeddings are regenerated when those inputs or settings change, preserving results for different configurations.

• Documentation

  • Added guidance on how positional-embedding caching behaves and the conditions required for cache reuse.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

The change adds a shared positional-encoding tensor cache and integrates it into diffusion model runners. Cache keys encode relevant values and tensor shapes. The runners reuse matching embeddings or generate and store them on a cache miss.

Changes: Positional-encoding cache

Layer / File(s)	Summary
Cache key and lookup src/core/pe_cache.h, src/core/pe_cache.cpp, docs/pe_cache.md	Adds PECacheKey value and shape encoding, plus PECache::get_or_create lookup and insertion behavior. The documentation describes cache keys, tensor requirements, and lifetime constraints.
Image runner integrations src/model/diffusion/boogu.hpp, src/model/diffusion/ernie_image.hpp, src/model/diffusion/flux.hpp, src/model/diffusion/krea2.hpp, src/model/diffusion/lens.hpp, src/model/diffusion/mage_flow.hpp, src/model/diffusion/ming_image.hpp, src/model/diffusion/pixart.hpp, src/model/diffusion/qwen_image.hpp, src/model/diffusion/z_image.hpp, src/model/diffusion/z_image_l2p.hpp	These runners use keyed cache lookups for positional embeddings instead of storing embedding vectors and setting up backend tensors on each graph build or computation.
Specialized embedding integrations src/model/diffusion/anima.hpp, src/model/diffusion/fibo2.hpp, src/model/diffusion/flux3.hpp, src/model/diffusion/ideogram4.hpp, src/model/diffusion/iris.hpp, src/model/diffusion/llada_image.hpp, src/model/diffusion/minit2i.hpp, src/model/diffusion/moss_image.hpp, src/model/diffusion/pid.hpp, src/model/diffusion/qwen_image_2_1.hpp	These runners cache one or more positional embeddings using keys based on their inputs and generation settings. MiniT2I also removes its runner-local backend cache and cleanup code.
Video and audio integrations src/model/diffusion/hunyuan.hpp, src/model/diffusion/lingbot_video.hpp, src/model/diffusion/ltxv.hpp, src/model/diffusion/wan.hpp	These runners use PECache for video, audio, cross-attention, or connector positional embeddings. LTXAV caches six embedding types.

Priority: Not assessed

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant Runner as Model runner
  participant PECache
  participant Builder as Embedding builder
  participant Graph as Model graph
  Runner->>PECache: get_or_create with key and builder
  PECache->>PECache: Compare name and encoded key
  PECache->>Builder: Build tensor on cache miss
  Builder-->>PECache: Return positional-encoding tensor
  PECache-->>Runner: Return cached or newly built tensor
  Runner->>Graph: Pass tensor as graph input

Loading

Possibly related PRs

• leejet/stable-diffusion.cpp#2201: Adds MOSS-Image positional-embedding reuse with a model-specific cache; this PR uses the shared PECache for the same model behavior.

Merge Risk: Minimal · up to 52c34

The reviewed cache changes are mergeable after normal checks; no actionable issue remains.

Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

🔧 Cppcheck (2.22.0)

src/pipeline/image.cpp

Cppcheck timed out; analysis of this file is incomplete

src/pipeline/diffusion_engine.cpp

Cppcheck timed out; analysis of this file is incomplete

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}