Repository navigation
feat: add Fibo 2.0 Dev and Edit Turbo support #2180
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| # How to Use | ||
|
|
||
| Fibo 2.0 uses a Fibo diffusion transformer, the standard FLUX.2 VAE, and Qwen3-VL 4B as the LLM text encoder. The Qwen tokenizer is built in. | ||
|
|
||
| ## Download weights | ||
|
|
||
| - Download Fibo 2.0 Dev | ||
| - safetensors: https://huggingface.co/Comfy-Org/Fibo-2.0/tree/main/diffusion_models | ||
| - Download Fibo 2.0 Edit Turbo | ||
| - safetensors: https://huggingface.co/Comfy-Org/Fibo-2.0/tree/main/diffusion_models | ||
| - Download vae | ||
| - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-dev/tree/main | ||
| - Download Qwen3-VL 4B | ||
| - safetensors: https://huggingface.co/Comfy-Org/Fibo-2.0/tree/main/text_encoders | ||
| - gguf: https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct-GGUF/tree/main | ||
|
|
||
| The original Diffusers weights from [briaai/Fibo-2.0-dev](https://huggingface.co/briaai/Fibo-2.0-dev) and [briaai/Fibo-2.0-edit-turbo](https://huggingface.co/briaai/Fibo-2.0-edit-turbo) are also supported. Pass the transformer's `diffusion_pytorch_model.safetensors.index.json` to `--diffusion-model` and keep its shards alongside it. | ||
|
|
||
| INT8 convrot weights load directly; see [INT8 convrot](./int8_convrot.md) for backend requirements. | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Fibo 2.0 Dev | ||
|
|
||
| Dev supports both text-to-image generation and reference-image editing. | ||
|
|
||
| ``` | ||
| .\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\fibo2_dev_int8_convrot.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --llm_vision ..\models\text_encoders\Qwen3-VL-4B-Instruct-mmproj-BF16.gguf --vae ..\models\vae\flux2_ae.safetensors -p "a lovely cat holding a sign says 'fibo2.cpp'" --cfg-scale 5.0 --sampling-method euler -v --offload-to-cpu | ||
| ``` | ||
|
|
||
| <img src="../assets/fibo2/t2i.png" /> | ||
|
There was a problem hiding this comment. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Add descriptive alternate text to both example images. Both Also applies to: 37-37 🧰 Tools🪛 markdownlint-cli2 (0.23.3)[warning] 31-31: Images should have alternate text (alt text) (MD045, no-alt-text) 🤖 Prompt for AI AgentsSource: Linters/SAST tools |
||
|
|
||
| ``` | ||
| .\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\fibo2_dev_int8_convrot.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --llm_vision ..\models\text_encoders\Qwen3-VL-4B-Instruct-mmproj-BF16.gguf --vae ..\models\vae\flux2_ae.safetensors -r ..\assets\flux\flux1-dev-q8_0.png -p "change 'flux.cpp' to 'fibo2.cpp'" --cfg-scale 5.0 --sampling-method euler -v --offload-to-cpu | ||
| ``` | ||
|
|
||
| <img src="../assets/fibo2/dev_edit.png" /> | ||
|
|
||
| ### Fibo 2.0 Edit Turbo | ||
|
|
||
| Edit Turbo is distilled for reference-image editing in 4 steps with CFG 1. | ||
|
|
||
| ``` | ||
| .\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\fibo2_edit_turbo_int8_convrot.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --llm_vision ..\models\text_encoders\Qwen3-VL-4B-Instruct-mmproj-BF16.gguf --vae ..\models\vae\flux2_ae.safetensors -r ..\assets\flux\flux1-dev-q8_0.png -p "change 'flux.cpp' to 'fibo2.cpp'" --cfg-scale 1 --steps 4 --sampling-method euler -v --offload-to-cpu | ||
| ``` | ||
|
|
||
| The BF16 files accept the same options. Use dimensions divisible by 16. Repeat `-r` to use multiple reference images; instructions can refer to `<image_1>`, `<image_2>`, etc. See [Image Editing](./edit.md) for reference-image preprocessing options. Separate GGUF text encoders need their vision projector passed with `--llm_vision` for editing. | ||
|
|
||
| The automatic `fibo2` reference preset combines `resize_single_ref_to_target=true`, `vae_size_buckets=true`, `vae_center_crop=true`, and `vlm_use_vae_input=true`. A single reference is scaled to cover the output dimensions and center-cropped. Multiple references use dimensions divisible by 16, chosen within ±10% of one megapixel to minimize cropping for each image's aspect ratio; `vae_input_max_pixels` overrides that target area. The VAE and vision encoder receive the same crops, with smaller copies used by the vision encoder. These are generic reference-image options and can be overridden individually: for example, `--ref-image-args vae_center_crop=false` uses direct resizing, while `resize_before_vae=false` skips this automatic crop and resize step entirely. | ||
|
|
||
| The vision encoder defaults to `vlm_resize_mode=longest_side,vlm_max_size=768`. Override these with `--ref-image-args`, for example `vlm_max_size=512`, or use `vlm_resize_mode=none` to skip size-based scaling. Vision inputs are still aligned to the encoder's patch grid. | ||
|
|
||
| Plain prompts and structured JSON captions are accepted. This integration uses the caption-only text-rendering path (`render_text_images=False`); it does not generate auxiliary glyph images or expand prompts with Bria VGL. Native Fibo mask preprocessing is not connected to `--mask`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| { | ||
| "schema_version": 1, | ||
| "id": "fibo2-dev", | ||
| "name": "Fibo 2.0 Dev", | ||
| "description": "Text-to-image generation and native reference-image editing. Select a separate vision projector for editing when it is not bundled in the text encoder.", | ||
| "model_url": "https://huggingface.co/briaai/Fibo-2.0-dev", | ||
| "slots": [ | ||
| { | ||
| "id": "diffusion_model", | ||
| "required": true | ||
| }, | ||
| { | ||
| "id": "llm", | ||
| "required": true, | ||
| "label": "Qwen 3 VL 4B Text Encoder" | ||
| }, | ||
| { | ||
| "id": "llm_vision", | ||
| "required": false, | ||
| "label": "Vision projector (required for separate GGUF text encoders)" | ||
| }, | ||
| { | ||
| "id": "vae", | ||
| "required": true, | ||
| "label": "FLUX.2 VAE" | ||
| } | ||
| ], | ||
| "defaults": { | ||
| "sampling_method": "euler", | ||
| "cfg_scale": 5, | ||
| "steps": 20, | ||
| "width": 1024, | ||
| "height": 1024 | ||
| }, | ||
| "features": [ | ||
| "flow_shift", | ||
| "ref_images" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| { | ||
| "schema_version": 1, | ||
| "id": "fibo2-edit-turbo", | ||
| "name": "Fibo 2.0 Edit Turbo", | ||
| "description": "Dedicated reference-image editing checkpoint; supply reference images. Select a separate vision projector when it is not bundled in the text encoder.", | ||
| "model_url": "https://huggingface.co/briaai/Fibo-2.0-edit-turbo", | ||
| "slots": [ | ||
| { | ||
| "id": "diffusion_model", | ||
| "required": true | ||
| }, | ||
| { | ||
| "id": "llm", | ||
| "required": true, | ||
| "label": "Qwen 3 VL 4B Text Encoder" | ||
| }, | ||
| { | ||
| "id": "llm_vision", | ||
| "required": false, | ||
| "label": "Vision projector (required for separate GGUF text encoders)" | ||
| }, | ||
| { | ||
| "id": "vae", | ||
| "required": true, | ||
| "label": "FLUX.2 VAE" | ||
| } | ||
| ], | ||
| "defaults": { | ||
| "sampling_method": "euler", | ||
| "cfg_scale": 1, | ||
| "steps": 4, | ||
| "width": 1024, | ||
| "height": 1024 | ||
| }, | ||
| "features": [ | ||
| "flow_shift", | ||
| "ref_images" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -158,22 +158,22 @@ static bool build_openai_edit_request(const httplib::Request& req, | |
| request.gen_params.height = height; | ||
| request.gen_params.batch_count = n; | ||
|
|
||
| std::string sd_cpp_extra_args_str = extract_and_remove_sd_cpp_extra_args(request.gen_params.prompt); | ||
| const json extra_args = sd_cpp_extra_args_str.empty() | ||
| ? json::object() | ||
| : json::parse(sd_cpp_extra_args_str, nullptr, false); | ||
| const bool has_extra_init_image_override = extra_args.is_object() && extra_args.contains("init_image"); | ||
| const bool has_extra_ref_images_override = extra_args.is_object() && extra_args.contains("ref_images"); | ||
| const bool has_extra_strength_override = extra_args.is_object() && | ||
| extra_args.contains("strength") && | ||
| extra_args["strength"].is_number(); | ||
| const bool has_extra_mask_image = extra_args.is_object() && | ||
| extra_args.contains("mask_image") && | ||
| !extra_args["mask_image"].is_null(); | ||
| const bool supports_ref_image_conditioning = sd_ctx_supports_ref_image_conditioning(runtime.sd_ctx); | ||
| const bool may_need_uploaded_init_image = !has_extra_init_image_override && | ||
| (!supports_ref_image_conditioning || !mask_bytes.empty() || | ||
| has_extra_mask_image || has_extra_strength_override || has_extra_ref_images_override); | ||
| SDImageOwner uploaded_image_for_init; | ||
| bool uploaded_image_was_first_reference = false; | ||
| for (auto& bytes : images_bytes) { | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Point the editing link to an existing heading.
docs/fibo2.mdhas no#editing-and-turboheading, so this link does not jump to the editing instructions. Link to the existing Edit Turbo heading instead.Proposed change
📝 Committable suggestion
🤖 Prompt for AI Agents