Fibo edit pipeline #1
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Introduced BriaFiboEditPipeline class with necessary backend requirements. * Updated import structures in relevant modules to include BriaFiboEditPipeline. * Ensured compatibility with existing pipelines and type checking.
* Added BriaFiboEditPipeline class for structured JSON-native image editing. * Created documentation for the new pipeline in bria_fibo_edit.md. * Updated import structures to include the new pipeline and its components. * Added unit tests for the BriaFiboEditPipeline to ensure functionality and correctness.
* Refined the Bria Fibo Edit model description for clarity and detail. * Added usage instructions for model authentication and login. * Implemented mask handling functions in the BriaFiboEditPipeline for improved image editing capabilities. * Updated unit tests to cover new mask functionalities and ensure input validation. * Adjusted example code in documentation to reflect changes in the pipeline's usage.
* Community Pipeline: Add z-image differential img2img * add pipeline for z-image differential img2img diffusion examples : run make style , make quality, and fix white spaces in example doc string. --------- Co-authored-by: r4inm4ker <[email protected]>
* fix wan 2.1 i2v context parallel * fix wan 2.1 i2v context parallel * fix wan 2.1 i2v context parallel * format
* Detect 2.0 vs 2.1 ZImageControlNetModel * Possibility of control_noise_refiner being removed
Updated CodeQL workflow to use reusable workflow from Hugging Face and simplified language matrix.
…ce#12892) * check attention mask * Apply style fixes * bugfix --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Sayak Paul <[email protected]>
…gface#12897) * [Flux.1] improve pos embed for ascend npu by setting it back to npu computation. * [Flux.2] improve pos embed for ascend npu by setting it back to npu computation. * [LongCat-Image] improve pos embed for ascend npu by setting it back to npu computation. * [Ovis-Image] improve pos embed for ascend npu by setting it back to npu computation. * Remove unused import of is_torch_npu_available --------- Co-authored-by: zhangtao <[email protected]>
* LTX Video 0.9.8 long multi prompt * Further align comfyui - Added the “LTXEulerAncestralRFScheduler” scheduler, aligned with [sample_euler_ancestral_RF](https://github.com/comfyanonymous/ComfyUI/blob/7d6103325e1c97aa54f963253e3e7f1d6da6947f/comfy/k_diffusion/sampling.py#L234) - Updated the LTXI2VLongMultiPromptPipeline.from_pretrained() method: - Now uses LTXEulerAncestralRFScheduler by default, for better compatibility with the ComfyUI LTXV workflow. - Changed the default value of cond_strength from 1.0 to 0.5, aligning with ComfyUI’s default. - Optimized cross-window overlap blending: moved the latent-space guidance injection to before the UNet and after each step, aligned with[KSamplerX0Inpaint]([ComfyUI/comfy/samplers.py at master · comfyanonymous/ComfyUI](https://github.com/comfyanonymous/ComfyUI/blob/master/comfy/samplers.py#L391)) - Adjusted the default value of skip_steps_sigma_threshold to 1. * align with diffusers contribute rule * Add new pipelines and update imports * Enhance LTXI2VLongMultiPromptPipeline with noise rescaling Refactor LTXI2VLongMultiPromptPipeline to improve documentation and add noise rescaling functionality. * Clean up comments in scheduling_ltx_euler_ancestral_rf.py Removed design notes and limitations from the implementation. * Enhance video generation example with scheduler Updated LTXI2VLongMultiPromptPipeline example to include LTXEulerAncestralRFScheduler for ComfyUI parity. * clean up * style * copies * import ltx scheduler * copies * fix * fix more * up up * up up up * up upup * Apply suggestions from code review * Update docs/source/en/api/pipelines/ltx_video.md * Update docs/source/en/api/pipelines/ltx_video.md --------- Co-authored-by: yiyixuxu <[email protected]>
* Add FSDP option for Flux2 * Apply style fixes * Add FSDP option for Flux2 * Add FSDP option for Flux2 * Add FSDP option for Flux2 * Add FSDP option for Flux2 * Add FSDP option for Flux2 * Update examples/dreambooth/README_flux2.md * guard accelerate import. --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…huggingface#12837) * feat: Add transformer cache context for conditional and unconditional predictions for skyreels-v2 pipes. * docs: Remove SkyReels-V2 FLF2V model link and add contributor attribution.
fix torchao typo.
…face#12871) * docs: add comprehensive docstrings and refine type hints for EDM scheduler methods and config parameters. * refactor: Add type hints to DPM-Solver scheduler methods.
num_frames and videos
* Initial LTX 2.0 transformer implementation * Add tests for LTX 2 transformer model * Get LTX 2 transformer tests working * Rename LTX 2 compile test class to have LTX2 * Remove RoPE debug print statements * Get LTX 2 transformer compile tests passing * Fix LTX 2 transformer shape errors * Initial script to convert LTX 2 transformer to diffusers * Add more LTX 2 transformer audio arguments * Allow LTX 2 transformer to be loaded from local path for conversion * Improve dummy inputs and add test for LTX 2 transformer consistency * Fix LTX 2 transformer bugs so consistency test passes * Initial implementation of LTX 2.0 video VAE * Explicitly specify temporal and spatial VAE scale factors when converting * Add initial LTX 2.0 video VAE tests * Add initial LTX 2.0 video VAE tests (part 2) * Get diffusers implementation on par with official LTX 2.0 video VAE implementation * Initial LTX 2.0 vocoder implementation * Use RMSNorm implementation closer to original for LTX 2.0 video VAE * start audio decoder. * init registration. * up * simplify and clean up * up * Initial LTX 2.0 text encoder implementation * Rough initial LTX 2.0 pipeline implementation * up * up * up * up * Add imports for LTX 2.0 Audio VAE * Conversion script for LTX 2.0 Audio VAE Decoder * Add Audio VAE logic to T2V pipeline * Duplicate scheduler for audio latents * Support num_videos_per_prompt for prompt embeddings * LTX 2.0 scheduler and full pipeline conversion * Add script to test full LTX2Pipeline T2V inference * Fix pipeline return bugs * Add LTX 2 text encoder and vocoder to ltx2 subdirectory __init__ * Fix more bugs in LTX2Pipeline.__call__ * Improve CPU offload support * Fix pipeline audio VAE decoding dtype bug * Fix video shape error in full pipeline test script * Get LTX 2 T2V pipeline to produce reasonable outputs * Make LTX 2.0 scheduler more consistent with original code * Fix typo when applying scheduler fix in T2V inference script * Refactor Audio VAE to be simpler and remove helpers (huggingface#7) * remove resolve causality axes stuff. * remove a bunch of helpers. * remove adjust output shape helper. * remove the use of audiolatentshape. * move normalization and patchify out of pipeline. * fix * up * up * Remove unpatchify and patchify ops before audio latents denormalization (huggingface#9) --------- Co-authored-by: dg845 <[email protected]> * Add support for I2V (huggingface#8) * start i2v. * up * up * up * up * up * remove uniform strategy code. * remove unneeded code. * Denormalize audio latents in I2V pipeline (analogous to T2V change) (huggingface#11) * test i2v. * Move Video and Audio Text Encoder Connectors to Transformer (huggingface#12) * Denormalize audio latents in I2V pipeline (analogous to T2V change) * Initial refactor to put video and audio text encoder connectors in transformer * Get LTX 2 transformer tests working after connector refactor * precompute run_connectors,. * fixes * Address review comments * Calculate RoPE double precisions freqs using torch instead of np * Further simplify LTX 2 RoPE freq calc * Make connectors a separate module (huggingface#18) * remove text_encoder.py * address yiyi's comments. * up * up * up * up --------- Co-authored-by: sayakpaul <[email protected]> * up (huggingface#19) * address initial feedback from lightricks team (huggingface#16) * cross_attn_timestep_scale_multiplier to 1000 * implement split rope type. * up * propagate rope_type to rope embed classes as well. * up * When using split RoPE, make sure that the output dtype is same as input dtype * Fix apply split RoPE shape error when reshaping x to 4D * Add export_utils file for exporting LTX 2.0 videos with audio * Tests for T2V and I2V (huggingface#6) * add ltx2 pipeline tests. * up * up * up * up * remove content * style * Denormalize audio latents in I2V pipeline (analogous to T2V change) * Initial refactor to put video and audio text encoder connectors in transformer * Get LTX 2 transformer tests working after connector refactor * up * up * i2v tests. * up * Address review comments * Calculate RoPE double precisions freqs using torch instead of np * Further simplify LTX 2 RoPE freq calc * revert unneded changes. * up * up * update to split style rope. * up --------- Co-authored-by: Daniel Gu <[email protected]> * up * use export util funcs. * Point original checkpoint to LTX 2.0 official checkpoint * Allow the I2V pipeline to accept image URLs * make style and make quality * remove function map. * remove args. * update docs. * update doc entries. * disable ltx2_consistency test * Simplify LTX 2 RoPE forward by removing coords is None logic * make style and make quality * Support LTX 2.0 audio VAE encoder * Apply suggestions from code review Co-authored-by: Sayak Paul <[email protected]> * Remove print statement in audio VAE * up * Fix bug when calculating audio RoPE coords * Ltx 2 latent upsample pipeline (huggingface#12922) * Initial implementation of LTX 2.0 latent upsampling pipeline * Add new LTX 2.0 spatial latent upsampler logic * Add test script for LTX 2.0 latent upsampling * Add option to enable VAE tiling in upsampling test script * Get latent upsampler working with video latents * Fix typo in BlurDownsample * Add latent upsample pipeline docstring and example * Remove deprecated pipeline VAE slicing/tiling methods * make style and make quality * When returning latents, return unpacked and denormalized latents for T2V and I2V * Add model_cpu_offload_seq for latent upsampling pipeline --------- Co-authored-by: Daniel Gu <[email protected]> * Fix latent upsampler filename in LTX 2 conversion script * Add latent upsample pipeline to LTX 2 docs * Add dummy objects for LTX 2 latent upsample pipeline * Set default FPS to official LTX 2 ckpt default of 24.0 * Set default CFG scale to official LTX 2 ckpt default of 4.0 * Update LTX 2 pipeline example docstrings * make style and make quality * Remove LTX 2 test scripts * Fix LTX 2 upsample pipeline example docstring * Add logic to convert and save a LTX 2 upsampling pipeline * Document LTX2VideoTransformer3DModel forward pass --------- Co-authored-by: sayakpaul <[email protected]>
huggingface#12928) docs: improve docstring scheduling_consistency_decoder.py
* Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 Co-authored-by: tcaimm <[email protected]> --------- Co-authored-by: tcaimm <[email protected]>
… LoRA Tests (huggingface#12962) * Improve incorrect LoRA format error message * Add flag in PeftLoraLoaderMixinTests to disable text encoder LoRA tests * Apply changes to LTX2LoraTests * Further improve incorrect LoRA format error msg following review --------- Co-authored-by: Sayak Paul <[email protected]>
* initial scheme of unified-sp * initial all_to_all_double * bug fixes, added cmnts * unified attention prototype done * remove raising value error in contextParallelConfig to enable unified attention * bug fix * feat: Adds Test for Unified SP Attention and Fixes a bug in Template Ring Attention * bug fix, lse calculation, testing bug fixes, lse calculation - switched to _all_to_all_single helper in _all_to_all_dim_exchange due contiguity issues bug fix bug fix bug fix * addressing comments * sequence parallelsim bug fixes * code format fixes * Apply style fixes * code formatting fix * added unified attention docs and removed test file * Apply style fixes * tip for unified attention in docs at distributed_inference.md Co-authored-by: Sayak Paul <[email protected]> * Update distributed_inference.md, adding benchmarks Co-authored-by: Sayak Paul <[email protected]> * Update docs/source/en/training/distributed_inference.md Co-authored-by: Sayak Paul <[email protected]> * function name fix * fixed benchmark in docs --------- Co-authored-by: KarthikSundar2002 <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Sayak Paul <[email protected]>
* initial * add kayers
* init * add * add 1 * Update __init__.py * rename * 2 * update * init with encoder * merge2pipeline * Update pipeline_glm_image.py * remove sop * remove useless func * Update pipeline_glm_image.py * up (cherry picked from commit cfe19a3) * review for work only * change place * Update pipeline_glm_image.py * update * Update transformer_glm_image.py * 1 * no negative_prompt for GLM-Image * remove CogView4LoraLoaderMixin * refactor attention processor. * update * fix * use staticmethod * update * up * up * update * Update glm_image.md * 1 * Update pipeline_glm_image.py * Update transformer_glm_image.py * using new transformers impl * support * resolution change * fix-copies * Update src/diffusers/pipelines/glm_image/pipeline_glm_image.py Co-authored-by: YiYi Xu <[email protected]> * Update pipeline_glm_image.py * use cogview4 * Update pipeline_glm_image.py * Update pipeline_glm_image.py * revert * update * batch support * update * version guard glm image pipeline * validate prompt_embeds and prior_token_ids * try docs. * 4 * up * up * skip properly * fix tests * up * up --------- Co-authored-by: zRzRzRzRzRzRzR <[email protected]> Co-authored-by: yiyixuxu <[email protected]>
…ingface#12974) * make transformers version check stricter for glm image. * public checkpoint.
* allow to * update version * fix version again * again * Update src/diffusers/pipelines/pipeline_utils.py Co-authored-by: Copilot <[email protected]> * style * xfail * add pr --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Sayak Paul <[email protected]>
* update * `disable_mmap` in `from_pretrained` --------- Co-authored-by: DN6 <[email protected]>
* up * style --------- Co-authored-by: [email protected] <[email protected]>
… component (huggingface#12963) * Don't attempt to move the text_encoder. Just move the generated_ids. * The inputs to the text_encoder should be on its device
* Add `ChromaInpaintPipeline` * Set `attention_mask` to `dtype=torch.bool` for `ChromaInpaintPipeline`. * Revert `.gitignore`.
* fix qwen-image cp * relax attn_mask limit for cp * CP plan compatible with zero_cond_t * move modulate_index plan to top level
* flux2-klein * Apply suggestions from code review Co-authored-by: Sayak Paul <[email protected]> * Klein tests (huggingface#2) * tests * up * tests * up * support step-distilled * Apply suggestions from code review Co-authored-by: dg845 <[email protected]> * Apply suggestions from code review Co-authored-by: dg845 <[email protected]> * doc string etc * style * more * copies * klein lora training scripts (huggingface#3) * initial commit * initial commit * remove remote text encoder * initial commit * initial commit * initial commit * revert * img2img fix * text encoder + tokenizer * text encoder + tokenizer * update readme * guidance * guidance * guidance * test * test * revert changes not needed for the non klein model * Update examples/dreambooth/train_dreambooth_lora_flux2_klein.py Co-authored-by: Sayak Paul <[email protected]> * fix guidance * fix validation * fix validation * fix validation * fix path * space --------- Co-authored-by: Sayak Paul <[email protected]> * style * Update src/diffusers/pipelines/flux2/pipeline_flux2_klein.py * Apply style fixes * auto pipeline --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: dg845 <[email protected]> Co-authored-by: Linoy Tsaban <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…#12980) * update mellonparams docstring to incude the acutal param definition render in mellon * style --------- Co-authored-by: [email protected] <[email protected]>
* LTX 2 transformer single file support * LTX 2 video VAE single file support * LTX 2 audio VAE single file support * Make it easier to distinguish LTX 1 and 2 models
…ted. (huggingface#12832) * gracefully error out when attn-backend x cp combo isn't supported. * Revert "gracefully error out when attn-backend x cp combo isn't supported." This reverts commit c8abb5d. * gracefully error out when attn-backend x cp combo isn't supported. * up * address PR feedback. * up * Update src/diffusers/models/modeling_utils.py Co-authored-by: Dhruv Nair <[email protected]> * dot. --------- Co-authored-by: Dhruv Nair <[email protected]>
…istep.py (huggingface#12936) * docs: improve docstring scheduling_cosine_dpmsolver_multistep.py * Update src/diffusers/schedulers/scheduling_cosine_dpmsolver_multistep.py Co-authored-by: Steven Liu <[email protected]> * Update src/diffusers/schedulers/scheduling_cosine_dpmsolver_multistep.py Co-authored-by: Steven Liu <[email protected]> * fix --------- Co-authored-by: Steven Liu <[email protected]>
…ingface#12986) Chore: Replace CONTRIBUTING.md with a symlink to documentation
The pipeline was ignoring the generator parameter and only using the seed parameter. This caused non-deterministic outputs in tests that pass a seeded generator.
This reverts commit 76f51a5.
make style to push new changes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes # (issue)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.