Add sdxl lightning quant use #992
Conversation
|
Warning Rate limit exceeded@lixiang007666 has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 9 minutes and 23 seconds before requesting another review. How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. WalkthroughThe recent changes introduce comprehensive documentation and new functionalities for using SDXL-Lightning with OneDiff. Key enhancements include a structured README for setup and execution, a command-line interface for image generation, performance monitoring tools, and updates to quantization scripts. These improvements aim to optimize model performance, facilitate user experience, and support advanced configurations, ultimately enhancing the utility of the Stable Diffusion XL model. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI
participant Model
participant Quantizer
User->>CLI: Run image generation command
CLI->>Model: Load specified model
Model-->>CLI: Return model instance
CLI->>Quantizer: Optional quantization
Quantizer-->>CLI: Return quantized model
CLI->>User: Save generated image
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Outside diff range, codebase verification and nitpick comments (4)
onediff_diffusers_extensions/examples/lightning/README.md (4)
20-23: Use markdown links instead of bare URLs.The setup instructions contain bare URLs. Consider using markdown links for better readability and consistency with markdown standards.
- Follow the instructions to set up OneDiff from the https://github.com/siliconflow/onediff?tab=readme-ov-file#installation. + Follow the instructions to set up OneDiff from the [OneDiff Installation Guide](https://github.com/siliconflow/onediff?tab=readme-ov-file#installation). - Follow the setup instructions for these backends from the https://github.com/siliconflow/onediff?tab=readme-ov-file#install-a-compiler-backend. + Follow the setup instructions for these backends from the [Compiler Backend Setup Guide](https://github.com/siliconflow/onediff?tab=readme-ov-file#install-a-compiler-backend).Tools
Markdownlint
20-20: null
Bare URL used(MD034, no-bare-urls)
23-23: null
Bare URL used(MD034, no-bare-urls)
32-33: Correct grammatical number in the note.The phrase "an 8 steps distillation model" should be corrected for grammatical consistency.
- Current test is based on an 8 steps distillation model. + Current test is based on an 8-step distillation model.Tools
LanguageTool
[uncategorized] ~33-~33: The grammatical number of this noun doesn’t look right. Consider replacing it.
Context: ...> [!NOTE] Current test is based on an 8 steps distillation model. ### Run 1024x1024 ...(AI_EN_LECTOR_REPLACEMENT_NOUN_NUMBER)
68-68: Use markdown links instead of bare URLs.The quantization instructions contain a bare URL. Consider using markdown links for better readability and consistency with markdown standards.
- For an introduction to the quantization parameters, refer to: https://github.com/siliconflow/onediff/blob/main/README_ENTERPRISE.md#diffusers-with-onediff-enterprise + For an introduction to the quantization parameters, refer to: [Quantization Parameters Guide](https://github.com/siliconflow/onediff/blob/main/README_ENTERPRISE.md#diffusers-with-onediff-enterprise)Tools
Markdownlint
68-68: null
Bare URL used(MD034, no-bare-urls)
122-122: Use markdown links instead of bare URLs.The quality section contains a bare URL. Consider using markdown links for better readability and consistency with markdown standards.
- https://github.com/siliconflow/odeval/tree/main/models/lightning + [Quality Evaluation Guide](https://github.com/siliconflow/odeval/tree/main/models/lightning)Tools
Markdownlint
122-122: null
Bare URL used(MD034, no-bare-urls)
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (5)
- onediff_diffusers_extensions/examples/lightning/README.md (1 hunks)
- onediff_diffusers_extensions/examples/lightning/text_to_image_sdxl_light.py (1 hunks)
- onediff_diffusers_extensions/examples/save_and_load_pipeline.sh (1 hunks)
- onediff_diffusers_extensions/onediffx/utils/performance_monitor.py (1 hunks)
- onediff_diffusers_extensions/tools/quantization/quantize-sd-fast.py (3 hunks)
Files skipped from review due to trivial changes (1)
- onediff_diffusers_extensions/examples/save_and_load_pipeline.sh
Additional context used
LanguageTool
onediff_diffusers_extensions/examples/lightning/README.md
[uncategorized] ~33-~33: The grammatical number of this noun doesn’t look right. Consider replacing it.
Context: ...> [!NOTE] Current test is based on an 8 steps distillation model. ### Run 1024x1024 ...(AI_EN_LECTOR_REPLACEMENT_NOUN_NUMBER)
Markdownlint
onediff_diffusers_extensions/examples/lightning/README.md
20-20: null
Bare URL used(MD034, no-bare-urls)
23-23: null
Bare URL used(MD034, no-bare-urls)
68-68: null
Bare URL used(MD034, no-bare-urls)
122-122: null
Bare URL used(MD034, no-bare-urls)
8-8: null
Link fragments should be valid(MD051, link-fragments)
9-9: null
Link fragments should be valid(MD051, link-fragments)
10-10: null
Link fragments should be valid(MD051, link-fragments)
11-11: null
Link fragments should be valid(MD051, link-fragments)
12-12: null
Link fragments should be valid(MD051, link-fragments)
13-13: null
Link fragments should be valid(MD051, link-fragments)
Ruff
onediff_diffusers_extensions/examples/lightning/text_to_image_sdxl_light.py
4-4:
timeimported but unusedRemove unused import:
time(F401)
14-14: Undefined name
diffusers(F821)
15-15: Local variable
eis assigned to but never usedRemove assignment to unused variable
e(F841)
Additional comments not posted (10)
onediff_diffusers_extensions/onediffx/utils/performance_monitor.py (1)
5-33: LGTM: Well-structured context manager for inference timing.The context manager
track_inference_timeis correctly implemented to handle both CUDA and non-CUDA scenarios. The use oftorch.cuda.Eventfor timing when CUDA is available is appropriate. The documentation is clear and concise.onediff_diffusers_extensions/tools/quantization/quantize-sd-fast.py (2)
69-80: LGTM: New command-line arguments for SDXL Lightning are well-integrated.The new arguments
--use_lightningand--lightning_ckptare correctly added with appropriate types and default values. The lambda function for boolean parsing is used effectively, and the documentation is clear.
120-134: LGTM: Conditional logic for SDXL Lightning model loading is correctly implemented.The conditional block properly checks the
args.use_lightningflag and loads the model using the specified checkpoint. The pipeline setup is consistent with the existing logic.onediff_diffusers_extensions/examples/lightning/README.md (1)
106-119: LGTM: Performance comparison section is well-structured.The performance comparison provides clear and useful metrics, enhancing the documentation's value.
onediff_diffusers_extensions/examples/lightning/text_to_image_sdxl_light.py (6)
18-57: Argument parsing is well-structured.The argument parser is comprehensive and covers various configuration options for the model.
170-194: Verify output image path.Ensure that the path for saving the output image is valid and accessible.
Verification successful
Output image path is valid and writable.
The directory for saving the output image is confirmed to be writable, ensuring that the image can be saved without any issues. No further action is needed.
Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the output image path. # Test: Check if the directory for saving the output image is writable. test -w "$(dirname 'sdxl-light-out.png')" && echo "Directory is writable."Length of output: 96
83-116: Verify calibration info file.Ensure that the
calibrate_info.txtfile exists and contains the correct format and data for quantization.
143-168: Verify backend configurations.Ensure that the backend configurations for
oneflowandnexfortare correctly handled and that the necessary options are provided.Verification successful
Backend configurations are correctly handled.
The
--compiler-configand--quantize-configarguments are present in the script, ensuring that the necessary configurations for thenexfortbackend are properly managed through command-line arguments. No issues found regarding the backend configurations.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify backend configurations for compilation. # Test: Check for the presence of necessary configurations in the argument parsing section. rg --type python 'compiler-config|quantize-config'Length of output: 83
Script:
#!/bin/bash # Description: Verify backend configurations for compilation. # Test: Check for the presence of necessary configurations in the argument parsing section. rg --type py 'compiler-config|quantize-config'Length of output: 1220
65-80: Verify file path handling and device allocation.Ensure that the file paths for model loading are correct and that the device allocation (
to("cuda")) is appropriate for the environment.
196-198: Verify graph saving directory.Ensure that the directory specified for saving graphs is valid and accessible.
| import argparse | ||
| import json | ||
| import os | ||
| import time |
There was a problem hiding this comment.
Remove unused import: time.
The time module is imported but not used in the code.
-import timeCommittable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import time |
Tools
Ruff
4-4:
timeimported but unusedRemove unused import:
time(F401)
|
|
||
| try: | ||
| USE_PEFT_BACKEND = diffusers.utils.USE_PEFT_BACKEND | ||
| except Exception as e: |
There was a problem hiding this comment.
Remove unused variable e.
The exception variable e is assigned but never used. Consider removing it.
-except Exception as e:
+except Exception:Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| except Exception as e: | |
| except Exception: |
Tools
Ruff
15-15: Local variable
eis assigned to but never usedRemove assignment to unused variable
e(F841)
| from safetensors.torch import load_file | ||
|
|
||
| try: | ||
| USE_PEFT_BACKEND = diffusers.utils.USE_PEFT_BACKEND |
There was a problem hiding this comment.
Fix undefined name: diffusers.
The variable diffusers is used without being defined or imported, causing a potential error.
- USE_PEFT_BACKEND = diffusers.utils.USE_PEFT_BACKEND
+ from diffusers import utils
+ USE_PEFT_BACKEND = utils.USE_PEFT_BACKENDCommittable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| USE_PEFT_BACKEND = diffusers.utils.USE_PEFT_BACKEND | |
| from diffusers import utils | |
| USE_PEFT_BACKEND = utils.USE_PEFT_BACKEND |
Tools
Ruff
14-14: Undefined name
diffusers(F821)
This PR is done:
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Chores