Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error with --half and onnx-low-memory #382

Open
HoopyFreud opened this issue Jun 15, 2023 · 1 comment
Open

Error with --half and onnx-low-memory #382

HoopyFreud opened this issue Jun 15, 2023 · 1 comment
Labels
model/diffusion provider/directml status/planned issues that have been planned but not started type/bug broken features
Milestone

Comments

@HoopyFreud
Copy link
Contributor

HoopyFreud commented Jun 15, 2023

with server on --half and onnx-low-memory disabled, the onnx runtime produces a matrix multiplication error during image generation

This error appears to be caused specifically by the enable_mem_reuse optimization for onnx-runtime. The other optimizations prevented by onnx-low-memory cause no errors. This error only occurs when onnxruntime-directml 1.15.0 is installed; error does not appear with onnxruntime-directml 1.14.1 installed.

Error:


[2023-06-14 19:51:43,680] ERROR: onnx-web worker: directml MainThread onnx_web.worker.worker: unrecognized error while running job
Traceback (most recent call last):
  File "G:\StableDiffusion\onnx-web\api\onnx_web\worker\worker.py", line 60, in worker_main
    job.fn(worker, *job.args, **job.kwargs)
  File "G:\StableDiffusion\onnx-web\api\onnx_web\diffusers\run.py", line 268, in run_txt2img_pipeline
    prompt_embeds = encode_prompt(
  File "G:\StableDiffusion\onnx-web\api\onnx_web\diffusers\utils.py", line 355, in encode_prompt
    return [
  File "G:\StableDiffusion\onnx-web\api\onnx_web\diffusers\utils.py", line 356, in <listcomp>
    pipe._encode_prompt(
  File "G:\StableDiffusion\onnx-web\api\onnx_env\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "G:\StableDiffusion\onnx-web\api\onnx_web\diffusers\utils.py", line 130, in expand_prompt
    text_result = self.text_encoder(input_ids=group.astype(np.int32))
  File "G:\StableDiffusion\onnx-web\api\onnx_env\lib\site-packages\diffusers\pipelines\onnx_utils.py", line 60, in __call__
    return self.model.run(None, inputs)
  File "G:\StableDiffusion\onnx-web\api\onnx_env\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 217, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running MatMul node. Name:'/text_model/encoder/layers.1/self_attn/out_proj/MatMul' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\MLOperatorAuthorImpl.cpp(2448)\onnxruntime_pybind11_state.pyd!00007FFED69FC715: (caller: 00007FFED69FD982) Exception(3) tid(2ad4) 80070057 The parameter is incorrect.```
@ssube ssube added this to the v0.10 milestone Jun 20, 2023
@ssube ssube added status/planned issues that have been planned but not started type/bug broken features model/diffusion provider/directml labels Jun 20, 2023
@ssube
Copy link
Owner

ssube commented Jun 20, 2023

This has been reported twice now in Discord, both times with a fresh install of the Python packages, most likely both using ORT 1.15.0.

For now, I've pinned the packages in the requirements file to what should be a working combination, based on what I've been using to build the Windows bundles. That specifies ORT 1.14.x for now, but I still plan on fixing this so we can upgrade.

@ssube ssube modified the milestones: v0.10, v0.11 Dec 3, 2023
@ssube ssube modified the milestones: v0.11, v0.12 Dec 22, 2023
@ssube ssube modified the milestones: v0.12, v0.13 Dec 31, 2023
@ssube ssube modified the milestones: v0.13, v0.14 Jan 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
model/diffusion provider/directml status/planned issues that have been planned but not started type/bug broken features
Projects
None yet
Development

No branches or pull requests

2 participants