Plans of continued development?

[jtabox]

Hi,

Out of curiosity, are there any plans to continue developing these nodes? I don't have any expectations, neither am I implying anything. Just wondering, since the last development activity was 2 years ago, and there's multiple unanswered issues, and of course a host of new models and architectures.

[NubeBuster]

The real question should be, to whom do we hand off this project? Or perhaps at this point, which of the forks do we agree to redirect the visitors of this abandoned resource? How do we convey this reliably?

If my quick observation is correct... wait what does comfyanonymous imply? This resource is in hands of the ComfyUI team? Well then I'd say we poke them to answer these questions. PRs aren't getting merged...

[jtabox]

Quite a few questions there, and, honestly, my initial question was a much more generic one. I was just cleaning up my Comfy custom nodes and wasn't sure if this node was worth keeping. For now I ended up disabling it.

You seem to have multiple questions, more complex and layered, and that are more suited for a discussion. Unfortunately at the moment it doesn't really feel that it's a discussion anyone is interested in having.

[lord-lethris]

Be interesting to see where this goes - because as of the last ComfyUI update, I cannot get this working any more.

After converting Models, this was significantly faster at rendering.

[NubeBuster]

I've made a VAE TensorRT implementation. Or rather, I've managed to let Claude manage to do so. The goal was to dip my toes in how complicated TensorRT is. We started with SDXL VAE. To my surprise in a XY Plot workflow this can squeeze out another 30% roughly on the workflow time compared to only SDXL TensorRT. So thar was a success.

Then the steep hill came. For some reason VAE encode is much more complicated than decode in Wan/Temporal contexts. After too much time we gave up on the Wan VAE encode implementation and only managed the VAE decode implementation.

Now that was solely for the VAE. So I reckon another Mount Everest is waiting in the actusl sampler implementation. That's a very rough guess though. I haven't paid enough attention to Claude's rambles to deeply understand what TensorRT is and where is can and cannot be applied. I wildly guess it depends on the domain and range of the operations. Lemme ask Gemini.

Does the ability to implement tensorrt on an ai model depend kn the domain and or range of the models architecture

Yes. When applying an ahead-of-time (AOT) compiler like TensorRT to an AI model, the mathematical concepts of "domain" and "range" translate directly to input tensor specifications and activation/output boundaries. TensorRT's ability to optimize and compile the model is strictly gated by these constraints.

I've always been good at catching information while barely paying attention. I still got it.

In short, if anyone is interested in SDXL VAE implementation and or Wan VAE Decode implementation lemme know and I'll take a minute to publish a repo. I'd say the SDXL VAE is great for XY Plot and otherwise very niche. Likewise the Wan VAE step, eh, it's a small performance bump.
But do not expect a high quality repo. It was for proof of concept and violates all ComfyUI LiteGraph principles - actually almost every comfy nodes repo does that.

I sometimes wonder if the comfyui abstraction is helping or not at all. I suppose the answer is to develop in Comfy and port to raw code in prod if you need performance.

Update

I loaded up the claude chat where I left off. Turns out I had already started on "DiT TRT test - ONNX CPU export blocker". I remember now that I asked a Claude agent to do some ground work while the VAE agent is churning (and failing). I did that nearing a Claude Quota Reset. Always a good opportunity to let an orchestrator agent spawn parallel Opus workers to spend those tokens and see what comes out of it.

---

Chat: "DiT TRT test - ONNX CPU export blocker"

It's been a few weeks. What is the state? Am I right the initial WAN VAE decode was slower and we are trying something now? And is it true the WAN VAE encode is not viable? Why?

● Let me refresh my memory from the project files.

---

Here's the state:

WAN VAE TRT — abandoned. Both decode and encode are dead ends:

• Decode: The split architecture puts TRT on the cheap latent-resolution layers, while the expensive pixel-resolution layers stay in PyTorch (because they need frame-by-frame temporal caching via CausalConv3d, which TRT can't do). Result: 2x slower than vanilla PyTorch. On top of that, fp16 GroupNorm produces dark, crushed output — a precision disaster.
• Encode: Same fundamental issue — temporal compression requires CausalConv3d caching. TRT full-pass mode preserves T instead of compressing it (33 frames → 33 instead of → 9). Since video encode always has multiple frames, there's no useful case. All WAN encode just delegates to the original PyTorch VAE.

Current focus: WAN DiT TRT — accelerating the diffusion backbone (14B model, 20–50 denoising steps). That's where TRT actually matters. We got through ONNX export + ORT validation, then this session was adding TRT engine build + inference test steps (Steps 4–5). The test hit a blocker: the dequantized FP8→fp16 model (~26 GB) gets pushed to CPU for ONNX tracing, but Conv3d has no CPU kernel (aten::cudnn_convolution). We were freeing disk space to see if keeping it on GPU was feasible, but 16 GB VRAM vs 26 GB model is the real constraint.

---

Okay. But as for the WAN VAEs, especially the decode — is it determined there is no speed to be gained using any TensorRT impl?

Not fully determined — there's one approach the roadblocks doc mentions as "not attempted" that could work:

Explicit cache I/O. The reason TRT can't do the cached decoder is that CausalConv3d maintains state between frames. But you could make those cache tensors explicit inputs and outputs of the TRT engine — similar to how TRT-LLM handles KV-cache in LLMs. Each frame call would be: (frame, cache_in) → (output, cache_out). TRT handles the heavy compute, cache state lives in regular CUDA tensors between calls.

This would let TRT run the full decoder per-frame, not just the cheap latent-res layers — addressing the "TRT accelerates the wrong part" problem.

Per-layer precision (GroupNorm in fp32, Conv3d in fp16) was also not attempted. TRT supports this via layer.precision constraints. Would fix the dark output.

Combined, those two could potentially give a real speedup. But:

1. The ONNX export would need significant rework — every CausalConv3d's cache becomes an explicit tensor pair (input + output), and there are many of them
2. VAE decode is called once per generation. The DiT runs 20–50 steps. Even a 3x VAE speedup barely moves the needle on total generation time
3. It's a lot of engineering for marginal end-to-end impact

So: not impossible, but the ROI is poor compared to DiT TRT, which is why we pivoted there.

So

I guess I'll let some agent poke at it on the side while I'm working. With some luck Wan DiT TRT Hack Magic is pretty straightforward and it'll be done swiftly. (Though likely sub-optimal initially)

[SveSop]

This sounds interesting tho, and kudos for doing testing of this 👍
I am a complete n00b to this, but i have been using the RIFE Tensorrt nodes + Tensorrt Upscaler nodes, and with some tweaks managed to make some updates for adding x2 and x1 support to various upscale models.
I prefer just using a x1-SkinDIFF model for some final polishing in a 30-60 second wan video over 2x or 4x upscaling, because i find the result being overly "smooth".

So, a TensorRT for VAE encode/decode - especially for wan - could very well be something interesting. I am all for a bit of tinkering and testing, but i come up very short when it comes to python and torch-whatever 😏

[NubeBuster]

So, a TensorRT for VAE encode/decode - especially for wan - could very well be something interesting. I am all for a bit of tinkering and testing, but i come up very short when it comes to python and torch-whatever 😏

Yeah but my Claude Opus and my findings are that the VAE for Wan is not significantly TensorRT-able, unfortunately.

Claude has predicted that the inference models, in contrast, are definitely optimizable using this tech.

I will try and make progress when I have time for it. Don't rely on me though. This is low-prio. It's just for fun. I don't need Wan to be faster for my personal use. I only wish to have some prod project where inference_time=money.