Does it work in ComfyUI?

#1
by juang3d - opened

I tested with apply controlnet and with diffsynth controlnet, but it does not seem to work, is there a way to make it work in comfy?

same problem

Alibaba-PAI org

We might need ComfyUI to add support for this. I'll ask about it tomorrow when I have time.

Thanks, let's hope they support it soon, I'm willing to do some testing of this, it looks great.

We might need ComfyUI to add support for this. I'll ask about it tomorrow when I have time.

thank you!

yeah~~hope this can support it

Alibaba-PAI org

If you have time, you could test this out - I just uploaded it.
https://github.com/aigc-apps/VideoX-Fun/tree/main/comfyui/qwenimage

If you have time, you could test this out - I just uploaded it.
https://github.com/aigc-apps/VideoX-Fun/tree/main/comfyui/qwenimage

Just tried, stuck at first step. Text encoder gets loaded to cpu causing Comfy to be killed (I only have 48GB RAM) so no way to see if the rest of the workflow works. Correct me if I'm wrong but comfyui's native memory logic uses gpu to load text encoder and then offloads to cpu to give space for diffusion model, which I think you should adapt to because most of us rely on maximum usage of both VRAM and RAM to run them big models today. Nonetheless thank you for following up this thread.

@bubbliiiingstill no news from comfy?

We need this to work with native nodes, unless I'm wrong the version fixes works only with those nodes.

Thanks!

Check this PR I made guys! it is working like a charm:

https://github.com/Comfy-Org/ComfyUI/pull/12359

@krigeta , does your update work with the Apply ControlNet node, or the QwenImageDiffsynthControlnet node?

@Secret-Abalone5614 Apply ControlNet node and no special node needed. Enjoy!

@krigeta , confirmed. Your update works great!

can someone share a workflow example please?

@jeramyhola, just use the image_qwen_image_instantx_controlnet template from Comfy, and replace the diffusion model and controlnet model with the 2512 versions.

@jeramyhola, just use the image_qwen_image_instantx_controlnet template from Comfy, and replace the diffusion model and controlnet model with the 2512 versions.

Will do, thank you!

@Secret-Abalone5614 Apply ControlNet node and no special node needed. Enjoy!

hey, i tried copying the files over but it gave me this error
module 'comfy.model_patcher' has no attribute 'CoreModelPatcher'

@jeramyhola, just use the image_qwen_image_instantx_controlnet template from Comfy, and replace the diffusion model and controlnet model with the 2512 versions.

Are there inpainting workflows available? I replaced the instantx inpaint workflow with this controlnet and I can barely see the differences between the result and the input image

Sign up or log in to comment