Hi everyone, ControlNet for SD3 is available on Comfy UI! Please read the instructions below:

1- In order to use the native 'ControlNetApplySD3' node, you need to have the latest Comfy UI, so update your Comfy UI.

2- Right now, there is 3 known ControlNet models, created by Instant-X team: Canny, Pose and Tile. This example is for Canny, but you can use the others as well.

Canny:  https://huggingface.co/InstantX/SD3-Controlnet-Canny/blob/main/diffusion_pytorch_model.safetensors

Pose:  https://huggingface.co/InstantX/SD3-Controlnet-Pose/blob/main/diffusion_pytorch_model.safetensors

Tile:  https://huggingface.co/InstantX/SD3-Controlnet-Tile/blob/main/diffusion_pytorch_model.safetensors

Thanks to the ComfyOrg & Instant-X teams!

Important Note: Make sure you are using 1024x1024 latent. Other resolutions may generate a lot of artifcats. Here is the explanation from the Instant-X Team:

Due to the fact that only 1024*1024 pixel resolution was used during the training phase, the inference performs best at this size, with other sizes yielding suboptimal results. We will initiate multi-resolution training in the future, and at that time, we will open-source the new weights.

P.s: you might need to adjust your 'strength' and 'threshold' parameters for getting the best results.