(This template is used for Workflow Contest)

What this workflow does

👉 Bring sketches to life or quickly inpaint with style!

How to use this workflow

1. Make sure you have all the custom nodes installed. If you don't, start with the manager: https://github.com/ltdrdata/ComfyUI-Manager then install missing nodes

2. The canvas node can be a little tricky to use in the way I am interacting with it. I recommend starting simple with a 512x512 canvas.

Simple sketch flow:

1. Not necessary, but reinforcement with prompts helps quite a bit for whatever you want to sketch

2. Disable the inpaint controlnet for a first pass, otherwise you will have to completely color in the mask in canvas (the IPAdaptor requires the mask too, so make sure there is some black on the mask if you want that to work)

3. Click Edit on the Canvas node. Create a new image in the same size you selected for the aspect ratio

4. Add a new layer and right click on it to make the red target appear (this will be where the result image gets placed for the next iteration)

5. Click on the base layer and start sketching

6. Queue it up (then follow the inpainting if you want)

Inpainting flow:

1. Make sure the inpainting controlnet is enabled

2. (optional) If you didn't already generate an image from sketch, right click on the base layer to mark it as target. Then go drag a background image onto the canvas node in the ComfyUI workspace. (best if it is the same size)

3. Create two new layers in canvas on top of the base, right click on the top one to make it the next target (this way you can always revert to a previous iteration if you want)

4. Click on the layer between that is blank to draw any new stuff. You may need to update your prompt to match whatever you are trying to inpaint.

5. Color in the mask layer where you want the inpainting to happen. Make sure the mask includes all the sketches you made or you might get weird unrendered parts)

(Note that currently Canvas will automatically queue the prompt every time you release the mouse when editing the mask layer, I think this is a bug, so hopefully you don't have that problem)

6. Repeat

(optional IPAdapter)

1. The IPAdapter is pretty great at adopting the style of whatever image you drop in there as reference.

Tips about this workflow

👉 Canvas

-- Canvas is freaking amazing, and also a little clunky at the time I built this workflow. Watch which layer you have targeted and which image you have active with the little green tab at the top right.

👉 Models/Sampler

--This is set up for SD1.5 models and ratios. I haven't tested on 2.0 or XL models.

--Use the Dreamshaper LCM model for speedy rendering. I use about 5-6 steps on the first pass when using upscale, and 7-9 steps on upscale sampler. When I disable upscale, I bump the steps up on the first sampler.

--For the LCM, be careful with the CFG, keep it around 2-2.5

--If you go much higher than 1.5 upscale factor, stuff starts to get weird

--Make sure your selected base image size in the aspect ratio box matches the canvas size selected in the ComfyUI Canvas Editor

--If you decide to try a different model that isn't LCM, just make sure you update all the samplers accordingly.

👉 IPAdapter

--The preprocessing I have is pretty basic. If you don't want your image to be cropped weirdly, pre-crop it to the reference you want, and scale it to 512

--If you add much noise beyond 0.05, it starts to affect the whole image during inpainting, not just the masked part. Might be what you want, but if you want the inpainting to be really focused, keep the noise low

🎥 Video demo link

👉 Spent too much time on the workflow. I will hopefully get around to this soon. Meantime, there are instructions here and in the workflow.