I work as a marketer in a company that sells interior doors, and I’ve created this workflow for showcasing our products. It’s my first experience working on a comfy workflow, and I spent more than a month on it. I feel like I've come to a pretty good result.

I am pretty sure this workflow can be adapted to other scenarios where you need to put an image of a real product in a generated scene. Also, I believe it's the only workflow that actually makes good use of the SDXL inpainting model. If I had more time before the competition's end, I would try to adapt it for more diverse use.

What this workflow does

It takes an image of a real door and creates an interior around it.

How to use this workflow

In order to reproduce my results, you will need:

• diffusers/stable-diffusion-xl-1.0-inpainting-0.1 (fp16 version) AND VAE for this model. (Download BOTH here:  https://huggingface.co/diffusers/stable-diffusion-xl-1.0-inpainting-0.1/tree/main )
• My custom loras (https://civitai.com/models/232509?modelVersionId=262449)
  The workflow will work without them, but the quality will be greatly diminished. All models I’ve tested are too bad at generating good marketing materials for interior doors.
• NewRealityXL model for the second pass. (https://civitai.com/models/161068/newrealityxl?modelVersionId=254534)

You can also use “Realistic Stock Photo” (https://civitai.com/models/139565/realistic-stock-photo?modelVersionId=154593), it’s also pretty good, but in my tests, NewRealityXL showed better results.

• Repair_slider.pt from here: https://sliders.baulab.info/weights/xl_sliders/
• And diffusers_xl_canny_full.safetensors for the control model (https://huggingface.co/lllyasviel/sd_control_collection/tree/main)
  Yes, it’s slow and heavy, but it’s actually the only model that gives me good results. You can try your luck with other models, but for the best results, use this one.

The whole workflow was built around these models, so every one of them is essential for replicating my results. Keep that in mind.

How to use the workflow:

1. Select the SDXL-inpaint model and appropriate VAE for the 1st pass.
2. Select the NewRealityXL model for the second pass.
3. There are two separate loras for each pass titled accordingly, select each lora for the appropriate pass. Same for Repair_slider.pt.
4. Select the control net model.
5. Put an image of the door on a transparent background in DOOR group nodes. Use only images with standard SDXL resolutions. (I’ve prepared several images for you here https://drive.google.com/drive/folders/10Y8hoYHoH6G4bkGdLN1gKMUDgB5sgrov )
6. Use “interior door” as a keyword for loras. Also use (at an angle:1.5) for generating the right perspective for images.
7. Adjust “Slerp Latent" value to your liking. 1 would give the same result as the “Composite Latent” node. Values close to 0 will remove details from the first pass. Use values above 0.7 for best results.
8. In order to use Hi-Res fix, put an image with double the initial resolution in “Hi-res” group nodes (also can be found in prepared images here  https://drive.google.com/drive/folders/10Y8hoYHoH6G4bkGdLN1gKMUDgB5sgrov ). Yes, you can just upscale the first image, but it’s essential to preserve the initial look of the door, so I use separate images. Or I guess you can just downscale the image to 1024 for first two passes.
9. Important note for Hi-res fix: “grow_mask_by” in “Vae Encode for inpainting” node is limited to 64 for some reason, but it’s actually working fine with values above that. In order to preserve all changes made in the first two passes, we need to set “grow_mask_by” to double what we had in the beginning. So, find the nodes.py file in your main ComfyUI folder, find “class VAEEncodeForInpaint” and set the “max” value for “grow_mask_by” to whatever you want. Then, inside of the workflow, convert “grow_mask_by” to input and connect it with the “Grow by Mask” node above it. This is how I use it.

This point is optional; you can use the standard 64 value.

All parameters were selected to give the best result, but in some cases, they need slight adjustments (for example, denoising of the Hi-res fix).

Tips about this workflow

Limitations:

1. Inpainting really tries to make the door longer by creating parts that don't exist on the real product. I haven’t found a good solution for that. The only thing that I’ve come up with is to add a black bar under the door slab (but not casings).
2. My Loras are not of the best quality, but they get the job done. And I have to use two separate loras for each pass. I have plans to train better models in the future.

I’ve tested a lot of approaches and I believe this is the best way of generating an interior for real-life interior doors. Hopefully, in the future, we can get better inpainting, but as for now, this is it.