Elevate Your Inpainting Game with Differential Diffusion in ComfyUI

Inpainting has long been a powerful tool for image editing, but it often comes with challenges like harsh edges and inconsistent results. Enter differential diffusion, a groundbreaking technique that introduces a more nuanced approach to inpainting. Instead of using a binary black-and-white mask, it applies changes on a per-pixel basis using a gradient mask. This results in smoother transitions and more natural-looking edits.

Key Benefits:

• Eliminates harsh edges between original and edited areas
• Produces more consistent and realistic results
• Works with standard generational checkpoints (inpainting checkpoints work as well too)

Essential Nodes for Differential Diffusion

To implement this technique in ComfyUI, you'll need these nodes:

1. Gaussian Blur Mask
2. Differential Diffusion
3. Inpaint Model Conditioning

Workflow Overview

1. Image Preparation: Load your base image. Create a mask using the Mask Editor.
2. Mask Processing: Pass the mask through the Gaussian Blur Mask node. Adjust kernel_size (brush size) and sigma (softness) to fine-tune the gradient.
3. Model Setup: Use a standard generational checkpoint. Pass the model through the Differential Diffusion node.
4. Conditioning: Use the Inpaint Model Conditioning node for prompts, mask, and other parameters.
5. Sampling: Adjust the denoise value in KSampler (typically 0.6-0.8 for inpainting). Fine-tune other sampling parameters as needed.

Tips:

Balance mask softness carefully – too soft can affect surrounding areas unintentionally. Experiment with denoise values to find the sweet spot between change and preservation. Include both current and desired elements in your prompt for best results.

Real-World Example

We tested this workflow by adding glasses to a portrait. The results showed seamless integration with the existing features, demonstrating the power of differential diffusion for subtle yet impactful edits.

Full demonstration is available here.