Video Demo:  https://www.patreon.com/posts/112239304

Hello Guyzzzz.... I present you a new workflow update, In this you can Denoise Subject and Background on different denoise values using the masks, Also can set their controlnets on different strength.

By this you can have stylized Subject with Stable backgrounds and . Have a smooth transition between subject and background with the Differential Diffusion Method using the inbuilt comfy node.

What's New in the workflow !! :

More Flexible controls:

1) Separate Subject and Background Controlnets Settings

2) Separate Subject and Background Denoising Settings

3) Re - Organizing and Shortcuts keys for Better and fast access.

• Since now this Workflow Uses Original Image as inputs, It provide great stability to the Renders, Giving it a vid2vid effect.
• So If you want to stylize only 25% or only 50% , you can do so, and even you can control this stylization levels of the subject and background differently.
• Plus, When Doing Rendering at max upto 50 - 70 % , Controlnets no need to be used at High settings. Lower can also give good results

Renders Made using this workflow:

1) https://www.youtube.com/shorts/_wq8FwEnVE4
2) https://www.youtube.com/shorts/Gea1eK0-HLs
3) https://www.youtube.com/shorts/-xjcpGTAkN4
4) https://www.youtube.com/shorts/yKwb0De7HiM

The New v4.5 workflows are uploaded in the Version 4 folder

Link: https://drive.google.com/drive/folders/1h2761jX1caH5KfjFABQv8Lk4qEACh5Zz

Or you can also find these in the attachments in the end of this post.

This is the full HD preview of the workflow, you can open this image in a new tab to zoom in.

1) Denoise Controls Preview

• They are dependent on Mask, so they play an important part here. Both raw and refine has these different denoise controls,
• In general you should keep the Denoise value of Refine sampler to under 0.5, so it don't deviate much
• If you change the prompt which has different element from the original say color of the cloths, then it will affect according to the denoise level.
• Denoise Value 1 will have the maximum stylization and change from the original which will be closer to the prompts
• Inpainting Trick : If you set 0 in the background denoise, you can have an inpainting workflow.

2) Controlnet Settings Preview

In the previous workflow, using mask limited the use of controlnet on the Background, by the above method you can control it too.

• Btw I've observed during my test, that if you are denoising up to 0.3-0.5.... controlnets are not needed that much. You can set a general low value.If you have moving background then you can set higher values, so it won't feel like "sliding"

Source Video : https://www.instagram.com/p/C9kmfKos2nd/
@ddanastan

Extract Passes From here: https://drive.google.com/drive/folders/1hLU5MhikUe6SnEnEPQc3tKTaNGmFT6p2

1) Raw

• Breakdown for the above is render from the 2_0) Animation Raw workflow with0.75 Denoise for subject and 0.3 Denoise for background - Raw0.45 Denoise for subject and 0.3 Denoise for background - Refine
• For Controlnets : Depth Anything v2 and Anyline Passes were Extracted for this video.You can watch the above thumbnail video to see the previews.
• Batch Counter - Batch Range were 50 and Overlap was 10 .After all frames were rendered, Seamless maker workflow v4 was used to make it seamless transition.
• Model Link: https://civitai.com/models/113447/helloflatcolorful2d
• Prompts - General Style prompt was used - like "2d, cartoon, flat vector art" . Rest, as it has all the color info, so AI model will nudge the details with that color into place.If you want to have same effect as my renders, you need to put these embeddings in your comfyui>embeddings folder too, to take effect which are triggered by the negative prompts.Precautions while using these embeddings in the negative prompts: Sometimes, it can override the style of the model or the lora you are using. Like 2D models will produce semi realistic render instead of 2D style, So remove them if you feel models aren't looking in the style as it suppose to be

2) Refine - No Refiner was used as quality was enough with proper hands

3) Face Fix - The Usual Face Fix Workflow

4) LivePortrait LipSync workflow was used to lip sync and add more facial expressions.


Then Finally Composed and Rendered Video is present here  and comparison here

Source Video: https://www.youtube.com/shorts/-fRKJa8UtI8
@yyyoungggggg

Extract Passes From here: https://drive.google.com/drive/folders/1hLU5MhikUe6SnEnEPQc3tKTaNGmFT6p2

1) Raw

• Breakdown for the above is render from the 2_1) Background Changer workflow with0.75 Denoise for subject and 1 Denoise for background as it will be anyways which will also help in accurate shadows. 0.5 Denoise for subject and 0.0 - 0.3 Denoise for background in the refine, so background is stable and also there are no artifacts.
• For Controlnets : Depth Anything v2 and Anyline Passes were Extracted for this video.with 0.8 weights.
• Batch Counter - Batch Range were 50 and Overlap was 10 .After all frames were rendered, Seamless maker workflow v4 was used to make it seamless transition.
• Model - Same as above:Link: https://civitai.com/models/113447/helloflatcolorful2d
• Prompts - " 1girl, red shirt, white skirt cartoon, flat vector art, anime, 2d, pointing fingers outwards , pointing index finger, sunset in the background, standing on road"Negative Prompts were usual
• Background used is this Since the original video has moving camera effects done in post, the subject in the render was looking sliding up and down.To fix it same camera movement was tracked in after effects and applied and exported as sequence and then Used as a sequence inside the workflow too.

2) Refine - No Refiner was used as quality was enough with proper hands

3) Face Fix - The Usual Face Fix Workflow

4) LivePortrait LipSync workflow was used to lip sync and add more facial expressions.

The Final Render : https://www.youtube.com/shorts/Gea1eK0-HLs
Comparison: https://www.youtube.com/shorts/ebjYR4lroYE

You can use these workflow as base, can take out a part or use in your workflows or you can use these however you like.

Working on new different type of project and will get back soon!

Hope these research helps!
- Jerry Davos  ❤️

Contacts

·        Email: [email protected]

·        Discord Channel: https://discord.gg/z9rgJyfPWJ

Download Testing Files in the Attachments Here: https://www.patreon.com/posts/v4-5-new-raw-112239304