[v2.0] AnimateDiff ControlNet Animation Refiner - LCM [Part 3]
5.0
0 reviewsDescription
New Update v2.1 !!! Available Here : https://www.patreon.com/posts/update-v2-1-lcm-95056616
.
.
A series of 5 workflows, for easy understanding, usage, and more clarity.
The Workflow is divided into 5 parts :
- Part 1 - ControlNet Passes Export
- Part 2 - Animation Raw - LCM
- Part 3 - AnimateDiff Refiner - LCM
- Part 4 - AnimateDiff Face Fix - LCM
- Part 5 - Batch Face Swap - ReActor [Optional] [Experimental]
What this workflow does
This workflow can Refine Bad looking images from [Part 2] into detailed videos, with the help of animateDiff used as an Upscaler and Refiner.
How to use this workflow
As you have already generated raw images from [Part 2] you can further enhance the details from this workflow.
- 1) Enter the Paths in Purple Directory Nodes of the Raw Images
- 2) Enter the Output path for saving the refined images.
- 3) Enter Batch Range
- 4) Overlapping Frames - 0 is Default (Put 5 or 10 for overlapping cross fade technique between batches)
- 5) Skip Frame is 0 by default (Increase it after every batch)
- 6) Batch Number Naming, Increase it by 1 after every batch for organizing.
- 7) Refiner Upscale - Upscale value is 1.4 by default ( Use 1.4 - 1.8 for best result for SD models)
- 8) Refiner Denoise - 0.45 by default (Use between 0.4 - 0.6 for best results, Higher value may change the image entirely) * Inject Additional Refiner Noise - Use value 0 - 1 (It enhances the minute details, use with add details lora)
- 9) Enter Prompts for more accuracy.
- 10) Choose Model checkpoint you want to use for refiner.
- 11) Loras work as usual (Some of not compatible with animatediff, test it first with 10 frames)
Tips about this workflow
You can also use this Refiner as :
1) Vid2Vid stylization
Stylized Video2Video Can also be made with this [Part 3] Refiner workflow:
- Generate the passes from [Part 1} and it will also produce original frames in a separate folder.
- Enter the ControlNet Softedge pass and Lineart pass in the controlnet directory node.
- Use these original frames as raw input in the refiner workflow with a denoise of 0.5-0.7
- Enter prompts and choose models, loras for stylization.
2) SVD / Normal Video Refiner
You can also refine Videos from SVD or even a low Res video of anything and convert it to a high res.
- Generate the passes from [Part 1} and it will also produce original frames in a separate folder.
- Enter the ControlNet Softedge pass and Lineart pass in the controlnet directory node.
- Put Value around 1.5 in the Upscale node
- Use these original frames as raw input in the refiner workflow with a denoise of 0.5-0.7
- Enter prompts for moreaccuracy and choose proper models , realistic for real photos, and anime for anime images ...etc.
SVD with this refiner already built can be found here : SVD with AnimateDiff Refiner
🎥 Video demo link (optional)
All the Recent Uploaded Shorts are refined from this workflow : https://www.youtube.com/@Ai_Davos/shorts
Important Documented Tutorial:
https://www.patreon.com/posts/update-animate-94523632
My Discord Server : https://discord.gg/z9rgJyfPWJ
Discussion
(No comments yet)
Loading...
Reviews
No reviews yet
Versions (1)
- latest (a year ago)
Node Details
Primitive Nodes (18)
Evaluate Integers (1)
Note (4)
PrimitiveNode (3)
Reroute (10)
Custom Nodes (56)
- ADE_AnimateDiffUniformContextOptions (1)
- ADE_AnimateDiffLoaderWithContext (1)
- CR Integer To String (3)
- CR Apply LoRA Stack (1)
- CR LoRA Stack (2)
ComfyUI
- KSampler (1)
- PreviewImage (6)
- CheckpointLoaderSimple (1)
- LoraLoader (1)
- ModelSamplingDiscrete (1)
- ControlNetApplyAdvanced (2)
- VAELoader (1)
- ImageScaleBy (1)
- VAEDecode (1)
- VAEEncode (1)
- BNK_InjectNoise (1)
- ControlNetLoaderAdvanced (2)
- LoadImagesFromDirectory (3)
- VHS_SplitImages (6)
- FloatConstant (4)
- ColorMatch (1)
- ShowText|pysssss (2)
- Int Literal (2)
- Cfg Literal (1)
- Seed Generator (1)
- smZ CLIPTextEncode (2)
- GetImageSize (1)
- Text Multiline (2)
- Image Save (1)
- Text Concatenate (1)
- Text String (1)
- String to Text (1)
Model Details
Checkpoints (1)
imp_v10.safetensors
LoRAs (1)
lcm_pytorch_lora_weights.safetensors