AnimateDiff Checkpoint Mixing
5.0
0 reviewsDescription
This workflow uses the latent noise mask node to enable the use of two different checkpoints for the subject and background. In these examples I mixed epicphotogasm and mistoonanime. It also allows you to use different loras for the subject/background as well.
The process is relatively simple, create an initial video using one model, create a mask for the subject, feed it into the latent noise mask and then reprocess the initial output with a new model. Finally, I added an adetailer pipeline to adjust the face to the subject model (optional). The process is actually relatively fast using AnimateLCM.
In the attached workflow I used COCO to generate a mask for the dancer, but you can use any method to generate your mask (like rembg for example) or just use your own mask through a video upload.
Discussion
(No comments yet)
Loading...
Reviews
No reviews yet
Versions (1)
- latest (2 years ago)
Node Details
Primitive Nodes (16)
GetNode (6)
Integer (2)
Note (1)
PrimitiveNode (1)
SetNode (6)
Custom Nodes (50)
- ADE_LoadAnimateDiffModel (1)
- ADE_ApplyAnimateDiffModelSimple (1)
- ADE_AnimateDiffSamplingSettings (1)
- ADE_StandardUniformContextOptions (1)
- ADE_UseEvolvedSampling (2)
ComfyUI
- PreviewImage (2)
- CLIPVisionLoader (1)
- SetLatentNoiseMask (1)
- CLIPTextEncode (4)
- LoadImage (1)
- MaskPreview+ (1)
- GetImageSize+ (1)
- UltralyticsDetectorProvider (1)
- SAMLoader (1)
- ImpactSimpleDetectorSEGS_for_AD (1)
- SEGSPaste (1)
- ToBasicPipe (1)
- SEGSDetailerForAnimateDiff (1)
- OneFormer-COCO-SemSegPreprocessor (1)
- DepthAnythingPreprocessor (1)
- DWPreprocessor (1)
- PiDiNetPreprocessor (1)
- IPAdapterModelLoader (1)
- IPAdapterAdvanced (2)
- PrepImageForClipVision (1)
- ACN_AdvancedControlNetApply (4)
- ControlNetLoaderAdvanced (3)
- VHS_VideoCombine (3)
- VHS_LoadVideo (1)
- LoRA Stacker (2)
- HighRes-Fix Script (1)
- Efficient Loader (2)
- KSampler (Efficient) (2)
- ColorToMask (1)
Model Details
Checkpoints (0)
LoRAs (1)
