OpenArt
OpenArt
Workflows

    My stuff

  • My Workflows

  • Liked Workflows

  • Following Workflows

Go to OpenArt main site
Upload workflow

Sparse Controlnet Animate Diff Video to Video +extras

5.0

1 reviews
131
32.9K
7.0K
19
Description

Beta 3 - extended

replaced the color of the padding to black to stop coco from detecting nothing.

Added cropping to end

added load video group to just before the upscaling to allow the reloading of previous videos or full.

more minor fixes


Beta 3

Fixed a lot with the first masking and proper background removal skip, it really frustrated me as nothing I rendered was nearly as good as when I first started.  Now it's back to proper clarity.

Masking combined for upscales

few other small fixes.


Beta 2 -seems to be bunk atm, when starting to compact masking it caused issues with the backgrounds.  Should have a new version out tomorrow if not tonight to fix that.


Beta 2 - needs more testing but throwing it out as i think its better atm than the first and it fixes a few bugs.

Added

-3 mask options, coco is great but slow. Can use any of the three mask options at all steps they are needed. Not sure if they help with memory.  Bypass masking groups you are not needing.

If you're smart and know a better masking solution, you can insert your own group to mask off things

-added anime/cartoon mask creation option

-added face numbered preview to the reactor area to find your needed images

-Added lora groups to all model areas

-Added second save option

-Added Cutoff nodes.  - allows better color control stability in general.

-Many more notes - too many notes... I talk too much.


This is a beta of my sparse control workflow for video to video.

Features

Sparse RGB or Scribble (any line art works) control nets

Full subject replacement

Full background replacement (both animated and static)

Face/body enhancements

Full upscale option

Cropping/padding allowing subject placements of subjects within the video.

Notes in the flow

Adding a few of the ref videos i used to the assets







Discussion

(No comments yet)

Loading...

Author

Ryan Dickinson

Ryan Dickinson

37
65.2K
746
295.3K
    anime.gif (2.6 MB)
    carlton.mp4 (137.1 kB)
    wet-pool.gif (13.9 MB)
    run.gif (4.3 MB)
    44.gif (4 MB)
    10.gif (1.7 MB)
    skirt-spin.gif (12.6 MB)
    2.gif (4.4 MB)
    444.gif (1.8 MB)

No reviews yet

  • - latest (2 years ago)

  • - v20240114-050729

  • - v20240112-081529

  • - v20240108-102217

Primitive Nodes (214)

Anything Everywhere3 (1)

FrameInterpolator (1)

Note (50)

PrimitiveNode (17)

Reroute (143)

SaveVideo (1)

SetMetadataForSaveVideo (1)

Custom Nodes (378)

AnimateDiff Evolved

  • - ADE_AnimateDiffLoaderWithContext (2)

  • - ADE_AnimateDiffUniformContextOptions (1)

  • - ADE_AnimateDiffSamplingSettings (1)

  • - ADE_AnimateDiffUnload (1)

ComfyBreakAnim

  • - GetKeyFrames (1)

ComfyUI

  • - MaskToImage (11)

  • - KSamplerAdvanced (7)

  • - ImageScale (28)

  • - ImageScaleBy (13)

  • - EmptyLatentImage (5)

  • - InvertMask (3)

  • - CLIPTextEncode (10)

  • - CheckpointLoaderSimple (2)

  • - CLIPSetLastLayer (2)

  • - VAEEncode (4)

  • - LoraLoader (12)

  • - RescaleCFG (4)

  • - SelfAttentionGuidance (7)

  • - VAEDecode (6)

  • - SetLatentNoiseMask (2)

  • - RebatchImages (15)

  • - LatentUpscaleBy (3)

  • - PreviewImage (16)

  • - VAELoader (1)

  • - LoadImage (12)

  • - ImagePadForOutpaint (2)

  • - VAEDecodeTiled (1)

  • - RebatchLatents (4)

  • - UpscaleModelLoader (2)

  • - RepeatImageBatch (1)

  • - ImageUpscaleWithModel (2)

ComfyUI Cutoff

  • - BNK_CutoffSetRegions (8)

  • - BNK_CutoffRegionsToConditioning_ADV (2)

  • - BNK_CutoffBasePrompt (2)

ComfyUI Frame Interpolation

  • - RIFE VFI (1)

ComfyUI Impact Pack

  • - ToBinaryMask (4)

  • - SegsToCombinedMask (4)

  • - UltralyticsDetectorProvider (2)

  • - ImpactSimpleDetectorSEGS (4)

  • - ImpactMakeImageBatch (1)

  • - ImpactSEGSRangeFilter (3)

  • - SAMLoader (1)

ComfyUI Nodes for Inference.Core

  • - AIO_Preprocessor (4)

  • - PixelPerfectResolution (4)

  • - ImageGenResolutionFromImage (4)

  • - AnimeFace_SemSegPreprocessor (1)

  • - TilePreprocessor (1)

  • - OneFormer-COCO-SemSegPreprocessor (1)

ComfyUI WD 1.4 Tagger

  • - WD14Tagger|pysssss (1)

ComfyUI-Advanced-ControlNet

  • - DiffControlNetLoaderAdvanced (1)

  • - ACN_AdvancedControlNetApply (12)

  • - ControlNetLoaderAdvanced (2)

  • - ACN_SparseCtrlLoaderAdvanced (2)

  • - ACN_SparseCtrlSpreadMethodNode (1)

  • - ScaledSoftControlNetWeights (4)

  • - ACN_SparseCtrlRGBPreprocessor (1)

ComfyUI-Image-Selector

  • - ImageSelector (5)

ComfyUI-post-processing-nodes

  • - Blur (6)

ComfyUI-Text_Image-Composite [WIP]

  • - AlphaChanelAddByMask (2)

ComfyUI-Video-Matting

  • - Robust Video Matting (2)

ComfyUI-VideoHelperSuite

  • - VHS_VideoCombine (11)

  • - VHS_GetImageCount (10)

  • - VHS_SelectEveryNthImage (1)

  • - VHS_LoadVideoPath (2)

Dream Video Batches

  • - Unwrap Frame Set [DVB] (1)

  • - Linear Camera Zoom [DVB] (1)

  • - Sine Camera Zoom [DVB] (1)

  • - Linear Camera Pan [DVB] (1)

  • - Create Frame Set [DVB] (1)

LoadLoraWithTags

  • - LoraLoaderTagsQuery (2)

Masquerade Nodes

  • - Image To Mask (9)

  • - Mask Morphology (14)

  • - Get Image Size (10)

  • - Cut By Mask (4)

  • - Change Channel Count (13)

  • - Mask To Region (2)

  • - Paste By Mask (2)

  • - Combine Masks (6)

  • - Mix Color By Mask (1)

  • - Separate Mask Components (1)

pythongosssss/ComfyUI-Custom-Scripts

  • - ShowText|pysssss (2)

ReActor Node for ComfyUI

  • - ReActorFaceSwap (1)

Use Everywhere (UE Nodes)

  • - Seed Everywhere (1)

WAS Node Suite

  • - Image Bounds (3)

  • - Bounded Image Crop (3)

  • - Image Select Color (3)

  • - Image Flip (2)

  • - Inset Image Bounds (3)

Checkpoints (2)

0001semirealistic_v54.safetensors

0001softrealistic_v154xxx.safetensors

LoRAs (14)

Gothpunk.safetensors

bellydance.safetensors

lcm\SD1.5\pytorch_lora_weights.safetensors

v3_sd15_adapter.ckpt