Go to OpenArt main site
Upload workflow

Loose Control Depth Animation using Voxel driven inputs in AD with SD1.5

5.0

0 reviews
7
3.6K
629
0
Description

A real awesome CN model to explore. Its a control net model called "Loose Depth".  

ControlNet Loose depth model

The interesting option is here this works very well with all voxel animation inputs as VID2VID with depth conversion in between to drive the final animation;

(1) voxel input video  (zoe depth here) also done inside comfy UI so we get depth maps to drive a new animation

(2) Voxel images can be generated using comfy UI models and Loras or can be done using Magica Voxel  a free program and exported as OBJ or FBX.

A simple voxel animation can be plugged using this workflow!


(3) Here is the  final work flow experimentation ! This uses a voxel generated using Magica and then sending it to the Mixamo for action and downloading an FBX with rigged model;


(4) This rigged model can be converted to mp4 using any standard tools or online tools to mp4

(5) Once converted this can be fed into the Loose control box animation using AD to generate some awesome results;



This is only a super basic workflow with no other CN in action and can be merged to make some awesome combo to make it RAD; A raw out put with no edit or upscale or interpolation applied! And adding some Prompt modifications to the same can result in animations that are different!

Start with a basic to get an idea and then push the frontiers further to see some amazing outcome ! Voxel creations can be very powerful and another super cool to help with same other than Magica is the GOXEL;

Goxel has a super cool interface and very nice workflow !

If you want to see some cool voxel  projects do dive into Sandbox


Explore and share your results and if any queries do let  me know. The workflows shared above are in not any general order as I had been working with tweaks and changes and will share the improvements as we go by !




Discussion

(No comments yet)

Loading...

Author

5
7.6K
136
39.8K
    ComfyUI_temp_ialso_00227_.png (108.6 kB)
    boxup-00_00012-2x-RIFE-RIFE4.6-24fps_stab_thm2_apo8_prob3.mp4 (136 MB)
    2024-01-12_21-32-42.mp4 (5 MB)
    2024-01-07_16-12-35.png (613.8 kB)
    2024-01-11_22-08-49.mp4 (2.4 MB)
    2024-01-08_16-26-09.gif (4 MB)
    ad_boxedbirdy001A.mp4 (16.7 MB)

No reviews yet

  • - latest (a year ago)

Primitive Nodes (30)

GetNode (12)

Note (6)

SetNode (12)

Custom Nodes (39)

AnimateDiff Evolved

  • - ADE_AnimateDiffUniformContextOptions (1)

  • - ADE_AnimateDiffSamplingSettings (1)

  • - ADE_AnimateDiffLoaderWithContext (1)

  • - ADE_AnimateDiffModelSettingsSimple (1)

  • - CR Apply LoRA Stack (1)

  • - CR SD1.5 Aspect Ratio (1)

  • - CR Integer To String (1)

  • - CR LoRA Stack (1)

ComfyUI

  • - CLIPTextEncode (2)

  • - SelfAttentionGuidance (1)

  • - PatchModelAddDownscale (1)

  • - FreeU_V2 (1)

  • - RepeatImageBatch (1)

  • - EmptyLatentImage (1)

  • - VAEDecode (1)

  • - ImageScale (1)

  • - ControlNetLoader (1)

  • - ControlNetApplyAdvanced (2)

  • - PreviewImage (1)

  • - SaveImage (2)

  • - CLIPSetLastLayer (1)

  • - KSampler (1)

  • - CheckpointLoaderSimple (1)

  • - VAELoader (1)

  • - GlobalSeed //Inspire (1)

  • - Zoe-DepthMapPreprocessor (1)

  • - OpenposePreprocessor (1)

  • - ControlNetLoaderAdvanced (1)

  • - VHS_VideoCombine (3)

  • - VHS_LoadImagesPath (1)

Hakkun-ComfyUI-nodes

  • - Image size to string (1)

Checkpoints (1)

SD15\calicomix_v75.safetensors

LoRAs (2)

ClayAnimationRedm.safetensors

disney.safetensors