A more complete workflow to generate animations with AnimateDiff. It uses ControlNet and IPAdapter, as well as prompt travelling.

Always check the "Load Video (Upload)" node to set the proper number of frames to adapt to your input video: frame_load_cape to set the maximum number of frames to extract, skip_first_frames is self explanatory, and select_every_nth to reduce the number of frames.

In this workflow, two types of ControlNets are used together: OpenPose that we saw earlier, and Depth, to capture better depth and shadows from the frames.

more: https://medium.com/@emabyte/prompt-travelling-and-animatediff-a-tutorial-for-comfyui-b5b4f701a7ef