What's this?

Reproduces a Lock-On Stabilization effect centered on an arbitrary object.

Workflow

• Extract the first frame from a video
• 🟪 Use Florence-2 to detect the ball via object detection
• 🟫 Since detecting only the ball also caused the arm to be included in the segmentation, detect the arm position with Florence-2 and pass it to SAM2 as a negative prompt
• 🟩 Use the position information from Florence-2 to segment the target object with SAM2
• 🟨 Use ComfyUI-Inpaint-CropAndStitch to create a cropped region with padding around the masked area, and turn it into a video
• Increasing context_from_mask_extend_factor will add more padding