Highly Experimental IPAdapter Tryon - Livestream workflow jam session
5.0
0 reviewsDescription
IMPORTANT
This workflow is the result of a live jam session during a livestream: https://youtube.com/live/UXpKVrLQmbI
As such, it's not production ready. I mean, it's not even polished - if you run it you can see that the second pass, during the final merge, results in some artifacts from the blending.
What it's intended for, however, is explaining the limiting factors of a VITON based on IPAdapter and CLIPVision, in regards to output resolution differences and the technical limits of CLIPVision in how it "sees" stuff.
How it works:
- The first pass needs a top and bottom garments, as well as two masks (one for the top, one for the bottom) drawn on top of a human posing.
The first pass is meant to highlight the capabilities of a IPAdapter based approach when generating a full body picture.
- The SAM pipeline segments the image into two parts, one for top and one for bottom, and provides two cropped images rescaled to 1024x1024, one for top and one for bottom, as well as the attention masks for the IPAdapter.
- The second pass regenerates the top and bottom garments separately, and should give better results for consistency. However, it's also meant to display the limits of CLIPVision, since it doesn't resolve smaller details, such as fine textures and logos, regardless of output size and crop size.
- The blend pass is the least polished, and you're *very* advised to find a better solution yourself. It's meant to patch the top and bottom together on top of the first pass output.
If you find a better solution, I'd love to hear it! You can either comment on the livestream link, or send me an email at andrea@andreabaioni.com
Overall, treat this workflow as a fun exercise rather than a working file.
Node Diagram
Discussion
(No comments yet)
Node Details
Primitive Nodes (9)
Note (9)
Custom Nodes (74)
ComfyUI
- VAEEncode (3)
- LoadImage (4)
- CLIPTextEncode (6)
- PreviewImage (10)
- CheckpointLoaderSimple (3)
- KSampler (3)
- ControlNetLoader (3)
- ControlNetApply (3)
- VAEDecode (3)
- MaskToImage (2)
- ImagePadForOutpaint (2)
- GrowMask (2)
- ImageResize+ (2)
- PreviewBridge (3)
- SAMLoader (2)
- DepthAnythingPreprocessor (3)
- IPAdapterUnifiedLoader (3)
- IPAdapterAdvanced (4)
- ImageGaussianBlur (2)
- Crop (mtb) (2)
- GroundingDinoModelLoader (segment anything) (2)
- GroundingDinoSAMSegment (segment anything) (2)
- JWImageResize (3)
- Image Blend by Mask (2)
Model Details
Checkpoints (3)
epicrealism_naturalSinRC1VAE.safetensors
LoRAs (0)