Weaknesses of ReActor

• ReActor can only create images of 128px × 128px, so the resolution is inherently low.
• To compensate for this, there are tools like Codeformer and GFPGAN, but they tend to make the skin look flat.
• Also, it lacks a sense of depth.
• We resolve this by using a Detailer with InstantID.

Workflow

• If the base image is too large, it will be automatically resized so that the shorter side becomes 2000px.
• Face Swap with ReActor.
• Generate a caption using JoyTag based on SEGS and use it as a prompt.
• Add InstantID.
• Run two cycles of sampling.
• Combine the SEGS, which has been processed by the Detailer, with the base image before resizing using the SEGSPaste node.

Unresolved Weaknesses of ReActor

• ReActor cannot fully respond to images taken from extreme angles, such as from directly above.
• If the base image shows the forehead but the reference image has bangs, it may not work well.
• It cannot keep up with large changes in facial expressions, such as winking or sticking out the tongue.

🚨Update

• 22/2/2024
• Due to the update of InstantID, the old workflow stopped working, so I have made corrections.
• I have modified the reference images to segment within the range of the face.
• I have changed the prompt generation from Joytag to llava-v1.6-mistral-7b.

日本語での解説

• https://scrapbox.io/work4ai/ReActor%E3%81%A7Face_Swap%E3%81%97%E3%81%9F%E3%81%82%E3%81%A8%E3%81%ABInstantID%E3%82%92%E4%BD%BF%E3%81%A3%E3%81%A6%E3%83%AA%E3%83%95%E3%82%A1%E3%82%A4%E3%83%B3%E3%81%99%E3%82%8B