ChatGPT
Perplexity
Claude
Gemini
Evaluated across 100 ad briefs and 500 videos, Director earns top rankings across all three quality categories and delivers the most consistent overall scores.
OpenArt Director has ranked #1 in Physion Labs' Arc Ads benchmark, an independent human evaluation of five AI ad-generation agents. It shares the top spot with Runway.
A strong video ad brings together a clear message, a compelling story, and polished execution. It needs to capture attention while keeping the product, offer, and creative direction faithful to the brief.
Physion's evaluation examines how well AI agents bring these requirements together. For OpenArt, the results mark another milestone in building creative intelligence that helps people turn ideas into finished creative work.
From an ad brief to a finished creative
Each agent received the same 100 ad briefs, producing 500 video ads assessed across 24 metrics.
Human evaluators checked whether each video followed the requested storyline, format, on-screen text, commercial claims, and audio. They also assessed consistency across scenes and compared ads side by side to judge their creative execution.
Director earned a top ranking in all three categories:
- Brief Fidelity and Coherence: Following the instructions and keeping the story, product, characters, and style consistent.
- Production Quality: Producing well-executed visuals, movement, audio, lighting, and color.
- Advertisement Effectiveness: Human assessments of attention, clarity, originality, brand impression, and persuasiveness.
The effectiveness category evaluates the creative itself through human judgments, rather than live campaign results.
Highest scores on four production essentials
Director scored higher than every other evaluated agent, including Runway, on four individual measures:
- Following the story: Including the required scenes and actions in the order specified by the brief.
- Delivering the requested sound and dialogue: Including the required audio elements and spoken lines.
- Rendering clean visuals: Avoiding malformed faces, bodies, objects, and other visual defects.
- Keeping audio stable: Maintaining steady levels without clipping, crackle, dropouts, or unintended silence.
These are practical details that determine whether a creative follows its brief and how much attention its execution may need. Director's highest scores on these measures reflect strengths in translating instructions into a coherent, polished ad.
The most consistent overall scores across 100 briefs
Director also recorded the lowest variation in overall quality scores among the five agents evaluated.
That matters because creative production involves different products, messages, and storytelling requirements. A useful workflow needs to deliver across those assignments.
The result provides evidence of Director's consistency across the tested briefs, alongside its top overall ranking.
Building creative intelligence for advertising
An ad brief establishes what a creative needs to communicate. Turning it into a finished video requires connected decisions about scenes, pacing, composition, and sound.
This is the creative intelligence we are building with Director: understanding a creator's intent and coordinating the choices needed to express it.
Director brings that process into a conversational workflow, with a professional timeline for refining the result. Creators can develop a concept, guide its execution, and shape the final edit while keeping the brief at the center of the work.
OpenArt's ambition is to lead the agentic shift for creative work, making capable creative production accessible to more people.
"Our mission is to make creativity accessible. A strong creative idea shouldn't depend on having a large production budget to bring it to life," said Coco Mao, CEO of OpenArt. "We're building Director to help people turn their vision into finished work, with control over the story, message, and final result."