PosterOmni- Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedback

如果无法正常显示,请先停止浏览器的去广告插件。
分享至:
1. Part 1. Poster Creation Model PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedback GitHub: https://github.com/MeiGen-AI/PosterOmni Web: https://ephemeral182.github.io/PosterOmni/ Model: https://github.com/MeiGen-AI/PosterOmni ArXiv: https://arxiv.org/abs/2602.12127 Presenter: Jianyu Lai MPhil at HKUST(GZ) ; Incoming PhD at HKUST ; Research Intern at Meituan 1
2. 1. Introduction – About Poster Creation CreatiDesign POSTAPosterCraft(CVPR’25)(ICLR’26)(ICLR’26) Use a modular MLLM design framework to ensure text accuracy and layout balance.Use a unified end-to-end framework to directly generate posters from prompts.Designed for multi-conditional image generation and can be naturally extended to editing tasks without additional training. 2
3. 1. Introduction – Motivation of PosterOmni Challenge: 1. Models must generate text, layout, style, and visual elements while preserving semantic fidelity and aesthetic coherence. 2. No open framework currently targets multi-task image-to-poster creation PosterOmni unifies local editing and global creation within a single image- to-poster generation framework. It covers six representative tasks— extending, filling, rescaling, identity- driven, layout-driven, and style-driven poster generation—enabling the model to achieve both fine-grained visual editing and holistic aesthetic composition. 3
4. 2. Method – Dataset Construction 4
5. 2. Method – Dataset Construction 5
6. 2. Method – Framework 1 Task-Specific SFT The six tasks were divided into two groups: local edit ing and global generation. Expert models were train ed for each group to achieve stronger task specializa tion capabilities in independent scenarios. 2 Task Distillation By distilling the knowledge of experts from differen t tasks into a unified model, we can reduce task int erference while taking into account both local editi ng and global generation. 3 PosterOmni reward training A unified reward model is constructed to model the preference between the aesthetic quality of the gen erated results and the task completion, providing a s table reward signal for subsequent reinforcement le arning. 4 Reward-guided reinforcement learning We utilize the preference signals output by the rewa rd model to perform Omni-Edit reinforcement learni ng, continuously optimizing the performance of the unified model in multi-task poster creation. 6
7. 2. Method – Framework Supervised fine-tuning can lead to shortcuts. We train a unified reward model to provide both general aesthetic and task-specific signals. We added labels to different tasks to help the reward model learn preference judgments for different tasks. Insight: Using different models to construct reward data may lead to reward hacking; it is best to use the SFT-enhanced model to construct preference data to ensure in-distribution. 7
8. 3. Experiments Gemini-2.5-Pro Pointwise Scoring User study on 150 samples Its performance metrics across all four dimensions / six tasks surpassed the baseline (Qwen-Image-Edit) and were on par with or exceeded the then-advanced editing model Seedream-4.0. 8
9. 3. Experiments A poste r for ... , featuring a tall ce ra mic vase on the left, a stac k of recycle d paper books in the ce nte r, a woode n donation box on the right, and a small terra cotta pot besides the box. The background is a soft sage gre en ... . At the top is the title \“N ature ’s Charity Fair\”; below is \“G ive Bac k to the Earth\". Type: Extending Image-to-Poster Prompt Thi s refi ned poster announces "SERENADE UN DER THE STARS: An Orch estral Evenin g." The central image i s a beautiful ly photographed silhoue tte of a symphon y orchestra performing on an ou tdoor stage … Th e event title, "SERENADE UNDER THE S TARS," is wri tten i n a graceful , ti meless, serif ... The su btitle, "AN ORCHEST RAL EVENING FEAT URING THE CELESTIAL PHILHARMONIC," is i n a small er, simi lar font below i t. Date and ven ue, "JULY 2 0TH, PINE GROVE AMPHITHEATER," are at the bottom i n a clean , legi ble seri f … Reference ImageFLUX-KontextQwen-Edit [2509]UniWorld-V2-QwenSeedream-4.0PosterOmni Reference ImageQwen-Edit [2509]UniWorld-V2-QwenGemini-2.5-ProSeedream-4.0PosterOmni Reference ImageBagelQwen-Edit [2509]FLUX-KontextSeedream-4.0PosterOmni Reference ImageUniWorld-V2-QwenQwen-Edit [2509]Seedream-3.0Seedream-4.0PosterOmni A pair of metal handcuffs, covered with reddish-brown rust and mottled oxidation marks, featuring circular cuffs and vis ibly worn links at the chain connections, res ts on a yellowed newspaper. The metal appears coarse, bearing the texture and signs of prolonged aging. Type: Filling Image-to-Poster Prompt Refer to the layout of this poster and create a new poster featuring an towel as the main subject, with a picture frame and a toothbrush placed beside it. The text sh ould in clude \"Home Decor Inspirations\" at the top and \"Create Your Space\" in the center. Type: Layout-driven Image-to-Poster Prompt Referencing the style of this poster, create an illustration featuring a colorful library at the center of the image, with a cartoon robot in the foreground, a beam-of-light tunnel at the upper right, and a giant Rubik's cube at the bottom. Include the text \"Future Tech Playground\" at the top and \"Explore Infinite Possibilities\" at the center of the image. Type: Style-driven Image-to-Poster Prompt 静物 摄影 风格 海报 。画面 左侧 放置 着一 枚精 钢表 壳、 深蓝 色表 盘、棕色 鳄鱼 皮表 带的 机械 腕表 。画面 右下 角,摆放 着一 部磨 砂黑 色机 身、精致 的智 能手 机。两 者均 静置 于温 润的 浅色 木质 台面 上。柔和 的自 然光 线 从侧 面洒 落,形成 细腻 的光 影变 化。背景 深度 虚化 , 呈现 出温 暖而 宁静 的家 居氛 围。构图 简洁 ,景深 显著 , 主体 锐利清 晰,海报 级画质 。主标 题:“品味 生活,时 间尽 显”,副标 题:“腕间 风尚,伴你 同行” Type: ID-driven Image-to-Poster Prompt Reference Images Qwen-Edit [2509] Seedream-3.0 Gemini-2.5-Pro Seedream-4.0 PosterOmni Rescale the poster from 4:3 (weight : height) to 3:4 Type: Rescaling Image-to-Poster Prompt Reference Image Bagel FLUX-Kontext Qwen-Edit [2509] UniWorld-V2-Qwen Seedream-3.0 Seedream-4.0 PosterOmni 9
10. 3. Experiments [Extending] A vibrant pop art poster for an IP limited edition product line, featuring a bold red s neaker with a graphic print on the left, a sleek blue smartphone case with a metallic logo in the center, and a gold collectible pin with a character design on the right—all set against a dynamic background of comic book dots and stripes. At the top is the title "IP Limited Edition Collection", and in the center is the text "Exclusive Drops Available Now". [Filling] rescale this poster from 4:3 to 9:16 [Filling] Three orange emergency kits, featuring reflective strips and black shoulder straps, neat ly arranged. Made from durable, sturdy materials and brightly colored to enhance visibility. 10
11. 3. Experiments [Layout Driven] Refer to the layout of this poster and create a new poster featuring a larg e beige speaker cabinet filled with plush toys. Next to it, place a metal high stool and a pai ntbrush with a wooden handle. On the right side, include a cluster of blooming light purpl e 3D-printed models. Add the text "Smart Space: Innovative Living" at the top and "Collab orative Exploration, Enhanced Experience" at the bottom. [Style Driven] Referencing the style of this poster, create an illustration featuring a col orful library at the center of the image, with a cartoon robot in the foreground, a bea m-of-light tunnel at the upper right, and a giant Rubik's cube at the bottom. Include th e text "Future Tech Playground" at the top and "Explore Infinite Possibilities" at the ce nter of the image. [Identity Driven] 3D anime-inspired poster with intricate details and vivid lighting, exuding a sense of aesthetic beauty. On the left side of the image, a lush potted bamboo plant feat ures vibrant, dew-kissed leaves gently swaying in the breeze, slender green stalks reaching upward, and a pot that embodies antiquity and elegance. In the lower right corner, a uniqu ely shaped cactus stands resilient in a simple terracotta pot, its surface covered with fine fu zz and sharp spines. The background showcases a bright, transparent indoor greenhouse sp ace, where soft natural light pours in through the top and sides, creating a warm and tranq uil atmosphere. The air is filled with the fresh scent of greenery …… 11
12. Thank You

ホーム - Wiki
Copyright © 2011-2026 iteam. Current version is 2.155.2. UTC+08:00, 2026-06-22 05:55
浙ICP备14020137号-1 $お客様$