OmniConsistency—新加坡国立大学推出图像风格迁移新模型

OmniConsistency 是新加坡国立大学开发的一款图像风格迁移模型，专注于解决复杂场景中风格化图像的一致性问题。该模型采用两阶段训练策略，将风格学习与一致性学习分离，确保在不同风格下保持语义、结构和细节的一致性。OmniConsistency 能够与任何特定风格的 LoRA 模块无缝集成，提供高效且灵活的风格化效果。在实验中，其性能与 GPT-4o 相当，并展示出更高的灵活性和泛化能力。

OmniConsistency Explained

OmniConsistency is an image style transfer model developed by the National University of Singapore. It addresses the issue of consistency in stylized images across complex scenes. The model is trained on large-scale paired stylized data using a two-stage training strategy that decouples style learning from consistency learning. This ensures semantic, structural, and detail consistency across various styles. OmniConsistency supports seamless integration with any style-specific LoRA module for efficient and flexible stylization effects. In experiments, it demonstrates performance comparable to GPT-4o, offering higher flexibility and generalization capabilities.

Key Features of OmniConsistency

• Style Consistency: Maintains style consistency across multiple styles without style degradation.
• Content Consistency: Preserves the original image's semantics and details during stylization, ensuring content integrity.
• Style Agnosticism: Seamlessly integrates with any style-specific LoRA (Low-Rank Adaptation) modules, supporting diverse stylization tasks.
• Flexibility: Offers flexible layout control without relying on traditional geometric constraints like edge maps or sketches.

Technical Underpinnings of OmniConsistency

• Two-Stage Training Strategy: Stage one focuses on independent training of multiple style-specific LoRA modules to capture unique details of each style. Stage two trains a consistency module on paired data, dynamically switching between different style LoRA modules to ensure focus on structural and semantic consistency while avoiding absorption of specific style features.
• Consistency LoRA Module: Introduces low-rank adaptation (LoRA) modules within conditional branches, adjusting only the conditional branch without interfering with the main network's stylization ability. Uses causal attention mechanisms to ensure conditional tokens interact internally while keeping the main branch (noise and text tokens) clean for causal modeling.
• Condition Token Mapping (CTM): Guides high-resolution generation using low-resolution condition images, ensuring spatial alignment through mapping mechanisms, reducing memory and computational overhead.
• Feature Reuse: Caches intermediate features of conditional tokens during diffusion processes to avoid redundant calculations, enhancing inference efficiency.
• Data-Driven Consistency Learning: Constructs a high-quality paired dataset containing 2,600 pairs across 22 different styles, learning semantic and structural consistency mappings via data-driven approaches.

Project Links for OmniConsistency

• GitHub Repository: http://github.com/showlab/OmniConsistency
• HuggingFace Model Library: http://huggingface.co/showlab/OmniConsistency
• arXiv Technical Paper: http://arxiv.org/pdf/2505.18445
• Online Demo Experience: http://huggingface.co/spaces/yiren98/OmniConsistency

Practical Applications of OmniConsistency

• Art Creation: Applies various art styles such as anime, oil painting, and sketches to images, aiding artists in quickly generating stylized works.
• Content Generation: Rapidly generates images adhering to specific styles for content creation, enhancing diversity and appeal.
• Advertising Design: Creates visually appealing and brand-consistent images for advertisements and marketing materials.
• Game Development: Quickly produces stylized characters and environments for games, improving development efficiency.
• Virtual Reality (VR) and Augmented Reality (AR): Generates stylized virtual elements to enhance user experiences.

[Note: All images remain in their original format.]

今天关于《OmniConsistency—新加坡国立大学推出图像风格迁移新模型》的内容介绍就到此结束，如果有什么疑问或者建议，可以在golang学习网公众号下多多回复交流；文中若有不正之处，也希望回复留言以告知！