Multi-gate-Mixture-of-Experts (MMoE) model architecture and knowledge distillation in Ads Engagement modeling development | by Pinterest Engineering | Pinterest Engineering Blog | Apr, 2025

Multi-gate-Mixture-of-Experts (MMoE) model architecture and knowledge distillation in Ads Engagement modeling development | by Pinterest Engineering | Pinterest Engineering Blog | Apr, 2025

摘要

多门混合专家模型（MMoE）通过动态分配计算资源提升效率，促进多任务学习。结合知识蒸馏，弥补数据保留期短导致的性能差距，增强新模型表现。实验验证，MMoE在广告用户匹配任务中表现优异，显著提升线上线下指标。混合精度推断和轻量门层设计有效降低基础设施成本，确保模型性能不降。