Taming the tail utilization of ads inference at Meta scale

Taming the tail utilization of ads inference at Meta scale

摘要

Tail utilization is a significant system issue and a major factor in overload-related failures and low compute utilization. The tail utilization optimizations at Meta have had a profound impact on …