Discord如何利用Go和C++每天调整1.5亿张图片的大小

Despite being a voice and text chat app, Discord sees over a hundred million images passing through its tubes every day. While we wish it was as simple as sending them out across the tubes to your friends, delivering these images creates some pretty large technical problems. Linking directly to images would leak users’ IP addresses to image hosts, and large images use up lots of bandwidth. To circumvent these problems, Discord needs an intermediary service to fetch images on behalf of users and then resize them to reduce bandwidth usage.

尽管是一个语音和文本聊天应用程序，但Discord每天都有超过一亿张图片通过其管道。虽然我们希望能像在管道中向你的朋友发送图片一样简单，但发送这些图片会产生一些相当大的技术问题。直接链接到图片会将用户的IP地址泄露给图片主机，而且大型图片会占用大量的带宽。为了规避这些问题，Discord需要一个中介服务来代表用户获取图片，然后调整其大小以减少带宽的使用。

Enter the Image Proxy

输入图像代理

To handle this job, we created a Python service creatively named Image Proxy. It fetched images from remote URLs and then used the pillow-simd package to do the heavy lifting of image resizing. Pillow-simd is wonderfully fast and uses x86 SSE instructions to accelerate resizing where it can. The Image Proxy would receive a HTTP request containing a URL to fetch, resize, and finally respond with the image.

为了处理这项工作，我们创造性地创建了一个名为Image Proxy的Python服务。它从远程URL获取图像，然后使用pillow-simd包来完成图像大小调整的重任。Pillow-simd的速度非常快，使用x86的SSE指令来加速调整大小。图像代理将收到一个包含URL的HTTP请求，以获取、调整大小，并最终响应图像。

On top of this, we setup a caching layer that would try to keep resized images around in memory and respond directly from cache when it could. A HAProxy layer routes requests based on a URL hash to the Nginx caching layer. The cache performs request coalescing in order to minimize the number of resize transformations required. This combination of cache and proxy was enough to scale our image proxy up well into millions of users.

在此基础上，我们设置了一个缓存层，尝试在内存中保留调整后的图片，并在可能的情况下直接从缓存中响应。HAProxy层根据URL哈希值将请求路由到Nginx缓存层。缓存执行请求凝聚，以尽量减少所需的大小转换的数量。缓存和代理的这种组合足以将我们的图像代理扩展到数百万用户。

Still, as Discord grew, the Image Proxy started to show signs of strain. The biggest problem was the Image Proxy did not have an even workload distribution, w...