How Hyperforce Edge Networking Handles 20M Domains on 30GB RAM
In our “Engineering Energizers” Q&A series, we shine a spotlight on the innovative engineers at Salesforce. Today, we feature Kshitij Dogra, whose Hyperforce Edge Networking team scaled the domain capacity from 3 million to over 20 million domains, while keeping the system memory usage under 30GB.
Discover how the team tackled significant scalability challenges, reengineered certificate management to replace Vault at scale, and maintained rapid deployment without sacrificing trust or security.
The Hyperforce Edge Networking team ensures a fast, secure, and reliable experience for Salesforce customers around the globe. Acting as an internal CDN, Edge service accelerates content delivery across all Salesforce Clouds, enabling low-latency access to platform services. With operations now spanning more than 20 regions — up from 8 or 9 before the adoption of Hyperforce and AWS — data centers are strategically located in countries like Australia, India, Israel, Brazil, Indonesia, South Korea, and others. This strategic placement guarantees proximity-based performance for a wide range of Salesforce applications.
Salesforce Edge presence across world map
To uphold trust, the Hyperforce Edge Networking team implements robust security measures, including TLS encryption, mutual TLS authentication, and caching schemes while respecting customer trust and web application firewalls (WAFs) for availability and security. In environments where availability, latency, reliability, and security are paramount, the Salesforce Edge platform consistently delivers enterprise-grade performance & security across all regions.
To support growth from 5 million to over 20 million domains, Edge needed to re-architect its storage model. Each domain’s metadata — certificate name, organization, Salesforce endpoint — was stored in a separate file, consuming 4096 bytes due to the filesystem block size, even though each entry required less than 400 bytes. At 20 million domains, total configuration size would exceed 80GB, far beyond what a memory-based file system could handle, and would significantly increase cost-to-serve.
Domain configuration translation to the list model
Additionally, generating and extracting the TAR bundle for 20 million individual files took 50+ minutes, severely slowing web proxy startup. This volume of small files introduced both operational bottlenecks and I/O strain.
Edge memory usage for a 20 million domain-list bundle generation
To address this, the team implemented a domain list model: grouping multiple domain configurations into a single file to fully utilize each 4096-byte block. This drastically reduced file count and improved memory efficiency. The team also switched to AWS GP3 EBS volumes for faster I/O when reading the new aggregated files.
Results:
- TAR size reduced: 80GB → 6GB
- File extraction time: 50+ min → under 5 min
- Memory usage: 78% → 7%
- Scalability goal achieved: 20 million domains
Scalability challenges are managed by balancing architectural redesign with infrastructure optimization. Instead of relying solely on vertical scaling, the focus is on creating a sustainable and controllable system. For example, one significant bottleneck was encountered with Vault, the secret store for the Edge service, which experienced scaling challenges beyond 25,000 certificates. To resolve this, Vault was replaced with a custom solution backed by AWS KMS, which provided better performance, control, and scalability.
Hyperforce Edge Networking has now reached a stage where it is the default CDN entry against wildcard DNS records for certain domain types. This involves processing large volumes of legitimate and NXDomain (non-existent domain) traffic. To accommodate this, subtle advancements in the platform include multi-process caching to identify NXDomains and reduce resource utilization, multithreaded configuration processing in the control plane, and a fine-grained dynamic load balancing approach to minimize proxy reload times.
These architectural innovations were essential in enabling Salesforce to onboard high-volume customers while ensuring the platform remained stable and efficient.
Rapid deployment is achieved through a robust approach to security and operational safety. The blue-green deployment model allows for the seamless introduction of new features while ensuring that the system can easily revert to a stable state if any issues arise. This method ensures that updates can be rolled out quickly without affecting the platform’s uptime.
All secrets and TLS certificates are encrypted using AWS Key Management Service (KMS), with access tightly controlled and fully auditable. No secrets or certificates are ever stored or transmitted in plain text. Automated compliance checks and vulnerability scans are integrated into the deployment pipeline for edge systems. Furthermore, all production changes must pass through strict safe-change protocols that ensure backward compatibility and reduce the risk of regression.
By combining best practices in encryption, automated scanning, and rigorous deployment processes, the Hyperforce Edge Networking platform continues to deliver rapid updates without compromising customer trust.
System integrity is safeguarded through continuous integration testing, real-world load simulations, and coordinated testing with other teams. An internal test suite covers all essential edge functionalities and is regularly updated to catch issues early in the development process.
Major changes are thoroughly tested under conditions that mimic production environments to ensure optimal performance in terms of latency, throughput, memory usage, and file system impact. Any feature that increases I/O or resource consumption is rigorously tested under full load before it goes live.
Simultaneously, we collaborate with other Salesforce engineering teams to conduct user acceptance testing in shared environments. For example, when features like WAF are rolled out, these integrations are tested in staging environments to ensure seamless end-to-end functionality across stakeholders.
This collaborative and comprehensive testing process ensures that each enhancement maintains the system’s reliability, performance, and customer satisfaction.
We primarily gather feedback from internal Salesforce engineering teams that use Edge for production workloads. These teams provide valuable insights into performance bottlenecks, latency issues, and availability concerns.
For example, in the past one significant challenge was the need to onboard new domains onto our platform and make them immediately available, with minimal operational effort. To achieve this, the team had to innovate and develop a proactive domain configuration lookup approach, enabling rapid CDN onboarding. Currently, a few test organizations are using an automated UI flow to register an account with Salesforce and have begun using Hyperforce Edge on the fly.
To monitor global performance, we use synthetic traffic simulations. These simulations mimic real-user traffic from various locations and help us detect early signs of issues in routing, certificate validation, and edge service availability.
This ongoing feedback loop directly shapes the platform’s development roadmap and accelerates the resolution of critical issues.