Scaling to 100,000 Concurrent Sandboxes: ComputeSDK's Technical Playbook

ComputeSDK's founder details the infrastructure decisions and operational insights that enabled their platform to support 100,000 concurrent sandboxes, emphasizing isolation and resource efficiency.

Garrison, founder of ComputeSDK, claims his platform has scaled to support 100,000 concurrent sandboxes, a technical milestone detailed in a recent blog post. This reported achievement highlights a specific approach to infrastructure design, prioritizing strong isolation, rapid startup times, and efficient resource utilization for on-demand execution environments.

Firecracker MicroVMs for Isolation

The core of ComputeSDK's architecture for achieving high concurrency relies on Firecracker microVMs. The founder reports that Firecracker provides a balance between the strong isolation of traditional virtual machines and the lightweight nature of containers. Each sandbox runs within its own Firecracker instance, ensuring process and memory separation. This choice addresses a fundamental requirement for sandboxed environments: preventing cross-tenant interference and maintaining security boundaries. The lightweight nature of Firecracker also contributes to faster startup times, which is critical for on-demand sandbox provisioning.

Custom Network and Filesystem Efficiency

To manage network isolation at scale, ComputeSDK developed custom kernel modules, specifically leveraging Linux network namespaces (netns). This approach allows each sandbox to operate within its own virtual network stack, preventing direct communication between sandboxes unless explicitly configured. For filesystem efficiency, the platform uses overlayfs. This copy-on-write filesystem allows for rapid provisioning of new sandboxes by layering a writable top layer over a shared, read-only base image, minimizing disk usage and speeding up sandbox creation. Resource limits for CPU and memory are enforced using cgroups.

Go Orchestration and Observability with eBPF

ComputeSDK's control plane, responsible for orchestrating these sandboxes, is built in Go. The founder states that Go's concurrency primitives and performance characteristics make it suitable for managing a large number of concurrent operations. For observability and security, the platform integrates eBPF (extended Berkeley Packet Filter). eBPF allows for dynamic, programmable tracing and monitoring of kernel events without modifying kernel code, providing deep insights into sandbox behavior and potential security anomalies at a low performance cost. This enables fine-grained control and debugging across the reported 100,000 concurrent instances.

What We'd Change

This playbook, while demonstrating significant technical prowess, presents a high barrier to entry for most teams. The reliance on custom kernel modules and deep systems-level engineering expertise for network isolation (netns) and eBPF integration is not a generalizable solution for many startups. Replicating this approach requires a specialized team with a deep understanding of Linux internals, which is a significant hiring and operational cost.

Furthermore, the decision to build a custom orchestration layer in Go, while effective for ComputeSDK's specific needs, might not be optimal for others. Managed container orchestration services (like Kubernetes) or serverless platforms (like AWS Lambda or Google Cloud Run) offer higher-level abstractions that reduce operational overhead, even if they introduce some overhead or less granular control. The trade-off between custom control and operational simplicity is a critical consideration that depends heavily on the specific product requirements and team capabilities. For most applications that do not require this extreme level of isolation or performance, off-the-shelf solutions would be more pragmatic.

The ComputeSDK experience underscores that achieving extreme scale and specific isolation requirements often necessitates bespoke, low-level engineering. This approach moves beyond readily available cloud primitives and managed services, demanding significant investment in core infrastructure development. For founders building similar developer tools or secure execution environments, the lesson is clear: off-the-shelf solutions will eventually hit a ceiling, requiring a pivot to custom systems engineering to push performance and isolation boundaries.

The investor read

The ability to manage 100,000 concurrent sandboxes signals a robust demand for highly isolated and performant execution environments, particularly within developer tooling, secure code execution, and cloud-native testing categories. This technical feat suggests a significant competitive moat for ComputeSDK, built on deep systems engineering expertise. Investors are increasingly looking for infrastructure plays that can abstract away this complexity for a broader market, or for companies capable of building such foundational technology. The use of Firecracker, eBPF, and custom kernel modules indicates a trend towards specialized, low-level optimization for performance-critical workloads, moving beyond general-purpose containerization. This level of engineering suggests a high capital expenditure in talent and R&D, potentially making it an attractive target for strategic acquisition by larger cloud providers or platform companies seeking to enhance their developer offerings.

Pull quote: “The ComputeSDK experience underscores that achieving extreme scale and specific isolation requirements often necessitates bespoke, low-level engineering.”