GPU Sandboxes
GPU-accelerated sandboxes and their capabilities.
GPU sandboxes are Linux sandboxes with one or more dedicated GPUs attached. Each sandbox has its own files, network connection, and fixed amount of processor, memory, and disk. The GPUs assigned to a sandbox are not shared with other sandboxes while it runs, and Daytona isolates each GPU sandbox from other sandboxes and from the host.
GPU sandboxes run workloads that require GPU acceleration, such as model inference, training, fine-tuning, and reinforcement learning. A create request asks for a number of GPUs, and Daytona places the sandbox on a host that has enough free GPUs.
A GPU sandbox is ephemeral: deleting it discards its filesystem. Its state can be saved as a checkpoint that includes its processes, memory, and GPU memory, and resumed in a new GPU sandbox. The available GPU models and the features and limits of GPU sandboxes may change over time.
| Term | Description |
|---|---|
| Sandbox | An isolated Linux machine with dedicated GPUs, created on demand to run code. |
| GPU | A graphics processing unit assigned to one sandbox at a time. |
| GPU count | The number of GPUs a sandbox requests when it is created. |
| Host | The physical server with GPUs that runs sandboxes. Daytona calls these servers runners. |
| Image | A packaged filesystem that defines what software a sandbox starts with. |
| GPU memory | The memory on a GPU that holds the data the GPU works on, such as model weights. |
| Checkpoint | A saved copy of a sandbox's processes, memory, and GPU memory that a new GPU sandbox resumes from. |
| Ephemeral | A sandbox whose filesystem is discarded when it is deleted. |