dstack

dstack is an open-source control plane for provisioning compute and running training, inference, and sandboxes across NVIDIA, AMD, TPU, and Tenstorrent GPUs—on clouds, Kubernetes, and bare-metal clusters. It provides a unified interface for managing AI infrastructure without requiring Kubernetes or Slurm expertise. Users define compute fleets, tasks, services, and dev environments using simple YAML configurations. dstack automates cluster provisioning, resource allocation, and job scheduling, and reports GPU utilization, memory usage, and health metrics during execution. It supports multi-GPU and multi-node distributed training, high-performance model inference with frameworks like SGLang and vLLM, and secure, auto-scaling OpenAI-compatible endpoints. dstack integrates with major cloud providers (AWS, GCP, Azure, Lambda, etc.) and can run on existing Kubernetes clusters or SSH-connected bare-metal servers. It is designed for both human engineers and AI agents, offering a CLI and a hosted version (dstack Sky) with GPU credits.