Carton

Carton is an open-source tool that provides a unified API to run machine learning models from any programming language. It wraps models without conversion, using a runner system to execute with the original framework. Carton is implemented in Rust with bindings for multiple languages, and supports x86_64 and aarch64 Linux and macOS, with WebAssembly for metadata access. It reduces complexity by decoupling inference code from specific ML frameworks.

Key Features

• Pack: Wrap a model with metadata into a .carton file without modifying the original.
• Load: Carton reads metadata to fetch and use the appropriate runner for the framework version.
• Run: Framework-agnostic inference via Carton's API.

Use Cases

• Deploying ML models across different programming languages without framework-specific code.
• Experimenting with multiple frameworks without conversion steps.

Who It's For

• Developers and data scientists who need to integrate ML models into applications using various programming languages.