local.ai

local.ai is a free and open-source native application designed for running AI models locally on your computer without a GPU. It simplifies the entire workflow of downloading, managing, verifying, and inferencing with large language models, all while keeping data private and offline. Built with a Rust backend, the app is memory efficient and compact (under 10MB).

Key Features

• CPU Inferencing: Performs AI inference on the CPU, automatically adapting to available threads and supporting GGML quantization (q4, q5.1, q8, f16).
• Model Management: Provides a centralized location to manage AI models with a resumable, concurrent downloader, usage-based sorting, and directory-agnostic storage.
• Digest Verification: Ensures model integrity using BLAKE3 and SHA256 digest computation, with a known-good model API, license chips, and an info card for each model.
• Inferencing Server: Starts a local streaming inference server in two clicks, along with a quick inference UI that writes conversations to.mdx files and supports remote vocabulary configuration.