Put your phone in airplane mode and start asking your AI assistant for things. What still works today is a decent map of what on-device AI has become: transcription and summaries keep running, live call translation keeps humming, and short writing help answers in a beat. What fails tells you the rest. Anything that needs a frontier model, a web search, or a photorealistic image quietly stops working, because it was never running on your phone in the first place.
That gap between the two is the whole subject of this piece. "On-device AI" is now a marketing phrase attached to almost every phone and laptop sold in 2026, so the useful question is no longer whether your device has AI. It is: what genuinely runs on your hardware, what phones home, and when does that distinction actually matter?
What on-device AI actually means right now
An on-device model is a compressed neural network that lives in your device's storage and runs on its silicon: the Neural Engine on iPhones and Macs, the neural processing unit (NPU) in Copilot+ PCs and modern Android flagships. Nothing about your request leaves the device: no prompt, screenshot, or audio file gets uploaded anywhere, because there is no server in the loop at all.
Three things made this practical rather than theoretical. Phone and laptop chips gained NPUs that handle AI workloads without waking the power-hungry main CPU. Model compression got good enough that 1-to-4-billion-parameter models, the size of Apple's foundation models, Google's Gemini Nano, and Microsoft's Phi Silica, run comfortably on a phone. And the big platforms stopped treating local AI as a hobby: Apple rebuilt Apple Intelligence for iOS 27 around on-device models, Google's AICore system ships Nano on Pixel and Galaxy flagships, and Microsoft documents Phi Silica as a first-class Windows AI component.
The important nuance: these small models are specialists. Phi Silica is genuinely good at summarizing, rewriting, reformatting, and short text generation, all locally. It is not going to write your strategy document or reason through a multi-step problem. That division of labor, quick private tasks on-device, heavy lifting in the cloud, is the architecture every major vendor converged on this year.
What genuinely runs offline, platform by platform
The airplane-mode test sorts every feature into two buckets. Here is where things stand.
Apple (iOS 27, macOS 27 Golden Gate). Apple Intelligence processes routine requests entirely on the device: writing tools, notification and mail summaries, smart replies, photo search, and features like Call Context, which surfaces helpful info about who you are calling and, per Apple, runs entirely on-device. When a request is too complex for the local model, Apple Intelligence hands it to Private Cloud Compute, its cryptographically auditable server system. Notably, iOS 27 opened Private Cloud Compute to third-party apps for the first time, though Apple limits that access to eligible developers, and meeting apps have already added it as a user-selectable processing option. You can also open Settings, then Privacy & Security, then Apple Intelligence Report, and export a JSON log of every request your iPhone sent to the cloud. That report is the single best tool for answering "what leaves my device?" on Apple platforms.
Android (Pixel, Galaxy). Gemini Nano runs locally through the AICore system service on flagship hardware, powering features like summarization, smart reply, and screenshot understanding with no server calls, and Google's own developer documentation is explicit that on-device generative AI eliminates server calls and enables offline functionality. Galaxy AI on recent Samsung flagships layers on Live Translate for calls, Chat Assist, and Note Assist; some of these run locally, others dip into the cloud depending on the feature and language. The honest summary: core assistance features survive airplane mode, the flashier generative ones mostly do not.
Windows (Copilot+ PCs). Microsoft's local stack is the most documented. On a Copilot+ PC, an NPU with at least 40 TOPS, Phi Silica handles text intelligence locally, Live Captions translates over 40 languages into English offline, and Recall (opt-in, local, Windows Hello-protected) indexes your screen history on the machine. Microsoft's Text Recognition (OCR) API also runs on the NPU, which makes on-device OCR a Copilot+ feature on Windows. A June 2026 experimental change lets developers run Phi Silica on non-Copilot+ PCs with an RTX 30-series or newer GPU and 6GB+ of VRAM, though Recall and Click to Do remain locked to NPU machines.
The cross-platform layer. Independent apps fill the gaps the platforms leave. Whisper, OpenAI's open-source speech model, is the backbone: Whisper Notes transcribes entirely offline on iPhone and Mac via the Neural Engine, MacWhisper wraps Whisper and Parakeet models on macOS, and Superwhisper and VoiceInk replace system dictation locally. On the desktop, Ollama, LM Studio, and Jan run full open-weight LLMs with zero cloud involvement. If you want a deeper comparison of those three, we covered them in LM Studio vs Ollama vs Jan.
The privacy difference: processing versus collection
Here is the distinction most "private AI" marketing blurs: where is my data processed, and is my data collected?
On-device processing answers both at once. If the audio never leaves your phone, there is no transcript on a vendor's server to retain, to breach, to train on, or to hand over. Apple's security team makes the point plainly: data that exists only on user devices is disaggregated and has no centralized point of attack.
Private Cloud Compute, the system Apple routes complex requests to, is the strongest attempt anyone has made at splitting the two. Data sent to PCC is used only to fulfill the request, never stored, and inaccessible even to Apple; independent researchers can inspect the code running on the servers. That is a far stronger promise than a typical cloud AI service, where your prompts may be retained for abuse monitoring and, in some cases, reviewed by humans.
But 2026 added a complication worth knowing about. As The Verge reported from WWDC, Apple's new cloud models are built in collaboration with Google, and PCC now runs on Google Cloud hardware with Nvidia GPUs and Intel CPUs, not just Apple's own silicon. Apple says it retains complete control of the software and maintains a verifiable ledger of the hardware involved. It is still a meaningful privacy guarantee. It is just no longer the pure "Apple silicon all the way down" story from 2024.
The practical takeaway: on-device processing is a hard privacy guarantee. Cloud AI with strong policy safeguards, PCC included, is a trust decision. For genuinely sensitive material, client meetings, medical documents, legal drafts, the hard guarantee is the one you want. For that scenario we cover it separately in private and local AI tools for sensitive documents.
Honest limits: battery, quality, and the feature matrix trap
Three limits keep on-device AI from being the whole answer.
Small models are small. A 3-billion-parameter model summarizes a paragraph well, drafts a passable email, and summarizes a meeting transcript credibly. Ask it to synthesize a 40-page contract, reason through a multi-step plan, or write something genuinely nuanced, and quality visibly degrades. This is why every platform keeps a cloud path, and why airplane mode confirms quickly which side a task falls on.
Battery is better than you fear, but not free. NPUs are engineered for sustained, power-efficient inference; that is their entire reason to exist next to the GPU. Short on-device tasks, a summary, a transcription chunk, are gentle on the battery in a way that laptop-GPU inference is not. Running a large open-weight model locally for an hour is another matter. The NPU efficiency claim is real for the workloads it was designed for, not a blank check.
"On-device" labels hide a mixed reality. Most flagship AI feature lists mix local and cloud features without saying which is which. The same phone will summarize notifications locally, generate images in a data center, and translate a call in real time with a hybrid split, all under one marketing umbrella. If on-device behavior matters to you, the airplane-mode test beats every spec sheet: if a feature works offline, it is local; if it spins and fails, it was never on your device. The same logic applies to screenshot search tools, which we compared in AI screenshot organizers you can search.
What to actually set up
If the airplane-mode test appeals to you, here is a lean stack that works offline across devices:
- Transcription: one offline Whisper app per platform (Whisper Notes on iPhone and Mac, MacWhisper on Mac, whisper.cpp on Linux). Record anything sensitive, transcribe locally, and the audio never existed anywhere else.
- Summaries and drafting: your platform's built-in on-device writing tools for short tasks, which cover most everyday summarizing and rewriting.
- A local LLM where it fits: Ollama or LM Studio on a Mac, PC, or flagship phone for private chat with your own documents. Expect useful, not frontier, quality.
- The hybrid pattern: transcription and short summaries on-device, then send only the non-sensitive parts to a cloud model. Many people find this beats both extremes.
Feature details are as published by each vendor and can change; confirm on the official site.
The verdict
On-device AI in late 2026 is real, useful, and finally honest about its own scope. The summarization, transcription, OCR, search, and short-drafting work that fills most of an ordinary day can now happen entirely on your hardware, offline, with a hard privacy guarantee rather than a policy promise. The frontier capability, deep reasoning, long-document synthesis, image generation, still lives in the cloud, now under more scrutiny and with better safeguards than two years ago, but in the cloud all the same.
The smart setup is not purist. Use on-device for everything sensitive and everything routine; reach for cloud models deliberately, with your eyes open, when the task genuinely demands them. The airplane-mode test tells you in ten seconds which one you are doing.
