Short answer: yes, many laptops sold in the last few years can run a capable AI model entirely on the machine, with no prompt or document leaving it. The longer answer is that "local" depends on three things most marketing glosses over — which model you run, how much memory you have, and whether the app you use quietly reaches out to a server anyway. Here is a practical guide for American users and IT buyers who want the privacy benefits without the guesswork.

What "running AI locally" actually means

A large language model is a file of numbers — the weights — plus software that does the math to turn your prompt into a reply. When ChatGPT or Gemini answers you in a browser, that math happens in a data center and your text travels there. Running locally means the weights sit on your drive and your own processor, graphics chip, or neural processing unit (NPU) does the work. Once the model is downloaded, it can run with Wi-Fi switched off. That last test is the simplest way to know you are truly local.

The hardware question: memory matters more than the logo

The single biggest constraint is memory. Models are commonly "quantized" — compressed to around 4 bits per weight — so a roughly 7- to 8-billion-parameter model typically needs on the order of 5 to 6 GB just for the weights, plus room for the conversation and your other apps. As a rough working guide:

8 GB of RAM: small models (1 to 4 billion parameters) for summarizing, rewriting, and simple Q&A. Usable, but expect limits. 16 GB: comfortable for 7–8B models, the current sweet spot for everyday writing and coding help. 32 GB or more: room for 13B–30B-class models and longer documents. Apple Silicon Macs share one pool of memory between CPU and GPU, which is why a 32 GB MacBook often handles larger models than a Windows laptop with a separate 8 GB graphics card.

NPUs are the newer piece. Microsoft set a 40+ TOPS NPU requirement for its Copilot+ PC category, announced in May 2024, and Intel, AMD, and Qualcomm now all ship chips that meet it. NPUs are power-efficient and good for built-in features like live captions and image effects, but many popular open-source tools still lean mainly on the CPU and GPU. An NPU is a plus, not a requirement.

The software: the tools people actually use

Three free tools dominate. Ollama runs models from the command line or through companion apps and is popular with developers. LM Studio offers a point-and-click desktop interface for browsing, downloading, and chatting with models. Both build on llama.cpp, the open-source engine that made efficient laptop inference practical. Model choice is wide: Meta released its compact Llama 3.2 1B and 3B models specifically for on-device use, according to its September 2024 announcement, and families from Google (Gemma), Microsoft (Phi), Mistral, and Alibaba (Qwen) come in laptop-friendly sizes.

Browsers are joining in. Google documents built-in AI in Chrome that runs Gemini Nano on the device for tasks like summarization, per its developer docs — useful, though the feature set and hardware requirements are still evolving.

Where "on-device" gets fuzzy

Built-in assistants often mix local and cloud processing. Apple is unusually explicit: Apple Intelligence handles many requests on the device and sends heavier ones to its Private Cloud Compute servers, which Apple says are designed so user data is not retained and can be inspected by outside researchers, per Apple's security blog. That is a meaningful privacy design — but it is not the same as never leaving the laptop. Similar hybrid behavior applies to other vendors' assistants, so read the settings rather than the slogan.

Open-source tools have their own quiet network calls: checking for updates, downloading models, or optional telemetry. And a "local" app that offers web search or plug-ins will send queries out when those features are used. None of this is sinister, but it matters if you handle client files, health records, or legal documents.

A five-minute privacy check

1. Download the model, then turn off Wi-Fi and confirm the app still answers. 2. Check settings for telemetry, crash reporting, and web-search toggles; switch off what you do not need. 3. Keep models from known publishers and download them through the tool or the official model page, since model files are software you are trusting. 4. Remember the conversation history is stored on your disk — use full-disk encryption (FileVault on Mac, BitLocker on Windows). 5. At work, ask IT before installing; a local model can still conflict with your company's data-handling policy.

What you give up

Local models are good, not magic. A laptop-sized model is noticeably weaker than the frontier systems running in the cloud on hard reasoning, current events, and long documents. It can still state wrong facts confidently — NIST's Generative AI Profile lists "confabulation" among the core risks regardless of where a model runs, per NIST AI 600-1. Expect slower replies on older machines and shorter battery life during heavy use. For many people the right setup is hybrid: local for anything sensitive, cloud for tasks where capability matters more than privacy.

Who should go local

Local AI makes the most sense for professionals bound by confidentiality — lawyers, clinicians, accountants, journalists protecting sources — for developers who want an offline coding assistant, and for anyone with poor connectivity. If your laptop has 16 GB of memory and was made in the past three or four years, you can try it this afternoon at no cost. If it has 8 GB, start with a small model and keep expectations modest.

Sources and scope

This guide draws on Apple's Private Cloud Compute security post, Microsoft's Copilot+ PC announcement, project documentation for Ollama, llama.cpp, and LM Studio, Meta's Llama 3.2 announcement, Chrome's built-in AI docs, and NIST AI 600-1. Memory figures are general rules of thumb; actual needs vary by model, quantization, and context length. We did not run original benchmarks for this piece.