Most people use AI through a website, with the actual computing happening in a company's data center. But you can also run capable AI models directly on your own computer, with no account and no internet required once it is set up. Here is why people do it, what it takes, and how to try it without turning into a system administrator.

Why run it locally

  • Privacy. Nothing you type leaves your machine. For sensitive notes, journaling, health or legal questions, or confidential work, that is a real difference, because there is no company server in the loop.
  • Cost. Once you have the hardware, using the model is free. No subscription, no per-use fees, no meter running.
  • Offline. It keeps working on a plane, in a cabin, or anywhere the Wi-Fi does not reach.
  • Control and curiosity. You choose the model, keep it as long as you like, and get to see what makes these systems tick. For a lot of people that last one is the whole appeal.

What 'open weights' means

You can only run a model locally if its maker has released it for you to download. When people say a model is open weights, they mean the trained model itself, the giant set of numbers (the 'weights') that make it work, is published for anyone to download and run. Well-known examples include Meta's Llama, Google's Gemma, Mistral, and DeepSeek.

One honest caveat about the words. 'Open weights' is not quite the same as 'open source.' Open weights means you get the finished model to run and adapt. Fully open source would also include the training code and data, the complete recipe, which most of these releases do not provide. For simply running a model at home, open weights is all you need; the distinction matters more to developers and researchers.

The opposite of open-weights models are the closed ones, the flagships behind ChatGPT, Claude, and Gemini, which you can only use through their makers' services. Those are usually the most capable models available, which is the central trade-off we will get to.

What you need

You do not need a supercomputer, but hardware is the honest gatekeeper. At a concept level, two things matter:

  • Memory (RAM), and ideally a decent graphics card. A model has to fit in your computer's memory to run, and bigger, smarter models need more. As a rough guide, around 16 GB of RAM is a comfortable floor for a small, capable model; more is better, and a modern GPU, or an Apple Silicon Mac, which shares fast memory, makes everything noticeably quicker.
  • A few gigabytes of disk space per model. Models are large files. You download one once, and it lives on your drive.

Models also come in different sizes, counted in parameters, of which there are billions. Smaller models, in the 3-to-8-billion range, run on ordinary laptops and are a sensible first test. Larger ones are smarter but demand serious hardware. And most local models are 'quantized,' a compression trick that shrinks them to fit consumer machines with only a small hit to quality. You do not need to master any of this to start, because the tools below handle the details.

The easy on-ramps

Two free tools have made local AI genuinely approachable for non-programmers:

  • LM Studio is a friendly desktop app. It has a built-in catalog you can browse, downloads a model in one click, and gives you a chat window that looks much like any online chatbot. If you want the least technical path, start here.
  • Ollama is free and open source, and a bit more minimalist. It leans on simple typed commands, which developers love, and a growing number of companion apps give it a friendly window if you prefer clicking to typing.

Either way the shape is the same: install the app, pick a model, wait for it to download, and start chatting, offline from then on. The first working reply from a model running on your own laptop, with the Wi-Fi switched off, is a small thrill worth having.

Setting your expectations

Here is the part the excitement sometimes skips. A model running on your laptop will generally not match the frontier models you get from a paid ChatGPT, Claude, or Gemini plan. The best cloud systems run on racks of specialized hardware no home machine can equal, and it shows, in reasoning, in accuracy, and in handling long and complex requests.

What local models are is remarkably good for their size, and improving fast. For summarizing, drafting, answering everyday questions, and tinkering privately, a good small model is more than capable. Just do not expect your laptop to out-think a data center. Think of it as the difference between a capable home kitchen and a professional one: both cook real meals, but only one caters a wedding.

Where to go from here

If your machine is modest, or you just want the best possible answers, sticking with the free tier of a cloud chatbot is a perfectly good choice, and no shame in it. But if the idea of a private, offline, no-subscription assistant appeals, downloading LM Studio and one small model is an afternoon well spent.

For the next layer, like choosing models, understanding quantization and context length, and running a local model as a private assistant for real work, [the intermediate track](/guides/) goes deeper. And you will not break anything by experimenting: worst case, a model is too slow or too big, you delete the file, and you are exactly where you started, only better informed.