How to run an AI model on your own laptop, offline, in fifteen minutes
A model that runs on your own machine cannot leak your chats, cannot hit a usage limit and works on a plane. It is also smaller and slower than the cloud ones. Here is how to set one up and when to use it.
I want AI that runs offline on my laptop. Install Ollama or LM Studio, pick a model that fits your memory (Gemma 4 for 8 GB, gpt-oss for 16 GB), run it, and know what a local model is and is not good for. No account, no subscription, nothing leaves your machine.
Check your memory, not your CPU
The only number that matters is memory. On a Mac with Apple silicon it is the unified memory in About This Mac. On a Windows or Linux PC with a graphics card it is the card's VRAM, which you can see in Task Manager under Performance, GPU. Without a graphics card it is your RAM, and the model will run, just slowly.
A model's download size is roughly the memory it needs, plus a few gigabytes for the conversation. So an 8 GB machine runs a 4 to 5 GB model well. A 16 GB machine runs a 14 GB model. A 24 GB or 32 GB machine opens the door to the 20 GB ones.
gemma4:e2b, 4.6 to 7.5 GB download, Ollama recommends 8 GB of VRAM or unified memory). It is small and quick. Keep conversations short.gemma4:e2b, at a few words per second. Fine for short questions and summaries. Not fine for long documents.Install Ollama
Ollama is the simplest way in. Download it from ollama.com/download for macOS, Windows or Linux and run the installer. On a Mac or Windows it installs an app that sits in your menu bar; on Linux it installs a command and you start the server with the command below if it is not already running.
LM Studio is the other good option if you prefer buttons to a terminal, and I cover it in step 5. Both are free. Neither needs an account for local models.
ollama serve
Pull a model and talk to it
Open a terminal (on a Mac, the Terminal app; on Windows, PowerShell) and run the first line. It downloads the model once. Then run the second line and you are in a chat. Type a question, press Enter, and the answer streams back from your own machine. Type /bye to leave.
The first answer can take a minute while the model loads into memory. After that it is quick. If it is painfully slow, you picked a model too big for your memory; go one size down.
ollama pull gemma4:e2b ollama run gemma4:e2b
Use the app instead of the terminal
You do not have to live in a terminal. The Ollama app has a chat window of its own: open it from the menu bar and pick the model from the dropdown. It also runs a local server on your machine, which is how other apps talk to it. Most desktop AI apps now have a setting to point at Ollama; search their settings for "local" or "Ollama".
The handful of commands below are all you need to manage models. Each one you download sits on disk until you remove it, and they are big, so remove the ones you do not use.
ollama list ollama rm gemma4:e2b ollama pull gpt-oss:20b
Or use LM Studio and let it tell you what fits
LM Studio does the same job with a window. Download it from lmstudio.ai, open it, and press Cmd+2 on a Mac or Ctrl+2 on Windows to open the Discover tab. Search for a model by name, such as gemma or qwen. The options are labelled with quantisation names like Q4_K_M; LM Studio's own advice is to choose a 4-bit option or higher if your machine can run it.
The thing LM Studio does better than Ollama for a beginner is tell you, before you download, whether a model will fit your machine. Trust that over any blog post, including this one.
Know what it is for
A local model is private, free and always available. It is also a year or two behind the cloud models on hard problems, it cannot search the web unless you add that yourself, and it knows nothing after its training date. So use it for what it is good at.
Good: summarising and rewriting your own documents, drafting emails, first-pass coding help, anything with a password, a contract or a patient in it, and anything you want to do on a plane. Bad: current events, hard maths, long research, anything where you would use the heavy model in the cloud.
Anything with a password, a contract or health details in it.
Offline work.
Unlimited drafts with no usage limit.
Learning how models behave, for free.
Hard reasoning, long code, research you will act on.
Very long documents, unless you have 32 GB or more.
Images and voice, mostly.
TipThe model name tells you its size. In gemma4:e2b, the E2B means it behaves like a 2-billion-parameter model. gpt-oss:20b is 20 billion. Bigger is smarter and slower, and above your memory it is just slower.
You're done when
Questions people are asking
Is Ollama free? Yes. The app and local models are free and need no account. Ollama also sells cloud-hosted models, marked with a cloud tag, which run on Ollama's servers rather than your machine.
Which local AI model is best for 8GB? Gemma 4 E2B. Ollama lists it at 4.6 to 7.5 GB and recommends 8 GB of VRAM or unified memory. On 16 GB, gpt-oss:20b at 14 GB is the step up.
Is a local model as good as ChatGPT? No. The 20 to 30 billion parameter models you can run at home are roughly where the cloud models were a year or two ago. They are good enough for rewriting, summarising and first drafts, and they are completely private.
Sources
- Ollama, Quickstart docs.ollama.com/quickstart
- Ollama, gemma4 model page with sizes ollama.com/library/gemma4
- Ollama, gpt-oss model page with sizes and memory note ollama.com/library/gpt-oss
- LM Studio, Download a model lmstudio.ai/docs/app/basics/download-model
How to let an AI agent use your browser without handing it your life
One email a week. Written by people.
New how-tos, the tools we tested and what we think, in one short email.