What is running a large model locally
Local deployment means downloading a large model to your own computer and running it there. What you say to it and the files you give it are all processed on your computer, without being sent to anyone else's server.
This section doesn't install anything.
What you'll be able to do after this section
- Explain the difference between local deployment and the web version
- Know what Ollama is for
- Judge whether your own computer is good enough
How it differs from the web version
| Web-based AI | Local deployment | |
|---|---|---|
| Where the model runs | The provider's servers | Your computer |
| Your prompts and files | Sent to the provider | Stay on your machine |
| Speed depends on | Network and the provider | Your graphics card |
| How large a model you can use | The provider's best | Whatever fits in your VRAM |
What local deployment buys you is keeping your data in your own hands; the cost is that the model size is limited by your graphics card.
What is Ollama
Ollama is a tool for downloading and running large models on your own computer. Once installed, a single command downloads a model and starts a conversation, and it also opens a local interface (http://localhost:11434) so other programs can call it too.
On privacy, the Ollama official FAQ says verbatim: Ollama runs locally, and when running locally they cannot see your prompts or data. Note that Ollama also offers cloud models, and when using cloud models your prompts pass through their service. If you only use local models, you can turn off the cloud feature in settings—see FAQ.
Quiz timeYou run deepseek-r1:8b locally with Ollama and ask a question about an internal policy. Will this policy be sent to Ollama's servers?
- I can state the biggest difference between local deployment and the web version
Is my computer good enough
Official system requirements:
| System | Requirements |
|---|---|
| Windows | Windows 10 22H2 or newer, Home or Pro edition; NVIDIA graphics driver 551.61 or newer |
| macOS | Sonoma (14) or newer; on Apple M-series chips both CPU and GPU can be used, on Intel chips only the CPU |
| Graphics card | NVIDIA cards require compute capability 5.0 or higher and driver 550 or newer; some AMD Radeon cards are also supported |
What really determines how large a model you can run is VRAM. The model runs fastest when it fits entirely in VRAM; if it doesn't fit, Ollama puts part of it in memory and computes with the CPU—it still runs, just much slower. The small tool on the course home page can do a rough calculation based on your VRAM; the how large a model to choose section covers this in detail.
- I know how much VRAM my graphics card has
Checkpoints
- I can explain local deployment in one sentence: the model runs on your own computer and the data stays on your machine
- I know Ollama is used to download and run models, and that it also opens a local interface
- I know that VRAM determines how large a model I can run
Common misconceptions
Thinking local deployment can run the full-strength DeepSeek. The deepseek-r1 on Ollama comes in multiple sizes from 1.5b to 671b; the 671b files are 404 GB, which an ordinary computer can't hold. Consumer graphics cards run the smaller-size versions.
DeepSeek's latest models are already the V4 / V4.1 series, and deepseek-r1 on Ollama is a model from a year ago. However, the V4 series on Ollama only has the cloud version (with :cloud in the name), and when using it, prompts are sent to the cloud; LM Studio says running V4-Flash locally requires at least 156 GB of memory. So what can still run locally on a home computer is the smaller size of deepseek-r1 — see Can DeepSeek V4 Run Locally for details.
Thinking that using Ollama means you never go online. Downloading models requires an internet connection, and Ollama also has cloud models. Only by using local models exclusively and turning off the cloud feature does your data stay entirely on your machine.