Skip to content

What is running a large model locally ​

Local deployment means downloading a large model to your own computer and running it there. What you say to it and the files you give it are all processed on your computer, without being sent to anyone else's server.

This section doesn't install anything.

What you'll be able to do after this section ​

  • Explain the difference between local deployment and the web version
  • Know what Ollama is for
  • Judge whether your own computer is good enough

How it differs from the web version ​

Web-based AILocal deployment
Where the model runsThe provider's serversYour computer
Your prompts and filesSent to the providerStay on your machine
Speed depends onNetwork and the providerYour graphics card
How large a model you can useThe provider's bestWhatever fits in your VRAM

What local deployment buys you is keeping your data in your own hands; the cost is that the model size is limited by your graphics card.

What is Ollama ​

Ollama is a tool for downloading and running large models on your own computer. Once installed, a single command downloads a model and starts a conversation, and it also opens a local interface (http://localhost:11434) so other programs can call it too.

On privacy, the Ollama official FAQ says verbatim: Ollama runs locally, and when running locally they cannot see your prompts or data. Note that Ollama also offers cloud models, and when using cloud models your prompts pass through their service. If you only use local models, you can turn off the cloud feature in settings—see FAQ.

Quiz timeYou run deepseek-r1:8b locally with Ollama and ask a question about an internal policy. Will this policy be sent to Ollama's servers?

  • I can state the biggest difference between local deployment and the web version

Is my computer good enough ​

Official system requirements:

SystemRequirements
WindowsWindows 10 22H2 or newer, Home or Pro edition; NVIDIA graphics driver 551.61 or newer
macOSSonoma (14) or newer; on Apple M-series chips both CPU and GPU can be used, on Intel chips only the CPU
Graphics cardNVIDIA cards require compute capability 5.0 or higher and driver 550 or newer; some AMD Radeon cards are also supported

What really determines how large a model you can run is VRAM. The model runs fastest when it fits entirely in VRAM; if it doesn't fit, Ollama puts part of it in memory and computes with the CPU—it still runs, just much slower. The small tool on the course home page can do a rough calculation based on your VRAM; the how large a model to choose section covers this in detail.

  • I know how much VRAM my graphics card has

Checkpoints ​

  • I can explain local deployment in one sentence: the model runs on your own computer and the data stays on your machine
  • I know Ollama is used to download and run models, and that it also opens a local interface
  • I know that VRAM determines how large a model I can run

Common misconceptions ​

Thinking local deployment can run the full-strength DeepSeek. The deepseek-r1 on Ollama comes in multiple sizes from 1.5b to 671b; the 671b files are 404 GB, which an ordinary computer can't hold. Consumer graphics cards run the smaller-size versions.

DeepSeek's latest models are already the V4 / V4.1 series, and deepseek-r1 on Ollama is a model from a year ago. However, the V4 series on Ollama only has the cloud version (with :cloud in the name), and when using it, prompts are sent to the cloud; LM Studio says running V4-Flash locally requires at least 156 GB of memory. So what can still run locally on a home computer is the smaller size of deepseek-r1 — see Can DeepSeek V4 Run Locally for details.

Thinking that using Ollama means you never go online. Downloading models requires an internet connection, and Ollama also has cloud models. Only by using local models exclusively and turning off the cloud feature does your data stay entirely on your machine.

Next section ​

Install Ollama.

gaodong.aiAboutPrivacyContact

Independent tutorials, not affiliated with any product mentioned. Product names and trademarks belong to their owners. Check official pages for prices and policies.