Local Models: Ollama, vLLM, and ds4

Up through the previous lesson, OpenClaw was connected entirely to cloud APIs. This lesson takes a different approach: OpenClaw local models—inference runs on your own hardware, with no API fees, and not a single byte of conversation data leaves your machine. In the official architecture, both models and agent harnesses (Claude, Codex, local models) are swappable plugins, and local backends are full first-class citizens: Ollama, vLLM, and SGLang are all built-in providers, LM Studio has dedicated onboarding, and ds4 (running DeepSeek V4 Flash locally on macOS Metal) connects via pure configuration. The whole process takes about 10 minutes; this tutorial is based on v2026.10.9.

What You'll Be Able to Do After This Lesson

Connect local models to OpenClaw using Ollama, with tool calls working properly—Claw can actually get work done, not just chat with you
Self-host open-weight models on GPU servers with vLLM / SGLang, with OpenClaw automatic discovery of the model directory
Run DeepSeek V4 Flash locally on macOS with ds4, configuring thinking tiers and on-demand startup
Use the official selection table to tell apart which of the seven local backends covers which scenario, so you won't get lost even on your first install
Use layered diagnostic commands to distinguish between "the model can respond" and "Gateway routing works," and set up a local-first + cloud-fallback hybrid plan

🎒 Prerequisites

Completed Model Connection Overview: The Provider System and Switching Models: you're clear on the provider/model reference rules and the three steps of "install plugin → configure credentials → pick a model"
OpenClaw is installed and the Gateway is running: openclaw gateway status produces output
Have a machine capable of running models—even the smallest official llama.cpp hosted recipe requires 8 GiB of host memory as a baseline (the 64K context recipe); actual requirements depend on the model weights, context, and runtime
Have at least one local inference backend installed (any of Ollama / vLLM / SGLang / LM Studio / ds4), or install one on the spot following the steps below

💡 Why use local models?

Three hard reasons: Free—inference happens on your CPU/GPU, with no per-token bills; Offline—once the model is pulled down it runs even without internet, friendly to intranets and classified environments; Privacy—conversations, memories, and files never leave your hardware. The trade-offs are also clear: quality and speed are limited by your hardware, and local models don't have a provider's safety filtering, so you need to tighten tool permissions yourself (covered in detail in Step 1).

Follow Along

Step 1: Choose a Backend—Find Your Match in the Official Selection Table

Why: Each local backend has its own role, and picking the wrong one means twice the work for half the results. The official local-models guide provides a selection table—find your match first:

Backend	When to Use It	How to Connect
Ollama	CLI workflows, rich model library, maintenance-free systemd service	Built-in provider (Step 2)
vLLM / SGLang / MLX	High-throughput self-hosting, OpenAI-compatible HTTP endpoint	vLLM, SGLang built-in providers (Step 3)
ds4	Run DeepSeek V4 Flash locally on macOS Metal, OpenAI-compatible tool calls	Pure configuration (Step 4)
LM Studio	First local deployment, GUI loader, native Responses API	onboarding wizard (step 4)
llama.cpp (plugin)	Hardware-aware model selection, download verification, OpenClaw managed server	After installing the llama.cpp plugin, run `openclaw onboard` and choose Managed local server
llmman	Pull models from OCI registries + llama.cpp/vLLM/MLX engines + local/hosted hybrid routing	Standalone provider
LiteLLM / custom OpenAI-compatible proxy	Put a layer of other model APIs in front	the previous lesson's custom flow

Three general rules that will apply to every step below:

Adapter: when a custom provider sets baseUrl but omits api, the default is openai-completions; if the backend supports the Responses API you can use api: "openai-responses" (LM Studio supports it). Ollama is a special case—it goes through the native adapter (api: "ollama"); see the /v1 warning in step 2.
Hardware requirements: memory needs depend on model weights, context, runtime, and other workloads on the machine; the official llama.cpp managed recipe uses 64K context, and even the smallest needs at least 8 GiB of host memory to start, with no guarantee it will run, let alone run fast. Leave plenty of headroom for OpenClaw's full prompts, tool schemas, history, and output.
Security: local models do not provide the provider's safety filters. Tighten tool permissions and prompt-injection defenses according to model capability—the weaker the model, the more easily it can be led astray by injected instructions in web pages or files.

Step 2: Connect Ollama (recommended starting point for local models)

Why: Ollama is the smoothest path for local models—OpenClaw has a built-in provider, Ollama pulls models with one command, and once installed it's a maintenance-free background service.

Option A (recommended): onboarding wizard

First install Ollama (ollama.com/download) and pull a model:

bash

ollama pull gemma4

Then run the wizard:

bash

openclaw onboard

Choose Ollama, then pick a mode: Cloud + Local (local models + :cloud cloud models when signed in) or Local only (local only).

What you should see: the wizard first checks the default or configured Ollama host; auto-discovery only recognizes models already loaded into memory (reported by /api/ps), and requires tool support + at least 16K of context (confirmed via /api/show). The selected route must pass a real completion before it is saved—discovery never automatically pulls or loads idle models. Installed-but-not-loaded models are not auto-discovered: in the desktop version's Model Setup, go to the Ollama card's Choose connection → Local only to prepare an installed idle model for the live check, without loading it manually first. When done, openclaw models list --provider ollama can list the models.

Non-interactive version (both custom parameters can be omitted):

bash

openclaw onboard --non-interactive --accept-risk --skip-health \
  --auth-choice ollama \
  --custom-base-url "http://ollama-host:11434" \
  --custom-model-id "qwen3.5:27b"

Omit the base URL to use the local default host; when the model ID is omitted, it prefers installed models that support tools and have ≥16K of context (favoring non-reasoning models, then smaller models), and if none qualify it tries the recommended gemma4.

Option B: manual configuration

bash

export OLLAMA_API_KEY="ollama-local"   # 本地/LAN 主机任意值即可
openclaw models set ollama/gemma4

Or write it into the config file:

json5

{
  agents: {
    defaults: {
      model: { primary: "ollama/gemma4" },
    },
  },
}

⚠️ Do not include /v1 in Ollama's baseUrl (the opposite of other backends)

OpenClaw talks to Ollama via the native /api/chat, not the OpenAI-compatible /v1. You must use the native address baseUrl: "http://host:11434"—including /v1 will break tool calls, with the model spitting out tool-call JSON as plain text. vLLM, SGLang, ds4, and LM Studio are exactly the opposite: their baseUrl must include /v1.

Credential rules: Ollama addresses on loopback, private networks, .local, or bare hostnames do not need a real token—OpenClaw marks them with ollama-local; only public remote hosts and https://ollama.com need a real OLLAMA_API_KEY or an auth profile.

Ollama Cloud (optional): the Ollama ecosystem without occupying local hardware

Don't want to run models locally but still want Ollama's hosted models? The standalone provider ollama-cloud connects directly to https://ollama.com and needs no local daemon:

bash

openclaw onboard --auth-choice ollama-cloud

The onboarding default model is ollama-cloud/minimax-m2.7; the hosted catalog includes IDs such as deepseek-v4-pro, glm-5.2, kimi-k3, minimax-m3, gpt-oss:20b. In hybrid mode, run ollama signin on the local host to use local models and :cloud cloud models (e.g., minimax-m2.7:cloud, kimi-k3:cloud) simultaneously through the same Ollama host.

openclaw models list --provider ollama lists your local models

Step 3: Connect vLLM or SGLang (GPU self-hosted, high throughput)

Why: you have a GPU server or a machine with lots of VRAM and want to self-host open weights at high throughput (the open weights of LongCat from the previous lesson fall into this category)—both vLLM and SGLang are built-in providers, OpenAI-compatible on /v1, with automatic model discovery.

Four steps to get vLLM working. First remember four properties: provider id vllm, API openai-completions, authentication VLLM_API_KEY, default baseUrl http://127.0.0.1:8000/v1.

bash

vllm serve <model-id>              # 1. 起服务，模型 ID 换成你要托管的
export VLLM_API_KEY="vllm-local"   # 2. 服务器不鉴权时任非空值，作为启用发现的 opt-in 信号

json5

// 3. 选模型
{
  agents: {
    defaults: {
      model: { primary: "vllm/your-model-id" },
    },
  },
}

bash

openclaw models list --provider vllm   # 4. 验证

What you should see: models list lists the models on vLLM—when VLLM_API_KEY is set and models.providers.vllm is not written, OpenClaw queries GET http://127.0.0.1:8000/v1/models to auto-generate model entries, and can also recognize DeepSeek V4 and Nemotron 3 series reasoning models (including aliases like yk_deepseek_v4). Note: once you write models.providers.vllm, only the models you hand-wrote are recognized—to stay dynamic, add the wildcard "vllm/*": {} in agents.defaults.models.

Non-interactive one-liner (for CI / scripted deployments):

bash

openclaw onboard --non-interactive --accept-risk --skip-health \
  --mode local \
  --auth-choice vllm \
  --custom-base-url "http://127.0.0.1:8000/v1" \
  --custom-api-key "vllm-local" \
  --custom-model-id "your-model-id"

When vLLM runs on a different machine/port, you need to pin context parameters, or you want to use a real key, write an explicit config (excerpted verbatim from the official example):

json5

{
  models: {
    providers: {
      vllm: {
        baseUrl: "http://127.0.0.1:8000/v1",
        apiKey: "${VLLM_API_KEY}",
        api: "openai-completions",
        timeoutSeconds: 300, // 可选：给慢的本地模型延长请求超时
        models: [
          {
            id: "your-model-id",
            name: "Local vLLM Model",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 128000,
            maxTokens: 8192,
          },
        ],
      },
    },
  },
}

🔧 Advanced (optional): priority scheduling for interactive requests

When one vLLM instance serves both interactive conversations and background tasks like cron / memory consolidation, first enable native priority scheduling on the server side:

bash

vllm serve <model-id> --scheduling-policy priority

Then enable the opt-in for the corresponding model:

json5

{
  agents: {
    defaults: {
      models: {
        "vllm/your-model-id": { params: { priorityScheduling: true } },
      },
    },
  },
}

Priority mapping: interactive turns and manual /compact get -100, delegated or unclassified work gets 0, cron / heartbeat / memory / simple auxiliary completions get 100 — smaller numbers are scheduled first, with ties broken by arrival order. This switch is off by default and only takes effect for the selected vllm/* models.

SGLang uses the same approach (built in and enabled by default, default baseUrl http://127.0.0.1:30000/v1):

bash

export SGLANG_API_KEY="sglang-local"   # 服务器不鉴权时任非空值
openclaw onboard --auth-choice sglang

The default placeholder model in onboarding is sglang/Qwen/Qwen3-8B. If you set SGLANG_API_KEY and don't write models.providers.sglang, it is likewise auto-discovered; to pin parameters, write an explicit configuration, whose shape is identical to vLLM (replace vllm with sglang, and change the port to 30000).

💡 MLX or other OpenAI-compatible local services can also be connected

Any local service that exposes an OpenAI-style /v1/chat/completions is within range: MLX (mlx_lm.server), LiteLLM, OAI-proxy, etc. Follow the custom flow from the previous lesson, and give --custom-provider-id a local id (e.g. mlx). Write the model id as the provider's internal key verbatim (e.g. mlx-community/Qwen3-30B-A3B-6bit), and add a prefix when referencing it: mlx/mlx-community/Qwen3-30B-A3B-6bit.

openclaw models list --provider vllm (or --provider sglang) lists the hosted models

Step 4: Connect ds4 or LM Studio (macOS DeepSeek and a beginner-friendly GUI)

Why: two featured backends cover two other categories of needs — ds4 gives Mac users a way to run DeepSeek V4 Flash locally; LM Studio gives people who don't want to touch the command line a GUI.

ds4: local DeepSeek V4 Flash on macOS Metal

First, let's be clear about what ds4 is: antirez's local project (github.com/antirez/ds4), which runs DeepSeek V4 Flash GGUF on the macOS Metal backend and exposes an OpenAI-compatible /v1. It is not a cloud service, nor an official DeepSeek product; on the OpenClaw side, there is no plugin — it integrates purely via configuration.

Start the service (replace <DS4_DIR> with the path to your ds4 checkout):

bash

<DS4_DIR>/ds4-server \
  --model <DS4_DIR>/ds4flash.gguf \
  --host 127.0.0.1 \
  --port 18000 \
  --ctx 32768 \
  --tokens 128

You should see: the response from curl http://127.0.0.1:18000/v1/models contains deepseek-v4-flash.

Then write the OpenClaw configuration (excerpted from the official full configuration):

json5

{
  agents: {
    defaults: {
      model: { primary: "ds4/deepseek-v4-flash" },
      models: { "ds4/deepseek-v4-flash": { alias: "DS4 local" } },
    },
  },
  models: {
    mode: "merge",
    providers: {
      ds4: {
        baseUrl: "http://127.0.0.1:18000/v1",
        apiKey: "ds4-local",
        api: "openai-completions",
        timeoutSeconds: 300,
        models: [
          {
            id: "deepseek-v4-flash",
            name: "DeepSeek V4 Flash (ds4)",
            reasoning: true,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 32768,
            maxTokens: 128,
            compat: {
              supportsUsageInStreaming: true,
              supportsReasoningEffort: true,
              maxTokensField: "max_tokens",
              supportsStrictMode: false,
              thinkingFormat: "deepseek",
              supportedReasoningEfforts: ["low", "medium", "high", "xhigh"],
            },
          },
        ],
      },
    },
  },
}

Two key points:

Start with --ctx 32768: OpenClaw's agent turns carry tool schemas and workspace context, --ctx 4096 can pass direct curl tests, but a fully loaded agent turn will report 500 prompt exceeds context. contextWindow aligns with --ctx, and maxTokens aligns with --tokens.
Think Max needs a large context: it only activates with ds4-server --ctx 393216 and a request of reasoning_effort: "max"; at that point the model metadata is synchronously changed to contextWindow: 393216, maxTokens: 384000, and supportedReasoningEfforts gains "max".

🔧 Start on demand: have OpenClaw launch ds4 only when it's used

Don't want ds4 resident and eating memory? Add localService to the same provider entry, and OpenClaw will automatically start the service when a ds4/... model is selected (official example, verbatim):

json5

{
  models: {
    providers: {
      ds4: {
        baseUrl: "http://127.0.0.1:18000/v1",
        apiKey: "ds4-local",
        api: "openai-completions",
        timeoutSeconds: 300,
        localService: {
          command: "<DS4_DIR>/ds4-server",
          args: [
            "--model",
            "<DS4_DIR>/ds4flash.gguf",
            "--host",
            "127.0.0.1",
            "--port",
            "18000",
            "--ctx",
            "32768",
            "--tokens",
            "128",
          ],
          cwd: "<DS4_DIR>",
          healthUrl: "http://127.0.0.1:18000/v1/models",
          readyTimeoutMs: 300000,
          idleStopMs: 0,
        },
        models: [
          {
            id: "deepseek-v4-flash",
            name: "DeepSeek V4 Flash (ds4)",
            reasoning: true,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 32768,
            maxTokens: 128,
            compat: {
              supportsUsageInStreaming: true,
              supportsReasoningEffort: true,
              maxTokensField: "max_tokens",
              supportsStrictMode: false,
              thinkingFormat: "deepseek",
              supportedReasoningEfforts: ["low", "medium", "high", "xhigh"],
            },
          },
        ],
      },
    },
  },
}

Don't get these three fields wrong: command must be an absolute executable path (no shell lookup, no ~ expansion); give readyTimeoutMs: 300000 a full 5 minutes — ds4 has a cold Metal warm-up and model preheating phase, so the first request is simply slow; use healthUrl with /v1/models for liveness probing.

LM Studio: GUI loader, fully automatic onboarding

LM Studio runs llama.cpp (GGUF) or MLX models via a GUI, and also has a headless llmster daemon. After installation, three steps:

bash

lms server start --port 1234        # 1. 起本地服务（headless 版用 lms daemon up）
export LM_API_TOKEN="your-lm-studio-api-token"   # 2. LM Studio 开了认证才需要，没开就留空
openclaw onboard                    # 3. 选 LM Studio，在 Default model 提示里挑模型

You should see: onboarding first queries the LM Studio host's /api/v1/models, and only models that LM Studio reports as "tool-trained + at least 16K of effective context" are automatically recommended (for loaded models, context is counted per the loaded instance, not the advertised maximum); after selection, one real completion must pass before saving, and the automatic check never downloads models for you.

LM Studio's model keys use the author/model-name format (e.g. qwen/qwen3.5-9b); when OpenClaw references them, add the provider prefix: lmstudio/qwen/qwen3.5-9b. To check exact keys, change defaults, or refresh after installing new models:

bash

curl http://localhost:1234/api/v1/models              # 看返回里的 key 字段
openclaw models set lmstudio/qwen/qwen3.5-9b
openclaw models list --provider lmstudio --refresh    # 装了新模型后刷新目录

In the default merge mode, --refresh adds newly discovered models and preserves the metadata of your hand-written entries (name, context limit, etc.). Non-interactive version: openclaw onboard --non-interactive --accept-risk --skip-health --auth-choice lmstudio; to explicitly supply all parameters, additionally add --custom-base-url http://localhost:1234/v1 --lmstudio-api-key "$LM_API_TOKEN" --custom-model-id qwen/qwen3.5-9b.

openclaw models list --provider ds4 (or --provider lmstudio) shows your models

Step 5: Verify the pipeline and set up hybrid fallback

Why: configuration written to disk ≠ the agent can do work. "The model can respond", "the Gateway routes correctly", and "tools actually execute" are three separate layers, and the official docs provide layered diagnostic commands — you'll know where to fix things whichever layer breaks.

bash

# 第一层：模型本身能不能回话（不带工具、不带 agent 上下文）
openclaw infer model run --local --model <provider/model> --prompt "Reply with exactly: pong" --json

# 第二层：Gateway 路由通不通（只发 prompt，跳过 transcript、工具与内置 MCP，
# 但仍走完整的 Gateway 路由、auth 与 provider 选择）
openclaw infer model run --gateway --model <provider/model> --prompt "Reply with exactly: pong" --json

Third layer: tool smoke test, run one full agent turn (using ds4 as an example):

bash

openclaw agent \
  --local \
  --session-id ds4-tool-smoke \
  --model ds4/deepseek-v4-flash \
  --thinking off \
  --message "Use the shell command pwd once, then reply exactly: tool-ok <output>" \
  --json \
  --timeout 240

You should see: executionTrace.winnerProvider is ds4, executionTrace.winnerModel is deepseek-v4-flash, toolSummary.calls is at least 1, and the visible reply starts with tool-ok — the tool actually executed, not the model reading JSON out loud to you.

Once everything passes, set your preferred model as the everyday default:

bash

openclaw models set <provider/model>

If you want to save money without service interruptions, add a layer of hybrid fallback — the recommended approach in the official local-models guide: local first, cloud as fallback; when a provider errors, it falls to the next fallback per turn (the official example, with primary/fallbacks order swapped):

json5

{
  agents: {
    defaults: {
      model: {
        primary: "lmstudio/my-local-model",   // 本地优先
        fallbacks: ["anthropic/claude-sonnet-4-6", "anthropic/claude-opus-4-6"],
      },
      models: {
        "anthropic/claude-sonnet-4-6": { alias: "Sonnet" },
        "lmstudio/my-local-model": { alias: "Local" },
        "anthropic/claude-opus-4-6": { alias: "Opus" },
      },
    },
  },
  models: {
    mode: "merge",   // merge 保住云端模型目录，随时 /model 切换
    providers: {
      lmstudio: {
        baseUrl: "http://127.0.0.1:1234/v1",
        apiKey: "lmstudio",
        api: "openai-responses",
        models: [
          {
            id: "my-local-model",
            name: "Local Model",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 196608,
            maxTokens: 8192,
          },
        ],
      },
    },
  },
}

For cloud first with local fallback, swap primary/fallbacks, keeping the providers block and models.mode: "merge" as is. Note that fallback is a per-turn switch that only triggers "on error"; if you want per-request routing that keeps small requests local and only sends oversized requests to the cloud, the official docs point to llmman's Hybrid inference.

All three verification layers pass: two levels of pong + tool smoke test toolSummary.calls ≥ 1

Checkpoint ✅

Chose the right backend against the official selection table: Ollama (CLI) / vLLM, SGLang (GPU high throughput) / ds4 (macOS DeepSeek) / LM Studio (GUI)
openclaw models list --provider <id> can list your local models
Ollama's baseUrl is http://host:11434 (without /v1); vLLM / SGLang / ds4 / LM Studio all include /v1
Both openclaw infer model run --local and --gateway return pong
In the agent smoke test, toolSummary.calls ≥ 1—tool calls are actually executed, not just JSON text being spat out
Want to save money without downtime: configure the fallbacks hybrid scheme, with models.mode: "merge" to keep the cloud model catalog

Pitfall Reminders

❌ Ollama baseUrl includes /v1—all tool calls break

Ollama uses the native /api/chat adapter, so the baseUrl must be http://host:11434 (without /v1). Including /v1 switches it to OpenAI compatibility mode, and tool calls break—the model spits out tool-call JSON as plain text. This is exactly the opposite of vLLM / SGLang / ds4 / LM Studio (all of which require /v1)—if you can't remember, just remember "Ollama marches to its own beat".

❌ ds4 --ctx 4096: curl works, but agent turns get 500

The reason for 500 prompt exceeds context: OpenClaw's agent turns carry tool schemas and workspace context, far larger than a single curl test message. Set ds4-server --ctx to at least 32768, and align models.providers.ds4.models[].contextWindow accordingly. Think Max should start from --ctx 393216.

❌ Local model "can chat", but tool calls show up as raw JSON/XML text

First follow the step-5 layered diagnosis. If both levels of infer model run pass but the full turn falls apart, troubleshoot in this order:

Local Ollama models, LM Studio, and hosted local services automatically enable structured Tool Search when tools.toolSearch is not set (defers schema delivery, preserves capability)—don't turn it off.
vLLM users should first confirm the correct tool-call parser and chat template at startup (vLLM official docs: Qwen2.5 uses hermes, Qwen3-Coder uses qwen3_xml).
For dedicated model entries that still fail, add params.extra_body.tool_choice: "required" to force tool calls—only for models that "should call tools every turn", don't set it as a global default.
Last resort: disable tools for that model via compat.supportsTools: false; or enable lean mode agents.defaults.experimental.localModelLean: true (removes big-schema tools like browser, automations, message, image_generate, music_generate, video_generate, tts, pdf).

❌ Slow models / cold models timing out

Slow first token on large local models, LM Studio cold loading, and ds4 cold warm-up are all common. Set timeoutSeconds on the provider entry (e.g. 300)—it covers connection, response headers, streaming body, and total abort, taking priority over the global agents.defaults.timeoutSeconds. For ds4 cold starts, also configure localService.readyTimeoutMs: 300000.

❌ WSL2 + Ollama + NVIDIA/CUDA: the VM keeps restarting

The official Ollama Linux installer enables a systemd service with Restart=always. In a WSL2 GPU environment, auto-start on boot reloads the previous model, pins host memory, and drags the WSL2 VM into a restart loop (the official troubleshooting page lists it as a WSL2 crash loop). Don't let Ollama auto-load models on boot inside WSL2.

❌ Misjudging the context budget: output gets truncated, only one token left near the limit

OpenClaw pre-checks against the model window: warns when headroom drops below 20% (8k floor), hard-blocks below 10% (4k floor). Locally discovered Ollama models default to an output cap of 8,192 token (OpenClaw sends the output cap as num_predict to native Ollama); to restore Ollama's unlimited default, set params: { num_predict: -1 }. For custom model entries, write real values for contextWindow/maxTokens instead of falling back to the default of 200000.

❌ After the Gateway runs as a daemon, exported Keys "don't take effect"

Repeating an old pitfall: launchd / systemd services can't see interactive shell environment variables. Put OLLAMA_API_KEY, VLLM_API_KEY, SGLANG_API_KEY, LM_API_TOKEN into ~/.openclaw/.env, or provide them to that process via env.shellEnv in the config.

❌ Treating local models as a "free cloud substitute", running naked on security

Local modelshave no provider-side safety filtering, and the weaker the model, the more easily it gets led astray by injection instructions in webpages or files. Narrow tool permissions, keep compaction on, and consider sandboxing group-chat sessions—in the official local-models guide's own words: keep tool permissions and prompt-injection defenses appropriate for the model and task. For hardening techniques, see Security Hardening (Part One): Access Control and Identity.

Next Steps

This wraps up the quick-start chapter: installation and model integration (domestic full suite + relay + local models) are all done. Up next:

Next Chapter Daily Use: Getting Along Efficiently with Your Claw: Control UI, chat commands, the memory system—making Claw truly part of your daily routine
Local model running on another machine? Server Deployment: Running 7×24 on Docker/VPS moves the Gateway and inference services onto a server together
Want your local Claw to remember you? Memory System: Making Claw Remember You—pure Markdown files + SQLite index, with your data equally in your own hands

Local Models: Ollama, vLLM, and ds4 ​

What You'll Be Able to Do After This Lesson ​

🎒 Prerequisites ​

Follow Along ​

Step 1: Choose a Backend—Find Your Match in the Official Selection Table ​

Step 2: Connect Ollama (recommended starting point for local models) ​

Option A (recommended): onboarding wizard ​

Option B: manual configuration ​

Ollama Cloud (optional): the Ollama ecosystem without occupying local hardware ​

Step 3: Connect vLLM or SGLang (GPU self-hosted, high throughput) ​

Step 4: Connect ds4 or LM Studio (macOS DeepSeek and a beginner-friendly GUI) ​

ds4: local DeepSeek V4 Flash on macOS Metal ​

LM Studio: GUI loader, fully automatic onboarding ​

Step 5: Verify the pipeline and set up hybrid fallback ​

Checkpoint ✅ ​

Pitfall Reminders ​

❌ Ollama baseUrl includes /v1—all tool calls break ​

❌ ds4 --ctx 4096: curl works, but agent turns get 500 ​

❌ Local model "can chat", but tool calls show up as raw JSON/XML text ​

❌ Slow models / cold models timing out ​

❌ WSL2 + Ollama + NVIDIA/CUDA: the VM keeps restarting ​

❌ Misjudging the context budget: output gets truncated, only one token left near the limit ​

❌ After the Gateway runs as a daemon, exported Keys "don't take effect" ​

❌ Treating local models as a "free cloud substitute", running naked on security ​

Next Steps ​

Related Tutorials ​

Local Models: Ollama, vLLM, and ds4

What You'll Be Able to Do After This Lesson

🎒 Prerequisites

Follow Along

Step 1: Choose a Backend—Find Your Match in the Official Selection Table

Step 2: Connect Ollama (recommended starting point for local models)

Option A (recommended): onboarding wizard

Option B: manual configuration

Ollama Cloud (optional): the Ollama ecosystem without occupying local hardware

Step 3: Connect vLLM or SGLang (GPU self-hosted, high throughput)

Step 4: Connect ds4 or LM Studio (macOS DeepSeek and a beginner-friendly GUI)

ds4: local DeepSeek V4 Flash on macOS Metal

LM Studio: GUI loader, fully automatic onboarding

Step 5: Verify the pipeline and set up hybrid fallback

Checkpoint ✅

Pitfall Reminders

❌ Ollama baseUrl includes /v1—all tool calls break

❌ ds4 --ctx 4096: curl works, but agent turns get 500

❌ Local model "can chat", but tool calls show up as raw JSON/XML text

❌ Slow models / cold models timing out

❌ WSL2 + Ollama + NVIDIA/CUDA: the VM keeps restarting

❌ Misjudging the context budget: output gets truncated, only one token left near the limit

❌ After the Gateway runs as a daemon, exported Keys "don't take effect"

❌ Treating local models as a "free cloud substitute", running naked on security

Next Steps

Related Tutorials