GGUF Quantized Models: Run Large Models on Low VRAM

Want to run 14B-class large models in ComfyUI but only have 8GB of VRAM? GGUF quantized models compress weights to lower bit depths, letting you run them without upgrading your GPU. This page walks through the full process of installing, downloading, and wiring up ComfyUI GGUF.

What You'll Be Able to Do

• Explain the difference between GGUF and ComfyUI's native quantized formats, and know when to choose which
• Install city96's ComfyUI-GGUF custom nodes
• Find and place GGUF weights from community quantization repositories (bullerwins, QuantStack)
• Replace the standard loader node in your original workflow with a GGUF loader and generate images successfully

Your Current Predicament

• "I want to run Wan2.2 14B, but my GPU only has 8GB of VRAM—the official fp16 weights simply won't fit."
• "I downloaded a .gguf file and put it in the models directory, but the loader can't find it."
• "I've heard quantization degrades image quality, and I don't know which bit level to choose."

When to Use This Approach

When you need to:

• Run large models that wouldn't otherwise fit on a low-VRAM GPU
• Use popular models the community has already quantized (such as the GGUF version of Wan2.2 A14B)

And you don't want to:

• Spend money upgrading your GPU
• Do the quantization conversion yourself

💡 Check the native format first

ComfyUI also has its own native quantized formats (fp8, int8, etc., stored directly as safetensors with no nodes required). If the target model has an official fp8 version, prefer it; GGUF is a community format that wins on covering more models and more bit levels.

🎒 Prerequisites

• You have finished readingWhere to Put Models: The models Directory and extra_model_paths Explained and understand the subdirectory structure under ComfyUI/models/
• ComfyUI can start normally and you can run a basic text-to-image workflow
• Your disk has enough free space (a single GGUF file ranges from a few GB to over twenty GB, depending on the bit level)
• Your network can access Hugging Face (if access is unstable in your region, you can download from mirror sites such as hf-mirror)

Core Idea

GGUF is a quantized weight format widely used by the community. ComfyUI does not natively support it—the official documentation states clearly: to use GGUF models, you need to install community custom nodes (such as city96's ComfyUI-GGUF).

Comparison	ComfyUI Native Quantization	GGUF Quantization
Requires installing nodes	No	Yes (city96/ComfyUI-GGUF)
File format	safetensors	.gguf
Common bit levels	fp8, int8, fp4, etc.	Multiple community quantization levels, pick by VRAM
How to obtain	Official releases, fp8 weights included	Hugging Face community repositories

In short: the native format works out of the box, while GGUF relies on nodes to fill the gap but offers finer-grained bit levels.

Follow Along

Step 1: Install the ComfyUI-GGUF custom nodes

GGUF is not a native format of ComfyUI, so you must first install city96's ComfyUI-GGUF nodes before the loaders will appear later.

Open ComfyUI's node manager (the Manage panel in the menu), search for GGUF, install city96/ComfyUI-GGUF, then restart ComfyUI as prompted.

You should see: after restarting, double-clicking on the canvas to search for nodes shows loader nodes with GGUF in their names.

• GGUF loaders can be found in the node list

Step 2: Download and place the GGUF quantized models

The community has already quantized popular large models into GGUF, so you can just download them—no need to convert them yourself.

Taking Wan2.2 A14B as an example, the community quantization repositories listed in the official tutorial include:

• bullerwins/Wan2.2-T2V-A14B-GGUF (text-to-video)
• bullerwins/Wan2.2-I2V-A14B-GGUF (image-to-video)
• QuantStack/Wan2.2 GGUFs (collection)

Once inside a repository, pick a bit level to download based on your VRAM (lower levels mean smaller files and less VRAM usage, but generally greater quality loss). After downloading, put the .gguf file into the directory for diffusion models, for example ComfyUI/models/unet/ (in newer versions of ComfyUI the diffusion model directory is models/diffusion_models/; both are usually recognized).

You should see: the file appears under the models/unet/ directory and has not been placed into other subdirectories such as checkpoints.

• .gguf file placed in the correct directory

Step 3: Replace the original loader with a GGUF loader

The standard loader cannot read GGUF, so you need a loader provided by ComfyUI-GGUF to take its place.

Open your original workflow, find the model loading node (such as Load Checkpoint or the node that loads diffusion models), double-click to search for and add the corresponding loader with GGUF in its name, select the .gguf file you just downloaded, connect its output to where the original model loading node's downstream was (the MODEL port and so on), then delete the old loading node.

Keep the remaining nodes—prompts, sampler, VAE, and so on—unchanged, and click run.

You should see: the queue executes normally, and once the progress bar finishes you get an image or video in the output directory, with VRAM usage noticeably lower than running full-precision weights.

• The workflow runs successfully and output files are generated

Checkpoint ✅

• ComfyUI-GGUF nodes are installed and the GGUF loader can be found
• .gguf files are placed in the correct models subdirectory
• The model loading in the workflow has been switched to the GGUF loader, with all ports wired up
• A generation task has been completed successfully under low VRAM

Pitfall reminders

❌ After putting the file into the directory, it cannot be found in the loader

Cause: the file was placed in the wrong subdirectory, or ComfyUI was not restarted/refreshed after placement.

Correct approach: verify the directory (put diffusion-model-type GGUF files in models/unet/), confirm the file extension is exactly .gguf, and if necessary restart ComfyUI so it rescans the models directory.

❌ Assuming GGUF is a native format and not installing the nodes

Cause: ComfyUI officially states that GGUF requires community custom node support; without installing ComfyUI-GGUF there is no corresponding loader.

Correct approach: complete step 1 to install the nodes first, then look for the loader.

❌ Expecting the template's "missing model auto-download" to download GGUF for you

Cause: the workflow template's model information only supports .safetensors/.sft formats; .gguf links are flagged as unsafe and not shown.

Correct approach: always download GGUF files manually and place them in the corresponding directory; do not rely on the template's auto-download.

❌ Blindly choosing the lowest bit-level option

Cause: the lower the level, the greater the image quality loss; different models and tasks have different sensitivity to quantization.

Correct approach: when VRAM is sufficient, prefer a higher bit-level option; if the generated images look wrong (broken colors or details), try one level higher first before investigating other causes.

Advanced tips

• The same repository often offers multiple bit-level options; you can download a small one first to verify the workflow runs, then decide whether to download the higher-quality version
• Text encoders and VAEs also have community GGUF quantized versions; when VRAM is extremely tight, you can use quantized weights for the whole set
• When you hit an out-of-memory error, first read Insufficient VRAM and lag: systematic solutions for OOM and then come back to adjust the level

Lesson summary

• ComfyUI does not natively support GGUF; you must first install city96's ComfyUI-GGUF nodes
• Download GGUF weights from community repositories: for the Wan2.2 series, see bullerwins and QuantStack
• Put files into the correct models subdirectory, and replace the original loader node with a loader whose name contains GGUF
• ComfyUI's native quantized formats (fp8 etc.) require no nodes; prefer them when official versions are available
• Template auto-download does not support .gguf; manual download is required

Key item	Recommended approach	Notes
Custom node	city96/ComfyUI-GGUF	The GGUF support node designated in the official documentation
Download repository	bullerwins / QuantStack	Community quantization resources listed in the Wan2.2 official tutorial
Storage directory	ComfyUI/models/unet/	Diffusion-model-type GGUF (follow the node README)
Level selection	Choose a higher bit level when VRAM allows	The lower the level, the greater the risk of image quality loss

Next lesson preview

Page won't run, nodes throw errors, model loading fails? The next lesson, Troubleshooting overview: check this page first when problems occur (60-second quick check), gives you a quick self-diagnosis process. You will learn:

• A 60-second quick diagnosis approach for common errors
• How to troubleshoot custom node installation failures
• What to do when a model cannot be found or fails to load
• Entry point to systematic solutions for insufficient VRAM