How to set up GLM 5.2: a practical guide for engineering teams

First, the one number that decides everything

GLM 5.2 is big. About 753 billion parameters big. The full weights land near 1.5 TB, and even the slimmed-down FP8 build is roughly 750 GB. The reference configuration for serving it at full precision is eight NVIDIA H200 GPUs working as one. So the honest starting point is simple: running the full model on a laptop is not realistic. Pick the path that matches the hardware actually on hand.

Path A: Run GLM 5.2 via the API (the fastest start)

No GPUs, no downloads, no weekend lost to CUDA. The endpoint is OpenAI-compatible, so it drops into existing tooling. For most teams this is the right place to begin: prove the model earns its place in the product first, then decide whether owning the infrastructure is worth it.

1. Create an account on the Z.ai API platform, or a third-party host like Fireworks, DeepInfra, Novita, or OpenRouter.
2. Generate an API key.
3. Point any OpenAI-compatible client at the endpoint: change the base URL and the key, and leave everything else alone.
4. Set the model name to glm-5.2 and send a request.

Path B: Run a quantized GLM 5.2 locally

Quantization compresses the weights to fit smaller machines, trading a little quality for a lot of portability. The Unsloth community ships GGUF builds for LM Studio and llama.cpp, roughly 376 GB at Q4 and 241 GB at Q2. Still large, but genuinely runnable on a high-end workstation or a Mac Studio with enough unified memory.

1. Install LM Studio (the easy route) or llama.cpp.
2. Search for unsloth/GLM-5.2-GGUF and pick a quant the machine can hold. Q2 is smallest; Q4 is the better quality-to-size balance.
3. Download, load, and chat, or turn on LM Studio’s local server to expose an OpenAI-compatible endpoint for other apps.

Path C: Self-host GLM 5.2 with vLLM (production, 8-GPU)

This is the official production route. Plan on roughly 8 high-memory GPUs (H200 class) and tensor parallelism. vLLM v0.23.0 is the minimum version.

1. Download the FP8 weights (~750 GB)

huggingface-cli download zai-org/GLM-5.2-FP8 \
  --local-dir /models/glm-5.2-fp8

2. Install vLLM

uv venv && source .venv/bin/activate
uv pip install "vllm==0.23.0" --torch-backend=auto
uv pip install "transformers>=5.9.0"

FP8 also needs DeepGEMM, installed via the repo’s install_deepgemm.sh.

3. Serve the model

vllm serve zai-org/GLM-5.2-FP8 \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 5 \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --served-model-name glm-5.2-fp8

4. Smoke-test it

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "glm-5.2-fp8",
       "messages": [{"role": "user", "content": "Say hi"}],
       "temperature": 1, "max_tokens": 4096}'

Connect GLM 5.2 to VS Code or a coding agent

Every path above ends in the same place: an OpenAI-compatible endpoint. So wiring it into an IDE or agent is identical regardless of how it is hosted.

1. In the IDE’s AI settings, choose “Custom / OpenAI-compatible endpoint.”
2. Paste the base URL (http://localhost:8000/v1 for local, or the provider’s URL) and the API key.
3. Set the model to glm-5.2. That is the whole integration.

Why teams set up GLM 5.2

New does not mean worth adopting. GLM 5.2 earns the attention for concrete reasons:

• It is genuinely ownable. MIT weights mean it can be downloaded, fine-tuned, and run commercially with no per-seat fee. Compute is bought once instead of tokens rented forever.
• The quality is frontier-class. It currently leads open-weight models on the Artificial Analysis Intelligence Index and competes with closed frontier models on long-horizon agentic coding.
• The cost curve is different. API pricing lands around $1.40 in / $4.40 out per million tokens, and self-hosting removes per-token cost entirely at volume.
• Data stays put. Running the weights on private infrastructure means prompts never leave the network, the only viable option for air-gapped or regulated work.
• No lock-in, plus 1M context. A version can be pinned forever so an upstream update never changes behavior overnight, and whole codebases or long documents fit in a single pass.

Where to get reliable information

Trust the primary sources over auto-generated tutorials, many of which contain errors:

• Official model card huggingface.co/zai-org/GLM-5.2
• FP8 model card huggingface.co/zai-org/GLM-5.2-FP8
• vLLM deployment recipe recipes.vllm.ai/zai-org/GLM-5.2