Z.ai published GLM-5.3-Flash under the MIT licence, which means you can download the weights and run them on your own hardware, commercially, with essentially no conditions. This page covers what that actually takes: memory requirements for a 320B/18B mixture of experts, the serving frameworks Z.ai documents, quantization trade-offs, and an honest breakeven analysis against paying per token.
The conclusion first, because it saves time: self-hosting this model is very rarely cheaper than the API. The pricing is low enough that the volumes required to justify dedicated hardware are enormous. Self-host because you need control over where your data goes — not because you expect to save money. The arithmetic for that claim is at the bottom.
What you are deploying
| Repository | zai-org/GLM-5.3-Flash on Hugging Face |
| Licence | MIT |
| Total parameters | 320B |
| Active per token | 18B — 8 of 288 routed experts, plus 1 shared expert |
| Layers | 45 — 34 linear attention, 11 sparse attention |
| Published precision | FP8 (E4M3) with dynamic activation scaling — attention, embeddings and lm_head held out of quantization |
| Download size | ~328 GB across 62 safetensors shards (72 files total) |
| Max positions | 1,048,576 |
| Vocabulary | 154,880 |
| Modalities | Text, image and video in; text out |
| Languages | English, Chinese |
The important structural point for anyone sizing hardware is the gap between total and active parameters. All 320 billion parameters must be resident in memory, because the router can select any expert for any token. Only 18 billion of them do arithmetic on a given token. Memory is sized by the total; throughput is governed by the active fraction. This model is therefore memory-hungry and compute-light relative to its size — an unusual profile, and one that rewards hardware chosen for capacity over raw FLOPs.
Memory requirements
Start from what Z.ai actually shipped, because this is where a lot of published guidance goes wrong. The repository is not bfloat16. The weights are distributed pre-quantized to FP8 in E4M3 format with dynamic activation scaling, and the download is about 328 GB across 62 shards. A naive “320 billion parameters times two bytes” calculation gives 640 GB and overstates the requirement by roughly double.
Not everything is quantized. The configuration explicitly excludes the attention modules, the hyper-connection parameters, the embedding table and lm_head, keeping those at higher precision — a common and sensible choice, since those components are disproportionately sensitive to precision loss. The figures below are computed from the published shard sizes and parameter count rather than measured on a live deployment.
| Precision | Weights | Practical floor incl. activations & KV cache | 80 / 141 / 192 GB accelerators |
|---|---|---|---|
| FP8 E4M3 — as published | ~328 GB | ~380 GB+ | 5 / 3 / 2 |
| 4-bit — community re-quantization | ~165 GB | ~190 GB+ | 3 / 2 / 1 |
| bfloat16 — upcast, not shipped | ~640 GB | ~740 GB+ | 10 / 6 / 4 |
The practical headline: the official release fits on five 80 GB accelerators, three 141 GB cards, or two 192 GB cards — demanding, but a long way short of what the parameter count implies at first glance. Shipping FP8 by default rather than leaving quantization to the community is a meaningful convenience, because it means the version everyone runs is the version the vendor validated.
Treat the “practical floor” column as a lower bound rather than a target. It assumes modest batching and nothing close to a full context window. The KV cache is the variable that will surprise you: it scales with both sequence length and concurrency, and on a model advertising a million-token window it can rival the weights for memory if you actually use that window under load. If you plan to serve long contexts to multiple concurrent users, budget for it explicitly and measure early.
The hybrid attention design helps here more than it would on a conventional architecture. With 34 of 45 layers using linear attention, per-token cache growth is meaningfully lower than a model of this context length would otherwise demand — that is much of the point of the design. It reduces the problem; it does not remove it.
Bandwidth and storage, before anything else
A 328 GB download is its own small project. Budget disk for at least the full repository plus working space, expect a meaningful transfer time on anything short of a fast datacentre link, and pull with a tool that resumes — 62 shards is 62 chances for a connection to drop at 90%. Mirror it internally once rather than re-downloading per node, and verify checksums before you commit hardware time to debugging what turns out to be a truncated shard.
Quantization trade-offs
Halving precision halves memory, which on a model this size is the difference between a plausible deployment and an implausible one. What it costs in quality is genuinely workload-dependent, and anyone quoting a single confident number has not tested enough workloads.
- FP8 E4M3 — the shipped baseline. This is what Z.ai published and validated, so it is the reference for anything you measure. Modern accelerators have hardware support for it, meaning it is often faster as well as smaller. Unless you have a specific reason to deviate, run what was shipped.
- Upcasting to bfloat16 — possible, doubles memory to roughly 640 GB, and recovers nothing that was lost when Z.ai quantized. There is no quality argument for it; the reason to do it is framework compatibility, if your serving stack cannot handle FP8 on your hardware.
- 4-bit — brings a 320B model within reach of a single high-memory accelerator, which is remarkable. Quality cost is real and uneven: degradation tends to show up first on long-horizon reasoning and precise long-context retrieval, which are exactly this model’s selling points. Test it on your own tasks before committing.
A caution specific to mixture-of-experts models: quantization error does not distribute evenly across experts. An expert that handles a narrow slice of inputs can degrade sharply while aggregate benchmark scores barely move, because the affected inputs are a small fraction of any general evaluation. If your workload is narrow — one language, one domain, one code style — a benchmark average is a poor proxy for what quantization did to your traffic. Evaluate on your own distribution.
Serving frameworks
Z.ai’s model card documents six deployment paths, each with its own recipe or cookbook:
- SGLang — a high-throughput serving engine with a published cookbook entry for this model. A sensible default for production serving.
- vLLM — the most widely deployed open inference server, with a published recipe for GLM-5.3-Flash. Strong choice if your team already runs vLLM.
- TokenSpeed — listed with its own recipe entry.
- KTransformers — notable for this model specifically. It targets CPU/GPU hybrid inference, keeping some experts in system RAM. Because only 8 of 288 experts activate per token, MoE models suit that approach unusually well, and it is the most realistic route to running this model without a large multi-GPU budget. There is a dedicated tutorial for GLM-5.3-Flash.
- Transformers — the reference implementation, via the
glm5_nextmodel class. Correct rather than fast: good for experimentation, not for serving. - Unsloth — documented for fine-tuning workflows.
If your constraint is budget rather than throughput, start with KTransformers. If it is throughput, start with SGLang or vLLM. Consult each project’s current recipe for this model rather than adapting a config from a different one — MoE serving is sensitive to expert-parallelism settings in ways dense models are not.
Two behaviours to carry across from the hosted API: reasoning is mandatory, with reasoning_effort accepting low, high and max and defaulting to max; and in the chat template clear_thinking defaults to false, so pass clear_thinking=true for chat use. Z.ai’s guidance is to keep the default max effort when reproducing published benchmark numbers.
Breakeven against the API
This is the section that decides it for most people, and the answer is unusually clear.
Take a blended API rate at an 80/20 input/output mix. At promotional pricing that is $0.11 per million tokens; at list pricing from September 10, $0.22 per million. Now divide a monthly hardware bill by those rates to find the volume at which self-hosting starts to win.
| Node cost per month | Breakeven at promo pricing | Breakeven at list pricing |
|---|---|---|
| $5,000 | ~45 billion tokens | ~23 billion tokens |
| $10,000 | ~91 billion tokens | ~45 billion tokens |
| $15,000 | ~136 billion tokens | ~68 billion tokens |
| $25,000 | ~227 billion tokens | ~114 billion tokens |
Those numbers are large. A $15,000-a-month deployment needs roughly 68 billion tokens a month at list pricing — about 2.3 billion tokens every single day — before it beats simply paying OpenRouter. For scale, 2.3 billion tokens is on the order of several thousand full-context requests a day, sustained, with no idle periods.
And this comparison flatters self-hosting, because the hardware line is only part of the real cost. It excludes engineering time to deploy and maintain the stack, on-call coverage, redundancy for the hours your single node is down, and the utilisation problem: an API bills only what you use, while a node costs the same at 3am on a Sunday. Realistic utilisation of 40–60% pushes the true breakeven substantially higher again.
When self-hosting is still right
Cost is the wrong reason. These are the right ones:
- Data residency and confidentiality. If prompts cannot leave your infrastructure — regulated industries, classified environments, client contracts that forbid third-party processing — this is not an optimisation, it is the only option. Before the MIT release there was no version of this conversation.
- Permanence. A hosted model can be deprecated, re-priced or withdrawn. Weights you hold cannot. If you are building something with a long support horizon, that is worth real money.
- Fine-tuning. The licence permits modification. If your edge comes from adapting the model to a proprietary domain, you need the weights.
- Determinism and pinning. Self-hosting means the model does not change under you. No silent updates, no shifting behaviour between releases.
- Air-gapped or edge deployment. No API reaches a disconnected environment.
- Genuinely extreme volume. If you are past the breakeven figures above with high utilisation, the economics do work.
For everyone else, the honest recommendation is to use the API and revisit if your volume or your compliance position changes. The MIT licence is valuable even if you never exercise it, because it means that option stays open — and that the model cannot be taken away from you later.
Frequently asked
Can I run GLM-5.3-Flash locally?
Yes — the weights are MIT-licensed on Hugging Face. It is a 320B-parameter model shipped pre-quantized to FP8, so “locally” means roughly 328 GB of memory, or about 165 GB if re-quantized to 4-bit. KTransformers, which keeps experts in system RAM, is the most realistic route on limited hardware.
What hardware do I need?
Computed from the published shard sizes: about 328 GB for the FP8 release as shipped, or about 165 GB re-quantized to 4-bit, before activations and KV cache. That is roughly five 80 GB accelerators, three 141 GB cards or two 192 GB cards for the official weights. Long contexts and high concurrency push these figures up.
Is self-hosting cheaper than the API?
Usually not. At list pricing a $15,000-a-month node needs roughly 68 billion tokens a month to break even, and that ignores engineering time, redundancy and idle capacity. Self-host for data control, permanence or fine-tuning rather than for cost.
Can I use it commercially?
Yes. The MIT licence permits commercial use, modification and redistribution with essentially no conditions beyond retaining the licence notice. Verify the current licence on the repository before relying on this for anything contractual.
Related: GLM-5.3-Flash full reference · Pricing and the live provider spread · API and migration guide · How Ox Alpha became GLM-5.3-Flash.
Sources and verification
Last substantively reviewed: August 27, 2026. Reviewed by: OxAlpha.chat Editorial Team.
How we verified this page
Hardware and deployment claims were checked against the official model card and exact framework recipes. The model is not described as practical for an ordinary desktop: published recipes require server-class memory and multiple accelerators or specialist CPU/GPU offload.
Primary sources
- Official weights: GLM-5.3-Flash model card — provides the checkpoints and supported serving frameworks.
- License: Official MIT license file — provides the reuse and modification terms.
- SGLang deployment: Official SGLang GLM-5.3-Flash cookbook — documents CUDA and ROCm deployment commands and features.
- vLLM deployment: Official vLLM GLM-5.3-Flash recipe — documents memory requirements and parallel serving options.
- Hybrid deployment: KTransformers GLM-5.3-Flash tutorial — documents CPU/GPU offload and large system-memory requirements.
Found a factual error or a changed price? Send us the page URL and supporting source; corrections follow our editorial policy.