This is the exact file that runs Moon-Beam-8B-A1B-Preview at 7.2 tok/s in real chat on a Xiaomi TV Box S (3rd Gen) . That box has 4× Cortex-A55 and ~1 GB of usable RAM. The model is an 8B mixture-of-experts with 128 experts. Experts are paged from flash on demand.
From the 🤏 smolmaxxing project. The full write-up is in EXPERIMENT.md §19 .
⚠️ This file needs the patched llama.cpp from the smolmaxxing repo. It has a pruned output head (60,000 of 157,184 vocabulary rows plus an output_ids tensor). Stock llama.cpp, LM Studio and Ollama will not load it. See Run .
⚠️ This file needs the patched llama.cpp from the smolmaxxing repo. It has a pruned output head (60,000 of 157,184 vocabulary rows plus an output_ids tensor). Stock llama.cpp, LM Studio and Ollama will not load it. See Run .
The general.name in the metadata is "Merged R1B". That comes from the source file and was left unchanged.
Measured on the TV box
Quality is English perplexity on a fixed 22 KB text, converted to bits per byte so models with different tokenizers compare fairly. Bonsai-8B scores 0.974 on the same text.
RAM: ~1.14 GB RSS in headless mode. On flash: the GGUF plus a 2.6 GB repack cache.
The repeat penalty matters. Q2_K experts drift into loops after ~250-300 words without --repeat-penalty 1.15 --repeat-last-n 256 .
to the other sizes
This model is the 8B MoE row. The best file of every size class is in hyperionaidev . The scoreboard has the full table and the kernels and memory tricks behind each step.
Build the patched llama.cpp first. It's llama.cpp at 8642541 with patches/llama.cpp-v0.8.0-tvbox.patch applied; Quick start has the exact steps. Then run:
On a machine with RAM to spare, drop the environment variables: the patched llama-server -m mb-q2-h60.gguf loads every expert normally. On the TV box, tv-llm-ui/start.sh in the repo does all of this, including a headless mode that frees Android's RAM.
LLAMA_MOE_USED : experts per token. 6 is the default; 8 gives better text at ~5 % less speed.
GGML_MOE_BIAS : how strongly routing prefers experts already in RAM.
GGML_MOE_POOL : experts kept in RAM per layer.
License: MIT, the same as the original model.
Model: tablinum/Moon-Beam-8B-A1B-Preview , built on inclusionAI's Ling-3.0.
Runtime: llama.cpp (MIT) plus the smolmaxxing patch.
We're not able to determine the quantization variants.
Model tree for hyperionaidev/Moon-Beam-8B-A1B-tvbox-GGUF
The full story
This article is one source in a clustered incident — the cluster page carries the summary, timeline and every other outlet covering it.
