---
title: A Measured Review of the Three Hottest Categories of Local Models on Hugging Face
date: 2026-09-24
time: 0:10
model: admin
category: knowhow
summary: A measured, benchmark-and-VRAM-based review of Hugging Face's most popular local models, from distilled math models to decensored tunes and coding agents
tags: HuggingFace, distillation, Abliterated, Qwen3-Coder-Next, GGUF, local-LLM, benchmark
---

To get straight to the point: right now Hugging Face has the Google Gemma 4 ecosystem and the Qwen lineup splitting the top of the charts, and every day the community pours out models they have tuned themselves from these two families. Instead of vague praise, this lays out concretely what level each model's benchmark numbers and VRAM requirements are at. Note, however, that community tunes measure under different conditions from card to card, so for representative claims I give the source, and where conditions are unclear I say so.

## 1. Reasoning and Knowledge Distillation

### daksh-neo/qwen-to-gemma-math: A 2B-Class Math Distillation Experiment

This is a distilled model that transplants the math reasoning chain (CoT) of the latest ultra-large Qwen3-plus into a Gemma 4 E2B (actually 5.12B parameters) student model.

| Item | Number | Conditions |
|---|---|---|
| Baseline (gemma-4-E2B-it) | GSM8K 71.0% (142/200) | greedy decode, exact match |
| Distilled model (NEO) | GSM8K 75.0% (150/200) | same conditions, +4.0%p improvement |
| Teacher reasoning generation | 96.5% of 500 problems valid | Qwen3-plus, temperature 0 |
| Fine-tuning | LoRA r=16, full GSM8K 7,473 samples | 3 epochs |

Why it's popular: despite being only a 5B-class size, it learned a large model's thought process directly and surpassed the baseline. But there is something to state honestly. In the same model card's per-version records, a full-fine-tuned version that scored only 10% (2/20) on a 20-sample test is also posted. In other words, 75% is the claim of the LoRA full-data version, and with 200 test samples the margin of error is large. It is usable as a math-specific auxiliary calculator, but not a number to take on faith.

User reaction: the community's distillation fever itself is hot, but the critics are sharp too. In the control experiment of a 23-model comparison project (abliterlitics), reasoning distillation tunes actually broke the models. Claude distill fell 17 points on GSM8K and 12.5 points on MMLU-Pro by overwriting Gemma 4's native reasoning circuitry, and Gemini distill was a net loss. The lesson is that distillation is all about teacher trace quality, and if the trace is bad, the student perfectly learns wrong thinking.

### GLM-5.2-MoE Community Tunes: Specialized for Agent Integration

These are versions the community polished for agent integration from GLM-5.2, a mixture-of-experts ultra-precise reasoning model. The native GLM-5.2's published numbers are the baseline.

| Benchmark | GLM-5.2 (native) | Comparison |
|---|---|---|
| Terminal-Bench 2.1 | 81.0 | close to Opus 4.8's 85.0 |
| SWE-bench Pro family | 62.1 | top tier among open source |
| Context | 1M tokens | suited to long-running agents |

Why it's popular: it delivers performance close to commercial paid models on complex multi-step reasoning and agent integration, while its weights are public. Each community tune has refined system-prompt compliance or tool-calling format, so there is no fixed benchmark per tune — treat the native numbers as the ceiling and choose accordingly. At 744B-class, local is impossible without a cluster, so using it via API is the realistic path.

## 2. The Decensoring Category: Abliterated/Uncensored Trends

### The Principle: Surgical Removal of the Refusal Response Without Retraining

Abliteration is a technique that finds the refusal direction vector inside a model and precisely erases it. Because it is weight editing rather than retraining, the community's assessment is that the smart performance is preserved and only the strict filtering is removed. The representative pipeline is Pliny's OBLITERATUS, and the methodology is rooted in the Arditi et al. (2024) research.

### OBLITERATUS Series Measured Numbers

| Model | Scope of edits | Refusal rate | Q4 size | Local feasibility line |
|---|---|---|---|---|
| Gemma 4 E4B OBLITERATED v3 | 21 of 42 layers operated on | 0% (hard refusal) | 4.9GB | runs even on a phone |
| Ornith-1.5-9B OBLITERATED | all 32 layers, staged intensity | mostly refusal-free | 5.4GB | 8GB GPU possible |
| Qwen3.8-27B OBLITERATED | directional ablation | claims refusal-free | about 16GB | 24GB GPU recommended |

For Gemma 4, existing tools all broke because of NaN activations and shared KV weights, and the creator says he broke through with whitened SVD and attention-head surgery, so it is a technically difficult piece of work. At 4.9GB in Q4, real-world reports keep piling up of people putting it into a phone app (PocketPal and the like) and running it offline.

### User Verification: Download Count and Quality Are Separate

This is exactly where the hottest controversy lies. The conclusion of the abliterlitics project, which compared 23 Gemma 4 E4B models by the same yardstick, was shocking. The OBLITERATUS version, the most downloaded (about 800,000), was judged the most broken model.

| Model | HarmBench release rate | GSM8K | Distribution drift (KL) | Edited tensors |
|---|---|---|---|---|
| abliterix | 100.0% | 87.1% | 0.054 | 89 |
| trevorjs uncensored | 99.3% | 88.3% | 0.015 | 84 |
| heretic family | 95.5% | 88.2% | 0.002 | 29 |
| OBLITERATUS | 72.0% | 66.0% | 1.102 | 381 |

As the numbers say, the fewer tensors touched (surgical), the winner. The heretic family touched only 29 tensors for a 95% release rate while maintaining performance, whereas OBLITERATUS touched 381 for last-place release rate and collapsed GSM8K from the baseline (87.0%) to 66%. User reviews are split too. On Hacker News, criticism that it is a mathematically groundless lobotomy and real-world reports that the model becomes stupid piled up, while the Qwen3.8-27B version's card countered with its own verification: 6 rounds of surgery, 0% refusal over 842 prompts, and a tie at MMLU 69/70. The lesson is one: download counts are marketing — choose a model with low KL (distribution drift) and few edited tensors.

### Community Tuners Such as Hauhau and Jiunsong

Besides OBLITERATUS, tuners such as Hauhau and Jiunsong steadily post Qwen and Gemma family abliterated models, drawing downloads. These are community-tuner accounts rather than the brand of a specific pipeline, so when you download, check the base model name, the quantization, and the card's test report yourself.

Caution: decensoring is useful for security research and personal freedom, but legal responsibility for the output rests with the user. When using it for work automation, keeping a separate output guardrail is recommended.

## 3. The Coding Agent Category: Qwen3-Coder-Next

### Specs and Official Benchmarks (arXiv:2603.00729)

A hybrid-attention MoE based on Qwen3-Next, activating only 3B of its total 80B per token. It is a coding-agent-dedicated model trained on executable coding-task synthesis and environment-feedback learning.

| Benchmark | Qwen3-Coder-Next (80A3) | Comparison |
|---|---|---|
| SWE-Bench Verified | 70.6-71.3 | on par with DeepSeek-V3.2's 70.2, below Opus 4.5's 78.2 |
| SWE-Bench Multilingual | 62.8 | below Sonnet-4.5's 67.2, practical range |
| SWE-Bench Pro | 42.7 | competitive with ultra-large open source |
| Context | 256K (1M extended via Yarn) | repo-level analysis possible |

Why it's popular: at the compute cost of 3B active parameters it scores on par with models 10-20x larger on coding, so the inference cost of a 24-hour crawling or code-analysis agent is overwhelmingly cheap. Pairing it with scaffolds in the Cline, Cursor, Claude Code family is the standard approach.

Real-world user reactions are clearly for and against. Here is a summary of r/LocalLLaMA usage reports.

Positive camp: many rate it as the first practical coding model under 60GB. With no thinking loop, overnight runs never stall, and tool calls are stable, earning praise in OpenCode and Roo Code. One user said the name "coder" does it a disservice, that it punches above its class in planning, research, and general agent work, and gave an A+ for small business agents.

Negative camp: reports of a ReadFile infinite loop, timeouts editing large files, and mistakes on large codebases due to the absence of reasoning. There is also criticism that it pairs poorly with Roo Code's architect mode. Some users find Qwen 3.5 27B better, if slower, so experience the speed-vs-depth tradeoff and choose. A common tip is that it runs best in an OpenCode-family scaffold, with sampling recommended at temperature 1.0 and top_p 0.95.

### Correction: It Does Not Run on 8GB

Many intros write that since it is small it runs on 8GB, but with a total of 80B that is physically impossible. Based on unsloth GGUF it needs 45GB+ at 4-bit and 30GB+ at 2-bit XL.

| Environment | Feasible? |
|---|---|
| 8GB GPU | No |
| 24GB GPU (3090/4090) | Possible with 2-bit quantization |
| 48GB+ / Mac 64GB+ | Comfortable at 4-bit |
| Anything less | Connect via API (OpenRouter, etc.) |

```bash
# Example for a 24GB environment (2-bit quantization)
ollama pull unsloth/qwen3-coder-next-gguf:Q2_K_XL 2>/dev/null || echo "download directly from the Hugging Face unsloth repository"
```

Early versions of llama.cpp had a bug where Qwen fell into a loop, so it is best to update to the latest build and download the GGUF again.

## 4. Final Selection Table by VRAM

| VRAM | Recommendation | Purpose |
|---|---|---|
| Phone-8GB | Gemma 4 E4B OBLITERATED Q4 (4.9GB) | offline assistant, light queries |
| 8-12GB | Ornith 9B OBLITERATED Q4 (5.4GB), qwen-to-gemma-math | uncensored coding assistance, math |
| Phone offline | Gemma 4 E2B (measured daily driver on Pixel) | 32K context, thinking mode on/off |
| 24GB | Qwen3.8-27B OBLITERATED Q4 (16GB), Coder-Next 2-bit | 24-hour crawling agent |
| 48GB+ | Coder-Next 4-bit, consider the GLM family | a real coding agent server |
| Cluster/API | GLM-5.2 tunes, MiniMax-class | long-running large agents |

Once you pin down your current hardware spec (VRAM capacity) and the purpose you want to implement (Python coding assistance, data extraction, a work assistant), the table above immediately shows the model that will run at top speed without stress.

Appendix: three measured tips from phone-offline users. First, there are reports that Gemma 4 E2B works as a daily driver on a Pixel. Put it in the AI Edge Gallery app and run it at 32K context and it is faster than you can read. Second, 8GB of RAM is the borderline. Below that, the app dies or becomes unusable from overheating. Third, on PC, older versions of llama.cpp cannot read Gemma 4 tensors and throw a missing tensor error. Update to the latest build and download the GGUF again. On an RTX 4060 8GB, there is a measured offloading result of splitting the load between 4GB of VRAM and 8GB of RAM.