Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
ManniX-ITAΒ 
posted an update Aug 21
Post
191
πŸš€ Qwen3.6-27B-A3B-CoderX β€” the long-horizon sibling to A3B-Coder. Same 256β†’184 expert budget (~35Bβ†’27B, A3B active), different selection: our saliency map picks the keep-set, a REAP-style per-layer floor (p=24) protects the tail, and the 72 evicted experts per layer are folded DERN-style into the survivors instead of discarded. No fine-tuning, no distillation.

πŸ“Š Q6_K + imatrix, llama.cpp b9700, greedy, one pinned geometry per bench, same host β€” CoderX / A3B-Coder / unpruned 256e:
⚑ LiveCodeBench v6 (77q, 24k think) β€” 72.73 / 61.04 / 61.04 β†’ +11.7pp over both
βœ… HumanEval+ (164) β€” 96.95 / 95.12 / 93.90 β†’ best of the three
🀝 MultiPL-E-100 (rs+java+js) β€” 88.67 / 89.00 / 91.00

⚠️ Read that last row honestly: a same-basis repeat of MultiPL-E moved 1.0pp on batch-scheduling nondeterminism alone. The 0.33pp CoderX↔Coder gap is INSIDE that band β€” a tie. The 2.33pp gap to the base is outside it and real. CoderX takes Rust (0.85 vs 0.81), gives up JS (0.92 vs 0.96).

🎯 Ships top-8, and that was measured, not assumed: MBPP-full 78.4 / 79.0 at top-8 vs 73.2 / 73.0 at top-10. Opposite call from A3B-Coder, which bakes top-10.

🧠 It thinks long β€” LCB median completion ~15.8k tokens vs ~2.2k for Coder. The length is where the win comes from; give it context headroom rather than clamping it.

πŸ”¬ Not measured yet: the canonical 9-bench. GPQA / MATH-500 / IFEval are deliberately NOT quoted β€” treat the non-code profile as unknown. Coder remains the one with a published 9-bench table.

πŸ“¦ bf16 safetensors (text-only) Β· 19 GGUF tiers, EVERY K/I-quant imatrix-built and verified by reading quantize.imatrix.* back out of each uploaded file Β· Ollama 39 tags (19 text + 19 vision-<tier> + :latest). MTP in every tier β€” draft_num_predict 3 gives 190β†’252 tok/s (+33%) on an RTX 5080.

πŸ”— ManniX-ITA/Qwen3.6-27B-A3B-CoderX

πŸ”— ManniX-ITA/Qwen3.6-27B-A3B-CoderX-MTP-GGUF

πŸ”— https://ollama.com/mannix/qwen3.6-27b-a3b-coderx

Your result is larger than your description of it. I went to check the DERN fold and could not find it in the weights.

Method, so you can redo it in about five minutes. Range requests on the safetensors: read the 8-byte header length, read the header JSON, then pull one tensor's byte span. No download of either model.

The router is a strict subset of the base, bit for bit

model.language_model.layers.L.mlp.gate.weight is [184, 2048] in CoderX and [256, 2048] in Qwen/Qwen3.6-35B-A3B. I hashed each 4096-byte row on both sides:

layer   rows matched to a base row   ascending   base rows absent
  0            184 / 184               yes            72
 20            184 / 184               yes            72
 39            184 / 184               yes            72

The keep-sets are genuinely per-layer, which is the part of your description that holds. Layer 0 keeps 0,1,3,4,5,7,8,10; layer 20 keeps 0,1,3,4,6,7,8,9; layer 39 keeps 1,2,3,5,6,8,9,10. Different maps, same budget. Your saliency pass is doing work.

The surviving experts are byte-identical to the base

layer 0    gate_proj, CoderX experts 0,1,2,5,7,20,50,100,150,183   10/10 identical
layer 0    up_proj,   expert 0                                      identical
layer 20   gate_proj + down_proj, experts 0,7,100,183                8/8 identical
layer 39   gate_proj + down_proj, experts 0,7,100,183                8/8 identical
shared_expert gate/up/down + shared_expert_gate, layers 0,20,39      identical
input_layernorm + post_attention_layernorm, layers 0,20,39           identical

Forty-five expert-side tensors sampled, plus the three routers above. Forty-eight comparisons, forty-eight exact matches against Qwen/Qwen3.6-35B-A3B after accounting for its fused [256, 1024, 2048] layout. Not close. Equal.

Your own config.json says the same thing. merge_args.merging is "none", next to merge_size: 184 and drop_map_184e_hybrid_p24.json.

Why I think this makes the model better, not worse

If a fold HAD landed, the untouched router row would be the bug. Survivor i would compute base_i plus whatever mass it inherited, while the gate still scores it with base_i's row and renormalizes over 184. The gate would be pricing a function the expert no longer computes.

Shipped as pure eviction, that inconsistency does not exist. The model is coherent exactly because nothing was merged.

And it moves the headline. +11.7pp on LiveCodeBench v6 over both A3B-Coder and the unpruned 256e, from deleting 28% of the experts and touching nothing else, is a much stronger claim than the same number with a merge step in it. Deletion has no free parameters to have gotten lucky in.

The honest reading of your MultiPL-E caveat supports that too. You called a 0.33pp gap a tie because your own repeat moved 1.0pp. Someone doing this to look good does not do that.

So the question is which artifact carries the numbers. Were the 19 GGUF tiers quantized from these exact safetensors, or from a merged checkpoint that did not get uploaded?

If it is these, the post is underselling itself and one line fixes it. If it is a different one, the safetensors repo and the GGUF repo are two different models and the LCB table belongs to only one of them.

Β·

I was on holidays and working thru the mobile app. Quite messy.
The HF model card reported the wrong recipe, folding was attempted and it was really bad.
Tested also the stock Samsung REAM and it's not getting close to REAP with protected experts.

Different result with Gemma-4 A4B where stock Samsung REAM produced a very competitive quant on evals but messed up in special token usage, leaking all the thinking into the message content when used in multi-turn agentic coding.
Merging survivors experts it's still not viable at this point.

The special-token leak is the part I would chase, not the eval numbers.

A quant that scores competitively and then leaks thinking into message content on multi-turn agentic coding has not degraded. It has degraded somewhere the benchmark cannot look. Single-turn suites score the content of one reply. They never check that the turn boundary survived. So a token-level regression prices in at zero.

That is the nastier failure, because it passes the gate that was supposed to catch it.

Two things I would want before trusting any REAM against REAP comparison on Gemma-4 A4B:

  • a multi-turn harness where the metric is "did any thinking-channel token appear in the message channel", counted per turn rather than per run
  • the same count on the unquantized base, because if the base leaks at a low rate too then the quant amplified something rather than introduced it

Your published A4B line is all 98e v5, v6, v7. The stock REAM build is not up, so the one artifact that shows the leak is the one nobody can reproduce. A quant that evals well and breaks the channel is a more useful thing to publish than another good one, if the card says so plainly.

You say stock Samsung REAM is nowhere near REAP with protected experts on the 27B, but competitive on A4B. Is that the method or the expert layout? A much smaller active set would make expert protection matter less, which would make it a property of the model rather than a result about the method.

Did the token leak show up on the 27B too, or only on A4B?

Β·

It was competitive on A4B on evals but when tested on my Cline fork I found out that it had completely broken the thinking channel.
Had to give up also on A4B.

You gave up on A4B, but you left the best artifact in the repo.

template_loop_unittest.py, sitting in both v7 GGUF repos. Different bug from your channel leak, I know. It is still a real detector with a sentinel and an exit code, and as far as I can tell nobody has pointed it at anything except the repo it ships in. So I ran it.

Every Gemma-4 instruct template Google publishes fails your test.

template sha    repos                                 sentinel re-injections
ae53464bf3be    gemma-4-26B-A4B-it, 31B-it, 12B-it    1   MAY BE VULNERABLE
0a2c8073c878    gemma-4-E4B-it, E2B-it                1   MAY BE VULNERABLE
d9f21aac4764    your v7 coder / coderx                0   OK

Your one-line change is the whole difference. Drop the unconditional loop.index0 > ns_turn.last_user_idx disjunct, keep Google's own preserve_thinking opt-in. 494 bytes against their 18,683.

Then I pointed it at your own namespace, and that is the part I would act on.

PASS   v7-coderx-it, v7-coder-it, v7-coderx-NVFP4A16
PASS   v6-coder-it, v5-coder-it, v5-coder-NVFP4A16
PASS   both v7 GGUF repos (embedded template, byte-identical to the side file)
FAIL   v7-coder-NVFP4A16
FAIL   v5-it, v4-it, v3-it, 31b-he1-it
FAIL   31b-he1-it-NVFP4A16, Gemma-4-26B-A4B-it-NVFP4A16, Gemma-4-31B-it-NVFP4A16

gemma-4-A4B-98e-v7-coder-it passes. gemma-4-A4B-98e-v7-coder-NVFP4A16 does not. Same model, and the NVFP4A16 is the vLLM build, which is the one that ends up in a long agentic loop. It also carries a template hash no other Gemma repo of yours uses (36e3a42e5cf1), so it reads like a build that missed the rebase rather than a decision.

I checked the GGUF path too, because a side file named chat_template.fixed.jinja usually implies the embedded one is not fixed. Not here. I pulled the first 40 MB of both CD-Q2_K files and parsed the GGUF KV block directly: tokenizer.chat_template is 19,177 bytes hashing to d9f21aac in both. A llama.cpp user gets the fix without passing anything. Worth one line on the card, because the file's name suggests the opposite.

None of this touches the channel leak, which is still the thing I would want counted per turn rather than per run. But it does change where your fix belongs. It is not a quant-repo detail. Google ships the failing template on five instruct models, and you are the only person I can find who has a test that catches it.

Have you put this upstream, or is there a reading where replaying history thinking is what they intended?