A design little sufficient to range from an approximately 5 GiB file on a CPU now declares a 98.2 on MATH-500. The most revealing line in its design card isn’t a rating. It’s a footnote stating the rival numbers were never ever re-run.
A Small Model With Big Claims
webAI launched TwIL-8B as an open-weight design on November 4, and its Hugging Face design card leads with enthusiastic numbers. The 8.8B-parameter thinking design is post-trained from IBM’s Granite 4.2 8B utilizing full-parameter monitored fine-tuning, followed by GRPO/DAPO support knowing. A last length-aware RL phase produces the “balanced” alternative, which is the default.
The card reports 98.2 on MATH-500 and 88.3 on AIME 2025. It states TwIL-8B posts the greatest MATH-500 and LiveCodeBench ratings in its contrast group, that includes Gemma 4 31B at about 3.5 times its size. That claim deserves screening, since little designs that beat bigger ones are precisely what open-weight users wish to hear.
What the Model Card Reports
On MATH-500, the card notes TwIL-8B at 98.2, Qwen3.5-9B at 96.6 and Gemma 4 31B at 87.9. On AIME 2025, TwIL-8B ratings 88.3 versus 87.1 for Gemma 4 31B. 
On LiveCodeBench, TwIL-8B ratings 81.6 versus 80.0, though that figure brings a dagger marking it as determined on the verbose version, not the default well balanced one.
The card is likewise more honest than many. It states TwIL-8B is not an understanding design. GPQA Diamond is available in at 70.2, tracking Gemma 4 31B by 14.1 points, and the design likewise tracks on MMLU-Pro and HLE. There is no security or choice tuning beyond what Granite brings, and the license is non-commercial. That sincerity makes the claims simpler to weigh.
The MATH-500 Gap Rests on One Borrowed Number
Here is where the heading gets made complex. TwIL-8B’s ratings originated from webAI’s own internal high-effort pass@1 runs. The card mentions that rival ratings are released figures and were not re-run.
Gemma 4 31B’s 87.9 on MATH-500 originates from Red Hat’s assessment of an FP8 developutilizing the lighteval harness, not from Google. The card itself keeps in mind that this figure is abnormally low and might show response extraction. Simply put, the 10.3-point space that anchors the heading compares one group’s own harness versus a 3rd party’s number that the card suspects might be downplayed.
None of this makes TwIL-8B’s outcome incorrect. It indicates the margin isn’t developed up until both designs run under the exact same harness, timely and tasting settings. For referral, Google’s Gemma 4 design card is the source for numerous of the other Gemma figures in the table.
Where Gemma 4 31B Still Wins
Check out the entire table and the image is more well balanced than the heading recommends. In the card’s head-to-head summary, TwIL-8B beats Gemma 4 31B on 3 of 14 shared standards and loses 11. It leads on MATH-500, LiveCodeBench and AIME 2025, however tracks on AIME 2026 (88.3 versus 89.2), HMMT Feb 2026 (68.2 versus 77.2), GPQA Diamond, MMLU-Pro and IFEval.
The AIME 2025 lead is the shakiest. The card states AIME and HMMT sets have 30 to 33 issues, so one issue deserves about 3 points, and spaces of that size sit within sound. A 1.2-point margin is well under one issue, so the outcome is finest checked out as a tie.
Versus the closest same-size competitor, Qwen3.5-9B, the card reveals the designs approximately level on contest mathematics, with TwIL-8B utilizing less tokens: about 19.5 k per AIME 2026 issue versus 31.2 k. That performance claim might end up being the more resilient one.
What a Home Re-Run Would Involve
The card makes duplication abnormally useful. The Q4_K_M well balanced GGUF weighs 4.98 GiB and goes through llama.cpp, with a command given up the card. Bigger Q8_0 and F16 develops exist just for the verbose version. The design likewise requires a huge generation spending plan, considering that a brief limitation truncates its thinking and ratings far even worse. The card recommends settings around 32,768 brand-new tokens and a 40,960-token context.
Anybody preparing a check must anticipate 3 catches. The card states all its ratings were determined on bf16 weights, not the GGUF develops, so a CPU run evaluates a somewhat various artifact, and the card anticipates little variances, most likely at Q4_K_M. Second, MATH-500 is 500 issues and each response can go to 10s of countless tokens, so CPU screening will be sluggish. Third, the translating setup matters, given that the card states none of its numbers utilize low-effort or thinking-off modes.
For a reasonable contrast, the very same timely, respond to extraction and tasting should be used to Gemma 4 31B too, preferably at complete accuracy.
How to Read the Result Until Someone Repeats It
The card likewise notes contamination cautions. Its training swimming pools were decontaminated versus AIME, HMMT Feb 2025 and 2026, MATH-500, GPQA, MMLU-Pro, HLE, IFEval, IFBench and LiveCodeBench utilizing 13-gram matching. HMMT Nov 2025 and the HumanEval, MBPP and CRUXEval suites weren’t in the decontamination set. The MATH-500 and AIME results fall inside the safeguarded group, which counts in the design’s favor.
In general, TwIL-8B appears like a really strong little design for contest mathematics, code and direction following, and the card is open about its weak points. The claim to beat Gemma 4 31B is narrower than the heading recommends: a clear lead on one criteria, a most likely tie on another and a loss on the majority of the rest.
The beneficial next action is a same-harness run of both designs, with the bf16 weights as a control for the GGUF construct. Up until independent outcomes appear, the 98.2 is finest priced estimate as webAI’s reported figure, and the size-beating framing is much better referred to as unverified than incorrect.
Disclosure: This is not trading or financial investment recommendations. Constantly do your research study before purchasing any cryptocurrency or investing in any services. Follow us on X @nulltxnews
Discover more from PMN S.P.O.R.T.S - A PRIME MEDIA NETWORK BRAND
Subscribe to get the latest posts sent to your email.
