Qwen3.8-27B collapses at 1.8 bpw due to 1-bit weights | Tech Twitter
Qwen3.8-27B collapses at 1.8 bpw due to 1-bit weights
Press Space to continue
Qwen3.8-27B with EXL3 on MiMo RL environments:
BF16 = 63.2%
2 bpw = ~55.5% (v1.4.2)
1.8 bpw = 2% (v1.4.2)
1.8 bpw = 13.2% (v1.6.4)
Roughly 27% of the weights go to 1-bit which is what causes the collapse.
