llama.cpp
d628a98f - tests : keep dummy ffn.act scales away from zero (#31940)

Commit
2 days ago
tests : keep dummy ffn.act scales away from zero (#31940) The dummy-model generator fills every tensor from a plain normal pool. `blk.N.ffn.act.scales` is a divisor in the mpt graph, so with seed 1224489258 it draws |v| ~ 1e-6, and `ffn_act = gelu(...) / scales` reaches 2.7e5. Metal stages both operands of `kernel_mul_mm_f32_f32` as half, so those activations overflow to inf, the norm downstream turns the inf into nan and the logits come out nan, while the CPU reference stays in f32 and stays finite. Both NMSE values then become nan and the check fails. Give `ffn.act` the same treatment as `rope_factors` and keep its magnitude away from zero. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB)
Author
Parents
Loading