llama.cpp

7874988b - spec : Warn when --spec-draft-p-min is used with probabilistic draft sampling (#31294)

spec : Warn when --spec-draft-p-min is used with probabilistic draft sampling (#31294) * spec : skip p_min for sampled MTP and DFlash2 drafts Under rejection sampling the acceptance at a position is 1 - TV(p, q), which the draft's top probability does not tell, so the p_min cutoff dropped draft tokens the target would have accepted. Skip it when MTP or DFlash2 samples the draft. draft-simple keeps it: each extra draft token costs a full draft model pass. * spec : replace the sampled-draft p_min skip with a warning p_min behaves as on master again for every drafter and sampling mode. Instead, log a warning when --spec-draft-p-min > 0 is used with --spec-draft-sampling probabilistic, suggesting p_min 0: under rejection sampling the draft's top probability does not tell how much of the draft the target accepts. The cut is not skipped because with MTP at long drafts it can still pay off. * spec : warn about p_min only for drafters that use it The n-gram drafters ignore p_min, and DSpark uses it as the cutoff of its confidence head, so the warning is skipped when only these are configured. --------- Co-authored-by: Pranesh Gonegandla <pgonegandla@nvidia.com>