llama.cpp
2c192444 - fit : account for skipped NextN layers when setting n_gpu_layers (#31569)

Commit
1 day ago
fit : account for skipped NextN layers when setting n_gpu_layers (#31569) * fit : account for skipped NextN layers when setting n_gpu_layers Without MTP, fit counts n_layer + 1 layers while the model loader counts n_layer_all + 1 for n_gpu_layers, including the NextN layers which hold no weights. A fit result that offloads all layers then leaves layer 0 on the CPU ("offloaded 65/66 layers" for Qwen3.8-27B on two GPUs). Add the skipped NextN layers to n_gpu_layers and to the tensor split of the device that holds the output layer so the loader assigns the layers that fit measured. Fixes #31531 * fit : pass hp_ngl_skip by reference like the other hparams * fit : add hp_ngl_skip to common_get_device_memory_data
Author
Parents
Loading