llama.cpp
458fcfc3 - docs: CANN backend supports BF16 on 910B, document GGML_CANN_NO_PINNED (#31249)

Commit
2 days ago
docs: CANN backend supports BF16 on 910B, document GGML_CANN_NO_PINNED (#31249) The note under 'Run the inference' still said that the CANN backend only supports FP16/Q4_0/Q8_0 models, while the DataType table in the same file, and ggml-cann.cpp (BF16 is enabled for MUL_MAT, GET_ROWS, CPY and CONT for every non-310P device), say otherwise. Spell out the per-device support and mark BF16 as unsupported on 310P instead of leaving the cell empty. GGML_CANN_NO_PINNED is read by ggml-cann.cpp when allocating host buffers and when advertising the host_buffer device capability, but it was not documented, so the escape hatch for a failing pinned host allocation was not discoverable. Signed-off-by: li-lizhe <147392333@qq.com>
Author
Parents
Loading