Numa #1556
ggerganov
merged
20 commits into
ggml-org:master
from
zrm:numa
detect NUMA systems and pin work threads to nodes (linux)
6fc5f17e
disable mmap prefetch/readahead for NUMA systems
0d23f8ce
avoid sending finalize op to thread pool if it does nothing
9d058c20
silence robot
2c1b5ae1
fix args
8502d517
github-actions
commented
on 2023-05-23
make --numa a param
bf83dcb2
recommendation that n_nodes evenly divide n_threads did not warrant s…
b71dfe63
lower synchronization overhead
adaad10e
statically allocate
c31d51d4
move numa state to g_state
2f5bb462
add description for --numa
4b945821
Merge branch 'master' into HEAD
90a0e65c
ggml : minor style changes
d0e35963
ggml : minor style + try fix sanitizer build
67ba34e8
Merge branch 'master' into HEAD
8f98035e
llama : allow to initialize backend with NUMA support
0fe4b00d
llama : avoid ggml include in llama-util.h
875a1e11
ggml : style / formatting
4a555b45
ggerganov
commented
on 2023-06-26
ggml : fix handling of ops with n_threads > n_tasks > 1
81a40e9d
ggerganov
approved these changes
on 2023-06-26
server : utilize numa parameter
9aec2b74
Assignees
No one assigned
Labels
performance
high priority
threading
Milestone
No milestone
Login to write a write a comment.
Login via GitHub