Threadpool: take 2 #8672
slaren
merged
48 commits into
ggml-org:master
from
CodeLinaro:threadpool
slaren
commented
on 2024-08-26
Introduce ggml_compute_threadpool
130adf84
Minor fixes
a0aae528
fixed use after release bug
d5c9c14d
fixed a harmless race condition
82224f84
Fix Android bulid issue
817eaf0c
fix more race conditions
57637326
fix deadlock for cases where cgraph.n_nodes == 1
3008b31b
threadpool: use cpu_get_num_math to set the default number of threadp…
96d6603d
bench: create fresh threadpool for each test
29534415
atomics: always use stdatomics with clang and use relaxed memory orde…
6fcc780b
threadpool: make polling the default to match openmp behavior
3b62f7c1
threadpool: do not wakeup threads in already paused threadpool
dfa63778
fix potential race condition in check_for_work
2e18f0d4
threadpool: do not create two threadpools if their params are identical
48aa8eec
threadpool: reduce pause/resume/wakeup overhead in common cases
494e27c7
threadpool: add support for hybrid polling
b630acdb
threadpool: reduce the number of barrier required
9d3e78c6
threadpool: remove special-casing for disposable threadpools
538bd9f7
threadpool: do not clear barrier counters between graphs computes (fi…
db45b6d3
threadpool: use relaxed order for chunk sync
307fece5
threadpool: remove abort_callback from threadpool state
63a0dad8
threadpool: better naming for thread/cpumask releated functions
2358bb36
threadpool: consistent use of int type for n_threads params
4a4d7150
threadpool: add support for ggml_threadpool_params_default/init
c4452edf
threadpool: move typedef into ggml.h
31541d74
threadpool: fix apply_priority() function name
40648601
threadpool: fix swift wrapper errors due to n_threads int type cleanup
f64c9751
threadpool: enable --cpu-mask and other threadpool related options on…
c506d7fc
threadpool: replace checks for compute_thread ret code with proper st…
8008463a
threadpool: simplify threadpool init logic and fix main thread affini…
49ac51f2
threadpool: update threadpool resume/pause function names
204377a0
threadpool: enable openmp by default for now
93f170d8
threadpool: don't forget to free workers state when omp is enabled
a7496bf7
threadpool: avoid updating process priority on the platforms that do …
8186e961
threadpool: update calling thread prio and affinity only at start/resume
658f16c3
llama-bench: turn threadpool params into vectors, add output headers,…
8d5ab9a5
llama-bench: add support for cool off between tests --delay
3bcc4dee
slaren
commented
on 2024-08-27
threadpool: move process priority setting into the apps (bench and cli)
5d4c0a13
threadpool: move all pause/resume logic into ggml
e3c22020
threadpool: futher api cleanup and prep for future refactoring
c6328bc0
threadpool: minor indent fixes
bead7d47
threadpool: improve setprioty error message
8e8f8ce4
Update examples/llama-bench/llama-bench.cpp
c6c27b14
threadpool: fix indent in set_threadpool call
b97bd67e
use int32_t for n_thread type in public llama.cpp API
cae35b9f
threadpool: use _new and _free instead of _create and _release
c49d6340
fix two more public APIs to use int32_t for n_threads
3b5f7c2a
build: set _GNU_SOURCE for Adroid
52aa6772
Reviewers
Assignees
No one assigned
Labels
testing
examples
Review Complexity : Medium
server
ggml
Milestone
No milestone
Login to write a write a comment.
Login via GitHub