metal: apple ane decoder #3848
chenqianhe
wants to merge
15 commits into
ggml-org:master
from
chenqianhe:apple-ane-decode
coreml : add experimental decoder path
eb4af57f
coreml : add decoder prefill and trace experiments
6a0972a3
coreml : prototype decoder cpu prefill
a8460c87
coreml : add step-level decoder trace and timing output
81e83115
coreml : add decoder prefill disable toggle
af3d5b96
coreml : add decoder compute-unit and state reuse experiments
a60e4c51
coreml : wire no-write decoder shards
df02af87
coreml : add opt-in 4-shard ANE greedy decode path and use explicit F…
5ec73b0d
coreml : add decoder build toggle and no-write shards
3f52325b
coreml : implement coreml_decoder_shard_plan function for dynamic lay…
0ff10e1d
coreml : support Core ML decoder multi-candidate state
6bfe2b78
coreml : document Core ML decoder capabilities
bff0e118
coreml : split Core ML encoder/decoder build and runtime toggles
408c537f
coreml : clean up model generation script indentation
2b480e5e
coreml : select first-shard decoder layer count
95af5ca3
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub