llama.cpp
ggml : support broadcast for ggml_soft_max_ext and ggml_flash_attn_ext
#14435

Merged

ggml : support broadcast for ggml_soft_max_ext and ggml_flash_attn_ext #14435

ggerganov merged 4 commits into master from gg/ggml-batch-soft-max-ops

github-actions added testing

github-actions added ggml

github-actions added Apple Metal

ggerganov force pushed 1 year ago

github-actions added Nvidia GPU

github-actions added Vulkan

github-actions added Ascend NPU

ggerganov force pushed 1 year ago

ggerganov force pushed to bdfd7b75 1 year ago

ggerganov marked this pull request as ready for review 1 year ago

github-actions added SYCL

ggerganov force pushed from bdfd7b75 to 461cb2f3 1 year ago

JohannesGaessler requested a review from

JohannesGaessler 1 year ago

ggml : support bcast ggml_soft_max_ext, ggml_flash_attn_ext (#14435)

32366701

vulkan: support softmax/FA batch and broadcast (#14449)

b7265648

CUDA: broadcasting for FlashAttention mask (#14500)

3045a1eb

CUDA: add softmax broadcast (#14475)

3b38afdf

ggerganov force pushed to 3b38afdf 1 year ago

ggerganov merged 55a1c5a5 into master 1 year ago

ggerganov deleted the gg/ggml-batch-soft-max-ops branch 1 year ago

Reviewers

JohannesGaessler

Assignees

No one assigned

Labels

testing Nvidia GPU Vulkan ggml SYCL Apple Metal Ascend NPU

Milestone

No milestone