PR #431 Reduce memory usage during Whisper inference

Reduce memory usage during Whisper inference #431

ggerganov merged 15 commits into master from mem

ggerganov force pushed from dadead51 to 34b8afe6 2 years ago

ggerganov force pushed from 50692634 to f1b9913b 2 years ago

ggerganov force pushed from 1f7cd041 to 60d0f9da 2 years ago

ggerganov marked this pull request as ready for review 2 years ago

ggerganov force pushed from b0c2268f to 41f31719 2 years ago

ggml : add "scratch" buffer support

60eff46b

ggml : support for scratch ring-buffer

0eea547a

ggml : bug fix in ggml_repeat()

18210579

ggml : error on scratch buffer overflow

0ba91b54

whisper : use scratch buffers during inference (base model only)

1a1dee46

whisper : update memory usage for all models

6cae05bd

whisper : fix encoder memory usage

79148a21

whisper : use whisper_context functions instead of macros

42d7dee4

whisper : fix FF + remove it from README

4e0e2520

ggml : reuse ggml_new_i32

d922aa4a

ggml : refactor the scratch buffer storage

62205aed

whisper : reorder scratch buffers in the decoder

01669ee8

main : add option to disable temp fallback

bdf21fa6

Update README.md

6ed13449

ggerganov force pushed from 41f31719 to 6ed13449 2 years ago

js : update whisper.js

a26f3a71

ggerganov merged f3ee4a96 into master 2 years ago

ggerganov deleted the mem branch 2 years ago

FELIXrobust approved these changes on 2024-07-01

Reviewers

FELIXrobust

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

whisper.cpp Reduce memory usage during Whisper inference #431 Merged

Reduce memory usage during Whisper inference #431

whisper.cpp
Reduce memory usage during Whisper inference
#431

Merged