ggerganov/llama.cpp

Pull Requests Commits

recurrent : rework graph inputs + add TODOs

ggerganov committed 1 year ago

Verified 59fee24c

refactor: Use a common build_recurrent_state method that is cache-agnostic

gabe-l-hart committed 1 year ago

faf41199

fix: Fix initialization of child states

gabe-l-hart committed 1 year ago

5046d412

fix: Fix resize vs reserve and skip null tensors in size computation

gabe-l-hart committed 1 year ago

9db44a2a

feat: Overhaul build_recurrent_state / build_inp_s_copy to match attention pattern

gabe-l-hart committed 1 year ago

11cd80d5

refactor: Use llama_memory_state_ptr for child states in hybrid memory state

gabe-l-hart committed 1 year ago

4ec4e6a8

fix: Remove llama_model_is_hybrid_Recurrent public API

gabe-l-hart committed 1 year ago

7ba463b3

fix: Remove logits_all after rebase

gabe-l-hart committed 1 year ago

1510016e

feat: Allow custom layer filters for hybrid recurrent

gabe-l-hart committed 1 year ago

d8c929ff

refactor: Remove n_embd_k/v_gqa from recurrent cache

gabe-l-hart committed 1 year ago

d5d7628b

refactor: Remove layer index from n_embd_k/v_s

gabe-l-hart committed 1 year ago

b42c8b43

refactor: Remove n_embd_k/v_s from unified cache

gabe-l-hart committed 1 year ago

1dd12133

fix: Use per-layer n_embd_k/v_s calls for mamba (1) layers

gabe-l-hart committed 1 year ago

833dfb54

fix: Remove errant virtual destructor leftover from previous impl attempt

gabe-l-hart committed 1 year ago

f6d5f055

fix: Update clear signature for data argument after rebase

gabe-l-hart committed 1 year ago

9c1a604a

fix: Add missing padding to n_ctx for hybrid cache construction

gabe-l-hart committed 1 year ago

de9297fd

fix: Fix status for init_update sig for recurrent cache state

gabe-l-hart committed 1 year ago

911e6944

fix: Update recurrent cache for changes to remove intermediate kv_cache interface

gabe-l-hart committed 1 year ago

d3699366

fix: Fix logic for initializing inputs and attn layers for hybrid caches

gabe-l-hart committed 1 year ago

a9b5fe98

feat: Support hybrid recurrent in llama-graph

gabe-l-hart committed 1 year ago

e3c16315

fix: Fix shift logic to defer to unified cache

gabe-l-hart committed 1 year ago

cf03d4ae

fix: Fix wrong bool condition for split equal in hybrid cache

gabe-l-hart committed 1 year ago

6c6ec000

feat: Construct hybrid recurrent cache for hybrid recurrent models

gabe-l-hart committed 1 year ago

423c8940

feat: First pass at llama_kv_cache_hybrid_recurrent

gabe-l-hart committed 1 year ago

c71eaa37

fix: Use per-layer sizing everywhere in kv caches

gabe-l-hart committed 1 year ago

13332a75

feat: Add layer filter to recurrent cache

gabe-l-hart committed 1 year ago

40e91878

refactor: rename *_is_hybrid -> *_is_hybrid_recurrent

gabe-l-hart committed 1 year ago

fb26e95a

feat: Auto-fill hparams.recurrent_layer_arr based on whether the model is recurrent

gabe-l-hart committed 1 year ago

fc9e0b57

feat: Add support for distinguishing recurrent vs non-recurrent layers in hparams

gabe-l-hart committed 1 year ago

05f19580

feat: Add c++ side constants for attention layer indices hparam

gabe-l-hart committed 1 year ago

5e2f2c38

Older