turboderp/exllama

Pull Requests Commits

Merge remote-tracking branch 'origin/master'

turboderp committed 2 years ago

3b013cd5

ExLlamaV2 announcement

turboderp committed 2 years ago

0558deaf

multi stoptoken (#283)

Kerushii committed 2 years ago

Verified 9cf6384d

Switch use of SMEM in kernel based on ROCm and CUDA version

turboderp committed 2 years ago

8a1d3305

Fix display name for sharded model file

turboderp committed 2 years ago

85b009c4

Merge branch 'master' into master2

turboderp committed 2 years ago

766e5823

Support for sharded models

turboderp committed 2 years ago

2a2ef8fb

Merge remote-tracking branch 'quarticcat/master' into master2

turboderp committed 2 years ago

b12acc60

Optimize q4_matmul: optimize non-128 group sizes

QuarticCat committed 2 years ago

Verified 8e37f2d8

Optimize q4_matmul: optimize non-128 group sizes

QuarticCat committed 2 years ago

Verified 93dbb266

Optimize q4_matmul: add missing assignment back

QuarticCat committed 2 years ago

Verified 215a7158

Optimize q4_matmul: fix dot_product_8 count

QuarticCat committed 2 years ago

Verified caf4de8d

Optimize q4_matmul: fix __syncthreads

QuarticCat committed 2 years ago

Verified d5df6a8f

Optimize q4_matmul: cache vector in shmem

QuarticCat committed 2 years ago

Verified f828764a

Command line argument to override rope_theta from config.json

turboderp committed 2 years ago

a5440858

Update re: CodeLlama rope_theta

turboderp committed 2 years ago

b7f3fac7

Merge remote-tracking branch 'origin/master'

turboderp committed 2 years ago

b8541ad0

Read rope_theta from config.json

turboderp committed 2 years ago

6020a1b3

Optimize and extend ws example for chatborts (#265)

Kerushii committed 2 years ago

Verified f8e9d7e3

Fix: Allow cache.max_seq_len < model.config.max_seq_len when fused_attn = False

turboderp committed 2 years ago

69216d9d

Fix: Allow rep penalty length = -1 to work as intended

turboderp committed 2 years ago

21f4a12b

refine json dicts for ws example (#255)

Kerushii committed 2 years ago

Verified 74e0ddd6

Fix: Always print the last output chunk

turboderp committed 2 years ago

9d4712b0

Alternative generator with streaming and flexible stop conditions, plus example for using it

turboderp committed 2 years ago

3cada7b6

Alternative generator with streaming and flexible stop conditions, plus example for using it

turboderp committed 2 years ago

4afd9c9f

recover unsaved modification (#250)

Kerushii committed 2 years ago

Verified 0610154b

Generation uses config.max_seq_len instead of default 2048 (#236)

flotos committed 2 years ago

Verified 82369c52

Merge remote-tracking branch 'origin/master'

turboderp committed 2 years ago

e9da6205

Option to set processor affinity

turboderp committed 2 years ago

0dacadb5

Fix HIP on recent PyTorch version (#224)

ardfork committed 2 years ago

Verified 3dff8fee

Older