PR #1727 Improve the defaults for the launcher

Improve the defaults for the launcher #1727

Narsil merged 15 commits into main from improve_defaults

dacorvo commented on 2024-04-11

drbh commented on 2024-04-11

Easier defaults for models stemmed from configs.

3c71d2f1

Better defaults (and LOG_COLORIZE).

bd01d448

Update default doc.

a4c86e86

No unwrap.

9ce9f39d

Making things work most of the time.

d43e10e0

Change things around when we don't have a tokenizer.

179ee4e2

Remove the override ?

9176ecbc

Adding some wiggle room.

289b0721

Narsil force pushed from 118516ad to 289b0721 2 years ago

Fixing default for BNB + cuda graphs (they don't work together).

c4ebcea7

Max_seq_len (old mpt config.)

cd072114

"Fixing t5" just use more RAM for this test.

1e5150f4

Smaller default for max_input_length.

e5955851

Forgot the doc again.

16386b83

dacorvo commented on 2024-04-12

Narsil commented on 2024-04-12

Update launcher/src/main.rs

b75bd5b7

Update the doc.

f66c9f34

Narsil merged 1b2670c8 into main 2 years ago

Narsil deleted the improve_defaults branch 2 years ago

Reviewers

dacorvo

drbh

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

text-generation-inference Improve the defaults for the launcher #1727 Merged

Improve the defaults for the launcher #1727

text-generation-inference
Improve the defaults for the launcher
#1727

Merged