PR #1727 Improve the defaults for the launcher

Easier defaults for models stemmed from configs.

Narsil committed 2 years ago

Better defaults (and LOG_COLORIZE).

Narsil committed 2 years ago

Update default doc.

Narsil committed 2 years ago

No unwrap.

Narsil committed 2 years ago

Making things work most of the time.

Narsil committed 2 years ago

Change things around when we don't have a tokenizer.

Narsil committed 2 years ago

Remove the override ?

Narsil committed 2 years ago

Adding some wiggle room.

Narsil committed 2 years ago

Fixing default for BNB + cuda graphs (they don't work together).

Narsil committed 2 years ago

Max_seq_len (old mpt config.)

Narsil committed 2 years ago

"Fixing t5" just use more RAM for this test.

Narsil committed 2 years ago

Smaller default for max_input_length.

Narsil committed 2 years ago

Forgot the doc again.

Narsil committed 2 years ago

Update launcher/src/main.rs

Narsil committed 2 years ago

Update the doc.

Narsil committed 2 years ago

text-generation-inference Improve the defaults for the launcher #1727 Merged