text-generation-inference
Attempt for cleverer auto batch_prefill values (some simplifications).
#2808

Merged

Commits

Attempt for cleverer auto batch_prefill values (some simplifications).

Narsil committed 1 year ago
Less flaky tests.

Narsil committed 1 year ago
Fixing typo insertion.

Narsil committed 1 year ago
Update launcher/src/main.rs

Narsil committed 1 year ago
Adding small comment for source of calculation.

Narsil committed 1 year ago
Adding L40.

Narsil committed 1 year ago
Adding L40s.

Narsil committed 1 year ago