text-generation-inference
Choosing input/total tokens automatically based on available VRAM?
#2673
Merged
Overview
Commits
13
Changes
View On GitHub

Choosing input/total tokens automatically based on available VRAM? #2673

Narsil merged 13 commits into main from auto_length
Narsil
drbh
drbh commented on 2024-10-21
HuggingFaceDocBuilderDev
Narsil Choosing input/total tokens automatically based on available VRAM?
a1aac784
Narsil Update doc.
79469f5f
Narsil Narsil force pushed from b2272ab7 to 79469f5f 1 year ago
Narsil Remove generated files.
a31db047
Narsil Trying to fix non chunking targets.
0a01dde9
Narsil Attempt #2
5c3efbc7
Narsil fix.
82a6cb82
Narsil QuantLinear is rocm compatible.
849d8821
Narsil Much simpler logic after the overhead.
10534511
Narsil Updating logic + non flash.
6994fa12
Narsil Revert doc text.
cacaba64
Narsil Simple updates.
199973cc
Narsil Fix integration mt0 (transformers update).
e3db5259
drbh
drbh dismissed these changes on 2024-10-25
OlivierDehaene OlivierDehaene requested a review from OlivierDehaene OlivierDehaene 1 year ago
Narsil Merge branch 'main' into auto_length
c3fb2ecd
Narsil Narsil dismissed their stale review via c3fb2ecd 1 year ago
Narsil Narsil merged 0c9b6cdd into main 1 year ago
Narsil Narsil deleted the auto_length branch 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum