PR #3186 feat: support logit bias in chat request

feat: support logit bias in chat request #3186

drbh wants to merge 14 commits into main from support-logit-bias-in-chat

Narsil commented on 2025-04-24

drbh requested a review from

Narsil 233 days ago

drbh force pushed from 88010bac to 2b996b0a 232 days ago

Narsil commented on 2025-05-02

feat: support logit bias in chat request

fae510b8

feat: include proto changes

da3f18e5

fix: adjust the NextTokenChooser logit bias processor

e44703d5

fix: include logit_bias in all ValidGenerateRequest's

61a50a81

fix: adjust imports

81656bd0

fix: remove deprecated test and fix typing

bb5c875f

fix: improve processor logic and refactor

9eeccbf9

fix: cleanup typos

b3ead6e9

fix: avoid zero'd logit bias mask

465294d3

fix: improve validation and transform logic

7659925d

drbh force pushed from a174f631 to 7659925d 227 days ago

fix: remove the bias padding

55d82d46

fix: read vocab size from tokenizer and add hacky patch for qwen2b

b32cd97b

fix: prefer patch to be vlm specific

783ca669

fix: linter

551ee3a3

Reviewers

Narsil

Assignees

No one assigned

Labels

None yet

Milestone

No milestone