[Core] [Bugfix] Add Input Embeddings #15428
vllm-bot
merged
80 commits into
vllm-project:main
from
qthequartermasterman:feature/vllm/add-input-embedding
qthequartermasterman
commented
on 2025-03-25
DarkLight1337
commented
on 2025-03-26
lzl-mt
commented
on 2025-03-27
(vllm) add input embedding
cef68948
improve embedding input
c51d8fb5
(vllm) fix import error
9564b402
(vllm) fix pre commit error
c60298a4
apply ruff and isort fixes
0c24a823
apply ruff and isort fixes
403a1655
styling
b1ac0721
fix missing imports from rebase
0390c334
typing fixes
0ca4daec
type fix
35320fe4
type fix
0a77630c
remove unnecessary changes
11b6c02a
remove unnecessary changes
cb92a3dd
re-add deleted whitespace
375bd5b4
Include unit tests from #6869.
c9d80248
remove unrelated qwen2 changes
a64e6274
guard clause around fully consumed prompt embeds to avoid returning e…
6ab349eb
use v0 for prompt embeds model runner tests
26c87840
fix batching of input embeddings
b71a13c2
style formatting
4aa9ade4
remove incorrect overload
e2c4c26d
remove incorrect overload
26d108ac
Update representations
af204355
remove unrelated changes to docs
25aaf3fe
remove unrelated typing change
bc058601
fix missing syntax
b55800d4
do not schedule prompt embeds and non-prompt embeds in the same batch
be42a17e
fix style linelength
c8fcfe41
Merge branch 'main' into feature/vllm/add-input-embedding
b21688fb
SnowCharmQ
commented
on 2025-04-08
Aaronhuang-778
commented
on 2025-04-09
propogate embeddings for sampled output tokens for decoding
1e359ae5
fix type check
59fbe702
do not schedule decode sequence groups with batches containing both p…
c152a3ac
Merge branch 'main' into feature/vllm/add-input-embedding
42ad8007
fix type check
e7ab2a22
add default value to optional parameter
911adbe1
remove unused comments
82d923d6
properly pass in placeholder token ids when testing prompt embeds
c9514792
do not test mixed token_ids/prompt_embeds batches in the model_runner
01e1a6eb
refactor cuda_prepare_decode test
193ad5cb
use correct expected input embeds length for prepare_decode_cuda_grap…
74bd9f46
add scheduler test to ensure prompt embeds and prompt tokens are not …
d949f1b0
support inputs_embeds in compiled mode
62bbc881
fix typing in test
1d1ae4ba
use corrector operator precedence for handling empty strings
1914676b
only test decoder models with input embeds in v0 backend
70198f62
Merge branch 'vllm-project:main' into feature/vllm/add-input-embedding
934ceae6
DarkLight1337
commented
on 2025-04-17
DarkLight1337
commented
on 2025-04-17
DarkLight1337
commented
on 2025-04-17
DarkLight1337
commented
on 2025-04-17
DarkLight1337
commented
on 2025-04-17
DarkLight1337
commented
on 2025-04-17
DarkLight1337
commented
on 2025-04-17
DarkLight1337
commented
on 2025-04-17
DarkLight1337
commented
on 2025-04-17
adjust type hints for modelinputforgpubuilder.build
5595b452
simplify conditional logic
3343d3e0
simplify compilation conditional logic
5010ea02
refactor decoder only language model tests to reduce number of times …
2075e538
break up multiple assignments for readability
9a4fb3c0
update type hints in scheduler
8ad40915
clear existing lists instead of instantiating new ones
9055daf7
preprocess tensors to handle batched/misshaped prompt embeds to avoid…
9a57acaf
use seperate Embedsprompt class for preprocessing inputs embeddings
bbfb0f0e
fix typing
933e5672
fix type errors
4e0d12f6
Merge branch 'vllm-project:main' into feature/vllm/add-input-embedding
164aeb5e
fix mistaken type change
9e6909e6
add missing type hint
90b950ad
add spaces for style
01d83f4d
seperate EmbedsInputs from TokenInputs and embeds_inputs from token_i…
69854521
fix docstrings for EmbedsInputs
e916551f
fix typing for token_type_ids
69f87250
fix typing for embeds_tokens in InputRegistry and InputsAdapter
9c2c89fb
remove prompts and prompt_token_ids from EmbedsPrompts
499dc6a7
Merge branch 'main' into feature/vllm/add-input-embedding
20668ca8
fight mypy to get correct typing for not embeds prompts
6712ba67
remove incorrect call to embeds_inputs
740b290b
wrestle with mypy and typeddict type narrowing
8f9bd513
wrestle with mypy and typeddict type narrowing
b8d36c69
DarkLight1337
commented
on 2025-04-29
DarkLight1337
approved these changes
on 2025-04-29
support indexing graph runners that with inputs_embeds
b764c193
disabled auto-merge 299 days ago
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
Merge branch 'main' into feature/vllm/add-input-embedding
cb6ff22f
support encoder decoder models with inputs_embeds
85642d0c
simplify redundant ternary statement
b226fd6b
explicitly remove support for inputs embeds with speculative decoding…
b738d3fd
fix occasional device mismatch errors when appending output tokens to…
2340119d
Merge branch 'main' into feature/vllm/add-input-embedding
92b3264c
Merge branch 'main' into feature/vllm/add-input-embedding
b9271c15
Fix a typo
28b0983b
gshtras
commented
on 2025-05-06
Reviewers
Assignees
Labels
frontend
speculative-decoding
ready
Milestone
No milestone
Login to write a write a comment.
Login via GitHub