openvino
[NPUW] Share kvcache between prefill and generate when chunking is enabled
#32642

Merged

Commits

Introduce lazy memory allocation for ireq's I/O

smirnov-alexey committed 314 days ago
Fix no tensor being present in the storage

smirnov-alexey committed 314 days ago
Merge branch 'master' of https://github.com/openvinotoolkit/openvino into as/npuw_lazy_io_alloc

smirnov-alexey committed 307 days ago
Address review comments

smirnov-alexey committed 306 days ago
Merge branch 'master' of https://github.com/openvinotoolkit/openvino into as/npuw_lazy_io_alloc

smirnov-alexey committed 303 days ago
Merge branch 'master' of https://github.com/openvinotoolkit/openvino into as/npuw_share_kvcache

smirnov-alexey committed 285 days ago
Copy Xiong's changes

smirnov-alexey committed 285 days ago
Remove copy

smirnov-alexey committed 282 days ago
WIP

smirnov-alexey committed 281 days ago
Fix concurrency issue with iterator invalidation

smirnov-alexey committed 280 days ago
Merge branch 'master' of https://github.com/openvinotoolkit/openvino into as/npuw_lazy_io_alloc

smirnov-alexey committed 280 days ago
Refactoring

smirnov-alexey committed 280 days ago
Fix merge

smirnov-alexey committed 280 days ago
Protect get_tensor by mutex

smirnov-alexey committed 280 days ago
Merge branch 'as/npuw_lazy_io_alloc' of https://github.com/smirnov-alexey/openvino into as/npuw_share_kvcache

smirnov-alexey committed 280 days ago
Disable kv cache sharing when one of the models is transposed

smirnov-alexey committed 280 days ago
Fix strides

smirnov-alexey committed 278 days ago
Handle strided tensors - copy on host

smirnov-alexey committed 275 days ago
Merge branch 'master' of https://github.com/openvinotoolkit/openvino into as/npuw_share_kvcache

smirnov-alexey committed 275 days ago
Handle strided tensors in pyramid attention

smirnov-alexey committed 275 days ago
Address review comments

smirnov-alexey committed 274 days ago
Merge branch 'as/npuw_lazy_io_alloc' of https://github.com/smirnov-alexey/openvino into as/npuw_share_kvcache

smirnov-alexey committed 274 days ago
Address review comments

smirnov-alexey committed 274 days ago
Merge branch 'master' into as/npuw_lazy_io_alloc

smirnov-alexey committed 274 days ago
Merge branch 'master' into as/npuw_share_kvcache

smirnov-alexey committed 274 days ago
Fix shape

smirnov-alexey committed 274 days ago
Merge branch 'master' of https://github.com/openvinotoolkit/openvino into as/npuw_lazy_io_alloc

smirnov-alexey committed 261 days ago
Remove is_io()

smirnov-alexey committed 261 days ago
Merge branch 'as/npuw_lazy_io_alloc' of https://github.com/smirnov-alexey/openvino into as/npuw_share_kvcache

smirnov-alexey committed 261 days ago
Move kv cache copy to second token time

smirnov-alexey committed 261 days ago
Fix empty tensors for pyramid attention

smirnov-alexey committed 261 days ago
Merge branch 'as/npuw_share_kvcache' of https://github.com/smirnov-alexey/openvino into as/npuw_share_kvcache

smirnov-alexey committed 261 days ago
Merge branch 'master' into as/npuw_share_kvcache

smirnov-alexey committed 258 days ago
Align code with latest pyramid changes merged

smirnov-alexey committed 258 days ago

Loading