PR #28392 Add FP8 KV-cache support

Add FP8 KV-cache support #28392

ORippler wants to merge 3 commits into osimons/ocp_fp8 from osimons/ocp_fp8_kv

github-actions added testing

github-actions added examples

github-actions added conversion

ORippler force-pushed the osimons/ocp_fp8 branch from d21c273f to cef6c528 35 days ago

ORippler force pushed from 460bbd81 to b6ccff9b 35 days ago

ORippler force pushed from b6ccff9b to e8b60dda 10 days ago

ORippler force-pushed the osimons/ocp_fp8 branch from cef6c528 to 060faf68 10 days ago

ORippler force pushed from 55a7e532 to b99e4ce0 9 days ago

ORippler force-pushed the osimons/ocp_fp8 branch from 060faf68 to 340a287d 9 days ago

ORippler force pushed from b99e4ce0 to f711e8c8 9 days ago

ORippler force-pushed the osimons/ocp_fp8 branch from 340a287d to 60dcb35a 9 days ago

ORippler force-pushed the osimons/ocp_fp8 branch from 60dcb35a to 91fe47f3 9 days ago

ORippler force pushed from f711e8c8 to d6c8682f 9 days ago

ORippler force-pushed the osimons/ocp_fp8 branch from 91fe47f3 to d904f345 8 days ago

ORippler force pushed from d6c8682f to 51f6a0f2 8 days ago

ORippler force pushed from 51f6a0f2 to b5f3dfd6 8 days ago

ORippler force-pushed the osimons/ocp_fp8 branch from d904f345 to b44ab5c3 8 days ago

ORippler force pushed from c8023359 to 4602e642 8 days ago

ORippler force-pushed the osimons/ocp_fp8 branch from 479c902f to 574eb984 8 days ago

Add FP8 KV-cache support

29995854

Add tests for scaled, FP8 KV-cache to test-backend-ops

287791c2

Fix get_*_scale mapping and dequnatize FP8 also before rope shift

10e80e62

ORippler force-pushed the osimons/ocp_fp8 branch from 574eb984 to 8efdff03 11 hours ago

ORippler force pushed from 4602e642 to 10e80e62 11 hours ago

Reviewers

No reviews

Assignees

No one assigned

Labels

testing examples conversion

Milestone

No milestone