onnxruntime
Add microbenchmark for layer normalization and improve latency
#22223
Merged
Merged
Add microbenchmark for layer normalization and improve latency #22223
amarin16
merged
36 commits into
microsoft:main
from
amarin16:dev/amarin16/layer_norm_microbench
Add microbenchmark for layer normalization
2b8cd170
github-advanced-security
commented
on 2024-09-25
fix warnings
0c89631e
initialize test input data at compile time
bca13ca0
remove unused specialization that fails on pipeline
680cf4fc
fix build on linux
f0df5263
convert all inputs to float efficiently if needed
87725c37
convert output buffer efficiently in layer_norm_impl
8aa80daa
convert output buffer efficiently in skip_layer_norm
295d6527
amarin16
changed the title
github-advanced-security
commented
on 2024-09-30
add inline and fix some lint issues
405a0a0c
fix some lint errors
245f298e
fix warning
f398b643
maybe_unused
a483ca48
Fix bug
19d225a0
separate MLFloat16 implementation in skip_layer_norm
05b5037b
fix linter issues
ab2e5f2e
fix precision warning
63e9644c
cast
11eb7fbc
separate implementation for MLFloat16 inside layer_norm_impl
46775a76
don't use vectors
fd904f69
reuse allocated arrays when possible
a41b802b
make_unique instead of new
6aece952
Revert "make_unique instead of new" for latency
766c4b23
lint
cb55d4bd
fix bug
2895f37f
fix bug
f93ccb7c
github-advanced-security
commented
on 2024-10-02
handle errors
4be02551
remove checks on tensor data
48ce9790
remove try/catch due to -fno-exceptions
3d6b990e
Prepack scale and bias in layer_norm_impl
f04aac04
Prepack skip, gamma, beta, bias in skip_layer_norm
1eaa63ff
return void from ComputeJob
26ddc6c1
lint
3231cffe
fajin-corp
commented
on 2024-10-11
fajin-corp
commented
on 2024-10-11
fajin-corp
commented
on 2024-10-11
fajin-corp
commented
on 2024-10-11
fajin-corp
commented
on 2024-10-11
Use GenerateArrayWithRandomValue in microbenchmark
2a37a92f
Use allocator instead of new
d8b11abb
lint
402b65d6
github-advanced-security
commented
on 2024-10-14
fajin-corp
commented
on 2024-10-14
fajin-corp
commented
on 2024-10-14
switch to IAllocator::MakeUniquePtr
57c3e637
fajin-corp
approved these changes
on 2024-10-14
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub