auto-round
1.8X speedup by disable_low_gpu_mem_usage and reduce memory usage by avoid using torch.cat
#106
Merged
Overview
Commits
47
Changes
View On GitHub

1.8X speedup by disable_low_gpu_mem_usage and reduce memory usage by avoid using torch.cat #106

wenhuach21 merged 47 commits into main from fix_performance
wenhuach21
wenhuach21 fix a bug in example
7d020db2
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
fbe69d5c
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
596a18f5
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
10add8ce
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
003b60a6
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
9d495140
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
3b7f386c
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
d3f14df6
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
76e4d909
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
08e46acb
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
15b756b0
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
40425bbd
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
f0b9ad00
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
04e70eca
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
43811bb9
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
c8817199
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
54920e56
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
e2c2f56c
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
4f718d45
wenhuach21 Merge branch 'main' of https://github.com/intel/auto-round
022988aa
wenhuach21 fix performance regression
446f52f8
wenhuach21 not using torch cat to avoid torch tricky gpu memory issue
d1c00771
pre-commit-ci[bot] [pre-commit.ci] auto fixes from pre-commit.com hooks
21508d33
wenhuach21 fix one issue
52df5141
wenhuach21 Merge branch 'fix_performance' of https://github.com/intel/auto-round…
9ff10837
wenhuach21 remove the torch.cuda.empty_cache in each iters, which improve the sp…
030c9264
pre-commit-ci[bot] [pre-commit.ci] auto fixes from pre-commit.com hooks
7a23be47
wenhuach21 wenhuach21 changed the title 2X speedup with disable_low_gpu_mem by fix performance regression 2X+ speedup by fix performance regression and avoid using torch.cat to reduce weird and large memory footprint 2 years ago
wenhuach21 fix some issue
945f3e6e
wenhuach21 torch.no_grad -> torch.inference_mode
2d935d43
wenhuach21 torch.no_grad -> torch.inference_mode
83261bdf
pre-commit-ci[bot] [pre-commit.ci] auto fixes from pre-commit.com hooks
205435d1
wenhuach21 tiny change
512263ce
wenhuach21 polish code and fix some issues
72305a83
wenhuach21 Merge branch 'fix_performance' of https://github.com/intel/auto-round…
9952271a
pre-commit-ci[bot] [pre-commit.ci] auto fixes from pre-commit.com hooks
06026ca2
wenhuach21 wenhuach21 changed the title 2X+ speedup by fix performance regression and avoid using torch.cat to reduce weird and large memory footprint speedup by disable_low_gpu_mem_usage and reduce memory usage by avoid using torch.cat 2 years ago
wenhuach21 wenhuach21 requested a review from WeiweiZhang1 WeiweiZhang1 2 years ago
wenhuach21 wenhuach21 requested a review from yintong-lu yintong-lu 2 years ago
wenhuach21 add trust_remote_code
a0236393
wenhuach21 Merge branch 'fix_performance' of https://github.com/intel/auto-round…
b77c3aeb
wenhuach21 fix chatglm3 bug
905085b8
pre-commit-ci[bot] [pre-commit.ci] auto fixes from pre-commit.com hooks
a905d9a6
wenhuach21 revert inference mode to no_grad as some issue occurs in phi2
595316f7
pre-commit-ci[bot] [pre-commit.ci] auto fixes from pre-commit.com hooks
2bdd602d
wenhuach21 fix a critical regression
ef52934c
wenhuach21 fix conv1d issue
13a82208
wenhuach21 add comments
c202979f
pre-commit-ci[bot] [pre-commit.ci] auto fixes from pre-commit.com hooks
b102fe78
wenhuach21 update readme
db29840d
wenhuach21 wenhuach21 changed the title speedup by disable_low_gpu_mem_usage and reduce memory usage by avoid using torch.cat ~80% speedup by disable_low_gpu_mem_usage and reduce memory usage by avoid using torch.cat 2 years ago
wenhuach21 wenhuach21 changed the title ~80% speedup by disable_low_gpu_mem_usage and reduce memory usage by avoid using torch.cat 1.8X speedup by disable_low_gpu_mem_usage and reduce memory usage by avoid using torch.cat 2 years ago
wenhuach21 fix typo
61d2b351
wenhuach21 wenhuach21 merged 23b60c3a into main 2 years ago
wenhuach21 wenhuach21 deleted the fix_performance branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum