DeepSpeed
Ds-inference Int8 support through ZeroQuant technology
#2217
Merged
Overview
Commits
25
Changes
View On GitHub
Commits
  • Fix the layer-past for GPT based models
    Reza Yazdani committed 4 years ago
  • add the Int8 support for ds-inference using ZeroQuant technology
    Reza Yazdani committed 4 years ago
  • fixing some issue with loading checkpoint and bias-add
    Reza Yazdani committed 4 years ago
  • adding the logic to store/restore scale for INT8 checkpoint
    Reza Yazdani committed 4 years ago
  • add empty quantization scale for different models to run with fp16
    Reza Yazdani committed 4 years ago
  • Empty-Commit
    Reza Yazdani committed 4 years ago
  • Merge branch 'master' into ds-inference/ZeroQuant-Int8
    RezaYazdaniAminabadi committed 4 years ago
  • fix sevral issues after merging with master
    Reza Yazdani committed 4 years ago
  • several fixes for generating the INT8 sharded checkpoint
    Reza Yazdani committed 4 years ago
  • Merge branch 'master' into ds-inference/ZeroQuant-Int8
    RezaYazdaniAminabadi committed 4 years ago
  • move quantizer declaration before inference branch
    Reza Yazdani committed 4 years ago
  • Merge branch 'master' into ds-inference/ZeroQuant-Int8
    RezaYazdaniAminabadi committed 4 years ago
  • fixing some part to catch up with latest update on HF side
    Reza Yazdani committed 4 years ago
  • Merge branch 'ds-inference/ZeroQuant-Int8' of github.com:microsoft/DeepSpeed into ds-inference/ZeroQuant-Int8
    Reza Yazdani committed 4 years ago
  • reducing the CPU memory usage when loading checkpoint (this solves the issue when there is not enough CPU memory to load large models
    Reza Yazdani committed 4 years ago
  • some minor modification to the ckpt names
    Reza Yazdani committed 4 years ago
  • remove masking and some configuration changes
    Reza Yazdani committed 4 years ago
  • remove dead code
    Reza Yazdani committed 4 years ago
  • Merge branch 'master' into ds-inference/ZeroQuant-Int8
    jeffra committed 4 years ago
  • Merge branch 'master' into ds-inference/ZeroQuant-Int8
    RezaYazdaniAminabadi committed 4 years ago
  • fix some issue with int8 ckpt-loading
    Reza Yazdani committed 4 years ago
  • Merge branch 'master' into ds-inference/ZeroQuant-Int8
    RezaYazdaniAminabadi committed 4 years ago
  • Merge branch 'master' into ds-inference/ZeroQuant-Int8
    RezaYazdaniAminabadi committed 4 years ago
  • Merge branch 'master' into ds-inference/ZeroQuant-Int8
    RezaYazdaniAminabadi committed 4 years ago
  • change the mp_size to tp_size at inference config & add some doc-string at init_inference
    Reza Yazdani committed 4 years ago
FAQ Terms Privacy Refunds Impressum
Loading