text-generation-inference
feat(server): Add exllama GPTQ CUDA kernel support #553
#666
Merged
Overview
Commits
30
Changes
View On GitHub
Commits
  • add exllama gptq kernel
    fxmarty committed 3 years ago
  • add attribution
    fxmarty committed 3 years ago
  • Merge branch 'main' into gptq-cuda-kernels
    fxmarty committed 3 years ago
  • some more cleanup
    fxmarty committed 3 years ago
  • Merge branch 'gptq-cuda-kernels' of https://github.com/fxmarty/text-generation-inference into gptq-cuda-kernels
    fxmarty committed 3 years ago
  • try-catch to load the cuda extension, quite ugly practice tbh
    fxmarty committed 3 years ago
  • have a single gptq quantization type
    fxmarty committed 3 years ago
  • move exllama buffer init to the top level
    fxmarty committed 3 years ago
  • cleanup
    fxmarty committed 3 years ago
  • support bits different than 4
    fxmarty committed 3 years ago
  • tests
    fxmarty committed 3 years ago
  • Merge branch 'main' into gptq-cuda-kernels
    fxmarty committed 3 years ago
  • fix test
    fxmarty committed 3 years ago
  • fix tests
    fxmarty committed 3 years ago
  • support all, test llama
    fxmarty committed 3 years ago
  • Merge branch 'main' into gptq-cuda-kernels
    fxmarty committed 3 years ago
  • fix the usual merge mess
    fxmarty committed 3 years ago
  • Merge branch 'main' into gptq-cuda-kernels
    fxmarty committed 3 years ago
  • fix per-column quantization
    fxmarty committed 3 years ago
  • Refactored a bit.
    Narsil committed 3 years ago
  • Small polish.
    Narsil committed 3 years ago
  • Give escape hatch to not use exllama kernels even if available.
    Narsil committed 3 years ago
  • Fixing GTPQ device santacoder.
    Narsil committed 3 years ago
  • Fix config.
    Narsil committed 3 years ago
  • Add kernel target.
    Narsil committed 3 years ago
  • Separate build process.
    Narsil committed 3 years ago
  • Update starcoder_gptq
    Narsil committed 3 years ago
  • Wtf gh.
    Narsil committed 3 years ago
  • Switching model for integration test llama gptq.
    Narsil committed 3 years ago
  • Getting closer to the non gptq test (stop sequence doesn't work).
    Narsil committed 3 years ago
FAQ Terms Privacy Refunds Impressum
Loading