huggingface/text-generation-inference

Pull Requests Commits
MoE Marlin: support `desc_act` for `groupsize != -1` (#2590)
danieldk committed 1 year ago
Verified 1c84a30f
Move flake back to tgi-nix `main` (#2586)
danieldk committed 1 year ago
Verified d1f257ac
feat: support phi3.5 moe (#2479)
drbh committed 1 year ago
Verified 93a7042d
Add support for GPTQ-quantized MoE models using MoE Marlin (#2557)
danieldk committed 1 year ago
Verified 90a1d04a
Update ROCM libs and improvements (#2579)
mht-sharma committed 1 year ago
Verified f9e561ec
Update architecture.md (#2577)
ulhaqi12 committed 1 year ago
Verified e790cfc0
Remove compute capability lazy cell (#2580)
danieldk committed 1 year ago
Verified afc7ded8
flashinfer: pass window size and dtype (#2574)
danieldk committed 1 year ago
Verified 1028996f
Improve support for GPUs with capability < 8 (#2575)
danieldk committed 1 year ago
Verified 5b6b74e2
Fix build with `--features google` (#2566)
alvarobartt committed 1 year ago
Verified 0aa66d69
Add LoRA adapters support for Gemma2 (#2567)
alvarobartt committed 1 year ago
Verified 0b7df771
remove LORA_ADAPTERS_PATH (#2563)
nbroad1881 committed 1 year ago
Verified 7efcb5e0
More tensor cores. (#2558)
Narsil committed 1 year ago
Verified dd8691b7
Cleanup Vertex + Chat (#2553)
Narsil committed 1 year ago
Verified c032280b
Hotfixing main. (#2562)
Narsil committed 1 year ago
Verified 75c8c54a
Adding note for private models in quick-tour document (#2548)
ariG23498 committed 1 year ago
Verified e6d29656
Simplify crossterm imports (#2545)
orhun committed 1 year ago
Verified 8024ded5
Update the link to the Ratatui organization (#2546)
orhun committed 1 year ago
Verified 03263f5e
Add `DenseMoELayer` and wire it up in Mixtral/Deepseek V2 (#2537)
danieldk committed 1 year ago
Verified 3f14cd14
Add support for scalar FP8 weight scales (#2550)
danieldk committed 1 year ago
Verified c29dc89c
Hotfixing main (#2556)
Narsil committed 1 year ago
Verified 0ff6ff60
Micro cleanup. (#2555)
Narsil committed 1 year ago
Verified 74d3ce10
Remove duplicated `RUN` in `Dockerfile` (#2547)
alvarobartt committed 1 year ago
Verified d31a6f75
chore: Add old V2 backend (#2551)
OlivierDehaene committed 1 year ago
Verified 10e6f292
nix: remove unused `_server.nix` file (#2538)
danieldk committed 1 year ago
Verified 9263817c
Preparing for release. (#2540)
Narsil committed 1 year ago
Verified 169178b9
fix: wrap python basic logs in debug assertion in launcher (#2539)
OlivierDehaene committed 1 year ago
Verified 7e2d1887
hotfix: ipex fails since cuda moe kernel is not supported (#2532)
sywangyi committed 1 year ago
Verified f478aa77
doc: clarify that `--quantize` is not needed for pre-quantized models (#2536)
danieldk committed 1 year ago
Verified abd24dd3
Update to moe-kenels 0.3.1 (#2535)
danieldk committed 1 year ago
Verified c1037601
Terms Of Use Privacy Policy Refund Policy Impressum