huggingface/text-generation-inference

Pull Requests Commits

feat: support llama 3.1 tooling and remove grammar schema

drbh committed 1 year ago

3f07ddb4

New release 2.3.1 (#2604)

Narsil committed 1 year ago

Verified f6e2f05b

Unroll notify error into generate response (#2597)

drbh committed 1 year ago

Verified d22b0c1f

CI (2592): Allow LoRA adapter revision in server launcher (#2602)

drbh committed 1 year ago

Verified 23354595

Max token capacity metric (#2595)

Narsil committed 1 year ago

Verified 0204946d

Mllama flash version (#2585)

Narsil committed 1 year ago

Verified d18ed5cf

nix: experimental support for building a Docker container (#2470)

danieldk committed 1 year ago

Verified 584b4d7a

MoE Marlin: support `desc_act` for `groupsize != -1` (#2590)

danieldk committed 1 year ago

Verified 1c84a30f

Move flake back to tgi-nix `main` (#2586)

danieldk committed 1 year ago

Verified d1f257ac

feat: support phi3.5 moe (#2479)

drbh committed 1 year ago

Verified 93a7042d

Add support for GPTQ-quantized MoE models using MoE Marlin (#2557)

danieldk committed 1 year ago

Verified 90a1d04a

Update ROCM libs and improvements (#2579)

mht-sharma committed 1 year ago

Verified f9e561ec

Update architecture.md (#2577)

ulhaqi12 committed 1 year ago

Verified e790cfc0

Remove compute capability lazy cell (#2580)

danieldk committed 1 year ago

Verified afc7ded8

flashinfer: pass window size and dtype (#2574)

danieldk committed 1 year ago

Verified 1028996f

Improve support for GPUs with capability < 8 (#2575)

danieldk committed 1 year ago

Verified 5b6b74e2

Fix build with `--features google` (#2566)

alvarobartt committed 1 year ago

Verified 0aa66d69

Add LoRA adapters support for Gemma2 (#2567)

alvarobartt committed 1 year ago

Verified 0b7df771

remove LORA_ADAPTERS_PATH (#2563)

nbroad1881 committed 1 year ago

Verified 7efcb5e0

More tensor cores. (#2558)

Narsil committed 1 year ago

Verified dd8691b7

Cleanup Vertex + Chat (#2553)

Narsil committed 1 year ago

Verified c032280b

Hotfixing main. (#2562)

Narsil committed 1 year ago

Verified 75c8c54a

Adding note for private models in quick-tour document (#2548)

ariG23498 committed 1 year ago

Verified e6d29656

Simplify crossterm imports (#2545)

orhun committed 1 year ago

Verified 8024ded5

Update the link to the Ratatui organization (#2546)

orhun committed 1 year ago

Verified 03263f5e

Add `DenseMoELayer` and wire it up in Mixtral/Deepseek V2 (#2537)

danieldk committed 1 year ago

Verified 3f14cd14

Add support for scalar FP8 weight scales (#2550)

danieldk committed 1 year ago

Verified c29dc89c

Hotfixing main (#2556)

Narsil committed 1 year ago

Verified 0ff6ff60

Micro cleanup. (#2555)

Narsil committed 1 year ago

Verified 74d3ce10

Remove duplicated `RUN` in `Dockerfile` (#2547)

alvarobartt committed 1 year ago

Verified d31a6f75

Older