llama.cpp
Add LLAMA_MOE_RANDOM_ROUTING=1 for benchmark-only random expert selection
#29539
Merged

Commits
  • Add LLAMA_MOE_RANDOM_ROUTING=1 for benchmark-only random expert selection
    gaugarg-nv committed 3 days ago
  • Handle expert caching
    gaugarg-nv committed 3 days ago
Loading