text-generation-inference

c3d12ae2 - Some protection against sharding (illegal access becuase of g_idx)