text-generation-inference

84eead21 - feat(backend): correctly setup llama_context providing n_threads and n_ubatch