transformers
Generate: Mistral/Mixtral FA2 cache fix when going beyond the context window
#28037

Merged

Commits

add breakpoint

gante committed 2 years ago
only slice if the cache has contents

gante committed 2 years ago
get the right slice

gante committed 2 years ago
.

gante committed 2 years ago
.

gante committed 2 years ago
tmp commit

gante committed 2 years ago
fix FA2 + mis(x)tral

gante committed 2 years ago
add self.layer_idx exception (as suggested by @tomaarsen)

gante committed 2 years ago