lighteval

b68d5bc9 - Autoscaling inference endpoints (#412)

Autoscaling inference endpoints (#412) * adding better management for restarts and resizes * upgraded autoscale * added pause option * fix to parallelism manager - no need for endpoint