vllm

b2c06509 - [P/D]Provide bucket algorithm rate limiter for proxy_server (#22643)

[P/D]Provide bucket algorithm rate limiter for proxy_server (#22643) Signed-off-by: frankie-ys <yongshengwang@cmbchina.com> Signed-off-by: frankie <wangyongsheng686@gmail.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Kuntai Du <kuntai@uchicago.edu>