Can vLLM be supported?

Hi, in local evaluation, to ensure vLLM runs correctly in Docker, the --ipc=host parameter needs to be added, according to the link. I would like to ask if using vLLM for evaluation is supported?


The same questions, it may be helpful for faster inference

1 Like

Hope this suggestion can be considered