Hi, in local evaluation, to ensure vLLM runs correctly in Docker, the --ipc=host parameter needs to be added, according to the link. I would like to ask if using vLLM for evaluation is supported?
2 Likes
The same questions, it may be helpful for faster inference
1 Like
Hope this suggestion can be considered