No kernel image is available for execution on the device

Thank you very much for the facility and environment support of from the organizers. However, during the runtime, I encountered the following issue:

The only two modification is on

1.apt.txt (adding g++)

2.Dockerfile(https://repo.anaconda.com/miniconda/Miniconda3-py38_22.11.1-1-Linux-x86_64.sh->https://repo.anaconda.com/miniconda/Miniconda3-py310_24.3.0-0-Linux-x86_64.sh) for a different version of python

However, the following error occurred:

  File "/home/aicrowd/.conda/lib/python3.10/site-packages/vllm/model_executor/layers/layernorm.py", line 59, in forward

    out = torch.empty_like(x)

RuntimeError: CUDA error: no kernel image is available for execution on the device

CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions

However, it seems that the server does not support docker for different python? Or maybe I changed the miniconda in the wrong way?

detailed information can be seen at AIcrowd

docker_run.sh can be successfully run on my own device.

Hi Did you solve the problem?