Need help about submission problem: python run.py then it's been killed

gcalgoz · March 27, 2024, 2:50pm

debug-logs shows me the error, I have no idea how to fix it, I’ve tried many times, but it’s not work ever.

reported detail error:
########################################

rollout-track1-validation

########################################
2024-03-27 13:53:09.630 | INFO | aicrowd_gym.clients.base_oracle_client:register_agent:210 - Registering agent with oracle…

2024-03-27 13:53:09.633 | SUCCESS | aicrowd_gym.clients.base_oracle_client:register_agent:226 - Registered agent with oracle

Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]/src/run.sh: line 3: 14 Killed python run.py

aicrowd_team · March 27, 2024, 2:59pm

@gcalgoz : Please set gpu: true in your aicrowd.json here, to be able to load the model into GPU memory instead of RAM. right now, your evaluations are killed because of high memory usage.

gcalgoz · March 28, 2024, 10:45am

@aicrowd_team thanks for your help, I set gpu: true in my aicrowd.json and rerun it this time, but it meets a timeout error and has no more detail info. please help

submission_hash : 8f4edcf8ee530855950f343919d0ddf59cf6205c.

error info:

issues: AIcrowd

logs: AIcrowd

gcalgoz · March 28, 2024, 3:59pm

it worked ! I changed Dockerfile, I think timeout reason is image is too large! tks

jinoooooooooo · March 29, 2024, 4:35pm

@gcalgoz hey, im facing the same issue, what did you have to change in the dockerfile, can you please help here, thanks