Insufficient cluster resources to launch trial

ava6969 · July 11, 2020, 8:08am

ray.tune.error.TuneError: Insufficient cluster resources to launch trial: trial requested 7 CPUs, 0.8999999999999999 GPUs but the cluster has only 2 CPUs, 1 GPUs, 1.37 GiB heap, 0.63 GiB objects (1.0 node:192.168.1.242). Pass queue_trials=True in ray.tune.run() or on the command line to queue trials until the cluster scales up or resources become available.

SO I understand what this error means but i don’t know if i need to repair it before i begin working on the project. I also need some advice on how to fix it.

shivam · July 11, 2020, 8:12am

Hi @ava6969,

Please modify you run.sh based on your system’s configuration.

github.com

AIcrowd/neurips2020-procgen-starter-kit/blob/master/run.sh#L19


export EXPERIMENT=${EXPERIMENT:-$EXPERIMENT_DEFAULT}

if [[ -z $AICROWD_IS_GRADING ]]; then
  ##########################################################################
  # This section contains commands you would like to run, when running     #
  # the codebase on your machines. During evaluation AICROWD_IS_GRADING    #
  # variable is set, due to which this block will be skipped.              #
  ##########################################################################

  export OUTPUTS_DIR=./outputs
  export RAY_MEMORY_LIMIT=1500000000
  export RAY_CPUS=2
  export RAY_STORE_MEMORY=1000000000

  # Cleaning output directory between multiple runs
  rm -rf ${OUTPUTS_DIR}
  mkdir ${OUTPUTS_DIR}
fi


export VALID_RUN=false