Error in evaluation phase

Learn-to-Race: Autonomous Racing Virtual Challenge
#1

I am unable to evaluate my model in the local machine due to the error in the evaluator.py file.

==============================================
Traceback (most recent call last):
File “rollout.py”, line 44, in
run_evaluation()
File “rollout.py”, line 40, in run_evaluation
evaluation_routine(evaluator)
File “rollout.py”, line 23, in evaluation_routine
scores = evaluator.evaluate()
File “/home/tduy/Workspace/competition/l2r-starter-kit/evaluator/evaluator.py”, line 88, in evaluate
self.register_metrics(metrics=info[“metrics”])
File “/home/tduy/Workspace/competition/l2r-starter-kit/evaluator/evaluator.py”, line 131, in register_metrics
self.metrics[idx][metric] = metrics[metric]
KeyError: 1

The error occurs because idx variable was assigned to the new value in line 127 of the code. I am not sure if this is a bug or I made some mistake in the agent file.

1 Like
#2

Hi, I also encounter this error:
shft_idx:3736, abs_idx: 3736, success:[False, False, True, True, True, 0, 0, 0, 0, 0], curr_seg:6, half:False
{0: {‘total_time’: 10.86, ‘total_distance’: 0.0, ‘average_speed_kph’: [0.0], ‘average_displacement_error’: [0.0], ‘trajectory_efficiency’: [3187502051.277], ‘trajectory_admissibility’: [1.0], ‘movement_smoothness’: [6.233], ‘timestep/sec’: [9.3], ‘reward’: [0.0], ‘num_infractions’: 45, ‘pct_complete’: 100}}
laps_completed
Traceback (most recent call last):
File “rollout.py”, line 43, in
run_evaluation()
File “rollout.py”, line 39, in run_evaluation
evaluation_routine(evaluator)
File “rollout.py”, line 22, in evaluation_routine
scores = evaluator.evaluate()
File “/home/hci-a4000/TIN/l2r-starter-kit/evaluator/evaluator.py”, line 88, in evaluate
self.register_metrics(metrics=info[“metrics”])
File “/home/hci-a4000/TIN/l2r-starter-kit/evaluator/evaluator.py”, line 133, in register_metrics
self.metrics[idx][metric] = metrics[metric]
KeyError: 5

1 Like
#3

Hi @hung_tran, do you want to team up? we’re both VIetnamese

1 Like
#4

The fix for this is underway will be available soon.

1 Like
#5

Hello, is the fix for this error done because I am experiencing it as well. Thank you.

1 Like
#6

We have pushed the fix, itll be available on aicrowd server publicly soon. You may notice a change in the leaderboard as a result.