5 seconds per timestep

Flatland
#1

Dear Flatland Organization Team,

I have a question regrading the 5 seconds per timestep. In run.py on submission git repo, there are two time time: time_taken_by_controller and time_taken_per_step. The 5 seconds limit per timestep is the sum of time_taken_by_controller and time_taken_per_step in each timestep, or only the time_taken_by_controller in each timestep?

Look forward to your reply and thanks,
Beibei

#2

Hey @beibei,

There are 3 things that can take time:

  • figuring out your next actions from the current observations (inference)
  • executing these actions in the environment (stepping)
  • building the observations from the new environment state (observation building)

The timeout per timestep limits the time taken by inference and observation building, but not stepping.

So it’s time_taken_by_controller + time_taken_per_step - stepping_time.

To measure the stepping time separately, you can create the environment with a dummy observation builder and build the observations separately from the call to step(). See eg here: https://gitlab.aicrowd.com/flatland/flatland-examples/blob/make-submittable/run.py#L67 (observations are built explicitly line 138).