Dear Flatland Organization Team,
I have a question regrading the 5 seconds per timestep. In run.py on submission git repo, there are two time time: time_taken_by_controller and time_taken_per_step. The 5 seconds limit per timestep is the sum of time_taken_by_controller and time_taken_per_step in each timestep, or only the time_taken_by_controller in each timestep?
Look forward to your reply and thanks,
There are 3 things that can take time:
- figuring out your next actions from the current observations (inference)
- executing these actions in the environment (stepping)
- building the observations from the new environment state (observation building)
The timeout per timestep limits the time taken by inference and observation building, but not stepping.
time_taken_by_controller + time_taken_per_step - stepping_time.
To measure the stepping time separately, you can create the environment with a dummy observation builder and build the observations separately from the call to
step(). See eg here: https://gitlab.aicrowd.com/flatland/flatland-examples/blob/make-submittable/run.py#L67 (observations are built explicitly line 138).