Hi there
How can I simulate the evaluation locally? If I use the flatland-evaluator, the order of the envs evaluated is random, and the largest ones will cause OOM. Is there any way to run them sequentially?
Thanks for your help.
You can use set the --shuffle parameter to False when calling the evaluator to order the environments.
To get the full Round 2 setup (infinite wave, new timeouts…) you should first update your pip version to the infinite_wave branch:
pip install -qq git+https://gitlab.aicrowd.com/flatland/flatland.git@infinite_wave
We’ll release it to pypi soon.