How to run evaluation locally?

The challenge has officially ended few months back.

But in case you are interested in playing with the problem you can run end to end submission on your local environment.

In order to run the starter kit locally for getting scores w.r.t. leaderboard you can do the following:

result, detailed_scores = real_robots.evaluate(
                SubmittedPolicy,
                intrinsic_timesteps=0,
                extrinsic_timesteps=10000,
                extrinsic_trials=50,
                visualize=False,
                goals_dataset_path=DATASET_PATH
            )

in your local_evaluation.py.

The scores will be comparable to the one on the leaderboards.
Let us know in case any more help is required!

Cheers,
Shivam

1 Like