Questions about evaluation set up

#1

Hi there. Can you elaborate a bit on the evaluation set up? In particular, I’m curious about what conditions will be different in Round 1 wrt to the environment we are given for development? Will it be exactly the same environment (including any parameters in the musculoskeletal model, noise level, etc.) and the only difference is the starting state and target field? Or will something else change?

Also, is there a time limit for computation?

Thanks!

#2

Hi, @luisenp. The evaluation in Round 1 will be done with the same environment with the last version of the development environment. When you submit your solution, our server runs multiple simulations with different target velocity fields, and the evaluation score is the mean of the cumulative rewards you receive in those simulations.

1 Like