I got the following error during evaluation: AIcrowd. My submission is for the intro track but why does the system try to load the Findcave inference? My debug submission did not encounter this error.
@slai7880 : the
Findcave reference in the logs for the intro track is a bug only specific to the logs, and because of how we are re-using the evaluators from the other track.
In case of the intro track, the
Findcave rollout is hot-patched to use the
ObtainDiamond env instead. So the logs show
Findcave even if it is actually using
ObtainDiamond. Sorry for the confusion.
Regarding the error at your end, this is because, of this line: AIcrowd
where during the evaluation phase, the participant submitted agents are not allowed to access the internal env specific parameters like
env.task. This is to prevent agents from interfering with the internal state of the env during the evaluation.