We have made a change in the evaluation setup, to allow for models that predict relative depth maps from mono image inputs. We now apply an alignment step to the model to align a relative depth prediction to the ground truth. For more details please refer to this paperTowards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer.
Updated code for local evaluation of your models can be found in the starter-kit . All valid old submissions will be re-evaluated. All future submissions will be evaluated with these updated metrics.
Thanks for pointing this out, the SILog had gone to NaN because your predictions were too large, I think you probably output the depth values without the scaling factor as shown in local_evaluation.py? The submission is now marked as failed.
Please check this function for reading depth images. The values should be closer to ~1-50.
@dipam, I tried twice to submit new weights with identical code to my last “evaluation successful” submission (everything the same but the weights) submitted a week ago, I am getting an error: ““Evaluate Scores: AssertionError:”” and no additional information. the logs are identical by the way.
Could you please tell me more about the error as I am out of ideas…