Lower or higher of metric is good?

NeurIPS 2019 : Disentanglement Challenge
#1

Hi,

I am a little bit confuse about rank of leader board
Lower or higher of metric is good???

Thanks

#2

Hi Namvi, the range of all the metrics is [0,1]. The higher score is better.

#3

so why???:
image

#4

@namvi: That was actually a typo I believe, and it has been fixed in the description. If there are other instances of the same mis-information, please do point us to it, and we will have it fixed.

But as @waleedgondal mentions, in case of all scores, the range of all the metrics is [0,1], and the higher score is better !

#5

Why are you sorting by FactorVAE on the leaderboard?

#6

@golovanov: This is an interim solution until we have a proper rank computation in place, which takes the mean of the rank of the submission across all the individual metrics.

Using the sum-of-all-metrics to rank the submissions is a bit misleading too, as the distributions of all the individual metrics are quite different, and some of the metrics can heavily dominate the whole sum-of-all-metrics.