Methods of evaluation of performance

Hi there,
I’m wondering a little about the reasons for the choice of submission format and evaluation process - will this be made clearer in subsequent rounds of this competition?

A much simpler method for evaluation would have been for everyone to submit a (num of molecules, total number of scent labels = 109) sized matrix (of 1 for a prediction of a scent label, 0 otherwise) and the just compute an overall Jaccard by computing:

Jaccard = TP/(FP+TP+FN)

This would be a fairer way to deal with molecules which varying label set cardinalities.