Today I found that the leaderboard have updated, what is top2 mean? Is it the first and second setence in our 5 sentence submission?
Top2 means that we look at only the first two sentences and not all the 5 sentences returned by your model.
I asked to add this extra metric in order to compare scores @ top 2 and @ top 5.
Then how the metric would be in the final round to determine the winner? Is it still the top5 value or the mean of top2 and top5 or anything others?
The final ranks are determined by the top-5 score. In the unlikely event of an exact tie in top-5 score, top-2 will be used as a tie breaker.