We also investigated our submissions again, I have two more findings
-
we seemed to only have tested
debug=Trueonce on Mar 30th and left it todebug=Falsesince then. We didn’t notice it back then but we can confirm that we got 6 submissions through on that day. As I remember, the reasons for enabling it, was to check weather we get more accurate readings on the runtime duration (not the case
) Here is the blame log for aicrowd.json.
-
The listing that @JusperLee posted above, showing that we got 7 submissions on May 8th UTC. seems not related to the debug issue. In fact, there seem to be two other reasons for it:
- AIcrowd | Cinematic Sound Demixing Track - CDX'23 | Submissions #220227 was not a May 8th UTC submission since the 24h fixed window was applied for the CDX task as explained above by @dipam.
- AIcrowd | Cinematic Sound Demixing Track - CDX'23 | Submissions #220233 submission 1
- AIcrowd | Cinematic Sound Demixing Track - CDX'23 | Submissions #220258 submission 2
- AIcrowd | Cinematic Sound Demixing Track - CDX'23 | Submissions #220278 submission 3
- AIcrowd | Cinematic Sound Demixing Track - CDX'23 | Submissions #220392 submission 4
- AIcrowd | Cinematic Sound Demixing Track - CDX'23 | Submissions #220397 submission 5
-
AIcrowd | Cinematic Sound Demixing Track - CDX'23 | Submissions #220402 this submission is identical to
220307. In fact we accidentially created a new tag for the exact same git commit. Funnily, it seems the submission system created identical timestamps and logs for both, so it might be the same submission and we accidentially discovered that you can do unlimited submission (when they are all the same)… I don’t think there is any way to benefit from this issue, right?
update: @dipam already mentioned the second issue in his post above