Based on multiple requests from participants, and internal discussions in the team, we are extending the deadline of the Code Submission Round of the competition to 20th of July, 23:59 UTC.
The timeouts for the Public and Private Test phases has also been increased to 120 mins each.
Additionally, we would like to add a Note that the final winners of the competition will be decided based on the performance of your submissions on the unseen private test set. We noticed that some participants are including locally generated predictions (on the public test set) in their submission and during the evaluation they are using those as a lookup table to generate predictions on the fly - the approach of course does not work on the unseen private test set, where either the submissions fail, or some of the submissions are simply generating random predictions for the private test set leading to a massive underperformance on the private leaderboard.
Thanks for providing a good challenge for us. But this news is really bad for me as our team are working on the submission which meet all the rules that requested including deadline and submission time limits. 30 minutes, 90 minutes and 120 minutes could yield significantly different solution and the changes of deadline will cause some schedule conflicts. I feel tortured when the rules changes every time. The arbitrary changes show no respect to those people who are working hard in this challenge.
Personally, I wish to keep the original schedule and time limits but I also respected other teams and host stand and opinions.
If possible, could we make a poll to let all active teams vote?
I feel very tired caused the changing rule, too… We think kddcup is the most prestigious competition in data mining and changing the rules is incredible
I GUESS THE TEAM SHOULD AT LEAST LAUNCH A POLL ON SUCH A BIG CHANGE, but it turns out you didn’t. “Based on multiple requests from participants”, how many is exactly “multiple”? Rules shouldn’t be kidnapped by some small group of people.
I understand that people leading the competition could be upset by such a decision. But considering that the code submission should have been launch May 20th, it’s normal to extend the competition end date to allow all participants to try to submit their code. I don’t think it will have a big impact on the leaderboard, same as the extended timeout, looking at the CSV submission leaderboard.
I participated in the competition recently, so the time is really tight, thank you for extending the deadlines to 20 July and increase timeout of 120 mins.
Many participants and teams were not very familiar with docker setup and code submissions, and needed a lot of help.
To accommodate all of them and to be fair to all the participants these decisions about the time limit and deadline dates were moved.
We understand that some of the other participants have some difficulties due to these extensions. But, we would also like to point out that such extensions will allow more people to provide stronger models and help us advance the core spirit of this competition - to collectively better understand the potential solutions to the tasks posed in this competition.
Rest assured, there will be no change in any of the competition rules or the deadline anymore.
Although I know that we have been unable to change your decision, I still want to tell you how your decision hurt many participants who have worked very hard from the beginning. We followed all the rules, tried to submit and have been waiting for the deadline. Now in the last two days of the game, you told us that we need to continue to fight for this game for another six days. It’s like telling a marathon runner who has run 40km that the finish line is 50km. Although the final ranking may not change much, we need to spend a lot of extra energy.
I started training models and submissions on the first day of the competition, so what is fairness? What were they doing when I stayed up late to get familiar with the submission process? What were they doing when I made model pruning to control the duration? The leader’s lead is earned by hard work, not cheating. This change is no less than redrawing the starting line, giving people who don’t work hard a new opportunity.
Just to share my case, I had been held out of competition for the majority of code submission round, due a synchronization issue between my aicrowd account and gitlab. (I couldn’t login to gitlab at all) I pointed out a potential cause and Shivam solved the issue by re-mapping my account. (so it wasn’t an issue I could’ve solved on my side) I was rank 5 in task 3 the last time I submitted during the prediction/csv submission round, btw. So extended time does partially fill the time I’d been held out. Anyways, all the best to all who are participating in this competition and hope submission works smoothly during the remaining period.
The submission pipeline is too slow, I have waited for 10 hours, I think we should extend deadline and I expect organizer can add submission computation node. @mohanty
