[Uni] Our competition experience

[azheng]

Finally, KDD competition is end. We have done code submissions for task2 and task3. We think the data provided by Amazon allows us to build a cross-encode language model for the rerank stage. As a top-level competition, the scenario provided is of great research value, meanwhile all contestants benefited a lot from the competition.
Actually, at the beginning of the competition, our team was ready to give up the competition… As we all know for a long time at the beginning, there is seemingly always a great impassible gap between the top tiers(0.85+) and others (0.79-) on the PB of tasks 2. We use the data of task2 to train a baseline model which score on PB of task 2 is around 0.775. We think there must be serious data leakage in task2 upon our internal review. The gap is replacing the answer of task2 with the training data of task1. But it just like a default hidden rule, no one made the news public. Even we train models with data of both task1 and task2 we still can not reach 0.83+, because of Bert models can not remember total answers of task1. Finally we get sick of this and get message to the competition organizer…
Then we thought that everyone was finally on the same starting line. We constantly improve models and get top 1 of task2 and task3 before the game deadline(7.15). We use two models get 0.825, and we use onnxruntime to optimize inferring time, which allows us to submit 5 models to get 0.8265+. The complete solution of 0.8275+ consists of 9 models(infoxlm-large of 5folds and deberta of 4 folds both with max_len=128 and 5 lgb models used in stacking). Then the competition was extended by about one week. We both feel tortured and tired about that. At the same time, we found that many players used external data (product pictures, categories, prices). This multilingual competition have been reduced to a multimodal competition. At the end of this competition ,we fail to crawl the entire data of all products in this match, and use only the information of training data of task2 to get 0.828+ on PB.
Thanks for sharing:

[ETS-Lab] Our solution

Thanks the AIcrowd team and the Amazon search team to organize this extensive competition. Finally, this game is ended. Our team learned a lot here and we believe this memorable period will help a lot in our future. Here we generally introduce our solution for this competition. General solution We trained 3 cross encoder models (DebertaV3, CocoLM, and Bigbird) for each language which differs in the pertained models, training method (e.g., knowledge distillation), and data splitting. In tota…

My solution (good-good-study, day-day-up)

Basic Solution All my models are based on the infoxlm large model. I concat the training set of Task1 and task2 as a new training set after de duplication. Then all next three tasks use the same model trained on the new training set. Finally, I used 8 models on Task1 and 4 models on task2 and task3. The output of the model can be submitted to different tasks after different processing: Task1: order the product by \hat P_{exact} +\hat P_ {substitute} * 0.1 + \hat P_ {completion} * 0.01. The …

What we just really didn’t expect is that there are still many leakages on the PB… anyway. Maybe the conclusion is:Infoxlm + Deberta + leak + external data + onnx + 8model stacking = Top1 ?
Although we learned a lot from the competition, the competition experience is really poor.

[yrquni]

I’m yrqUni from the Uni team, and in our solution, we only used the training set of task2, which should be amazing.