We are constantly trying to improve this challenge for you and would appreciate any feedback you might have!
Please reply to this thread with your suggestions and feedback on making the challenge better for you!
- What have been your major pain points so far?
- What would you like to see improved?
Cheers,
Team Mosquito Alert 2023
Thank you for organizing this competition!
I would like to provide feedback on the ground truth bounding boxes.
I compared the ground truth bounding boxes of the training data against the bounding boxes of a pre-trained model called Owl-ViT, and below are some issues which, as I am convinced, adversely effect the quality of the competition:
Bounding Box Quality
In some cases, the quality of the ground truth bounding boxes are bad, leading to low IoU between the ground truth and Owl-ViT, although Owl-ViT arguably leads to better bounding boxes. In those cases the IoU between the two boxes would even be below the required threshold for scoring.
Multiple Objects in One Image
In case of multiple objects, sometimes the ground truth bounding box is around the smaller object. I argue that one should either split those images in two such that no image contains more than one object, or have the ground truth around the largest object.
Bounding Boxes Around Multiple Objects
From my point of view, each ground truth bounding box should be around one object only.
Since the challenge metric involves thresholding the IoU between the predicted and (arguably bad) ground truth bounding boxes, I fear the scope of the challenge might change from trying to locate mosquitos as best as possible to reproducing the ground-truth bounding boxes as best as possible to achieve a good challenge score.
I therefore suggest to update the ground truth bounding boxes, or adjust the challenge metric as to not include the IoU.
5 Likes