Thanks for setting up this competition. I suppose one approach is to fine tune controlnet for better score given the performance metrics. i have question about the test images and prompts.
“You will be provided a total of 1800 seconds to run the complete private set of images containing 40 images. Your models also need to”
I suppose there is only one prompt per images?
(i.e. only 40 test samples)
i also want to confirm the public leaderboard results is only based on public 3 images only.
on a side note, it might be better to provide a few more examples of prompt or at least some keywords, or some references (paper, other public datasets, blogs, etc). We do not know how big is the scope of the objects, style, descriptions in the prompt.
some open dataset:
1 Like
I suppose there is only one prompt per images?
(i.e. only 40 test samples)
We confirm that for each test set image there is a corresponding prompt. So, the test set currently contains 40 image prompt pairs.
Thank you for pointing out the ambiguity. We will clarify the same in the documentation.
i also want to confirm the public leaderboard results is only based on public 3 images only.
The three public test images are only used for integration testing of your code with the rest of the evaluation system; the scores on the leaderboard are generated using the whole test set of 40 images.
on a side note, it might be better to provide a few more examples of prompt or at least some keywords, or some references (paper, other public datasets, blogs, etc). We do not know how big is the scope of the objects, style, descriptions in the prompt.
We are considering your recommendation and are currently deliberating the possibility of providing a more extensive validation set (images along with their corresponding prompts). This would enable a more thorough local evaluation of your solutions. We aim to share further updates on this matter shortly.
1 Like
Follow-up questions.
Do all prompts in the test-set focus on living room/bedroom creation? Or also bathrooms, and kitchens?