Hello everyone!
We are getting closer to the end of the Generative Interior Design Challenge on April 1. Make a final push and become one of the top-3 teams traveling to Dubai and competing for the 1st prize at the Machines Can See summit on April 17!
To help you on this way, we here provide a few hints and share information on the test data in terms of room categories, example images and text prompts.
Development set.
In the final phase, your solutions will be run and evaluated on six room categories: bedroom, living room, home office, dining room, children’s room, and kitchen. We have released a new a Development set containing an 3 images and corresponding text prompts for each of the six room types, totaling to an additional 18 image-prompt pairs. We hope this will help you to identify and improve weak points of your solutions.
You can download the development set in the Resources Section of the challenge.
Text prompt structure.
To improve your method, it’s a good idea to expand the set of text prompts for training and testing. The typical structure of text prompts used in our test set contains the following three parts: [Room Type][General Style][Description of furniture items and their arrangement in the room].
Example text prompt:
A functional home office space characterized by industrial design elements, featuring a sturdy metal desk, a comfortable ergonomic chair, open shelving units displaying a mix of professional and personal items, and a tall floor lamp providing ample light.
More examples can be found in the development set.
Tips and tricks
Below we provide a few ideas that may enhance your current solutions. Note that these are merely suggestions—testing their effectiveness will be up to you.
- Our suggested baseline is composed of the following steps: Room segmentation → Identification of masks that should not change → Generation of an image based on the textual description in the specified area. Each of these stages can be improved independently.
- To preserve the room layout, our baseline prevents modifications of window regions in the image. This often causes issues for spaces with large panoramic windows, where image edits become restricted only to small image areas. Come up with better methods enabling changes in the window regions of the image without modifying locations and the style of original windows.
- Do you need more data? Apart from downloading images of either furnished or empty rooms, consider generating these images using tools such as MidJourney and Stable Diffusion.
- Exploring pre-trained models for each step of your pipeline may prove beneficial. Platforms such as civitai.com could be a good starting point.
- Extend your focus and creativity beyond Computer Vision components. Why not improve image generation by modifying original text prompts using heuristics or LLMs? For example, pay attention to the negative prompts.
- Note that the quality of generated images is typically affected by the image resolution and the number of iterations in diffusion models.
Wondering if it’s worth investing more resources into this competition? We are pleased to announce that additional prizes are expected by Nvidia. But shhh, it’s still a secret.
Good luck and hope to see you in Dubai!
Generative Interior Design 2024 organizing team