We’ve launched Round of the challenge, as you know, this comes with an updated the dataset and leaderboard. Along with this, we introduced some new major updates, as mentioned below.
Prompt Engineering Track - We have introduced the option to use OpenAI API for prompt engineering based submissions. This will significantly lower the barrier to entry to make submissions. Check out latest documentation starter kit to find out how to make submissions with access to OpenAI API.
Changes in function format - The
generate_responsesfunction in the agent class has an updated interface to support the prompt engineering track and gpu track simultaneously. Please make the changes to the agent class according to the new format. Check this starter kit commit for the updates to the documentation. It’s recommended that you pull the latest updates made on the starter kit to your submission repos.
Removal of persona A information - The organizers have asked for the removal of persona A information from the input data. This brings the benchmark closer to a real world setting where the persona of the conversation partner may not be known. If your agent used persona A information, please update it accordingly.