Hi!
Seeking clarification on Taks 1 and the historical dialogue.
From the Task1 readme.md , we read that
For each batch of conversations, the first set of turns will be provided to your model. After the response is receieved the further turns of the same conversation will be provided. Each conversation will have exactly 7 turns.
Now, looking at the sample discussion:
My understanding is that the model is first provided with message from Anne (persona A).
“Oh , what a long queue ! I need to arrive at the seminar in 30 minutes but there 's not much hope now … you seem to be in a rush too , are you alright ?”
From the sample notes, it is clear at this point that the gold reference is invisible. My understanding is that the gold reference is an appropriate response.
Gold Reference:
“Oh hi ! I am OK but I 'm in a bit of a rush too actually . I have a meeting in an hour and I wanted to arrive early so that I could rehearse my presentation again . I 'm David by the way . What kind of seminar are you attending ?”
Let's say my model outputs this response to Anne's first message (instead of the gold ref):
“Hi, I understand the frustration of being in a long queue, but it’s important to stay patient. I’m actually in a bit of a rush as well.”
Ultimately, for performance evaluation, my model response is compared to the Gold Reference.
I seek clarification on the following turns.
The model will be provided with the historical dialogue.
Will the historical dialogue be constructed using:
-
both Anne’s question and the previous Gold References?
or - both Anne’s question and my model actual previous responses?
Thanks!