The evaluation dataset questions consists of stock prices, but none of the answers are accurate as of Feb 16th, which is the last date given in the dataset. I used Nasdaq and MarketWatch to test the prices on that day, but none of them matched.
you can check the closing price on https://www.nasdaq.com/market-activity/stocks/tfx
For example, the closing price on feb16th is 251.07$ but the answer is given as 249.07$
“interaction_id”: 0,
“query”: “what’s the current stock price of Teleflex Incorporated Common Stock in USD?”,
“answer”: “249.07 USD”
Hi saidinesh_pola,
Thank you for your question. Please note that there is a query_time provided for each question, corresponding to the time when the query was made. In the main page, we noted “The ground truth is the answer that was correct at the point when the question was posed and data were collected.”
Also, we updated the data to v2, which resolved some low quality questions and answer errors. Please update the data when you can. Thank you!
Please refer to: AIcrowd | Meta Comprehensive RAG Benchmark: KDD Cup 2024 | Challenges
Best Regards,
The CRAG Team