Why Does Private Web Search v0.6 search_pipeline() return None?

[Chris_Deotte]

@yilun_jin8 @jyotish

I have noticed that validation web search v0.6 and public test web search v0.6 never return None when we call search_pipeline() locally. This makes sense because a RAG search uses cosine similarity, and it can always find and return k chunks.

But during submission, the private web search v0.6 search_pipeline() returns None. The private web search v0.5 did not return None.

Today, AIcrowd is re-running all our submissions. All of my re-run submissions are failing because of these None (but worked fine with v0.5). Is something going wrong with private web search v0.6 search_pipeline()? What would cause it to return None? And is it only returning a few None or is every call returning None?

[zijian_zhang]

+1 ， I am very confused about this and don’t know when I can submit it stably. Continuing like this, I feel that the end of the game will not be a stable state.

[Jiaqi]

@Chris_Deotte @zijian_zhang

The web search should never return None. The “None” returned error was caused by an issue in the web content fetching step in the latest deployment.

The issue has been fixed now. Team is running tests to ensure the reliability of the evaluation. Once confirmed, we will re-evaluate the affected submissions.

Apologies for any confusions or frustrations!