Changing the Dataset in local_evaluation.py?

[master-pc9845x]

The dataset by default in the local_evaluation.py file is development.json.

If I need to test my model, can I change the file name into whatever dataset that is in the local_evaluation into whatever dataset that I use?

And if I create a new file to test my model, then do I have to implement the overall score calculation by scratch?

[yilun_jin]

They are basically your choice. The development.json is sampled from the real test data, so they would be more indicative.

If you follow the format in development.json, I don’t think you need to implement the score calculation from scratch.