Do models with lower RMSE perform better for the Profit leaderboard? Mapping Average profit vs RMSE

Hi!

This is Vrushank from AIcrowd. We played around with the week 2 leaderboard data little bit, and came up with some charts that the you might be interested in.

Do models that have lower RMSE perform better for the Profit leaderboard? Or do you have to optimize for market profit? Check out what the trend looks below.


Thoughts?

2 Likes

Considering that you can deviate from the baseline strategy of setting price = expected * 2, this is not unexpected