Do models with lower RMSE perform better for the Profit leaderboard? Mapping Average profit vs RMSE

Insurance pricing game
#1

Hi!

This is Vrushank from AIcrowd. We played around with the week 2 leaderboard data little bit, and came up with some charts that the you might be interested in.

Do models that have lower RMSE perform better for the Profit leaderboard? Or do you have to optimize for market profit? Check out what the trend looks below.

all
all2632×1637 148 KB

top-10
top-102252×1376 145 KB

Thoughts?

2 Likes
#2

Considering that you can deviate from the baseline strategy of setting price = expected * 2, this is not unexpected