How are you doing? (Part 2)

Thank you all for the comments!

@Calico Yes I was applying a loading, however same method for all models… simply multiplying by 1.15.

@simon_coulombe I’ll keep working on this, but so far, I’m seeing an average loss ratio of 120% when there’s absolutely no loading. Probably sensitive to the number of models in compete? To be continued.

@alfarzan First time I’m hearing this kitchen-sink nomenclature. Funny. Will look into your suggestion. Didn’t think of creating bad model on purpose.

@tom_snowdon Thanks for the suggestion, and results are below…

 

Disclosure

To produce the charts earlier, I left R and went into Excel…
…and I got fooled by Excel’s =rank() function…
Did you know Microsoft decided on returning a decreasing order by default?!. :roll_eyes:

Now I know, and so do you.
Some of the charts were (very) wrong, sorry for that.

 

Profit (same as before)
profit_rank
 

RMSE, MAE revisited, and GINI :bar_chart:

Added standard-deviation row-wise, and the average of the stds.

Gini seems much more sensitive to the seed than the other two metrics.
@tom_snowdon what’s your take on that?
 

Correlations

correlation_test

 

Some Thoughts

  • With the fixed chart, Model L is safe from my relentless bashing.
    It’s just an overall poor model.

  • Still looking for a metric that could be better correlated with profit. Open to suggestions!

  • The previous erroneous chart did influence my decision for this week leaderboard.
    Scored 99th position lol.
    Thanks Microsoft.

6 Likes