Description of weekly feedback

Insurance pricing game
, ,
#1

In a real insurance market, when you win some policies, you get some feedback.

From tonight onward you will be receiving this market-level feedback in your weekly email! :minidisc:

Note: remember that your model plays in thousands of markets each week (see evaluation metric).

1 Plot and KPIs :bar_chart:

1.1 Plot: Profit Vs Market share

sample_market
sample_market3291Ă—1586 393 KB

In this plot you see your profits and market shares plotted against each other for each market you participated in.

So what are the blue, yellow, and red dots?

We put all of your markets into 20 groups, from your most profitable (group 1) to your least profitable (group 5). The blue, yellow, and red dots represent sample markets from groups 1, 10 and, 20. In other words:

Blue dot. Randomly selected market from your 5% most profitable markets.

Red dot. Randomly selected market from your 5% least profitable markets.

Yellow dot. Randomly selected market from the middle 5% of your markets.

Note that these 3 groupings are quantiles and so each contains the same number of markets.

1.2 The KPIs next to the plot :balance_scale:

For a description of how the KPI are computed please see this post.

2 The tables :floppy_disk:

You will see 6 tables below the plot that look like:

image
image1308Ă—353 66.7 KB

2.1 Table rows :rowing_woman:

First let’s understand what the rows mean:

  1. Policies won most often/sometimes/rarely: xx - yy%. Since your model plays in many of markets for the same set of policies, we can compute how frequently you win a specific policy. In other words, if you win the policy PLXXXXX in 20 out of 100 markets, then we can say you win this policy 20% of the time. We call this the conversion rate. We then split your policies based on their conversion rate into up to 3 groups of equal size from most often won to rarely won. The conversion rate range is represented by xx - yy.
  2. Policies won in sample profitable/average/losing market. These represent contracts won in one of the three sample markets represented by dots on your plot (see the section on plots).

2.2 Table columns :classical_building:

These represent summary statistics about the policies that you have won.

  • For continuous columns: the mean value is displayed.
  • For categorical columns: the most common category is displayed. In parenthesis we also show the proportion of that label within the contracts represented by that row. So for example, if you see M (55%) for the gender column on the Policies won in sample profitable market row. That means that of the policies you won in your sample profitable market, the most common label was male, making up 55% of the contracts that you won.

2.3 Data within the tables :writing_hand:

Dataset summary tables

The first 4 tables are related to the dataset features:

  1. Policy summaries
  2. Driver summaries
  3. Vehicle summaries
  4. Location summaries

Financial summary tables

The final two tables are related to the financial performance of your model.

  1. Financials by conversion rate summaries. This table has 5 columns:
    1. Claim frequency. Fraction of contracts that made a claim
    2. Premiums. Mean value of premiums you have won.
    3. Conversion rate. The mean value of the conversion rate.
    4. Profit per policy. This is the mean value of profit per policy weighted by the conversion rate. To give you an idea of how profitable each segment is.
    5. Profit margin. This is the mean value of your profit margin calculated as ( predicted premium - predicted expected claim ) / predicted premium
  2. Financials in sample markets summaries. This table includes detailed information about the 3 sample markets and should be relatively self-explanatory.

If you have any questions or comments please mention them here.

Good luck in the markets!
Ali

9 Likes
Week 3 leaderboard is out!
Profit leaderboard
#2

Hi, this will be valuable feedback i think! However I have a couple of questions:

  1. Am I understanding correctly that this model only wins policies for which there is a driver 2?

  2. In the point 2 often and sometimes have the same interval. Is this an error?

  3. Also am I understanding correctly that in this model case, every policy was at least won in 1.1% of the markets?

  4. Finally, is this based on the second pas which the model is only compared against the 10% best competitors?

2 Likes
#3

Hi @joycelc

Glad to hear :slight_smile: To answer your questions:

  1. Yes it seem so as indicated by the (100%). But that could be distributional feature as well, that the vast majority of the data might have a second driver registered.
  2. It is not an error. We provide up to three groups. Sometimes there are not enough distinct conversion rates to create 3 groupings with equal size, in these cases we default to giving you 2 groups or even 1 group but with useful data. Otherwise it would just be very small and unrepresentative of your performance.
  3. Not every policy. It means that for policies that they did win, they won them in at least 1.1% of the markets. We don’t include information about policies that were never won.
  4. Yes! All data that we provide to you is always based on the second pass.
3 Likes
#4

Hi alfarzan, it’s great you are providing more details each week but have you already sent the feedback for week 3? I haven’t got the email (I’ve verified all folders in my mail box).

#5

Nope! not yet.

Since this is the first week we are sending it out we are sense-checking the values a bit more rigerously.
They should be rolling out from tonight :incoming_envelope:

#6

Hi @alfarzan
Could you explain me how market share and coversion rate differ?

From the information I found:

MARKET SHARE:This is the mean value of the market-share in terms of the number of policies you have won

CONVERSION RATE:how frequently you win a specific policy

Shouldn’t be similar in the “Financials In Sample Markets Summaries”?
I have totally different number in week4.
For example if I have a conversion rate of 100% then should I also have a market share of 100%?

I have also a second question, I have feedback from week2 by email and feedback of week4 in the leaderboard…but what about week3?

#7

Hi @pejo92

Yes they are different.

Let’s take the “Financials in Sample Markets Summaries” example and let’s look at one row in that table. Say the “sample profitable market”.

  1. Market share. This is the number of policies that you won in that sample profitable market, divided by the total number of policies up for grabs.
  2. Conversion rate. This is how often you normally win the policies that you ended up winning in the sample profitable market. So if you have a conversion rate of 100% here, that means that you always win the policies that you have won in this market. But that says nothing about your market share. You could only win one policy and still have a 100% conversion rate for that one policy.

The market share in one sample market and the over all conversion rate of a set of policies are not interchangeable. Is that a little clearer?

#8

About your second question on the week 3 feedback: that was sent out last week in your emails. Have you not received it?

And if not, can you give me your submission ID from that week’s leaderboard so I can check?

#9

Thank you for the quick answer.

For question number 1 about market share vs conversion now is super clear yes.

For question number 2 the submission ID is 113844. I’ve checked again and I have no email for it.

1 Like
#10

That’s quite strange, can you also check with your teammate if they have received the email? I am checking with our team as well.

In the meantime I have sent you a private message with the feedback so you can conduct your analysis :rocket: