Description of weekly feedback

Insurance pricing game
, ,
#1

In a real insurance market, when you win some policies, you get some feedback.

From tonight onward you will be receiving this market-level feedback in your weekly email! :minidisc:

Note: remember that your model plays in thousands of markets each week (see evaluation metric).

1 Plot and KPIs :bar_chart:

1.1 Plot: Profit Vs Market share

sample_market
sample_market3291×1586 393 KB

In this plot you see your profits and market shares plotted against each other for each market you participated in.

So what are the blue, yellow, and red dots?

We put all of your markets into 20 groups, from your most profitable (group 1) to your least profitable (group 5). The blue, yellow, and red dots represent sample markets from groups 1, 10 and, 20. In other words:

Blue dot. Randomly selected market from your 5% most profitable markets.

Red dot. Randomly selected market from your 5% least profitable markets.

Yellow dot. Randomly selected market from the middle 5% of your markets.

Note that these 3 groupings are quantiles and so each contains the same number of markets.

1.2 The KPIs next to the plot :balance_scale:

For a description of how the KPI are computed please see this post.

2 The tables :floppy_disk:

You will see 6 tables below the plot that look like:

image
image1308×353 66.7 KB

2.1 Table rows :rowing_woman:

First let’s understand what the rows mean:

  1. Policies won most often/sometimes/rarely: xx - yy%. Since your model plays in many of markets for the same set of policies, we can compute how frequently you win a specific policy. In other words, if you win the policy PLXXXXX in 20 out of 100 markets, then we can say you win this policy 20% of the time. We call this the conversion rate. We then split your policies based on their conversion rate into up to 3 groups of equal size from most often won to rarely won. The conversion rate range is represented by xx - yy.
  2. Policies won in sample profitable/average/losing market. These represent contracts won in one of the three sample markets represented by dots on your plot (see the section on plots).

2.2 Table columns :classical_building:

These represent summary statistics about the policies that you have won.

  • For continuous columns: the mean value is displayed.
  • For categorical columns: the most common category is displayed. In parenthesis we also show the proportion of that label within the contracts represented by that row. So for example, if you see M (55%) for the gender column on the Policies won in sample profitable market row. That means that of the policies you won in your sample profitable market, the most common label was male, making up 55% of the contracts that you won.

2.3 Data within the tables :writing_hand:

Dataset summary tables

The first 4 tables are related to the dataset features:

  1. Policy summaries
  2. Driver summaries
  3. Vehicle summaries
  4. Location summaries

Financial summary tables

The final two tables are related to the financial performance of your model.

  1. Financials by conversion rate summaries. This table has 5 columns:
    1. Claim frequency. Fraction of contracts that made a claim
    2. Premiums. Mean value of premiums you have won.
    3. Conversion rate. The mean value of the conversion rate.
    4. Profit per policy. This is the mean value of profit per policy weighted by the conversion rate. To give you an idea of how profitable each segment is.
    5. Profit margin. This is the mean value of your profit margin calculated as ( predicted premium - predicted expected claim ) / predicted premium
  2. Financials in sample markets summaries. This table includes detailed information about the 3 sample markets and should be relatively self-explanatory.

If you have any questions or comments please mention them here.

Good luck in the markets!
Ali

9 Likes
Week 3 leaderboard is out!
Profit leaderboard
#2

Hi, this will be valuable feedback i think! However I have a couple of questions:

  1. Am I understanding correctly that this model only wins policies for which there is a driver 2?

  2. In the point 2 often and sometimes have the same interval. Is this an error?

  3. Also am I understanding correctly that in this model case, every policy was at least won in 1.1% of the markets?

  4. Finally, is this based on the second pas which the model is only compared against the 10% best competitors?

2 Likes
#3

Hi @joycelc

Glad to hear :slight_smile: To answer your questions:

  1. Yes it seem so as indicated by the (100%). But that could be distributional feature as well, that the vast majority of the data might have a second driver registered.
  2. It is not an error. We provide up to three groups. Sometimes there are not enough distinct conversion rates to create 3 groupings with equal size, in these cases we default to giving you 2 groups or even 1 group but with useful data. Otherwise it would just be very small and unrepresentative of your performance.
  3. Not every policy. It means that for policies that they did win, they won them in at least 1.1% of the markets. We don’t include information about policies that were never won.
  4. Yes! All data that we provide to you is always based on the second pass.
3 Likes
#4

Hi alfarzan, it’s great you are providing more details each week but have you already sent the feedback for week 3? I haven’t got the email (I’ve verified all folders in my mail box).

#5

Nope! not yet.

Since this is the first week we are sending it out we are sense-checking the values a bit more rigerously.
They should be rolling out from tonight :incoming_envelope: