Eligibility for weekly evaluation

Insurance pricing game
#1

Hi,

My last submission passes the non-negative training profit test and all prices are positive (>0). But it gets the mention

ELIGIBLE_FOR_WEEKLY_EVALUATION=FALSE.

I can’t understand the problem.

Could you please specify what to check ?

Thank’s for your help.

(my submission # 119805)

#2

Hi @AlainBugnon

This flag is specifically checking whether your submission passes the training profit rule. What it means is that the sum of your premiums does not exceed to the sum of the claims in the training data. You should hike up the prices a bit :slight_smile:

The purpose of this rule is to prevent models that would effectively “ruin” a market by pricing at extremely low levels and obtaining unrealistically high market-share while losing lots of money.

#3

Hi @alfarzan

thank you for your prompt response. I raised the price a bit and it’s now OK. :grinning:

However, I am sure that the test was OK before as well. Is it possible that I did the test with the 50’000 cut :hocho:and you without the cut ?

I have understood the need to guarantee a representative market by testing the average of the premiums in relation to the claims. Nevertheless, it is questionable whether this test is really optimal. An insurer can offer an unrealistic dumping premium on a segment, which will be sold, when it can be compensated by an unrealistically high premium in an other segment, which will not be sold. The respect of same average is therefore theorical and not reflected in reality. Another option could be:

  • no imposed price constraint or the frustrating 5% market capture
  • calculation of results by simulation over the entire portfolio
  • elimination of the insurers with the most losses (e.g. 10% share, without eliminating the insurers with no market since they are possibly stopped from this unhealthy insurers)
  • second round in the same way with insurance companies that are not eliminated
  • and so on until the final for the top 10% as planned on the rules.

At the end, this could also be done by elimination until all insurers are in the positive.

This would ultimately guarantee a representative market without imposing any constraints (except a competitiv premium… but against healthy companies).

It would also provide an indicator of the timing of exit from competition.

Perhaps a proposal to be transferred to the post “Ideas” :bulb: to have the opinion of the favourites that emerge in this competition.

1 Like
#4

Hi @AlainBugnon

I will investigate the question regarding the cut shortly :thinking:

The elimination style proposal :ok_hand:

The elimination style is a very interesting proposal actually! :trophy:

When we started thinking about this, some version of what you suggest was initially our picked metric. We ultimately decided against this because of two reasons:

  1. Inclusiveness. Many people would be left out of markets and would not appear on the leaderboards if we go with this elimination style, specially at the very beginning.
  2. Feedback. We wanted people to be able to receive some feedback about their performance each time that is is not very complex to understand. I think we’ve hit a happy medium with this, but we can still improve.

What is the 5% rule? :pie:

From initial experiments before launch, we knew that the markets would need time to calibrate and become profitable. In the interim, that would mean that most models would fall victim to adverse selection (and have negative profits). In such a case, if you didn’t participate at all (profit of zero) you could do well on the leaderboard. Hence, enter the 5% rule. But now we can see that markets are actually more and more profitable every week :muscle:

To clarify, the 5% rule is not a 5% market share rule. The rule is that a model must have a nonzero market share in at least 5% of markets. This was only an issue in the very first week where a few people were not participating. In weeks 2 onward you can see only a very small number of models (<5) end up not participating and failing to enter the leaderboard.

Note that these models are likely extremely noncompetitive in their pricing, hence they never offer the cheapest price!

The spirit of the nonnegative training profit rule :straight_ruler:

The idea for the training profit rule is to help everyone calibrate their prices somehow before the markets are “stable”. I believe now we have reached that stability more or less. So this rule is not very informative. However, it still creates a healthy barrier against a race to the bottom.

Now, you mention a workaround:

An insurer can offer an unrealistic dumping premium on a segment, which will be sold, when it can be compensated by an unrealistically high premium in an other segment, which will not be sold.

Technically speaking you could do something like that if you wanted to, but it would defeat the purpose of the sense check we have in place. But let’s say it does happen with some models.

What happens then?

In the worst case scenario, a lot of models end up employing this strategy, then what you will observe is that in almost every random market there are one or two participant models with extremely low prices (like you mention) and they will win close to 100% market-share leaving others to not participate.

So what we as the organisers would see is market participation (i.e. number of markets you win at least one policy) dropping for many people.

To mitigate your worries, what we actually see is market participation has been increasing, with almost everyone participating in 100% of markets :raised_hands:

But what if it happens later?

If we get close to that situation and participation in markets for models drops, then we have a situation where:

  1. Those “bad” low-price models will win a lot of market share most of the time. And lose a LOT of money pushing them to the bottom of the leaderboard.
  2. Those normal models will still win some policies (e.g. in markets where the bad models are not present) and because the prices are better the will be more profitable being pushed up the leaderboard.

So from your perspective it will still not be a terrible situation in the worse case.

We are aware of this risk and are constantly monitoring for it, so far, people are playing normally like a real company would :slight_smile: One or two models have tried doing what you mention but they are almost at the very bottom of the leaderboard.

1 Like
#5

Hi @alfarzan

thank you for the precisions and explanations of the selected rules. :+1:

In all cases, the competition reproduces the constraint of profit and competitiveness in a market with a strong random component. This is why it is very interesting on an actuarial and strategic level.

1 Like