I will investigate the question regarding the cut shortly
The elimination style proposal
The elimination style is a very interesting proposal actually!
When we started thinking about this, some version of what you suggest was initially our picked metric. We ultimately decided against this because of two reasons:
Inclusiveness. Many people would be left out of markets and would not appear on the leaderboards if we go with this elimination style, specially at the very beginning.
Feedback. We wanted people to be able to receive some feedback about their performance each time that is is not very complex to understand. I think we’ve hit a happy medium with this, but we can still improve.
What is the 5% rule?
From initial experiments before launch, we knew that the markets would need time to calibrate and become profitable. In the interim, that would mean that most models would fall victim to adverse selection (and have negative profits). In such a case, if you didn’t participate at all (profit of zero) you could do well on the leaderboard. Hence, enter the 5% rule. But now we can see that markets are actually more and more profitable every week
To clarify, the 5% rule is not a 5% market share rule. The rule is that a model must have a nonzero market share in at least 5% of markets. This was only an issue in the very first week where a few people were not participating. In weeks 2 onward you can see only a very small number of models (<5) end up not participating and failing to enter the leaderboard.
Note that these models are likely extremely noncompetitive in their pricing, hence they never offer the cheapest price!
The spirit of the nonnegative training profit rule
The idea for the training profit rule is to help everyone calibrate their prices somehow before the markets are “stable”. I believe now we have reached that stability more or less. So this rule is not very informative. However, it still creates a healthy barrier against a race to the bottom.
Now, you mention a workaround:
An insurer can offer an unrealistic dumping premium on a segment, which will be sold, when it can be compensated by an unrealistically high premium in an other segment, which will not be sold.
Technically speaking you could do something like that if you wanted to, but it would defeat the purpose of the sense check we have in place. But let’s say it does happen with some models.
What happens then?
In the worst case scenario, a lot of models end up employing this strategy, then what you will observe is that in almost every random market there are one or two participant models with extremely low prices (like you mention) and they will win close to 100% market-share leaving others to not participate.
So what we as the organisers would see is market participation (i.e. number of markets you win at least one policy) dropping for many people.
To mitigate your worries, what we actually see is market participation has been increasing, with almost everyone participating in 100% of markets
But what if it happens later?
If we get close to that situation and participation in markets for models drops, then we have a situation where:
- Those “bad” low-price models will win a lot of market share most of the time. And lose a LOT of money pushing them to the bottom of the leaderboard.
- Those normal models will still win some policies (e.g. in markets where the bad models are not present) and because the prices are better the will be more profitable being pushed up the leaderboard.
So from your perspective it will still not be a terrible situation in the worse case.
We are aware of this risk and are constantly monitoring for it, so far, people are playing normally like a real company would One or two models have tried doing what you mention but they are almost at the very bottom of the leaderboard.