How to use this notebook 📝¶
- Copy the notebook. This is a shared template and any edits you make here will not be saved. You should copy it into your own drive folder. For this, click the "File" menu (top-left), then "Save a Copy in Drive". You can edit your copy however you like.
- Link it to your AICrowd account. In order to submit your code to AICrowd, you need to provide your account's API key (see "Configure static variables" for details).
Stick to the function definitions. The submission to AICrowd will look for the pre-defined function names:
Anything else you write outside of these functions will not be part of the final submission (including constants and utility functions), so make sure everything is defined within them, except for:
- Define your preprocessing. In addition to the functions above, anything in the cell labelled "Define your data preprocessing" will also be imported into your final submission.
Your pricing model 🕵️¶
In this notebook, you can play with the data, and define and train your pricing model. You can then directly submit it to the AICrowd, with some magic code at the end.
Setup the notebook 🛠¶
!bash <(curl -sL https://gitlab.aicrowd.com/jyotish/pricing-game-notebook-scripts/raw/master/python/setup.sh) from aicrowd_helpers import *
⚙️ Installing AIcrowd utilities... Running command git clone -q https://gitlab.aicrowd.com/yoogottamk/aicrowd-cli /tmp/pip-req-build-wrp81y3p ✅ Installed AIcrowd utilities 💾 Downloading training data... ✅ Downloaded training data
class Config: TRAINING_DATA_PATH = 'training.csv' MODEL_OUTPUT_PATH = 'model.pkl' AICROWD_API_KEY = '' # You can get the key from https://aicrowd.com/participants/me ADDITIONAL_PACKAGES = [ 'numpy', # you can define versions as well, numpy==0.19.2 'pandas', 'sklearn', ]
Download dataset files 💾¶
Import here all the packages you need to define your model. You will need to include all of these packages in
Config.ADDITIONAL_PACKAGES for your code to run properly once submitted.
%%track_imports import numpy as np import pandas as pd import pickle
from global_imports import * # do not change this line
Loading the data 📲¶
df = pd.read_csv(Config.TRAINING_DATA_PATH) X_train = df.drop(columns=['claim_amount']) y_train = df['claim_amount']
How does the data look like? 🔍¶
166662 0.0 201461 0.0 174658 0.0 72559 0.0 Name: claim_amount, dtype: float64
Training the model 🚀¶
You must first define your first function:
fit_model. This function takes training data as arguments, and outputs a "model" object -- that you define as you wish. For instance, this could be an array of parameter values.
Define your data preprocessing¶
You can add any class or function in this cell for preprocessing. Just make sure that you use the functions here in the
predict_premium functions if necessary. italicised text
%%aicrowd_include # This magical command saves all code in this cell to a utils module. # include your preprocessing functions and classes here.
from utils import * # do not change this line
Define the training logic¶
def fit_model(X_raw, y_raw): """Model training function: given training data (X_raw, y_raw), train this pricing model. Parameters ---------- X_raw : Pandas dataframe, with the columns described in the data dictionary. Each row is a different contract. This data has not been processed. y_raw : a Numpy array, with the value of the claims, in the same order as contracts in X_raw. A one dimensional array, with values either 0 (most entries) or >0. Returns ------- self: this instance of the fitted model. This can be anything, as long as it is compatible with your prediction methods. """ # TODO: train your model here. return None # By default, nothing was trained, return nothing.
Train your model¶
trained_model = fit_model(X_train, y_train)
Important note: your training code should be able to run in under 10 minutes (since this notebook is re-run entirely on the server side).
If you run into an issue here we recommend using the zip file submission (see the challenge page). In short, you can simply do this by copy-pasting your
predict_premium functions to the
Note that if you want to perform extensive cross-validation/hyper-parameter selection, it is better to do them offline, in a separate notebook.
Saving your model¶
You can save your model to a file here, so you don't need to retrain it every time.
def save_model(model_path): with open(model_path, 'wb') as target_file: pickle.dump(trained_model, target_file)
If you need to load it from file, you can use this code:
def load_model(model_path): with open(model_path, 'rb') as target: return pickle.load(target)
trained_model = load_model(Config.MODEL_OUTPUT_PATH)
Predicting the claims 💵¶
The second function,
predict_expected_claim, takes your trained model and a dataframe of contracts, and outputs a prediction for the (expected) claim incurred by each contract. This expected claim can be seen as the probability of an accident multiplied by the cost of that accident.
This is the function used to compute the RMSE leaderboard, where the model best able to predict claims wins.
def predict_expected_claim(model, X_raw): """Model prediction function: predicts the expected claim based on the pricing model. This functions estimates the expected claim made by a contract (typically, as the product of the probability of having a claim multiplied by the expected cost of a claim if it occurs), for each contract in the dataset X_raw. This is the function used in the RMSE leaderboard, and hence the output should be as close as possible to the expected cost of a contract. Parameters ---------- model: a Python object that describes your model. This can be anything, as long as it is consistent with what `fit` outpurs. X_raw : Pandas dataframe, with the columns described in the data dictionary. Each row is a different contract. This data has not been processed. Returns ------- avg_claims: a one-dimensional Numpy array of the same length as X_raw, with one expected claim per contract (in same order). These expected claims must be POSITIVE (>0). """ # TODO: estimate the expected claim of every contract. return np.full( (len(X_raw.index),), 1000 ) # Estimate that each contract will cost 1000 (bad, do not do this).
To test your function, run it on your training data:
array([1000, 1000, 1000, ..., 1000, 1000, 1000])
Pricing contracts 💰💰¶
The third and final function,
predict_premium, takes your trained model and a dataframe of contracts, and outputs a price for each of these contracts. You are free to set this prices however you want! These prices will then be used in competition with other models: contracts will choose the model offering the lowest price, and this model will have to pay the cost if an accident occurs.
This is the function used to compute the profit leaderboard: your model will participate in many markets of size 10, populated by other participants' model, and we compute the average profit of your model over all the markets it participated in.
def predict_premium(model, X_raw): """Model prediction function: predicts premiums based on the pricing model. This function outputs the prices that will be offered to the contracts in X_raw. premium will typically depend on the average claim predicted in predict_average_claim, and will add some pricing strategy on top. This is the function used in the average profit leaderboard. Prices output here will be used in competition with other models, so feel free to use a pricing strategy. Parameters ---------- model: a Python object that describes your model. This can be anything, as long as it is consistent with what `fit` outpurs. X_raw : Pandas dataframe, with the columns described in the data dictionary. Each row is a different contract. This data has not been processed. Returns ------- prices: a one-dimensional Numpy array of the same length as X_raw, with one price per contract (in same order). These prices must be POSITIVE (>0). """ # TODO: return a price for everyone. return predict_expected_claim(model, X_raw) # Default: price at the pure premium with no pricing strategy.
To test your function, run it on your training data.
prices = predict_premium(trained_model, X_train)
Profit on training data¶
In order for your model to be considered in the profit competition, it needs to make nonnegative profit over its training set. You can check that your model satisfies this condition below:
print('Income:', prices.sum()) print('Losses:', y_train.sum()) if prices.sum() < y_train.sum(): print('Your model is invalid: it loses money on its training data!') else: print('Your model passes the test!')
Income: 228216000 Losses: 26057988.080000006 Your model passes the test!
Ready? Submit to AIcrowd 🚀¶
If you are satisfied with your code, run the code below to send your code to the AICrowd servers for evaluation! This requires the variable
trained_model to be defined by your previous code.
Make sure you have included all packages needed to run your code in the "Packages" section.
🚀 Preparing to submit... ❗️ AICROWD_API_KEY should not be empty
An exception has occurred, use %tb to see the full traceback. SystemExit: 1
/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py:2890: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D. warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)