Ranking not showing up

aalok_devkota · February 7, 2021, 2:05pm

My submission did not get scored. I do not see on the rankings. Submission #[119847]

alfarzan · February 7, 2021, 3:15pm

It seems your submission failed on one of the leaderboard sets. This should have been clear upon your submission but it hasn’t for some reason. I will investigate further and get back to you here.

In the meantime I have sent you your error traceback privately as well.

aalok_devkota · February 7, 2021, 7:09pm

I have resubmitted as I am not getting any error on my side. The submission number is 120039. If this does not work, can I just get the one from Week 6 scored and I will try to fix the issue next week.

zhisheng_wang · February 8, 2021, 2:45am

same issue for my submission 119142, could you pls check for me @alfarzan?
thanks!

jyotish · February 8, 2021, 3:37am

Hello @zhisheng_wang

The submission failed to generate prices during the evaluation. This error should have been caught during RMSE leaderboard evaluation. We are looking into this.

Also, shared the error traceback privately.

aalok_devkota · February 8, 2021, 5:00am

Looks like similar issue. I was able to submit and it was graded and I received a RMSE score, When I run it on jupyter notbook it works fine and i did not receive any error.

aalok_devkota · February 8, 2021, 5:21am

zhisheng_wang · February 8, 2021, 6:24am

Hi, @jyotish I know the reason now, when calculating new features it lost one column in profit evaluation since there are not enough data comparing to rmse calculation

aalok_devkota · February 8, 2021, 8:04am

@zhisheng_wang, so how did you solve the issue when adding a new field?

MathB · February 8, 2021, 10:41am

Same question for submission 119612.

zhisheng_wang · February 9, 2021, 2:42am

just prefix the features in the beginningg of the preprocess
for example: FEATURES=[‘x’,‘y’] then df[FEATURES]=0

aalok_devkota · February 9, 2021, 5:49pm

Any update on the issue? Is it possible to score the Week 6 submission, I wanted to see how I did so that I can modify/update the model.

alfarzan · February 11, 2021, 12:22pm

Apologies for the delay everyone!

For all the submissions listed here the problem was either:

Feature mismatch between the expected data and the data that comes out of the proeprocessor. Or
The model generated NaNs.

Both of these should generally be caught upon submission and so if you fix these issues I may be able to privately generate the leaderboards for you, and if the rankings are not substantially different, give you the resulting feedback

To address the questions:

@aalok_devkota I believe I replied privately, but the error remains the same. I suggest hardcoding the features as @zhisheng_wang has mentioned.
@MathB this submission generated NaN values. Generally a check for this before sending out the prices would help fix the issue. We will also be checking from our side as well.