Storage in submission

Amazon KDD Cup 2024: Multi-Task Online Shopping Ch
#1

Hi.

I tried git clone(lfs) in Dockerfile.
because I can’t upload using git lfs for huggingface models.
I face storage problem during submission.
Build Packages And Env: The node was low on resource: ephemeral-storage. Container wait was using 84Ki, which exceeds its request of 0.

I would like to know submission environment.

  1. What limit of storage in Docker?(100GB?, 150GB? or more?)
  2. Do Docker and the files to be submitted (Gitlab Repository files) use different storage?

@yilun_jin

debug log(admin)
https://dashboard.aicrowd.com/explore?orgId=1&left={"datasource":"[AWS%20Internal]%20Loki","queries":[{"refId":"A","expr":"{aicrowd_com_submission_id%3D\"251691\",%20aicrowd_com_logs%3D\"true\",%20container%3D\"main\"}"}],"range":{"from":"now-2d","to":"now"}}

#2

submission link:AIcrowd

#3

I was told that the issue is caused by excessively large docker images. However, it seems that you correctly use .dockerignore, and the model checkpoints are not larger than the limit (25G).

Asking @spmohanty for help as I don’t spot any errors here.

#4

This is caused by: AIcrowd

where you are trying to include the whole model from huggingface into your docker image.
Is there any specific issue you are facing in terms of checking in the model weights to your repository ?

#5

@yilun_jin @aicrowd_team
Thank you for reply.
I modified my Dockerfile(remove .git), but validation is failed.

Is there any specific issue you are facing in terms of checking in the model weights to your repository ?

yes, I need to upload 20GB over .safetensor, I try to upload using git lfs, but it’s failed.(timeout…?)
so I use git clone during Docker build
log output is that upload size is 200GB. why…?

if you need more detail, I can tell detail situation using email.

#6

@tereka It is strange that I did succeed in checking in a Mistral-7B without any problem.

I see that your later submissions did succeed, so was the problem solved?

#7

@yilun_jin
No, I would like to upload other files
I can’t upload safetensors(one file have 23GB)

#8

@tereka I am sure whether I understand your question correctly, but if you want to upload a safetensor with 23GB size, please make sure that the total repo size is below 25GB.

We will loosen the limit in the near future, but right now please stick to the limit.