Instructions to include non-standard packages

#1

I am using a package directly from git (as it is not yet there in PyPI or conda).

When I do a conda export it doesn’t include the package in the list, is there an alternative way to fix this?

Link to the issue.

Thanks!

#2

[Edit: This won’t work]
A suggestion (not sure if it will work):
Did you try cloning the repo and installing (before run.py is invoked in run.sh), make sure git is installed as well
For eg,
Building torchvision from source (inside run.sh):

git clone https://github.com/pytorch/vision
cd vision
python setup.py install
cd …
python3 run.py

#3

The container would not have network access, hence cloning the repository would not work.
But you can always install everything you need in an Anaconda environment, and dump the environment as specified here

#4

The Anaconda environment dump does not seem to work for packages installed directly from git. Can you suggest other alternative ways to handle this please?

#5

For packages installed directly from git, you should be able to manually replace the relevant line (from the pip subsection in the environment.yml), with something like :

- git+https://github.com/google-research/disentanglement_lib.git

Also, do remember to build the image locally and see if it worked.
This is unfortunately a bug with anaconda, and we will see if we can do anything about it in the short term.

#6

I tried the same, it throws git not found error.
Link to the logs

#7

@mohanty how to install git before running the conda environment.yml file?

#8

You can include a file called as apt.txt and include “git” as its content.

Please try using aicrowd-repo2docker to build the images before submitting.

1 Like
#9

I did the same, the apex package needs access to GPU to run, I have set gpu: true in the aicrowd.json file, anything else to be done?

#10

@mohanty I somehow need to fix this to make my final submission :grinning:

Any help will be appreciated :hugs:

#11

@mohanty can you please look into this?