(Solved) Running test_submission.py runs into _pickle.UnpicklingError: invalid load key, 'v'

Hy, please help me out <3

When I run test_submission.py i get following error. This happens locally on my ubuntu20.04 laptop, and also when I start within a docker container running
docker run -it -v pwd:/home/aicrowd --gpus=‘all’ fairnle/challenge:dev

and also when I use the provided Dockerfile.
This is so strange, I have python 3.8 on my ubuntu 20.04 laptop.

Can you give me a hint how to fix this?

(venv) master@wx-87:~/PycharmProjects/justpaulsai$ python test_submission.py 
Traceback (most recent call last):
  File "test_submission.py", line 37, in <module>
    evaluate()
  File "test_submission.py", line 25, in evaluate
    agent = Agent(num_envs, batched_env.num_actions)
  File "/home/master/PycharmProjects/justpaulsai/agents/torchbeast_agent.py", line 20, in __init__
    self.model = load_model(MODEL_DIR, self.device)
  File "/home/master/PycharmProjects/justpaulsai/nethack_baselines/torchbeast/models/__init__.py", line 54, in load_model
    checkpoint_states = torch.load(flags.checkpoint, map_location=device)
  File "/home/master/PycharmProjects/justpaulsai/venv/lib/python3.8/site-packages/torch/serialization.py", line 608, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/master/PycharmProjects/justpaulsai/venv/lib/python3.8/site-packages/torch/serialization.py", line 777, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

This is the docker log:

(venv) master@wx-87:~/PycharmProjects/justpaulsai$ docker run -it -v `pwd`:/home/aicrowd --gpus='all' fairnle/challenge:dev
root@dd2e492df141:/home/aicrowd# python -V
Python 3.8.5
root@dd2e492df141:/home/aicrowd# python test_submission.py 
Traceback (most recent call last):
  File "test_submission.py", line 37, in <module>
    evaluate()
  File "test_submission.py", line 25, in evaluate
    agent = Agent(num_envs, batched_env.num_actions)
  File "/home/aicrowd/agents/torchbeast_agent.py", line 20, in __init__
    self.model = load_model(MODEL_DIR, self.device)
  File "/home/aicrowd/nethack_baselines/torchbeast/models/__init__.py", line 54, in load_model
    checkpoint_states = torch.load(flags.checkpoint, map_location=device)
  File "/opt/conda/lib/python3.8/site-packages/torch/serialization.py", line 608, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/opt/conda/lib/python3.8/site-packages/torch/serialization.py", line 777, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.
root@dd2e492df141:/home/aicrowd#

Ok wow after 8 hours I fixed it <3

This is a problem with git lfs or so… I downloaded my checkpoints directly from the repository via my web-browser. Now it works :slight_smile:

It feel a bit crazy that I found this solution…

1 Like

Hi @paul_puntschart,

Happy that you managed to resolve it! :smiley:

Meanwhile, in case anyone is reading this in the future. :arrow_heading_down:

This might be happening because the file isn’t properly uploaded via git-lfs (or not uploaded at all). Please browse your repository in the browser to verify the file. You can do the following to fix the problem:

  • reupload by git lfs push origin master
  • verify .gitattributes if your lfs entry is mentioned (at repository root)
  • clone repository freshly in new folder, and try uploading from that (in case your current repository’s settings are messed up locally)

[More about git-lfs]

1 Like