Conda env creation errors...UPDATED: later EOF error when running evaluator

Flatland
,
#1

I am following the 10 minute instructions and I get an error message such as:
ResolvePackageNotFound:

  • libuuid=2.32.1
  • ncurses=6.1
  • readline=7.0
  • libedit=3.1.20181209

I tried copying these to under pip on the yaml file such as:

  • pip:
    • libuuid==2.32.1
    • ncurses==6.1
    • readline==7.0
    • libedit==3.1.20181209
      Then I get following error:
      Pip subprocess error:
      ERROR: Could not find a version that satisfies the requirement libuuid==2.32.1 (from -r C:\Users\Memo\Desktop\NEURips_Flatland\neurips2020-flatland-starter-kit\condaenv.zkqzvdv3.requirements.txt (line 1)) (from versions: 1.0.0)
      ERROR: No matching distribution found for libuuid==2.32.1 (from -r C:\Users\Memo\Desktop\NEURips_Flatland\neurips2020-flatland-starter-kit\condaenv.zkqzvdv3.requirements.txt (line 1))

Then I remove the version number with libuuid such as:
pip:
- libuuid
- ncurses==6.1
- readline==7.0
- libedit==3.1.20181209

I get the exact same error as above but this time with “ncurses” so I remove version numbers from ncurses as well and still get the same “ERROR: Could not find a version that satisfies the requirement ncurses”

I am using windows 10 64 bit. What am I supposed to do? Any help is greatly appreciated

1 Like
#2

Hi @MemoAI,

Thanks for pointing it out. The packages you mentioned are not available for Windows. (Ex: libuuid)

Let us check and update the environment.yml to be compatible with windows. Meanwhile, do try to create conda environment removing those unresolved packages, it should ideally work.

1 Like
#3

Actually, the new Flatland release has much fewer dependencies, so you can ignore the environment.yml file.

You can simply create a new conda environment, then install Flatland with pip:
pip install flatland-rl

You can even skip conda altogether. However conda makes it easier to package your solution if you want to use specific dependencies, and you need to keep the environment.yml in your submission repository in any case.

1 Like
#4

@shivam , @MasterScrat ,

Thank you I am currently trying that exact solution. However it has been waiting on:

Collecting flatland-rl
Using cached https://files.pythonhosted.org/packages/e1/6f/16b4085907c4eef5e908840e26a7fcf85d4edc03039bec1bd1119eb81db3/flatland-rl-2.2.1.tar.gz

for about 10 minutes now. is this normal?
could it take such a long time?

#5

Hey @MemoAI , please check your internet connection or retry doing it (seems to be stuck on download from pypi), it shouldn’t happen.

1 Like
#6

Ok I will retry and will let you know

#7

It did also take very long for me on Windows but eventually worked, not sure why.

1 Like
#8

I am trying on two computers. I manually create conda env and pip install flatland-rl

On one computer it is taking forever and didn’t see it finish.

On the other it installed with following errors:

ERROR: ludwig 0.2.1 requires tensorflow==1.14.0, which is not installed.
ERROR: tox 3.15.2 has requirement six>=1.14.0, but you’ll have six 1.12.0 which is incompatible.
ERROR: python-gitlab 2.3.1 has requirement requests>=2.22.0, but you’ll have requests 2.18.4 which is incompatible.

I have pip installed six and requests appropriate versions however on my pip list I see tensorflow 1.14 gpu installed. Do I need cpu version only? will gpu version work? and work faster?

#9

Currently I am at where @jakkes was on discord and I am getting the same error when trying to run the evaluator :

-envs/
[‘Test_5\Level_1.pkl’, ‘Test_2\Level_0.pkl’, ‘Test_9\Level_0.pkl’, ‘Test_8\Level_1.pkl’, ‘Test_9\Level_1.pkl’, ‘Test_4\Level_0.pkl’, ‘Test_11\Level_0.pkl’, ‘Test_7\Level_1.pkl’, ‘Test_3\Level_1.pkl’, ‘Test_0\Level_1.pkl’, ‘Test_10\Level_1.pkl’, ‘Test_12\Level_0.pkl’, ‘Test_13\Level_0.pkl’, ‘Test_11\Level_1.pkl’, ‘Test_1\Level_0.pkl’, ‘Test_12\Level_1.pkl’, ‘Test_0\Level_0.pkl’, ‘Test_1\Level_1.pkl’, ‘Test_7\Level_0.pkl’, ‘Test_5\Level_0.pkl’, ‘Test_10\Level_0.pkl’, ‘Test_4\Level_1.pkl’, ‘Test_8\Level_0.pkl’, ‘Test_6\Level_1.pkl’, ‘Test_6\Level_0.pkl’, ‘Test_3\Level_0.pkl’, ‘Test_2\Level_1.pkl’, ‘Test_13\Level_1.pkl’]
Listening at : flatland-rl::FLATLAND_RL_SERVICE_ID::commands
Traceback (most recent call last):
File “c:\users\memo\appdata\local\programs\python\python36\lib\runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “c:\users\memo\appdata\local\programs\python\python36\lib\runpy.py”, line 85, in run_code
exec(code, run_globals)
File "C:\Users\Memo\AppData\Local\Programs\Python\Python36\Scripts\flatland-evaluator.exe_main.py", line 9, in
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\click\core.py”, line 764, in call
return self.main(*args, **kwargs)
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\click\core.py”, line 717, in main
rv = self.invoke(ctx)
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\click\core.py”, line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\click\core.py”, line 555, in invoke
return callback(*args, **kwargs)
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\flatland\cli.py”, line 87, in evaluator
grader.run()
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\flatland\evaluators\service.py”, line 839, in run
command = self.get_next_command()
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\flatland\evaluators\service.py”, line 422, in get_next_command
command = _get_next_command(self.command_channel, _redis)
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\timeout_decorator\timeout_decorator.py”, line 91, in new_function
return timeout_wrapper(*args, **kwargs)
File “c:\users\memo\appdata\local\programs\python\python36\lib\site-packages\timeout_decorator\timeout_decorator.py”, line 146, in call
self.__process.start()
File “c:\users\memo\appdata\local\programs\python\python36\lib\multiprocessing\process.py”, line 105, in start
self._popen = self._Popen(self)
File “c:\users\memo\appdata\local\programs\python\python36\lib\multiprocessing\context.py”, line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File “c:\users\memo\appdata\local\programs\python\python36\lib\multiprocessing\context.py”, line 322, in _Popen
return Popen(process_obj)
File “c:\users\memo\appdata\local\programs\python\python36\lib\multiprocessing\popen_spawn_win32.py”, line 65, in init
reduction.dump(process_obj, to_child)
File “c:\users\memo\appdata\local\programs\python\python36\lib\multiprocessing\reduction.py”, line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can’t pickle local object ‘FlatlandRemoteEvaluationService.get_next_command.._get_next_command’
Traceback (most recent call last):
File “”, line 1, in
File “c:\users\memo\appdata\local\programs\python\python36\lib\multiprocessing\spawn.py”, line 105, in spawn_main
exitcode = _main(fd)
File “c:\users\memo\appdata\local\programs\python\python36\lib\multiprocessing\spawn.py”, line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

I have installed “wsl 1” and ubuntu 18.02 LTS, Redis is running with ubuntu prompt and I try to run evaluator from cmd prompt. Is wsl 1 enough? and Running evaluator from cmd prompt shouldn’t be a problem right?

I also tried with admin powershell, same error

Any suggestions?

1 Like
#10

I was able to get it all to work by installing WSL and then Ubuntu from windows app store and launching an instance of redis on one ubuntu prompt, Evaluator on another ubuntu prompt and the agent on an anaconda prompt.
Thank you everyone for their help

#11

Same here.

In windows, I can only install the environment via: pip install flatland-rl

However, it failed to run the evaluator, error same as in MemoAI’s post. (… EOFError: Ran out of input)

#12

What i did was give up and use Linux instead. I guess it’s difficult to make a tutorial that covers both Linux and Windows 10

#13

One can use Ubuntu prompt within windows, that solved the issue for me. all you have to do is install WSL(I used wsl 1) and then Ubuntu from windows store

1 Like
#14

That might help indeed! They also just announced GPU acceleration support!
https://blogs.windows.com/windowsdeveloper/2020/06/17/gpu-accelerated-ml-training-inside-the-windows-subsystem-for-linux/

I think rendering with Pyglet from WSL is problematic though, let us know if you find a solution around that

#15

Definitely not that simple. I already had WSL installed for other reasons.

The setup is a bit more difficult than that. Also having to synchronize 3 different command line instances (according to your post) isn’t ideal. Linux made life much easier + i am not sure what other errors might show up, but other posts suggests there will be more and the challenge is hard enough as it is.

The overall setup is very time consuming if i also add the problems stemming from using windows then this is really no fun.

Are you using WSL1 or WSL2 btw?
WSL 2: Getting started - YouTube

#16

I use wsl1 but according to what MasterScrat has posted WSL 2 has gpu support so I might upgrade to that.

Don’t I have to setup a vm to install linux tho? I never set one up before. is it easy?

I for got the step to enable windows subsystem for linux. in powershell in administrator mode.

here is the link to how to do it from windows for anyone who wants to go that route:

#18

You could dual boot, which involves some risks, but it worked for me so far. Lots of videos on that.
https://www.youtube.com/watch?v=-iSAyiicyQY for Ubuntu 20.04

Or you could use Virtual Box, which is risk free and you can use linux inside a window basically, it’s alright if you have enough resources, but those resources could be spent on training instead.
https://itsfoss.com/virtualbox-guest-additions-ubuntu/

#19

Thank you for the info

1 Like
#20

I am using WSL2 with Ubuntu(16.04) and docker.

It works well so far.

For the visualization, I have tried two ways, both work for me:

  1. Install GUI and XServer for WSL2.
    Some links I found helpful:
  1. After getting frames in png format, use the following function to generate a video:
    https://gitlab.aicrowd.com/flatland/flatland/blob/master/flatland/evaluators/aicrowd_helpers.py#L108

Overall, I feel the 2nd method is simpler and I am currently using it for visualization.

2 Likes