Getting a timeout when running the flatland-evaluator

Getting a timeout when I run flatland-evaluator --tests ./scratch/test-envs/

The redis server is running and I could get a response with redis-cli

`[‘Test_12/Level_1.pkl’, ‘Test_2/Level_1.pkl’, ‘Test_13/Level_1.pkl’, ‘Test_9/Level_0.pkl’, ‘Test_8/Level_1.pkl’, ‘Test_5/Level_1.pkl’, ‘Test_1/Level_0.pkl’, ‘Test_4/Level_0.pkl’, ‘Test_3/Level_1.pkl’, ‘Test_0/Level_1.pkl’, ‘Test_11/Level_1.pkl’, ‘Test_2/Level_0.pkl’, ‘Test_8/Level_0.pkl’, ‘Test_10/Level_1.pkl’, ‘Test_0/Level_0.pkl’, ‘Test_7/Level_1.pkl’, ‘Test_6/Level_0.pkl’, ‘Test_10/Level_0.pkl’, ‘Test_4/Level_1.pkl’, ‘Test_12/Level_0.pkl’, ‘Test_11/Level_0.pkl’, ‘Test_3/Level_0.pkl’, ‘Test_7/Level_0.pkl’, ‘Test_9/Level_1.pkl’, ‘Test_6/Level_1.pkl’, ‘Test_5/Level_0.pkl’, ‘Test_13/Level_0.pkl’, ‘Test_1/Level_1.pkl’]
Listening at : flatland-rl::FLATLAND_RL_SERVICE_ID::commands
Traceback (most recent call last):
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/flatland/evaluators/service.py”, line 422, in get_next_command
command = _get_next_command(self.command_channel, _redis)
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/timeout_decorator/timeout_decorator.py”, line 81, in new_function
return function(*args, **kwargs)
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/flatland/evaluators/service.py”, line 417, in _get_next_command
command = _redis.brpop(command_channel)[1]
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/redis/client.py”, line 1635, in brpop
return self.execute_command(‘BRPOP’, *keys)
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/redis/client.py”, line 839, in execute_command
return self.parse_response(conn, command_name, **options)
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/redis/client.py”, line 853, in parse_response
response = connection.read_response()
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/redis/connection.py”, line 683, in read_response
response = self._parser.read_response()
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/redis/connection.py”, line 295, in read_response
response = self._buffer.readline()
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/redis/connection.py”, line 227, in readline
self._read_from_socket()
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/redis/connection.py”, line 171, in _read_from_socket
data = recv(self._sock, socket_read_size)
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/redis/_compat.py”, line 58, in recv
return sock.recv(*args, **kwargs)
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/timeout_decorator/timeout_decorator.py”, line 72, in handler
_raise_exception(timeout_exception, exception_message)
File “/home/sumedh/anaconda3/envs/flatland-rl/lib/python3.6/site-packages/timeout_decorator/timeout_decorator.py”, line 45, in _raise_exception
raise exception()
timeout_decorator.timeout_decorator.TimeoutError: ‘Timed Out’

Structure of my scratch folder
scratch
└── test-envs
├── metadata.csv
├── Test_0
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_1
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_10
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_11
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_12
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_13
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_2
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_3
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_4
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_5
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_6
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_7
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_8
│ ├── Level_0.pkl
│ └── Level_1.pkl
└── Test_9
├── Level_0.pkl
└── Level_1.pkl

Hey, what agent are you running? Is this using the default random agent from the starter kit? What are the logs on the agent’s side?

You can try flushing redis data, that may be the problem :thinking:
https://flatland.aicrowd.com/getting-started/first-submission.html#env-client-step-called-before-env-client-env-create-call

1 Like

Hi,

Flushing redis worked.

Thanks!