Thank you for pointing this out. We actually realized this ourselfes when training the agents.
Our suggestions would be to penalize the agents yourself for waiting. In future releases we will have more complex schedules where it is ok for agents to wait before entering the environment.
We suggest the following two solutions:
- Penalize waiting agents by looking at the info returned by the environment
if env.agents[a].status == 0: all_rewards[a] -= 1 - Use other algorithms to decide what agent should enter when and only us RL for agents in the environment.
Does this help? Otherwise we can discuss further if this needs to be implemented and push it in the next update.
Best regards,
Erik