Reset simulator to specific state (e.g., state previously seen during an episode)

luisenp · July 29, 2019, 4:12pm

Hi. Is there a way to reset the environment to a previous state, and restart the simulation from that point forward?

arjun_nemani · July 29, 2019, 6:50pm

Nudge @lukasz_kidzinski !

He should be able to answer this.

Best of Luck!

lukasz_kidzinski · July 30, 2019, 12:59am

Hi! These two functions should do the job

github.com

stanfordnmbl/osim-rl/blob/master/osim/env/osim.py#L243-L250


def get_state(self):
    return opensim.State(self.state)


def set_state(self, state):
    self.state = state
    self.istep = int(self.state.getTime() / self.stepsize) # TODO: remove istep altogether
    self.reset_manager()

You can access the OpenSim model through

github.com

stanfordnmbl/osim-rl/blob/master/osim/env/osim.py#L269


# The amin purpose of this class is to provide wrap all 
# the functions of OpenAI gym. It is still an abstract
# class but closer to OpenSim. The actual classes of
# environments inherit from this one and:
# - select the model file
# - define the rewards and stopping conditions
# - define an obsernvation as a function of state
class OsimEnv(gym.Env):
    action_space = None
    observation_space = None
    osim_model = None
    istep = 0
    verbose = False


    visualize = False
    spec = None
    time_limit = 1e10


    prev_state_desc = None


    model_path = None # os.path.join(os.path.dirname(__file__), '../models/MODEL_NAME.osim')

I hope that helps!

luisenp · July 30, 2019, 1:36am

Thanks, for the reply @lukasz_kidzinski! Looks like that’s working

luisenp · August 10, 2019, 11:50am

Hi @lukasz_kidzinski. Is there any way to serialize or correctly populate the environment state if one wants to set copies of the environment running on different process/machines to exactly the same state? Seems like a simple pickle does not work for SWIG objects, so I was thinking if it was possible to serialize the values themselves, and if so, which ones are important.

lukasz_kidzinski · August 10, 2019, 2:56pm

Hi, you would need to loop through all variables of the state, serialize them and reinitialize the state. We don’t have a function for that yet, but it should be technically possible. Here is some useful starting point

Please follow up in that issue (or start another one in github) – it will be easier for me to bring in developers of OpenSim to the discussion.