Data Pipeline Freezes

olliejday · September 14, 2019, 11:44am

Hi,

I am using the mineRL data like so.

class Data:
def __init__(self, minerl_data):
    self.data = minerl_data  # the minerl data object
def get_data():
    data = []
    for current_states, a, _, next_states, _ in self.data.sarsd_iter(num_epochs=-1):
        # gather data
    return data

Have used the verify or download and data seems ok. But after a few iterations of calling the function above, it freezes at this point where on the previous iteration it would continue to log that it was loading and enqueuing data just fine:

2019-09-14 11:56:21 ollie-pc root[9325] DEBUG Gathering expert data.
2019-09-14 11:56:21 ollie-pc minerl.data.data_pipeline[9325] DEBUG Starting seq iterator on data/MineRLObtainDiamond-v0
2019-09-14 11:56:21 ollie-pc minerl.data.data_pipeline[9325] DEBUG 464

Anyone experienced this or got any insight?

Thanks!