It looks like these yaml files have a predefined format and we can’t introduce new keys. Is there a spec or docs describing the expected format? Is this a file we shouldn’t try to use with our custom models?
Hello @laxatives
Did you try using custom_options? https://github.com/AIcrowd/neurips2020-procgen-starter-kit/blob/master/experiments/procgen-starter-example.yaml#L158
Some resources that might be helpful,
Custom models using tensorflow: https://ray.readthedocs.io/en/stable/rllib-models.html#custom-models-tensorflow
You can find some example implementations at https://github.com/ray-project/ray/tree/master/rllib/examples