You may have noticed some changes on the leaderboard however, it is nothing to worry about. We have included a grid cost, D in addition to the normalized price, C and normalized emission, G costs. The grid cost is to encourage cooperative multi-agent RL solutions that when evaluated at the aggregated district level, improve grid-level objectives on average. The grid-level objectives that we have introduced are the reduction of
1 - load factor. See the following sections for more details on the changes to the evaluation and leaderboard.
R is the smoothness of the district’s load profile. Low R means there is gradual increase in grid electricity demand even after self-generation becomes unavailable in the evening and early morning. High R means abrupt change in load on the grid that may lead to unscheduled strain on grid infrastructure and blackouts as a result of supply deficit that is unable to meet the rapid increase in demand. It is calculated as the sum of the absolute difference of net electricity consumption between consecutive time steps:
L is the efficiency of electricity consumption and is bounded between 0 (very inefficient) and 1 (highly efficient). Thus, the goal is to minimize 1 - L. L is the average ratio of monthly average and maximum net electricity consumption and is calculated as:
There are two new columns on the leaderboard.
Grid Cost displays the value of D and
Average Score (with grid cost) is the average of normalized C, normalized G and D:
Average Score retains its original definition as the average of the normalized price, C and normalized emission, G costs:
In Phase I and Phase II,
Average Score is used to rank submissions to keep the competition fair. Only by Phase III will
Average Score (with grid cost) be used for ranking.
Phase I and Phase II submissions that were made before ~ 2022-09-14 15:00:00 UTC will have
Grid Cost and
Average Score (with grid cost) columns blank. Later submissions will have values for
Grid Cost and
Average Score (with grid cost) however, neither of these scores will be used for ranking until the beginning of Phase III.
Submissions made after ~ 2022-09-14 15:00:00 UTC will have values for
Grid Cost and
Average Score (with grid cost) but will not be used for ranking until the beginning of Phase III as with previous submissions.
All you need to do for now is to:
- Update to the latest CityLearn v1.3.6 to be able to view the grid cost in your local evaluation:
pip install git+https://github.com/intelligent-environments-lab/CityLearn.email@example.com
- Design your agent to minimize the grid cost as done for the price and emission costs. While the grid cost is not utilized in Phase I and II ranking, it will become relevant by Phase III so best to start planning for it now !