Clarification of evaluation metric

NeurIPS 2022: CityLearn Challenge
#1

Hello again, I have two questions regarding the reward :

  1. How are the normalized C[entry] and G[entry] of each building put together to form a unique metric ?
  2. In the starter kit, in get_reward.py, I don’t understand why the reward is only calculated from electricity_price, which is just the price of electricity in the area. It should be calculated from electricity_cost. Maybe the doc is wrong and the variable “electricity_price” means “electricity_cost” ?
1 Like
#2

Hi @joseph_amigo and thanks for your question.

  1. C[entry] is the sum of electricity bill ($) for all buildings in the district while G[entry] is the sum of CO2 emission (kg_co2) for all buildings in the district. They are first divided by the C[no_battery] and G[no_battery] and then added with equal weights to get the final score. See the source code for the calculation of C[entry] and G[entry].

  2. The reward in the starter kit is calculated from both electricity_price and carbon_emission. Also, I think we use electricity_cost and electricity_price to mean the same thing as they both have same unit [$]. The challenge also allows you to define an appropriate reward function so while designing your agent, you can also, modify get_reward.py to suit your needs.

Hope these answer your questions otherwise, I am happy to further clarify.

#3

Yeah that’s clear thank you.