Confusion on airborne object names and range_distance_m

Airborne Object Tracking Challenge
#1

@shivam @zontakm9

Hi!

I am confused about “airborne objects”. As we known, Airplane2, Bird1, Helicopter1,… are occured in grouthtruth.json. Could you provide more details about “airborne objects”?

By the way, I noticed that range_distance_m may be nan. Should the objects be considerd as airborne objects?

Thanks!

1 Like
🚀 Discussion on Starter Kit
#2

Hi @octo,

The main purpose of classes in the dataset/ground truth is unique identifiers for each sequence, for different objects.

The name does tell about the object type i.e. Airplane, Helicopter, Bird, etc, but the class name don’t have an impact on your submissions.

But here is the full list and distribution of airborne objects:

image
image762×713 22.7 KB

Snippet to generate yourself 
all_object_names = {}
for flight_id in dataset.get_flight_ids():
    flight = dataset.get_flight(flight_id)
    for obj in flight.detected_objects:
        if obj not in all_object_names:
          all_object_names[obj] = 0
        all_object_names[obj] += 1

Clipboard friendly list: :smile:

['Airborne1', 'Airborne10', 'Airborne11', 'Airborne12', 'Airborne14', 'Airborne15', 'Airborne16', 'Airborne17', 'Airborne18', 'Airborne2', 'Airborne3', 'Airborne4', 'Airborne5', 'Airborne6', 'Airborne7', 'Airborne9', 'Airplane1', 'Airplane10', 'Airplane2', 'Airplane3', 'Airplane4', 'Airplane5', 'Airplane6', 'Airplane7', 'Airplane8', 'Bird1', 'Bird10', 'Bird15', 'Bird2', 'Bird23', 'Bird3', 'Bird4', 'Bird5', 'Bird6', 'Bird7', 'Bird8', 'Bird9', 'Drone1', 'Flock1', 'Flock2', 'Flock3', 'Helicopter1', 'Helicopter2', 'Helicopter3', 'Helicopter4']

Regarding the range_distance_m, as shared in challenge’s overview:

If the label corresponds to a planned airborne object, its distance information may be available. Note that distance data is not available for other non-planned airborne objects in the scene.

i.e. those nan ones are unplanned objects, hence distance information isn’t available.

Let us know in case you have any follow up questions. :smiley:

2 Likes
#3

@shivam
As we known, Airplane2, Bird1, Helicopter1,… are occured in grouthtruth.json.

Does all categories(except for bird) are evaluated as airbone objects?

If not, which categories are evaluated?

Thanks!

2 Likes
#4

Hi @octo,

Copying from the challenge description:

Additional details on detection evaluation and false alarms calculation

[…]

Note that dataset videos and the provided ground truth labels might contain other airborne objects that are not planned, or planned airborne objects that do not belong to valid encounters.

The airborne metrics does not consider those objects for detection rate calculation and treats them as ‘don’t care’ (i.e., those detections will not be counter towards false alarms). Frame-level metrics consider non-planned objects and planned objects at range > 700m as ‘don’t care’.

Requesting @zontakm9 to confirm the information.

3 Likes