Error in local evaluation

SUADD'23- Scene Understanding for Autonomous Drone
#1

When I run the file local_evaluation.py, I get the following error:

Predicting Segmentation Masks: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 1786/1786 [08:53<00:00,  3.35it/s]
Evaluating results:  74%|██████████████████████████████████████████████████████████████████████████▏                         | 1324/1786 [03:07<01:05,  7.06it/s]
Traceback (most recent call last):
  File "/Users/sourish/Projects/suadd-2023-semantic-segmentation-starter-kit/local_evaluation.py", line 100, in <module>
    evaluate(LocalEvalConfig)
  File "/Users/sourish/Projects/suadd-2023-semantic-segmentation-starter-kit/local_evaluation.py", line 76, in evaluate
    all_metrics[fname] = calculate_metrics(semantic_annotation, semantic_prediction)
  File "/Users/sourish/Projects/suadd-2023-semantic-segmentation-starter-kit/local_evaluation.py", line 38, in calculate_metrics
    mean_iou_score  = mean_iou(semantic_annotation, semantic_prediction)
  File "/Users/sourish/Projects/suadd-2023-semantic-segmentation-starter-kit/local_evaluation.py", line 28, in mean_iou
    numer = np.sum(class_annotation & class_prediction, axis=(0,1))
ValueError: operands could not be broadcast together with shapes (2250,1550) (2200,1550)

This is using the default random model.

#3

Hi Sourish @sourishg

Happy New Year!

The error is not in local evaluation. Seems one of the files in semantic annotations has a different shape, we’ll work with the AWS team and get that fixed ASAP.

This is the file
semantic_annotations/a219be9df80b4431bd952700b840262f-1622741974400009778.png

About the Semantic Segmentation category
#4

is this issue fixed? I am also getting same error during local evaluation

#5

Hi @sagar_rathod

We’ve updated the file in the Resources section - suadd_23_semantic_annotations-v0.2.zip

Let me know if you still get the error with the updated dataset.