Baseline - MINILEAVES

Baseline for MINILEAVES Challenge on AIcrowd

Author : Sharada Mohanty

To open this notebook on Google Computing platform Colab, click below!

Open In Colab

Download Necessary Packages

In [18]:
import sys
!{sys.executable} -m pip install numpy
!{sys.executable} -m pip install pandas
!{sys.executable} -m pip install scikit-learn
Requirement already satisfied: numpy in /home/mohanty/miniconda3/lib/python3.7/site-packages (1.18.3)
Requirement already satisfied: pandas in /home/mohanty/miniconda3/lib/python3.7/site-packages (1.0.3)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/mohanty/miniconda3/lib/python3.7/site-packages (from pandas) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in /home/mohanty/miniconda3/lib/python3.7/site-packages (from pandas) (2020.1)
Requirement already satisfied: numpy>=1.13.3 in /home/mohanty/miniconda3/lib/python3.7/site-packages (from pandas) (1.18.3)
Requirement already satisfied: six>=1.5 in /home/mohanty/miniconda3/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas) (1.14.0)
Requirement already satisfied: scikit-learn in /home/mohanty/miniconda3/lib/python3.7/site-packages (0.22.2.post1)
Requirement already satisfied: joblib>=0.11 in /home/mohanty/miniconda3/lib/python3.7/site-packages (from scikit-learn) (0.14.1)
Requirement already satisfied: scipy>=0.17.0 in /home/mohanty/miniconda3/lib/python3.7/site-packages (from scikit-learn) (1.4.1)
Requirement already satisfied: numpy>=1.11.0 in /home/mohanty/miniconda3/lib/python3.7/site-packages (from scikit-learn) (1.18.3)

Download data

The first step is to download our train and test datasets. We will be training a classifier on the training dataset and make predictions on test dataset.

In [19]:
#Donwload the datasets
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/minileaves/v0.1/train-images.npy
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/minileaves/v0.1/train-labels.npy
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/minileaves/v0.1/test-images.npy
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/minileaves/v0.1/all_classes.txt
--2020-05-01 12:40:18--  https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/minileaves/v0.1/train-images.npy
Resolving s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)... 130.117.252.10, 130.117.252.11, 130.117.252.13, ...
Connecting to s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)|130.117.252.10|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 133708928 (128M) [binary/octet-stream]
Saving to: ‘train-images.npy.1’

train-images.npy.1  100%[===================>] 127.51M  19.6MB/s    in 6.7s    

2020-05-01 12:40:26 (19.1 MB/s) - ‘train-images.npy.1’ saved [133708928/133708928]

--2020-05-01 12:40:26--  https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/minileaves/v0.1/train-labels.npy
Resolving s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)... 130.117.252.12, 130.117.252.16, 130.117.252.13, ...
Connecting to s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)|130.117.252.12|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 348328 (340K) [binary/octet-stream]
Saving to: ‘train-labels.npy.1’

train-labels.npy.1  100%[===================>] 340.16K   784KB/s    in 0.4s    

2020-05-01 12:40:27 (784 KB/s) - ‘train-labels.npy.1’ saved [348328/348328]

--2020-05-01 12:40:27--  https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/minileaves/v0.1/test-images.npy
Resolving s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)... 130.117.252.12, 130.117.252.16, 130.117.252.13, ...
Connecting to s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)|130.117.252.12|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 33113216 (32M) [binary/octet-stream]
Saving to: ‘test-images.npy.1’

test-images.npy.1   100%[===================>]  31.58M  17.0MB/s    in 1.9s    

2020-05-01 12:40:30 (17.0 MB/s) - ‘test-images.npy.1’ saved [33113216/33113216]

--2020-05-01 12:40:30--  https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/minileaves/v0.1/all_classes.txt
Resolving s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)... 130.117.252.12, 130.117.252.16, 130.117.252.13, ...
Connecting to s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)|130.117.252.12|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 886 [text/plain]
Saving to: ‘all_classes.txt.1’

all_classes.txt.1   100%[===================>]     886  --.-KB/s    in 0s      

2020-05-01 12:40:31 (15.6 MB/s) - ‘all_classes.txt.1’ saved [886/886]

Import packages

In [20]:
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.neural_network import MLPClassifier
from sklearn.svm import SVC
from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score
import matplotlib.pyplot as plt

Load Data

In [21]:
train_images_path = "train-images.npy" #path where data is stored
train_labels_path = "train-labels.npy"

train_images = np.load(train_images_path)
train_labels = np.load(train_labels_path)

# Load Class mapping
class_names = [x.strip() for x in open("all_classes.txt").readlines()]
In [ ]:

Visualize the data

In [22]:
from mpl_toolkits.axes_grid1 import ImageGrid

def random_data_sample():
    """
    Returns a random data sample from the training set
    """
    global class_name, train_images, train_labels
    random_index = np.random.randint(0, train_images.shape[0])
    # Render the image
    image_array = train_images[random_index]
    class_label = train_labels[random_index] # an integral number in [0-38)
    class_name = class_names[class_label] # a human readable class name
    return image_array, class_label, class_name

"""
Render a 7x7 grid of 49 randomly sampled images
from the training set
"""
grid_shape = (7, 7)
fig = plt.figure(figsize=(10, 10))
grid = ImageGrid(fig, 111,  # similar to subplot(111)
                 nrows_ncols=grid_shape,  # creates 2x2 grid of axes
                 axes_pad=0.1,  # pad between axes in inch.
                 )

data_samples = [random_data_sample() for _ in range(grid_shape[0] * grid_shape[1])]
for ax, data_sample in zip(grid, data_samples):
    image_array, class_label, class_name = data_sample
    ax.imshow(image_array)
    ax.axis('off')

plt.savefig("image-grid.png")

Dataset Distribution

In [23]:
from collections import Counter

data_counter = Counter(train_labels)
unique_class_indices = data_counter.keys()

for _class_index in unique_class_indices:
    print("Class Index : ", _class_index)
    print("Class Name : ", class_names[_class_index])
    print("Number of images in the dataset : ", data_counter[_class_index])
    print("="*100)
# TODO : Add a bar plot of the distribution of the classes here
Class Index :  32
Class Name :  tomato-leaf-mold
Number of images in the dataset :  776
====================================================================================================
Class Index :  34
Class Name :  tomato-spider-mites-two-spotted-spider-mite
Number of images in the dataset :  1349
====================================================================================================
Class Index :  7
Class Name :  corn-maize-cercospora-leaf-spot-gray-leaf-spot
Number of images in the dataset :  418
====================================================================================================
Class Index :  14
Class Name :  grape-leaf-blight-isariopsis-leaf-spot
Number of images in the dataset :  869
====================================================================================================
Class Index :  12
Class Name :  grape-esca-black-measles
Number of images in the dataset :  1107
====================================================================================================
Class Index :  13
Class Name :  grape-healthy
Number of images in the dataset :  333
====================================================================================================
Class Index :  26
Class Name :  strawberry-healthy
Number of images in the dataset :  363
====================================================================================================
Class Index :  37
Class Name :  tomato-tomato-yellow-leaf-curl-virus
Number of images in the dataset :  4273
====================================================================================================
Class Index :  28
Class Name :  tomato-bacterial-spot
Number of images in the dataset :  1692
====================================================================================================
Class Index :  6
Class Name :  cherry-including-sour-powdery-mildew
Number of images in the dataset :  848
====================================================================================================
Class Index :  5
Class Name :  cherry-including-sour-healthy
Number of images in the dataset :  685
====================================================================================================
Class Index :  29
Class Name :  tomato-early-blight
Number of images in the dataset :  781
====================================================================================================
Class Index :  36
Class Name :  tomato-tomato-mosaic-virus
Number of images in the dataset :  302
====================================================================================================
Class Index :  9
Class Name :  corn-maize-healthy
Number of images in the dataset :  939
====================================================================================================
Class Index :  27
Class Name :  strawberry-leaf-scorch
Number of images in the dataset :  871
====================================================================================================
Class Index :  10
Class Name :  corn-maize-northern-leaf-blight
Number of images in the dataset :  777
====================================================================================================
Class Index :  16
Class Name :  peach-bacterial-spot
Number of images in the dataset :  1851
====================================================================================================
Class Index :  2
Class Name :  apple-cedar-apple-rust
Number of images in the dataset :  233
====================================================================================================
Class Index :  1
Class Name :  apple-black-rot
Number of images in the dataset :  499
====================================================================================================
Class Index :  25
Class Name :  squash-powdery-mildew
Number of images in the dataset :  1477
====================================================================================================
Class Index :  8
Class Name :  corn-maize-common-rust
Number of images in the dataset :  966
====================================================================================================
Class Index :  17
Class Name :  peach-healthy
Number of images in the dataset :  291
====================================================================================================
Class Index :  30
Class Name :  tomato-healthy
Number of images in the dataset :  1281
====================================================================================================
Class Index :  21
Class Name :  potato-healthy
Number of images in the dataset :  119
====================================================================================================
Class Index :  11
Class Name :  grape-black-rot
Number of images in the dataset :  955
====================================================================================================
Class Index :  24
Class Name :  soybean-healthy
Number of images in the dataset :  4098
====================================================================================================
Class Index :  3
Class Name :  apple-healthy
Number of images in the dataset :  1310
====================================================================================================
Class Index :  31
Class Name :  tomato-late-blight
Number of images in the dataset :  1510
====================================================================================================
Class Index :  35
Class Name :  tomato-target-spot
Number of images in the dataset :  1125
====================================================================================================
Class Index :  18
Class Name :  pepper-bell-bacterial-spot
Number of images in the dataset :  787
====================================================================================================
Class Index :  15
Class Name :  orange-haunglongbing-citrus-greening
Number of images in the dataset :  4415
====================================================================================================
Class Index :  33
Class Name :  tomato-septoria-leaf-spot
Number of images in the dataset :  1419
====================================================================================================
Class Index :  0
Class Name :  apple-apple-scab
Number of images in the dataset :  513
====================================================================================================
Class Index :  22
Class Name :  potato-late-blight
Number of images in the dataset :  813
====================================================================================================
Class Index :  20
Class Name :  potato-early-blight
Number of images in the dataset :  796
====================================================================================================
Class Index :  4
Class Name :  blueberry-healthy
Number of images in the dataset :  1204
====================================================================================================
Class Index :  23
Class Name :  raspberry-healthy
Number of images in the dataset :  304
====================================================================================================
Class Index :  19
Class Name :  pepper-bell-healthy
Number of images in the dataset :  1176
====================================================================================================

Split Data into Train and Validation

Now we eventually want to see how well our classifier is performing, but we dont have the test data labels with us to check. What do we do ? We split our dataset into a training set and a validation set. The idea is that we test our classifier on validation set in order to get an idea of how well our classifier works. This way we can also ensure that we dont overfit on the training dataset.

In [24]:
X_train, X_val= train_test_split(train_images, test_size=0.2, random_state=42)
y_train, y_val= train_test_split(train_labels, test_size=0.2, random_state=42) 
## Note : Given that we use the same random state in both the splits, 
## and both `train_images` and `train_labels` have the same number of (aligned) elements, 
## the overall alignment between X_train and the corresponding Y_train will be retained.

Here we have selected the size of the testing data to be 20% of the total data. You can change it and see what effect it has on the accuracies. To learn more about the train_test_split function click here.

Define the Classifier

Now we come to the juicy part. We have fixed our data and now we train a classifier. The classifier will learn the function by looking at the inputs and corresponding outputs. There are a ton of classifiers to choose from some being Logistic Regression, SVM, Random Forests, Decision Trees, etc.
Tip: A good model doesnt depend solely on the classifier but on the features(columns) you choose. So make sure to play with your data and keep only whats important.

In [25]:
classifier = MLPClassifier(hidden_layer_sizes=(20, 20))

#from sklearn.linear_model import LogisticRegression
# classifier = LogisticRegression()

## NOTE : This is definitely not the most optimal approach for this problem. 
## This code is provided to help you get started quickly, and should be treated
## as suck

We have used a very simple Multi Layer Perceptron as a classifier here and set few of the parameteres. But one can set more parameters and increase the performance. To see the list of parameters visit here.

We can also use other classifiers. To read more about sklean classifiers visit here. Try and use other classifiers to see how the performance of your model changes. Try using Logistic Regression or MLP and compare how the performance changes.

Define data pre-processors

In [26]:
"""
Some of the models have different requirements in terms of what form of the data they can consume.

In this example, we will use 
a very simple Multi Layer Perceptron (explained further in the notebook), 
which expects a single dimensional representation for every data point. 

Hence, we will define some pre-processors which can help flatten the data,
and also normalize the data as necessary.
"""

def preprocess_X(X_array):
    assert X_array.shape[1:] == (32, 32, 3) # as the images are of the dimension 32x32x3
    
    # We normalize the whole array to a [0-1] range. As the pixel RGB values are uint8 values in the range[0, 256], 
    # we will normalize simply by dividing the whole array by 255
    X_array = X_array.astype(np.float32) / 255.0
    
    # Now, we will reshape the ndarray of shape (N, 32, 32, 3) to (N, 32*32*3), where N is the number of samples
    # present in the dataset
    X_array = X_array.reshape(X_array.shape[0], 32*32*3)
    return X_array

X_train = preprocess_X(X_train)
X_val = preprocess_X(X_val)

Train the classifier

In [27]:
classifier.fit(X_train, y_train)
/home/mohanty/miniconda3/lib/python3.7/site-packages/sklearn/neural_network/_multilayer_perceptron.py:571: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.
  % self.max_iter, ConvergenceWarning)
Out[27]:
MLPClassifier(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,
              beta_2=0.999, early_stopping=False, epsilon=1e-08,
              hidden_layer_sizes=(20, 20), learning_rate='constant',
              learning_rate_init=0.001, max_fun=15000, max_iter=200,
              momentum=0.9, n_iter_no_change=10, nesterovs_momentum=True,
              power_t=0.5, random_state=None, shuffle=True, solver='adam',
              tol=0.0001, validation_fraction=0.1, verbose=False,
              warm_start=False)

Predict on Validation

Now we predict our trained classifier on the validation set and evaluate our model

In [28]:
y_pred = classifier.predict(X_val)

Evaluate the Performance

We use the same metrics as that will be used for the test set.
F1 score and Log Loss are the metrics for this challenge

In [29]:
precision = precision_score(y_val,y_pred,average='micro')
recall = recall_score(y_val,y_pred,average='micro')
accuracy = accuracy_score(y_val,y_pred)
f1 = f1_score(y_val,y_pred,average='macro')
In [30]:
print("Accuracy of the model is :" ,accuracy)
print("Recall of the model is :" ,recall)
print("Precision of the model is :" ,precision)
print("F1 score of the model is :" ,f1)
Accuracy of the model is : 0.67972429638139
Recall of the model is : 0.67972429638139
Precision of the model is : 0.67972429638139
F1 score of the model is : 0.5774074568690114

Prediction on Evaluation Set

Load Test Set

In [13]:
test_file_path = "test-images.npy"
test_images = np.load(test_file_path)

Predict Test Set

The moment of truth! Predict on test set and then we can make the submission.

In [14]:
# We first pre-process the test images, the same way we pre-process the train images
test_images = preprocess_X(test_images)

submission = classifier.predict(test_images)

Save the prediction to csv

In [15]:
#change the header according to the submission guidelines
In [16]:
import pandas as pd
submission = pd.DataFrame(submission)
submission.to_csv('submission.csv',header=['class_index'],index=False)

Note: Do take a look at the submission format.The submission file should contain a header.

To download the generated csv in colab run the below command

In [ ]:
# This block is only relevant if you are running this notebook on Google Colab
from google.colab import files
files.download('submission.csv')

Go to platform. Participate in the challenge and submit the submission.csv.

In [ ]: