Deep Learning to Recognize Traffic Signs with Python GUI: Part 1

This content is powered by Balige Publishing. Visit this link (collaboration with Rismon Hasiholan Sianipar)

Step 1: Now, you will create a GUI to implement how to recognizing traffic signs. Open Qt Designer and choose Main Window template. Save the form as gui_traffic.ui.

Step 2: Put one Widget from Containers panel onto form and set its ObjectName property as widgetGraph. You will use this widget to plot graph.

Step 3: Put two Push Button widgets onto form. Set their text properties as LOAD DATA and TRAINING MODEL. Set their objectName properties as pbLoad and pbTraining.

Step 4: Put a Group Box widget onto form and set its objectName property as gbHistogram. Inside it, put two Radio Button widgets and set their objectName properties as rbHistTraining and rbHistTesting. Set their text properties as Training Data and Testing Data.

Step 5: Put another Group Box widget onto form and set its objectName property as gbCorr. Inside it, put two Radio Button widgets and set their objectName properties as rbCorrTraining and rbCorrTesting. Set their text properties as Training Data and Testing Data.

Step 6: Put one Combo Box widget onto form and set its objectName property as cbAccuracy. Double click on this widget and populate it with two items as shown in Figure below.

Step 7: Put the third Group Box widget onto form and set its objectName property as gbDataset. Inside it, put two Radio Button widgets and set their objectName properties as rbDataTraining and rbDataTesting. Set their text properties as Training Data and Testing Data.

Step 8: Put one Table Widget onto the right side of form and set its objectName property as twData. Save the form. It now looks as shown in Figure below.

Step 9: Right click on widgetGraph and choose Promote to …. Set Promoted class name as widgetClass. Click Add and Promote button. In Object Inspector window, you can see that widgetGraph is now an object of widget_class as shown in Figure below.

Step 10: Define the definition of widget_class and save it as widget_class.py as follows:

#widget_class.py

from PyQt5.QtWidgets import*
from matplotlib.backends.backend_qt5agg import FigureCanvas
from matplotlib.figure import Figure
    
class widget_class(QWidget):    
    def __init__(self, parent = None):
        QWidget.__init__(self, parent)        
        self.canvas = FigureCanvas(Figure())
       
        vertical_layout = QVBoxLayout()
        vertical_layout.addWidget(self.canvas)
        
        self.canvas.axis1 = self.canvas.figure.add_subplot(111)
        self.canvas.figure.set_facecolor("xkcd:neon yellow")
        self.setLayout(vertical_layout)

Step 11: Write this Python script and save it as recognize_traffic_sign.py:

#recognize_traffic_sign.py
from PyQt5.QtWidgets import *
from PyQt5.uic import loadUi
from matplotlib.backends.backend_qt5agg import (NavigationToolbar2QT as NavigationToolbar)
from matplotlib.colors import ListedColormap

class DemoGUI_TrafficSign(QMainWindow):   
    def __init__(self):       
        QMainWindow.__init__(self)
        loadUi("gui_traffic.ui",self)

        self.setWindowTitle("GUI Demo of Recognizing Traffic Signs")
        self.addToolBar(NavigationToolbar(self.widgetGraph.canvas,\
            self))
                      
if __name__ == '__main__':
    import sys
    app = QApplication(sys.argv)
    ex = DemoGUI_TrafficSign()
    ex.show()
    sys.exit(app.exec_())

Step 12: Run recognize_traffic_sign.py and see the form as shown in Figure below.

Step 13: Define set_state() method to determine the state of some widgets. This method is used to disble some widgets when the form first runs:

def set_state(self,state):
    self.gbHistogram.setEnabled(state)
    self.gbCorr.setEnabled(state)
    self.pbTraining.setEnabled(state)
    self.cbAccuracy.setEnabled(state)
    self.gbDataset.setEnabled(state)

Step 14: Call set_state() method and put it inside __init__() method:

def __init__(self):       
    QMainWindow.__init__(self)
    loadUi("gui_traffic.ui",self)
    self.setWindowTitle("GUI Demo of Recognizing Traffic Signs")
    self.addToolBar(NavigationToolbar(self.widgetGraph.canvas, self))
    self.set_state(False)

Step 15: Import all needed libraries into recognize_traffic_sign.py using the following statements:

import numpy as np 
import pandas as pd 
import matplotlib.pyplot as plt
import seaborn as sns
import cv2
from PIL import Image
import os
from sklearn.model_selection import train_test_split
from keras.utils import to_categorical
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential,load_model
from tensorflow.keras.layers import Dense, Dropout, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Flatten, Dropout
from tensorflow.keras import backend as K
from sklearn.metrics import accuracy_score

Step 16: Define load_data() and create_dataset_dataframe() methods to load training and testing data and to create dataframes:

def load_data(self):
    self.data = []
    self.labels = []
    classes = 43
    self.curr_path = os.getcwd()

    #Retrieving the images and their labels 
    for i in range(classes):
        path = os.path.join(self.curr_path,'train',str(i))
        images = os.listdir(path)
        for a in images:
            try:
                image = Image.open(path + '\\'+ a)
                image = image.resize((30,30))
                image = np.array(image)
                self.data.append(image)
                self.labels.append(i)
            except:
                print("Error loading image")

    #Converting lists into numpy arrays
    self.data = np.array(self.data)
    self.labels = np.array(self.labels)
    print(self.data.shape, self.labels.shape)

    table = {'image_path': path, 'target': self.labels}
    self.df = pd.DataFrame(data=table)
    self.df = self.df.sample(frac = 1).reset_index(drop=True) 
        
    #Creates dataset and dataframe
    self.create_dataset_dataframe()
        
    #Disables pbLoad widget
    self.pbLoad.setEnabled(False)
        
    #Enables back widgets
    self.set_state(True)
        
    #Checks rbDataTraining widget
    self.rbDataTraining.setChecked(True)

def create_dataset_dataframe(self):
    #Splitting training and testing dataset
    self.X_train, self.X_test, self.y_train, self.y_test = \
        train_test_split(self.data, self.labels, test_size=0.2, \
        random_state=42)
    print(self.X_train.shape, self.X_test.shape, \
        self.y_train.shape, self.y_test.shape)

    #Converting the labels into one hot encoding
    self.y_train = to_categorical(self.y_train, 43)
    self.y_test = to_categorical(self.y_test, 43)
        
    #Creates testing dataframe
    self.test_df = pd.read_csv(str(self.curr_path) +'/Test.csv')

    #Creates training dataframe
    self.df_train = pd.read_csv(str(self.curr_path) +'/Train.csv')
    nRow, nCol = self.df_train.shape
    print(f'There are {nRow} rows and {nCol} columns')
    print(self.df_train.head(5))

Step 17: Define display_table() method to display testing and training data on table widget as follows:

def display_table(self,df, tableWidget):
    # show data on table widget
    self.write_df_to_qtable(df,tableWidget)
        
    styleH = "::section {""background-color: red; }"
    tableWidget.horizontalHeader().setStyleSheet(styleH)

    styleV = "::section {""background-color: red; }"
    tableWidget.verticalHeader().setStyleSheet(styleV)  

# Takes a df and writes it to a qtable provided. df headers 
# become qtable headers
@staticmethod
def write_df_to_qtable(df,table):
    headers = list(df)
    table.setRowCount(df.shape[0])
    table.setColumnCount(df.shape[1]) 
    table.setHorizontalHeaderLabels(headers)

    # getting data from df is computationally costly 
    # so convert it to array first
    df_array = df.values
    for row in range(df.shape[0]):
        for col in range(df.shape[1]):
            table.setItem(row, col, \                                
                QTableWidgetItem(str(df_array[row,col])))

Step 18: Invoke display_table() method and put it at the end of load_data():

#Displays data on table widget
self.display_table(self.df_train,self.twData)

Step 19: Run recognize_traffic_sign.py to see that training data has been displayed on table widget as shown in Figure below.

Step 20: Define rb_dataset() method to read text property of rbDataTraining and rbDataTesting widgets to determine which data will be displayed on twData widget:

def rb_dataset(self,b):	
    if b.text() == "Training Data":
        if b.isChecked() == True:
            self.display_table(self.df_train,self.twData)
        else:
            self.display_table(self.test_df,self.twData)
                
    if b.text() == "Testing Data":
        if b.isChecked() == True:
            self.display_table(self.test_df,self.twData)
        else:
            self.display_table(self.df_train,self.twData)

Step 21: Connect toggled() event of both rbDataTraining and rbDataTesting widgets with rb_dataset method and put them inside __init__() method as follows:

def __init__(self):       
    QMainWindow.__init__(self)
    loadUi("gui_traffic.ui",self)
    self.setWindowTitle("GUI Demo of Recognizing Traffic Signs")
    self.addToolBar(NavigationToolbar(self.widgetGraph.canvas, self))
    self.set_state(False)
    self.pbLoad.clicked.connect(self.load_data)
    self.rbDataTraining.toggled.connect(\
        lambda:self.rb_dataset(self.rbDataTraining))
    self.rbDataTesting.toggled.connect(\
        lambda:self.rb_dataset(self.rbDataTesting))

Step 22: Run recognize_traffic_sign.py to see that training data has been displayed on table widget. Then click on Testing Data radio button on gbDataset group box to see testing data displayed on table widget as shown in Figure below.

Step 23: Define display_histogram() method to display distribution of number of images in each class:

def display_histogram(self, widget, df, xlabel, ylabel, title):   
    widget.canvas.axis1.clear()
    x=df.target.value_counts()
    sns.barplot(x.index,x, ax=widget.canvas.axis1)
    widget.canvas.axis1.set_xlabel(xlabel)
    widget.canvas.axis1.set_ylabel(ylabel)
    widget.canvas.axis1.set_title(title)
    widget.canvas.axis1.grid()
    widget.canvas.draw()

Step 24: Invoke display_histogram() method and make rbHistTraining widget checked when form first runs. Put them at the end of load_data() method:

#Displays histogram of training data
hist_train = self.df.target.value_counts()
self.display_histogram(self.widgetGraph, hist_train, \
    'Class', 'Samples', \
    'The distribution of number of training samples in each class')
#Checks rbDataTraining widget
self.rbHistTraining.setChecked(True)

Step 25: Run recognize_traffic_sign.py to see distribution of number of images in each class of training data as shown in Figure below.

Step 26: Define rb_histogram() method to read text property of rbHistTraining and rbHistTesting widgets to determine which data will be displayed on widgetGraph widget:

def rb_histogram(self,b):
    hist_train = self.df.target.value_counts()
    hist_test = self.test_df.ClassId.value_counts()
    if b.text() == "Training Data":
        if b.isChecked() == True:
            self.display_histogram(self.widgetGraph, hist_train, \
                'Class', 'Samples', \
                'The distribution of number of training samples in 
                each class')
        else:
            self.display_histogram(self.widgetGraph, hist_test, \
                'Class', 'Samples', \
                'The distribution of number of testing samples 
                in each class')
                
    if b.text() == "Testing Data":
        if b.isChecked() == True:
            self.display_histogram(self.widgetGraph, hist_test, \
                'Class', 'Samples', \
                'The distribution of number of testing samples 
                in each class')
        else:
            self.display_histogram(self.widgetGraph, hist_train, \
                'Class', 'Samples', 'The distribution of number of\
                training samples in each class')

Step 27: Connect toggled() event of both rbHistTraining and rbHistTesting widgets with rb_histogram method and put them inside __init__() method as follows:

def __init__(self):       
    QMainWindow.__init__(self)
    loadUi("gui_traffic.ui",self)
    self.setWindowTitle("GUI Demo of Recognizing Traffic Signs")
    self.addToolBar(NavigationToolbar(self.widgetGraph.canvas, self))
    self.set_state(False)
    self.pbLoad.clicked.connect(self.load_data)
    self.rbDataTraining.toggled.connect(\
        lambda:self.rb_dataset(self.rbDataTraining))
    self.rbDataTesting.toggled.connect(\
        lambda:self.rb_dataset(self.rbDataTesting))       
    self.rbHistTraining.toggled.connect(\
        lambda:self.rb_histogram(self.rbHistTraining))
    self.rbHistTesting.toggled.connect(\
        lambda:self.rb_histogram(self.rbHistTesting))

Step 28: Run recognize_traffic_sign.py to see distribution of number of images in each class of training data. Then, click on Testing Data in gbHistogram group box. You will see the histogram of testing data as shown in Figure below.

Deep Learning to Recognize Traffic Signs with Python GUI: Part 2