LEARN FROM SCRATCH SIGNAL AND IMAGE PROCESSING WITH PYTHON GUI

You can buy this book from Amazon or Google Books.

In this book, you will learn how to use OpenCV, NumPy library and other libraries to perform signal processing, image processing, object detection, and feature extraction with Python GUI (PyQt). You will learn how to filter signals, detect edges and segments, and denoise images with PyQt. You will also learn how to detect objects (face, eye, and mouth) using Haar Cascades and how to detect features on images using Harris Corner Detection, Shi-Tomasi Corner Detector, Scale-Invariant Feature Transform (SIFT), and Features from Accelerated Segment Test (FAST).

In Chapter 1, you will learn: Tutorial Steps To Create A Simple GUI Application, Tutorial Steps to Use Radio Button, Tutorial Steps to Group Radio Buttons, Tutorial Steps to Use CheckBox Widget, Tutorial Steps to Use Two CheckBox Groups, Tutorial Steps to Understand Signals and Slots, Tutorial Steps to Convert Data Types, Tutorial Steps to Use Spin Box Widget, Tutorial Steps to Use ScrollBar and Slider, Tutorial Steps to Use List Widget, Tutorial Steps to Select Multiple List Items in One List Widget and Display It in Another List Widget, Tutorial Steps to Insert Item into List Widget, Tutorial Steps to Use Operations on Widget List, Tutorial Steps to Use Combo Box, Tutorial Steps to Use Calendar Widget and Date Edit, and Tutorial Steps to Use Table Widget.

In Chapter 2, you will learn: Tutorial Steps To Create A Simple Line Graph, Tutorial Steps To Create A Simple Line Graph in Python GUI, Tutorial Steps To Create A Simple Line Graph in Python GUI: Part 2, Tutorial Steps To Create Two or More Graphs in the Same Axis, Tutorial Steps To Create Two Axes in One Canvas, Tutorial Steps To Use Two Widgets, Tutorial Steps To Use Two Widgets, Each of Which Has Two Axes, Tutorial Steps To Use Axes With Certain Opacity Levels, Tutorial Steps To Choose Line Color From Combo Box, Tutorial Steps To Calculate Fast Fourier Transform, Tutorial Steps To Create GUI For FFT, Tutorial Steps To Create GUI For FFT With Some Other Input Signals, Tutorial Steps To Create GUI For Noisy Signal, Tutorial Steps To Create GUI For Noisy Signal Filtering, and Tutorial Steps To Create GUI For Wav Signal Filtering.

In Chapter 3, you will learn: Tutorial Steps To Convert RGB Image Into Grayscale, Tutorial Steps To Convert RGB Image Into YUV Image, Tutorial Steps To Convert RGB Image Into HSV Image, Tutorial Steps To Filter Image, Tutorial Steps To Display Image Histogram, Tutorial Steps To Display Filtered Image Histogram, Tutorial Steps To Filter Image With CheckBoxes, Tutorial Steps To Implement Image Thresholding, and Tutorial Steps To Implement Adaptive Image Thresholding.

In Chapter 4, you will learn: Tutorial Steps To Generate And Display Noisy Image, Tutorial Steps To Implement Edge Detection On Image, Tutorial Steps To Implement Image Segmentation Using Multiple Thresholding and K-Means Algorithm, and Tutorial Steps To Implement Image Denoising.

In Chapter 5, you will learn: Tutorial Steps To Detect Face, Eye, and Mouth Using Haar Cascades, Tutorial Steps To Detect Face Using Haar Cascades with PyQt, Tutorial Steps To Detect Eye, and Mouth Using Haar Cascades with PyQt, and Tutorial Steps To Extract Detected Objects.

In Chapter 6, you will learn: Tutorial Steps To Detect Image Features Using Harris Corner Detection, Tutorial Steps To Detect Image Features Using Shi-Tomasi Corner Detection, Tutorial Steps To Detect Features Using Scale-Invariant Feature Transform (SIFT), and Tutorial Steps To Detect Features Using Features from Accelerated Segment Test (FAST).

Learn From Scratch Neural Networks Using PyQt: Part 11

This content is powered by Balige Publishing. Visit this link (collaboration with Rismon Hasiholan Sianipar) PART 1  PART 2  PART 3  PART 4  PART 5  PART 6  PART 7 PART 8 PART 9 PART 10

Tutorial Steps To Implement Linear Discriminant Analysis (LDA)  with Perceptron, Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), K-Nearest Neighbor (KNN) classifiers Using Scikit-Learn with PyQt

Step 1: Open Scikit_Classifier_Feature.py and add the following import statement:

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA

Step 2: Now, you will implement logistic regression (LR) classifier with LDA feature extractor.  Define draw_exp_var_LDA() function to display explained variance ratio of LDA:

def draw_exp_var_LDA(self,extractor,ylabelStr,xlabelStr,axisWidget):
    eigen_vals, eigen_vecs = np.linalg.eig(extractor.covariance_)
    tot = sum(eigen_vals.real)
    discr = [(i / tot) for i in sorted(eigen_vals.real, \
        reverse=True)]
    cum_discr = np.cumsum(discr)

    axisWidget.axis1.clear()
    axisWidget.axis1.bar(range(1,14), discr, alpha=0.5, \
        align='center',label='individual explained variance')
    axisWidget.axis1.step(range(1,14), cum_discr, \
        where='mid',label='cumulative explained variance')
    axisWidget.axis1.set_ylabel(ylabelStr)
    axisWidget.axis1.set_xlabel(xlabelStr)
    axisWidget.axis1.legend(loc='best')
    axisWidget.draw()

Step 3: Define lda_feature() function to implement LDA feature extractor into every model used (LR as first tested model):

def lda_feature(self):
    iterNum = self.sbIter.value()
    self.dsbRate.setDecimals(5)
    learningRate = self.dsbRate.value()
    depth = self.sbDepth.value()
    neighbor = self.sbDepth.value()
    ratio = self.dsbRatio.value()
        
    self.load_data_ratio(ratio)
        
    item = self.listAlgorithm.currentItem()
    strList = item.text()
        
    if strList == 'Logistic Regression':
        self.sbIter.setEnabled(True)  
        self.dsbRate.setEnabled(True)  
        self.sbDepth.setEnabled(False) 
        self.sbNeighbor.setEnabled(False)
        self.dsbRatio.setEnabled(True)
            
        #Trains LR model
        lda = LDA(n_components=2, store_covariance=True)      
        lr = make_pipeline(StandardScaler(), \
            SGDClassifier('log',max_iter=iterNum,eta0=learningRate, \
            tol=1e-3))
        self.X_train_lda = \
            lda.fit_transform(self.X_train_std,self.y_train)
        self.X_test_lda = lda.transform(self.X_test_std)
        lr.fit(self.X_train_lda, self.y_train)
            
        #Displays explained variance
        ylabelStr = 'Explained variance ratio'
        xlabelStr = 'Principal component index'
        self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\
            self.widgetVariance.canvas)
    
        #Displays decision regions    
        strTitle = 'LR Classifier with LDA Extractor: ' + \
            str(ratio*100) + '% Data Ratio '
        strTitle += ' and Learning Rate ' +str(learningRate)
        self.display_decision(self.X_train_lda, self.y_train, \
            classifier=lr,axisWidget=self.widgetDecision.canvas,\
            title=strTitle)

        #display accuracy graph
        self.graph_LR(self.widgetEpoch.canvas, self.accuracy_LR)

Step 4: Modify choose_feature() to involve Linear Discriminant Analysis (LDA) in cboFeature widget:

 def choose_feature(self): 
    strCB = self.cboFeature.currentText()
    if strCB == 'Principal Component Analysis (PCA)':
        self.pca_feature()  
    if strCB == 'Linear Discriminant Analysis (LDA)':
        self.lda_feature()

Step 5: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Logistic Regression from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of LR model with LDA extractor as shown in Figure below.

Change data ratio to 0.5 and learning rate 0.1, you will see the decision regions as shown in Figure below.

Step 6: Now, you will implement Perceptron classifier with LDA feature extractor.  Add the following code to the end of lda_feature() function to implement Perceptron classifier with LDA feature extractor:

if strList == 'Perceptron':
    self.sbIter.setEnabled(True)  
    self.dsbRate.setEnabled(True)  
    self.sbDepth.setEnabled(False) 
    self.sbNeighbor.setEnabled(False)
    self.dsbRatio.setEnabled(True)
            
    #Trains perceptron
    lda = LDA(n_components=2, store_covariance=True)
    ppn = Perceptron(max_iter=iterNum, eta0=learningRate, \
        random_state=1)
    self.X_train_lda = \
        lda.fit_transform(self.X_train_std,self.y_train)
    self.X_test_lda = lda.transform(self.X_test_std)
    ppn.fit(self.X_train_lda, self.y_train)   
            
    #Draws explained variance
    ylabelStr = 'Explained variance ratio'
    xlabelStr = 'Principal component index'
        self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\      
        self.widgetVariance.canvas)
        
    #Displays decision regions
    strTitle = 'Perceptron Classifier with LDA Extractor: ' + \
        str(ratio*100) + '% (DR) '
    strTitle += ' and ' +str(learningRate) + ' (LR)'
            
    self.display_decision(self.X_train_lda, self.y_train, \
        classifier=ppn,axisWidget=self.widgetDecision.canvas,\
        title=strTitle)
        
    #display graph
    self.graph_LR(self.widgetEpoch.canvas, self.accuracy_PPN)

Step 7: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Perceptron from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of perceptron model with LDA extractor as shown in Figure below.

Change data ratio to 0.5 and learning rate 0.01, you will see the decision regions as shown in Figure below.

Step 8: Then, you will implement Support Vector Machine (SVM) classifier with LDA feature extractor.  Add the following code to the end of lda_feature() function to implement SVM classifier with LDA feature extractor:

if strList == 'Support Vector Machine (SVM)':
    self.sbIter.setEnabled(True)  
    self.dsbRate.setEnabled(True)  
    self.sbDepth.setEnabled(False) 
    self.sbNeighbor.setEnabled(False)
    self.dsbRatio.setEnabled(True)
            
    #Trains SVM model
    lda = LDA(n_components=2, store_covariance=True)
    svm = make_pipeline(StandardScaler(), \
        SGDClassifier('hinge',max_iter=iterNum,eta0=learningRate, \
        tol=1e-3))
    self.X_train_lda = \
        lda.fit_transform(self.X_train_std,self.y_train)
    self.X_test_lda = lda.transform(self.X_test_std)
    svm.fit(self.X_train_lda, self.y_train)        
            
    #Draws explained variance
    ylabelStr = 'Explained variance ratio'
    xlabelStr = 'LDA component index'
    self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\
        self.widgetVariance.canvas)
            
    #Displays decision regions
    strTitle = 'SVM Classifier with PCA Extractor:' + \
        str(ratio*100) + '% (DR) '
    strTitle += ' and ' +str(learningRate) + ' (LR)'        
    self.display_decision(self.X_train_lda, self.y_train, \
        classifier=svm,axisWidget=self.widgetDecision.canvas,\
        title=strTitle)
                
    #display graph
    self.graph_LR(self.widgetEpoch.canvas, self.accuracy_SVM)

Step 9: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Support Vector Machine (SVM) from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of SVM model with LDA extractor as shown in Figure below.

Change data ratio to 0.5 and learning rate 0.5, you will see the decision regions as shown in Figure below.

Step 10: Then, you will implement Decision Tree (DT) classifier with LDA feature extractor.  Add the following code to the end of lda_feature() function to implement DT classifier with LDA feature extractor:

if strList == 'Decision Tree':
    self.dsbRatio.setEnabled(True)
    self.sbIter.setEnabled(False)  
    self.dsbRate.setEnabled(False)  
    self.sbDepth.setEnabled(True) 
    self.sbIter.setEnabled(False)
    self.sbNeighbor.setEnabled(False)
            
    #Trains Decision Tree model
    lda = LDA(n_components=2, store_covariance=True)
    tree = DecisionTreeClassifier(criterion='gini', \
        max_depth=depth,random_state=1)            
    self.X_train_lda = \
        lda.fit_transform(self.X_train_std,self.y_train)
    self.X_test_lda = lda.transform(self.X_test_std)
    tree.fit(self.X_train_lda, self.y_train)              
            
    #Draws explained variance
    ylabelStr = 'Explained variance ratio'
    xlabelStr = 'LDA component index'
    self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\
        self.widgetVariance.canvas)
            
    #Displays decision regions
    strTitle = 'DT Classifier with PCA Extractor: (DR)=' + \
        str(ratio*100) 
    strTitle += ' and Max Depth=' +str(depth)
    self.display_decision(self.X_train_lda, self.y_train,  \ 
        classifier=tree,axisWidget=self.widgetDecision.canvas,\
        title=strTitle)
                
    #display accuracy graph
    self.graph_DT(self.widgetEpoch.canvas, self.accuracy_DT)

Step 11: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Decision Tree from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of DT model with LDA extractor as shown in Figure below.

Change data ratio to 0.5 and max depth to 5, you will see the decision regions as shown in Figure below.

Step 12: Then, you will implement Random Forest (RF) classifier with LDA feature extractor.  Add the following code to the end of lda_feature() function to implement RF classifier with LDA feature extractor:

if strList == 'Random Forest':
    self.dsbRatio.setEnabled(True)
    self.sbIter.setEnabled(False)  
    self.dsbRate.setEnabled(False)  
    self.sbDepth.setEnabled(True) 
    self.sbIter.setEnabled(False)
    self.sbNeighbor.setEnabled(False)
            
    #Trains Random Forest model
    lda = LDA(n_components=2, store_covariance=True)
    forest = RandomForestClassifier(criterion='gini', \
        n_estimators=25,max_depth=depth,random_state=1)          
    self.X_train_lda = \
        lda.fit_transform(self.X_train_std,self.y_train)
    self.X_test_lda = lda.transform(self.X_test_std)
    forest.fit(self.X_train_lda, self.y_train)
           
    #Draws explained variance
    ylabelStr = 'Explained variance ratio'
    xlabelStr = 'LDA component index'
    self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\
        self.widgetVariance.canvas)
            
    #Draws explained variance
    strTitle = 'Random Forest Classifier with (DR)=' + str(ratio*100)
    strTitle += ' and Max Depth =' +str(depth)
    self.display_decision(self.X_train_lda, self.y_train, \
        classifier=forest,axisWidget=self.widgetDecision.canvas,\
        title=strTitle)            
                
    #display accuracy graph
    self.graph_DT(self.widgetEpoch.canvas, self.accuracy_RF)

Step 13: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Random Forest from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of RF model with LDA extractor as shown in Figure below.

Change data ratio to 0.5 and max depth to 5, you will see the decision regions as shown in Figure below.

Step 14: Lastly, you will implement Nearest Neighbor (KNN) classifier with LDA feature extractor.  Add the following code to the end of lda_feature() function to implement KNN classifier with LDA feature extractor:

if strList == 'Nearest Neighbor':
    self.dsbRatio.setEnabled(True)
    self.sbIter.setEnabled(False)  
    self.dsbRate.setEnabled(False)  
    self.sbDepth.setEnabled(False) 
    self.sbIter.setEnabled(False)
    self.sbNeighbor.setEnabled(True)
            
    #Trains Nearest Neighbor model
    lda = LDA(n_components=2, store_covariance=True)
    knn = KNeighborsClassifier(n_neighbors=neighbor, \
        p=2, metric='minkowski')          
    self.X_train_lda = \
        lda.fit_transform(self.X_train_std,self.y_train)
    self.X_test_lda = lda.transform(self.X_test_std)
    knn.fit(self.X_train_lda, self.y_train)
            
    #Draws explained variance
    ylabelStr = 'Explained variance ratio'
    xlabelStr = 'LDA component index'
    self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\
        self.widgetVariance.canvas)
            
    strTitle = 'KNN Classifier with (DR)=' + str(ratio*100)
    strTitle += ' and Neihbors =' +str(neighbor)
    self.display_decision(self.X_train_lda, self.y_train, \
        classifier=knn,axisWidget=self.widgetDecision.canvas,\
        title=strTitle)            
                
    #display accuracy graph
    self.graph_KNN(self.widgetEpoch.canvas, self.accuracy_KNN)

Step 15: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Nearest Neighbor from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of KNN model with LDA extractor as shown in Figure below.

Change data ratio to 0.5 and number of neighbors to 7, you will see the decision regions as shown in Figure below.

Learn From Scratch Neural Networks Using PyQt: Part 12