This content is powered by Balige Publishing. Visit this link (collaboration with Rismon Hasiholan Sianipar) PART 1 PART 2 PART 3 PART 4 PART 5 PART 6 PART 7 PART 8 PART 9 PART 10
Tutorial Steps To Implement Linear Discriminant Analysis (LDA) with Perceptron, Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), K-Nearest Neighbor (KNN) classifiers Using Scikit-Learn with PyQt
Step 1: Open Scikit_Classifier_Feature.py and add the following import statement:
Step 2: Now, you will implement logistic regression (LR) classifier with LDA feature extractor. Define draw_exp_var_LDA() function to display explained variance ratio of LDA:
Step 3: Define lda_feature() function to implement LDA feature extractor into every model used (LR as first tested model):
Step 4: Modify choose_feature() to involve Linear Discriminant Analysis (LDA) in cboFeature widget:
Step 5: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Logistic Regression from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of LR model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and learning rate 0.1, you will see the decision regions as shown in Figure below.
Step 6: Now, you will implement Perceptron classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement Perceptron classifier with LDA feature extractor:
Step 7: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Perceptron from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of perceptron model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and learning rate 0.01, you will see the decision regions as shown in Figure below.
Step 8: Then, you will implement Support Vector Machine (SVM) classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement SVM classifier with LDA feature extractor:
Step 9: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Support Vector Machine (SVM) from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of SVM model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and learning rate 0.5, you will see the decision regions as shown in Figure below.
Step 10: Then, you will implement Decision Tree (DT) classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement DT classifier with LDA feature extractor:
Step 11: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Decision Tree from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of DT model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and max depth to 5, you will see the decision regions as shown in Figure below.
Step 12: Then, you will implement Random Forest (RF) classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement RF classifier with LDA feature extractor:
Step 13: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Random Forest from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of RF model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and max depth to 5, you will see the decision regions as shown in Figure below.
Step 14: Lastly, you will implement Nearest Neighbor (KNN) classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement KNN classifier with LDA feature extractor:
Step 15: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Nearest Neighbor from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of KNN model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and number of neighbors to 7, you will see the decision regions as shown in Figure below.
Learn From Scratch Neural Networks Using PyQt: Part 12
Tutorial Steps To Implement Linear Discriminant Analysis (LDA) with Perceptron, Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), K-Nearest Neighbor (KNN) classifiers Using Scikit-Learn with PyQt
Step 1: Open Scikit_Classifier_Feature.py and add the following import statement:
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
Step 2: Now, you will implement logistic regression (LR) classifier with LDA feature extractor. Define draw_exp_var_LDA() function to display explained variance ratio of LDA:
def draw_exp_var_LDA(self,extractor,ylabelStr,xlabelStr,axisWidget): eigen_vals, eigen_vecs = np.linalg.eig(extractor.covariance_) tot = sum(eigen_vals.real) discr = [(i / tot) for i in sorted(eigen_vals.real, \ reverse=True)] cum_discr = np.cumsum(discr) axisWidget.axis1.clear() axisWidget.axis1.bar(range(1,14), discr, alpha=0.5, \ align='center',label='individual explained variance') axisWidget.axis1.step(range(1,14), cum_discr, \ where='mid',label='cumulative explained variance') axisWidget.axis1.set_ylabel(ylabelStr) axisWidget.axis1.set_xlabel(xlabelStr) axisWidget.axis1.legend(loc='best') axisWidget.draw()
Step 3: Define lda_feature() function to implement LDA feature extractor into every model used (LR as first tested model):
def lda_feature(self): iterNum = self.sbIter.value() self.dsbRate.setDecimals(5) learningRate = self.dsbRate.value() depth = self.sbDepth.value() neighbor = self.sbDepth.value() ratio = self.dsbRatio.value() self.load_data_ratio(ratio) item = self.listAlgorithm.currentItem() strList = item.text() if strList == 'Logistic Regression': self.sbIter.setEnabled(True) self.dsbRate.setEnabled(True) self.sbDepth.setEnabled(False) self.sbNeighbor.setEnabled(False) self.dsbRatio.setEnabled(True) #Trains LR model lda = LDA(n_components=2, store_covariance=True) lr = make_pipeline(StandardScaler(), \ SGDClassifier('log',max_iter=iterNum,eta0=learningRate, \ tol=1e-3)) self.X_train_lda = \ lda.fit_transform(self.X_train_std,self.y_train) self.X_test_lda = lda.transform(self.X_test_std) lr.fit(self.X_train_lda, self.y_train) #Displays explained variance ylabelStr = 'Explained variance ratio' xlabelStr = 'Principal component index' self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\ self.widgetVariance.canvas) #Displays decision regions strTitle = 'LR Classifier with LDA Extractor: ' + \ str(ratio*100) + '% Data Ratio ' strTitle += ' and Learning Rate ' +str(learningRate) self.display_decision(self.X_train_lda, self.y_train, \ classifier=lr,axisWidget=self.widgetDecision.canvas,\ title=strTitle) #display accuracy graph self.graph_LR(self.widgetEpoch.canvas, self.accuracy_LR)
Step 4: Modify choose_feature() to involve Linear Discriminant Analysis (LDA) in cboFeature widget:
def choose_feature(self): strCB = self.cboFeature.currentText() if strCB == 'Principal Component Analysis (PCA)': self.pca_feature() if strCB == 'Linear Discriminant Analysis (LDA)': self.lda_feature()
Step 5: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Logistic Regression from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of LR model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and learning rate 0.1, you will see the decision regions as shown in Figure below.
Step 6: Now, you will implement Perceptron classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement Perceptron classifier with LDA feature extractor:
if strList == 'Perceptron': self.sbIter.setEnabled(True) self.dsbRate.setEnabled(True) self.sbDepth.setEnabled(False) self.sbNeighbor.setEnabled(False) self.dsbRatio.setEnabled(True) #Trains perceptron lda = LDA(n_components=2, store_covariance=True) ppn = Perceptron(max_iter=iterNum, eta0=learningRate, \ random_state=1) self.X_train_lda = \ lda.fit_transform(self.X_train_std,self.y_train) self.X_test_lda = lda.transform(self.X_test_std) ppn.fit(self.X_train_lda, self.y_train) #Draws explained variance ylabelStr = 'Explained variance ratio' xlabelStr = 'Principal component index' self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\ self.widgetVariance.canvas) #Displays decision regions strTitle = 'Perceptron Classifier with LDA Extractor: ' + \ str(ratio*100) + '% (DR) ' strTitle += ' and ' +str(learningRate) + ' (LR)' self.display_decision(self.X_train_lda, self.y_train, \ classifier=ppn,axisWidget=self.widgetDecision.canvas,\ title=strTitle) #display graph self.graph_LR(self.widgetEpoch.canvas, self.accuracy_PPN)
Step 7: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Perceptron from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of perceptron model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and learning rate 0.01, you will see the decision regions as shown in Figure below.
Step 8: Then, you will implement Support Vector Machine (SVM) classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement SVM classifier with LDA feature extractor:
if strList == 'Support Vector Machine (SVM)': self.sbIter.setEnabled(True) self.dsbRate.setEnabled(True) self.sbDepth.setEnabled(False) self.sbNeighbor.setEnabled(False) self.dsbRatio.setEnabled(True) #Trains SVM model lda = LDA(n_components=2, store_covariance=True) svm = make_pipeline(StandardScaler(), \ SGDClassifier('hinge',max_iter=iterNum,eta0=learningRate, \ tol=1e-3)) self.X_train_lda = \ lda.fit_transform(self.X_train_std,self.y_train) self.X_test_lda = lda.transform(self.X_test_std) svm.fit(self.X_train_lda, self.y_train) #Draws explained variance ylabelStr = 'Explained variance ratio' xlabelStr = 'LDA component index' self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\ self.widgetVariance.canvas) #Displays decision regions strTitle = 'SVM Classifier with PCA Extractor:' + \ str(ratio*100) + '% (DR) ' strTitle += ' and ' +str(learningRate) + ' (LR)' self.display_decision(self.X_train_lda, self.y_train, \ classifier=svm,axisWidget=self.widgetDecision.canvas,\ title=strTitle) #display graph self.graph_LR(self.widgetEpoch.canvas, self.accuracy_SVM)
Step 9: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Support Vector Machine (SVM) from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of SVM model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and learning rate 0.5, you will see the decision regions as shown in Figure below.
Step 10: Then, you will implement Decision Tree (DT) classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement DT classifier with LDA feature extractor:
if strList == 'Decision Tree': self.dsbRatio.setEnabled(True) self.sbIter.setEnabled(False) self.dsbRate.setEnabled(False) self.sbDepth.setEnabled(True) self.sbIter.setEnabled(False) self.sbNeighbor.setEnabled(False) #Trains Decision Tree model lda = LDA(n_components=2, store_covariance=True) tree = DecisionTreeClassifier(criterion='gini', \ max_depth=depth,random_state=1) self.X_train_lda = \ lda.fit_transform(self.X_train_std,self.y_train) self.X_test_lda = lda.transform(self.X_test_std) tree.fit(self.X_train_lda, self.y_train) #Draws explained variance ylabelStr = 'Explained variance ratio' xlabelStr = 'LDA component index' self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\ self.widgetVariance.canvas) #Displays decision regions strTitle = 'DT Classifier with PCA Extractor: (DR)=' + \ str(ratio*100) strTitle += ' and Max Depth=' +str(depth) self.display_decision(self.X_train_lda, self.y_train, \ classifier=tree,axisWidget=self.widgetDecision.canvas,\ title=strTitle) #display accuracy graph self.graph_DT(self.widgetEpoch.canvas, self.accuracy_DT)
Step 11: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Decision Tree from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of DT model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and max depth to 5, you will see the decision regions as shown in Figure below.
Step 12: Then, you will implement Random Forest (RF) classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement RF classifier with LDA feature extractor:
if strList == 'Random Forest': self.dsbRatio.setEnabled(True) self.sbIter.setEnabled(False) self.dsbRate.setEnabled(False) self.sbDepth.setEnabled(True) self.sbIter.setEnabled(False) self.sbNeighbor.setEnabled(False) #Trains Random Forest model lda = LDA(n_components=2, store_covariance=True) forest = RandomForestClassifier(criterion='gini', \ n_estimators=25,max_depth=depth,random_state=1) self.X_train_lda = \ lda.fit_transform(self.X_train_std,self.y_train) self.X_test_lda = lda.transform(self.X_test_std) forest.fit(self.X_train_lda, self.y_train) #Draws explained variance ylabelStr = 'Explained variance ratio' xlabelStr = 'LDA component index' self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\ self.widgetVariance.canvas) #Draws explained variance strTitle = 'Random Forest Classifier with (DR)=' + str(ratio*100) strTitle += ' and Max Depth =' +str(depth) self.display_decision(self.X_train_lda, self.y_train, \ classifier=forest,axisWidget=self.widgetDecision.canvas,\ title=strTitle) #display accuracy graph self.graph_DT(self.widgetEpoch.canvas, self.accuracy_RF)
Step 13: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Random Forest from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of RF model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and max depth to 5, you will see the decision regions as shown in Figure below.
Step 14: Lastly, you will implement Nearest Neighbor (KNN) classifier with LDA feature extractor. Add the following code to the end of lda_feature() function to implement KNN classifier with LDA feature extractor:
if strList == 'Nearest Neighbor': self.dsbRatio.setEnabled(True) self.sbIter.setEnabled(False) self.dsbRate.setEnabled(False) self.sbDepth.setEnabled(False) self.sbIter.setEnabled(False) self.sbNeighbor.setEnabled(True) #Trains Nearest Neighbor model lda = LDA(n_components=2, store_covariance=True) knn = KNeighborsClassifier(n_neighbors=neighbor, \ p=2, metric='minkowski') self.X_train_lda = \ lda.fit_transform(self.X_train_std,self.y_train) self.X_test_lda = lda.transform(self.X_test_std) knn.fit(self.X_train_lda, self.y_train) #Draws explained variance ylabelStr = 'Explained variance ratio' xlabelStr = 'LDA component index' self.draw_exp_var_LDA(lda,ylabelStr,xlabelStr,\ self.widgetVariance.canvas) strTitle = 'KNN Classifier with (DR)=' + str(ratio*100) strTitle += ' and Neihbors =' +str(neighbor) self.display_decision(self.X_train_lda, self.y_train, \ classifier=knn,axisWidget=self.widgetDecision.canvas,\ title=strTitle) #display accuracy graph self.graph_KNN(self.widgetEpoch.canvas, self.accuracy_KNN)
Step 15: Run Scikit_Classifier_Feature.py and choose Linear Discriminant Analysis (LDA) from cboFeature widget and Nearest Neighbor from listAlgorithm widget. You will see explained variance ratio, decision regions, and prediction accuracy of KNN model with LDA extractor as shown in Figure below.
Change data ratio to 0.5 and number of neighbors to 7, you will see the decision regions as shown in Figure below.
Learn From Scratch Neural Networks Using PyQt: Part 12
No comments:
Post a Comment