This content is powered by Balige Publishing. Visit this link (collaboration with Rismon Hasiholan Sianipar) PART 1
In this tutorial, you will learn how to use Pandas, NumPy and other libraries to perform simple classification using perceptron and Adaline (adaptive linear neuron). The dataset used is Iris dataset directly from the UCI Machine Learning Repository.
Tutorial Steps To Implement Adaline (ADAptive LInear NEuron) with PyQt
Step 1: Open gui_perceptron.ui form with Qt Designer. Add a Spin Box widget and set its minimum, maximum, singleStep, and value properties to 10, 1000, 5, and 10. Set its objectName property to sbIter.
Step 2: Add a List Widget onto form and its objectName property to listAlgorithm. Double click on the widget and add four items as shown in Figure below.
The form now looks as shown in Figure below.
Step 3: In Perceptron.py, modify display_epoch() and display_decision() functions by adding new input parameter to give title:
Step 4: Modify load_data() function to invoke new defined algo_NN() function as follows:
Step 5: Connect valueChanged() signal of sbIter and dsbRate widgets with algo_NN() function and put them inside __init__() method:
Step 6: Run Perceptron.py to see the result as shown in Figure below.
You now can change the value of data length, number of iterations, and learning rate, as shown in Figure below.
Step 7: Create a new class, AdalineStochastic, and save it as AdalineStochastic_Class.py:
Step 8: Put import statement below in Perceptron.py:
Step 9: Modify algo_NN() function to read selected item of listAlgorithm widget and determine action accordingly as follows:
Step 10: Run Perceptron.py and click on Load Data button. From list widget, choose Adaline Gradient Descent. Set iteration to 30 and learning rate to 0.0001. The result is shown in Figure below.
Then, choose Adaline Gradient Descent with Feature Scaling. Set iteration to 10 and learning rate to 0.01070. The result is shown in Figure below.
Then, choose Adaline Stochastic Gradient Descent. Set iteration to 10 and learning rate to 0.3. The result is shown in Figure below.
Below is the full script of Perceptron.py:
Learn From Scratch Neural Networks Using PyQt: Part 3
In this tutorial, you will learn how to use Pandas, NumPy and other libraries to perform simple classification using perceptron and Adaline (adaptive linear neuron). The dataset used is Iris dataset directly from the UCI Machine Learning Repository.
Tutorial Steps To Implement Adaline (ADAptive LInear NEuron) with PyQt
Step 1: Open gui_perceptron.ui form with Qt Designer. Add a Spin Box widget and set its minimum, maximum, singleStep, and value properties to 10, 1000, 5, and 10. Set its objectName property to sbIter.
Then, add a Double Spin Box widget and set its minimum, maximum, singleStep, and value properties to 0.0001, 0.5, 0.0001, and 0.1. Set its objectName property to dsbRate.
Add a Group Box widget and set its objectName property to gbNNParam. Put sbLength, sbIter, and dsbRate widgets inside this group box.
The form now looks as shown in Figure below.
Step 3: In Perceptron.py, modify display_epoch() and display_decision() functions by adding new input parameter to give title:
def display_epoch(self,ppn,xVal, yVal, title): self.widgetEpoch.canvas.axis1.clear() self.widgetEpoch.canvas.axis1.plot(xVal,yVal, marker='o') self.widgetEpoch.canvas.axis1.set_xlabel('Epochs') self.widgetEpoch.canvas.axis1.set_ylabel('Number of updates') self.widgetEpoch.canvas.axis1.set_title(title) self.widgetEpoch.canvas.axis1.grid() self.widgetEpoch.canvas.draw() def display_decision(self,ppn,title): self.widgetDecision.canvas.axis1.clear() # setup marker generator and color map markers = ('s', 'x', 'o', '^', 'v') colors = ('red', 'blue', 'lightgreen', 'gray', 'cyan') cmap = ListedColormap(colors[:len(np.unique(self.y))]) # plot the decision surface x1_min, x1_max = self.X[:, 0].min() - 1, self.X[:, 0].max() + 1 x2_min, x2_max = self.X[:, 1].min() - 1, self.X[:, 1].max() + 1 xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, 0.01), np.arange(x2_min, x2_max, 0.01)) Z = ppn.process(np.array([xx1.ravel(), xx2.ravel()]).T) Z = Z.reshape(xx1.shape) self.widgetDecision.canvas.axis1.contourf(xx1, \ xx2, Z, alpha=0.5, cmap=cmap) self.widgetDecision.canvas.axis1.set_xlim(xx1.min(), xx1.max()) self.widgetDecision.canvas.axis1.set_ylim(xx2.min(), xx2.max()) # plot class samples for idx, cl in enumerate(np.unique(self.y)): self.widgetDecision.canvas.axis1.scatter(\ x=self.X[self.y == cl, 0], y=self.X[self.y == cl, 1], alpha=0.8, c=colors[idx], marker=markers[idx], label=cl, edgecolor='black') self.widgetDecision.canvas.axis1.set_xlabel('sepal length [cm]') self.widgetDecision.canvas.axis1.set_ylabel('petal length [cm]') self.widgetDecision.canvas.axis1.legend(loc='upper left') self.widgetDecision.canvas.axis1.set_title(title) self.widgetDecision.canvas.draw()
Step 4: Modify load_data() function to invoke new defined algo_NN() function as follows:
def load_data(self): df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data',header=None) #df = pd.read_csv('iris.data',header=None) self.dataLength = self.sbLength.value() # select setosa and versicolor y = df.iloc[0:self.dataLength, 4].values self.y = np.where(y == 'Iris-setosa', -1, 1) # extract sepal length and petal length self.X = df.iloc[0:self.dataLength, [0, 2]].values #display data on table self.display_table(df) #invoke algorithm type self.algo_NN() self.gbNNParam.setEnabled(True) self.pbLoad.setEnabled(False) def algo_NN(self): iterNum = self.sbIter.value() self.dsbRate.setDecimals(5) learningRate = self.dsbRate.value() #Perceptron #display errors ppn = Perceptron(eta=learningRate, n_iter=iterNum) ppn.train(self.X, self.y) strTitle = 'Perceptron: ' + str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(ppn.errors_) + 1) yVal = ppn.errors_ self.display_epoch(ppn,xVal,yVal,strTitle) #display decision self.display_decision(ppn,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle)
Step 5: Connect valueChanged() signal of sbIter and dsbRate widgets with algo_NN() function and put them inside __init__() method:
def __init__(self): QMainWindow.__init__(self) loadUi("gui_perceptron.ui",self) self.setWindowTitle("GUI Demo of Perceptron") self.pbLoad.clicked.connect(self.display_data) self.addToolBar(NavigationToolbar(self.widgetData.canvas, self)) self.sbLength.valueChanged.connect(self.display_data) self.sbIter.valueChanged.connect(self.algo_NN) self.dsbRate.valueChanged.connect(self.algo_NN) self.gbNNParam.setEnabled(False)
Step 6: Run Perceptron.py to see the result as shown in Figure below.
You now can change the value of data length, number of iterations, and learning rate, as shown in Figure below.
Step 7: Create a new class, AdalineStochastic, and save it as AdalineStochastic_Class.py:
#AdalineStochastic_Class.py import numpy as np class AdalineStochastic(object): """ADAptive LInear NEuron classifier with Stochastic gradient descent rule Parameters ------------ eta : float Learning rate (between 0.0 and 1.0) n_iter : int Passes over the training dataset. shuffle : bool (default: True) Shuffles training data every epoch if True to prevent cycles. random_state : int Random number generator seed for random weight initialization. Attributes ----------- w_ : 1d-array Weights after training. cost_ : list Sum-of-squares cost function value in each epoch. """ def __init__(self, eta=0.01, n_iter=50, shuffle=True, \ random_state=None): self.eta = eta self.n_iter = n_iter self.w_initialized = False self.shuffle = shuffle self.random_state = random_state def train(self, X, y): """Trains NN Parameters ---------- X : {array-like}, shape = [n_samples, n_features] Training vectors, where n_samples is the number of samples and n_features is the number of features. y : array-like, shape = [n_samples] Target values. Returns ------- self : object """ self._initialize_weights(X.shape[1]) self.cost_ = [] for i in range(self.n_iter): if self.shuffle: X, y = self._shuffle(X, y) cost = [] for xi, target in zip(X, y): cost.append(self._update_weights(xi, target)) avg_cost = sum(cost) / len(y) self.cost_.append(avg_cost) return self def partial_train(self, X, y): """Fit training data without reinitializing the weights""" if not self.w_initialized: self._initialize_weights(X.shape[1]) if y.ravel().shape[0] > 1: for xi, target in zip(X, y): self._update_weights(xi, target) else: self._update_weights(X, y) return self def _shuffle(self, X, y): """Shuffle training data""" r = self.rgen.permutation(len(y)) return X[r], y[r] def _initialize_weights(self, m): """Initialize weights to small random numbers""" self.rgen = np.random.RandomState(self.random_state) self.w_ = self.rgen.normal(loc=0.0, scale=0.01, size=1 + m) self.w_initialized = True def _update_weights(self, xi, target): """Apply Adaline learning rule to update the weights""" output = self.activation(self.NN_input(xi)) error = (target - output) self.w_[1:] += self.eta * xi.dot(error) self.w_[0] += self.eta * error cost = 0.5 * error**2 return cost def NN_input(self, X): #Calculate net input return np.dot(X, self.w_[1:]) + self.w_[0] def activation(self, X): """Compute linear activation""" return X def process(self, X): #passing training data to net to get output return np.where(self.activation(\ self.NN_input(X)) >= 0.0, 1, -1)
Step 8: Put import statement below in Perceptron.py:
from AdalineStochastic_Class import AdalineStochastic
Step 9: Modify algo_NN() function to read selected item of listAlgorithm widget and determine action accordingly as follows:
def algo_NN(self): iterNum = self.sbIter.value() self.dsbRate.setDecimals(5) learningRate = self.dsbRate.value() item = self.listAlgorithm.currentItem() strList = item.text() if strList == 'Perceptron': #Perceptron ppn = Perceptron(eta=learningRate, n_iter=iterNum) ppn.train(self.X, self.y) #display errors strTitle = 'Perceptron: ' + str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(ppn.errors_) + 1) yVal = ppn.errors_ self.display_epoch(ppn,xVal,yVal,strTitle) #display decision self.display_decision(ppn,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle) if strList == 'Adaline Gradient Descent': #Adaline Gradient Descent ada = Adaline(n_iter=iterNum, \ eta=learningRate).train(self.X, self.y) #display errors strTitle = 'Adaline Gradient Descent: ' + \ str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(ada.cost_) + 1) yVal = ada.cost_ self.display_epoch(ada,xVal,yVal,strTitle) #display decision self.display_decision(ada,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle) if strList == 'Adaline Gradient Descent with Feature Scaling': #Adaline Gradient Descent with Feature Scaling X_std = np.copy(self.X) X_std[:,0] = \ (self.X[:,0] - self.X[:,0].mean()) / self.X[:,0].std() X_std[:,1] = \ (self.X[:,1] - self.X[:,1].mean()) / self.X[:,1].std() ada = Adaline(eta=learningRate, \ n_iter=iterNum).train(X_std, self.y) #display errors strTitle = 'Adaline GD with Feature Scaling: ' \ + str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(ada.cost_) + 1) yVal = ada.cost_ self.display_epoch(ada,xVal,yVal,strTitle) #display decision self.display_decision(ada,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle) if strList == 'Adaline Stochastic Gradient Descent': #Adaline Stochastic Gradient Descent X_std = np.copy(self.X) X_std[:,0] = \ (self.X[:,0] - self.X[:,0].mean()) / self.X[:,0].std() X_std[:,1] = \ (self.X[:,1] - self.X[:,1].mean()) / self.X[:,1].std() adaSto = AdalineStochastic(eta=learningRate, \ n_iter=iterNum, random_state=1) adaSto.train(X_std, self.y) #display errors strTitle = 'Adaline Stochastic Gradient Descent: ' + \ str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(adaSto.cost_) + 1) yVal = adaSto.cost_ self.display_epoch(adaSto,xVal,yVal,strTitle) #display decision self.display_decision(adaSto,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle)
Step 10: Run Perceptron.py and click on Load Data button. From list widget, choose Adaline Gradient Descent. Set iteration to 30 and learning rate to 0.0001. The result is shown in Figure below.
Then, choose Adaline Gradient Descent with Feature Scaling. Set iteration to 10 and learning rate to 0.01070. The result is shown in Figure below.
Then, choose Adaline Stochastic Gradient Descent. Set iteration to 10 and learning rate to 0.3. The result is shown in Figure below.
Below is the full script of Perceptron.py:
#Perceptron.py from PyQt5.QtWidgets import * from PyQt5.uic import loadUi from matplotlib.backends.backend_qt5agg import (NavigationToolbar2QT as NavigationToolbar) from matplotlib.colors import ListedColormap from Perceptron_Class import Perceptron from Adaline_Class import Adaline from AdalineStochastic_Class import AdalineStochastic import numpy as np import pandas as pd class DemoGUIPerceptron(QMainWindow): def __init__(self): QMainWindow.__init__(self) loadUi("gui_perceptron.ui",self) self.setWindowTitle("GUI Demo of Perceptron") self.pbLoad.clicked.connect(self.display_data) self.addToolBar(NavigationToolbar(self.widgetData.canvas, self)) self.sbLength.valueChanged.connect(self.display_data) self.sbIter.valueChanged.connect(self.algo_NN) self.dsbRate.valueChanged.connect(self.algo_NN) self.gbNNParam.setEnabled(False) self.listAlgorithm.setEnabled(False) self.listAlgorithm.clicked.connect(self.algo_NN) def load_data(self): df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data',header=None) #df = pd.read_csv('iris.data',header=None) self.dataLength = self.sbLength.value() # select setosa and versicolor y = df.iloc[0:self.dataLength, 4].values self.y = np.where(y == 'Iris-setosa', -1, 1) # extract sepal length and petal length self.X = df.iloc[0:self.dataLength, [0, 2]].values #display data on table self.display_table(df) #invoke algorithm type self.listAlgorithm.setCurrentRow(0) self.algo_NN() self.gbNNParam.setEnabled(True) self.pbLoad.setEnabled(False) self.listAlgorithm.setEnabled(True) def algo_NN(self): iterNum = self.sbIter.value() self.dsbRate.setDecimals(5) learningRate = self.dsbRate.value() item = self.listAlgorithm.currentItem() strList = item.text() if strList == 'Perceptron': #Perceptron ppn = Perceptron(eta=learningRate, n_iter=iterNum) ppn.train(self.X, self.y) #display errors strTitle = 'Perceptron: ' + str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(ppn.errors_) + 1) yVal = ppn.errors_ self.display_epoch(ppn,xVal,yVal,strTitle) #display decision self.display_decision(ppn,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle) if strList == 'Adaline Gradient Descent': #Adaline Gradient Descent ada = Adaline(n_iter=iterNum, eta=learningRate).train(self.X, self.y) #display errors strTitle = 'Adaline Gradient Descent: ' + str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(ada.cost_) + 1) yVal = ada.cost_ self.display_epoch(ada,xVal,yVal,strTitle) #display decision self.display_decision(ada,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle) if strList == 'Adaline Gradient Descent with Feature Scaling': #Adaline Gradient Descent with Feature Scaling X_std = np.copy(self.X) X_std[:,0] = (self.X[:,0] - self.X[:,0].mean()) / self.X[:,0].std() X_std[:,1] = (self.X[:,1] - self.X[:,1].mean()) / self.X[:,1].std() ada = Adaline(eta=learningRate, n_iter=iterNum).train(X_std, self.y) #display errors strTitle = 'Adaline GD with Feature Scaling: ' + str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(ada.cost_) + 1) yVal = ada.cost_ self.display_epoch(ada,xVal,yVal,strTitle) #display decision self.display_decision(ada,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle) if strList == 'Adaline Stochastic Gradient Descent': #Adaline Stochastic Gradient Descent X_std = np.copy(self.X) X_std[:,0] = (self.X[:,0] - self.X[:,0].mean()) / self.X[:,0].std() X_std[:,1] = (self.X[:,1] - self.X[:,1].mean()) / self.X[:,1].std() adaSto = AdalineStochastic(eta=learningRate, n_iter=iterNum, random_state=1) adaSto.train(X_std, self.y) #display errors strTitle = 'Adaline Stochastic Gradient Descent: ' + str(iterNum) + ' Iterations' strTitle += ' and Learning Rate ' +str(learningRate) xVal = range(1, len(adaSto.cost_) + 1) yVal = adaSto.cost_ self.display_epoch(adaSto,xVal,yVal,strTitle) #display decision self.display_decision(adaSto,strTitle) self.widgetDecision.canvas.axis1.set_title(strTitle) def display_data(self): self.load_data() dataHalf = int(self.dataLength/2) # plot data self.widgetData.canvas.axis1.clear() self.widgetData.canvas.axis1.scatter(self.X[:dataHalf, 0], \ self.X[:dataHalf, 1], color='red', marker='o', label='setosa') self.widgetData.canvas.axis1.scatter(self.X[dataHalf:self.dataLength, \ 0], self.X[dataHalf:self.dataLength, 1], color='blue', marker='x', label='versicolor') self.widgetData.canvas.axis1.set_xlabel('sepal length [cm]') self.widgetData.canvas.axis1.set_ylabel('petal length [cm]') self.widgetData.canvas.axis1.legend(loc='upper left') title = 'sepal and petal length with data length ' + \ str(self.sbLength.value()) self.widgetData.canvas.axis1.set_title(title) self.widgetData.canvas.draw() self.algo_NN() def display_table(self,df): # show data on table widget self.write_df_to_qtable(df,self.tableData) self.tableData.setHorizontalHeaderLabels(['0', '1', '2', '3', 'Name']) self.tableData.setColumnWidth(0, 40) self.tableData.setColumnWidth(1, 40) self.tableData.setColumnWidth(2, 40) self.tableData.setColumnWidth(3, 40) styleH = "::section {""background-color: red; }" self.tableData.horizontalHeader().setStyleSheet(styleH) styleV = "::section {""background-color: yellow; }" self.tableData.verticalHeader().setStyleSheet(styleV) # Takes a df and writes it to a qtable provided. df headers become qtable headers @staticmethod def write_df_to_qtable(df,table): table.setRowCount(df.shape[0]) table.setColumnCount(df.shape[1]) # getting data from df is computationally costly so convert it to array first df_array = df.values for row in range(df.shape[0]): for col in range(df.shape[1]): table.setItem(row, col, \ QTableWidgetItem(str(df_array[row,col]))) def display_epoch(self,ppn,xVal, yVal, title): self.widgetEpoch.canvas.axis1.clear() self.widgetEpoch.canvas.axis1.plot(xVal,yVal, marker='o') self.widgetEpoch.canvas.axis1.set_xlabel('Epochs') self.widgetEpoch.canvas.axis1.set_ylabel('Sum-squared-error') self.widgetEpoch.canvas.axis1.set_title(title) self.widgetEpoch.canvas.axis1.grid() self.widgetEpoch.canvas.draw() def display_decision(self,ppn,title): self.widgetDecision.canvas.axis1.clear() # setup marker generator and color map markers = ('s', 'x', 'o', '^', 'v') colors = ('red', 'blue', 'lightgreen', 'gray', 'cyan') cmap = ListedColormap(colors[:len(np.unique(self.y))]) # plot the decision surface x1_min, x1_max = self.X[:, 0].min() - 1, self.X[:, 0].max() + 1 x2_min, x2_max = self.X[:, 1].min() - 1, self.X[:, 1].max() + 1 xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, 0.01), np.arange(x2_min, x2_max, 0.01)) Z = ppn.process(np.array([xx1.ravel(), xx2.ravel()]).T) Z = Z.reshape(xx1.shape) self.widgetDecision.canvas.axis1.contourf(xx1, xx2, Z, alpha=0.5, cmap=cmap) self.widgetDecision.canvas.axis1.set_xlim(xx1.min(), xx1.max()) self.widgetDecision.canvas.axis1.set_ylim(xx2.min(), xx2.max()) # plot class samples for idx, cl in enumerate(np.unique(self.y)): self.widgetDecision.canvas.axis1.scatter(x=self.X[self.y == cl, 0], y=self.X[self.y == cl, 1], alpha=0.8, c=colors[idx], marker=markers[idx], label=cl, edgecolor='black') self.widgetDecision.canvas.axis1.set_xlabel('sepal length [cm]') self.widgetDecision.canvas.axis1.set_ylabel('petal length [cm]') self.widgetDecision.canvas.axis1.legend(loc='upper left') self.widgetDecision.canvas.axis1.set_title(title) self.widgetDecision.canvas.draw() if __name__ == '__main__': import sys app = QApplication(sys.argv) ex = DemoGUIPerceptron() ex.show() sys.exit(app.exec_())
Learn From Scratch Neural Networks Using PyQt: Part 3
No comments:
Post a Comment