This is the product of Balige Academy, North Sumatera.
Balige Academy Team
Vivian Siahaan
RIsmon Hasiholan Sianipar
HORASSS!!!
This "Data Visualization, Time-Series Forecasting, and Prediction using Machine Learning with Tkinter" project is a comprehensive and multifaceted application that leverages data visualization, time-series forecasting, and machine learning techniques to gain insights into bitcoin data and make predictions. This project serves as a valuable tool for financial analysts, traders, and investors seeking to make informed decisions in the stock market.
The project begins with data visualization, where historical bitcoin market data is visually represented using various plots and charts. This provides users with an intuitive understanding of the data's trends, patterns, and fluctuations. Features distribution analysis is conducted to assess the statistical properties of the dataset, helping users identify key characteristics that may impact forecasting and prediction.
One of the project's core functionalities is time-series forecasting. Through a user-friendly interface built with Tkinter, users can select a stock symbol and specify the time horizon for forecasting. The project supports multiple machine learning regressors, such as Linear Regression, Decision Trees, Random Forests, Gradient Boosting, Extreme Gradient Boosting, Multi-Layer Perceptron, Lasso, Ridge, AdaBoost, and KNN, allowing users to choose the most suitable algorithm for their forecasting needs. Time-series forecasting is crucial for making predictions about stock prices, which is essential for investment strategies.
The project employs various machine learning regressors to predict the adjusted closing price of bitcoin stock. By training these models on historical data, users can obtain predictions for future adjusted closing prices. This information is invaluable for traders and investors looking to make buy or sell decisions. The project also incorporates hyperparameter tuning and cross-validation to enhance the accuracy of these predictions.
These models employ metrics such as Mean Absolute Error (MAE), which quantifies the average absolute discrepancy between predicted values and actual values. Lower MAE values signify superior model performance. Additionally, Mean Squared Error (MSE) is used to calculate the average squared differences between predicted and actual values, with lower MSE values indicating better model performance. Root Mean Squared Error (RMSE), derived from MSE, provides insights in the same units as the target variable and is valued for its lower values, denoting superior performance. Lastly, R-squared (R2) evaluates the fraction of variance in the target variable that can be predicted from independent variables, with higher values signifying better model fit. An R2 of 1 implies a perfect model fit.
In addition to close price forecasting, the project extends its capabilities to predict daily returns. By implementing grid search, users can fine-tune the hyperparameters of machine learning models such as Random Forests, Gradient Boosting, Support Vector, Decision Tree, Gradient Boosting, Extreme Gradient Boosting, Multi-Layer Perceptron, and AdaBoost Classifiers. This optimization process aims to maximize the predictive accuracy of daily returns. Accurate daily return predictions are essential for assessing risk and formulating effective trading strategies.
Key metrics in these classifiers encompass Accuracy, which represents the ratio of correctly predicted instances to the total number of instances, Precision, which measures the proportion of true positive predictions among all positive predictions, and Recall (also known as Sensitivity or True Positive Rate), which assesses the proportion of true positive predictions among all actual positive instances. The F1-Score serves as the harmonic mean of Precision and Recall, offering a balanced evaluation, especially when considering the trade-off between false positives and false negatives. The ROC Curve illustrates the trade-off between Recall and False Positive Rate, while the Area Under the ROC Curve (AUC-ROC) summarizes this trade-off. The Confusion Matrix provides a comprehensive view of classifier performance by detailing true positives, true negatives, false positives, and false negatives, facilitating the computation of various metrics like accuracy, precision, and recall. The selection of these metrics hinges on the project's specific objectives and the characteristics of the dataset, ensuring alignment with the intended goals and the ramifications of false positives and false negatives, which hold particular significance in financial contexts where decisions can have profound consequences.
Overall, the "Data Visualization, Time-Series Forecasting, and Prediction using Machine Learning with Tkinter" project serves as a powerful and user-friendly platform for financial data analysis and decision-making. It bridges the gap between complex machine learning techniques and accessible user interfaces, making financial analysis and prediction more accessible to a broader audience. With its comprehensive features, this project empowers users to gain insights from historical data, make informed investment decisions, and develop effective trading strategies in the dynamic world of finance.
#main_program.py import os import tkinter as tk from tkinter import * from PIL import Image, ImageTk from design_window import Design_Window from process_data import Process_Data from helper_plot import Helper_Plot from regression import Regression from machine_learning import Machine_Learning class Main_Program: def __init__(self, root): self.initialize() def initialize(self): self.root = root width = 1560 height = 790 self.root.geometry(f"{width}x{height}") self.root.title("TIME-SERIES ANALYSIS, FORECASTING, AND PREDICTION USING MACHINE LEARNING") #Creates necessary objects self.obj_window = Design_Window() self.obj_data = Process_Data() self.obj_plot = Helper_Plot() self.obj_reg = Regression() self.obj_ML = Machine_Learning() #Places widgets in root self.obj_window.add_widgets(self.root) #Reads dataset self.df = self.obj_data.preprocess() #Create dummy dataset for visualization self.df_dummy = self.df.copy() self.df_dummy = self.obj_data.create_dummy(self.df_dummy) #Binds event self.binds_event() #Computes technical indicators self.df_final = self.df.copy() self.df_final = self.obj_data.technical_indicators(self.df_final) #Extracts input and output variables for regression self.obj_reg.splitting_data_regression(self.df_final) #Extracts input and output variables for prediction self.obj_ML.oversampling_splitting(self.df_final) #turns off combo_reg and combo_pred after splitting is done self.obj_window.combo_reg['state'] = 'disabled' self.obj_window.combo_pred['state'] = 'disabled' def binds_event(self): #Binds button1 to shows_table() function #Shows table if user clicks LOAD DATASET self.obj_window.btn_load.config(command = lambda:self.obj_plot.shows_table(self.root, self.df, 1250, 600, "Bitcoin Dataset")) #Binds listbox to choose_list_widget() function self.obj_window.listbox.bind("<<ListboxSelect>>", self.choose_list_widget) # Binds combo_year to choose_combo_year() self.obj_window.combo_year.bind("<<ComboboxSelected>>", self.choose_combo_year) # Binds combo_month to choose_combobox_month() self.obj_window.combo_month.bind("<<ComboboxSelected>>", self.choose_combobox_month) #Binds btn_reg to split_regression() function self.obj_window.btn_reg.config(command = self.split_regression) #Binds combo_pred to split_prediction() function self.obj_window.btn_pred.config(command = self.split_prediction) # Binds combo_tech to choose_combo_tech() self.obj_window.combo_tech.bind("<<ComboboxSelected>>", self.choose_combo_tech) # Binds combo_reg to choose_combo_reg() self.obj_window.combo_reg.bind("<<ComboboxSelected>>", self.choose_combo_reg) # Binds combo_pred to choose_combo_pred() self.obj_window.combo_pred.bind("<<ComboboxSelected>>", self.choose_combo_pred) def choose_list_widget(self, event): chosen = self.obj_window.listbox.get(self.obj_window.listbox.curselection()) print(chosen) self.obj_plot.choose_plot(self.df, self.df_dummy, chosen, self.obj_window.figure1, self.obj_window.canvas1, self.obj_window.figure2, self.obj_window.canvas2) def choose_combo_year(self, event): chosen = self.obj_window.combo_year.get() year_data_mean, year_data_ewm, year_norm = self.obj_data.normalize_year_wise_data(self.df) print(year_data_mean) self.obj_plot.choose_year_wise(self.df, year_data_mean, year_data_ewm, year_norm, chosen, self.obj_window.figure1, self.obj_window.canvas1, self.obj_window.figure2, self.obj_window.canvas2) def choose_combobox_month(self, event): chosen = self.obj_window.combo_month.get() month_data_mean, month_data_ewm, month_norm = self.obj_data.normalize_month_wise_data(self.df) self.obj_plot.choose_month_wise(self.df, self.df_dummy, month_data_mean, month_data_ewm, month_norm, chosen, self.obj_window.figure1, self.obj_window.canvas1, self.obj_window.figure2, self.obj_window.canvas2) def choose_combo_tech(self, event): chosen = self.obj_window.combo_tech.get() self.obj_plot.choose_plot_technical_indicators(self.df_final, chosen, self.obj_window.figure1, self.obj_window.canvas1, self.obj_window.figure2, self.obj_window.canvas2) def split_regression(self): file_path = os.getcwd()+"/X_final_reg.pkl" if os.path.exists(file_path): self.X_Ori, self.X_final_reg, self.X_train_reg, self.X_test_reg, \ self.X_val_reg, self.y_final_reg, self.y_train_reg, \ self.y_test_reg, self.y_val_reg = self.obj_reg.load_regression_files() else: self.obj_reg.splitting_data_regression(self.df_final) self.X_Ori, self.X_final_reg, self.X_train_reg, self.X_test_reg, self.X_val_reg, self.y_final_reg, self.y_train_reg, self.y_test_reg, self.y_val_reg = self.obj_reg.load_regression_files() print("Loading regression files done...") #turns on combo_reg after splitting is done self.obj_window.combo_reg['state'] = 'normal' self.obj_window.btn_reg.config(state="disabled") def choose_combo_reg(self, event): chosen = self.obj_window.combo_reg.get() self.obj_plot.choose_plot_regression(chosen, self.X_final_reg, self.X_train_reg, self.X_test_reg, self.X_val_reg, self.y_final_reg, self.y_train_reg, self.y_test_reg, self.y_val_reg, self.obj_window.figure1, self.obj_window.canvas1, self.obj_window.figure2, self.obj_window.canvas2) def split_prediction(self): file_path = os.getcwd()+"/X_train.pkl" if os.path.exists(file_path): self.X_train, self.X_test, self.y_train, self.y_test = self.obj_ML.load_files() else: self.obj_ML.oversampling_splitting(self.X, self.y) self.X_train, self.X_test, self.y_train, self.y_test = self.obj_ML.load_files() print("Loading files done...") #turns on combo_pred after splitting is done self.obj_window.combo_pred['state'] = 'normal' self.obj_window.btn_pred.config(state="disabled") def choose_combo_pred(self, event): chosen = self.obj_window.combo_pred.get() self.obj_plot.choose_plot_ML(self.root, chosen, self.X_train, self.X_test, self.y_train, self.y_test, self.obj_window.figure1, self.obj_window.canvas1, self.obj_window.figure2, self.obj_window.canvas2) if __name__ == "__main__": root = tk.Tk() app = Main_Program(root) root.mainloop() #design_window.py import tkinter as tk from tkinter import ttk from matplotlib.figure import Figure from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg class Design_Window: def add_widgets(self, root): #Set styles self.set_style(root) #Adds button(s) self.add_buttons(root) #Adds canvasses self.add_canvas(root) #Adds labels self.add_labels(root) #Adds listbox widget self.add_listboxes(root) #Adds combobox widget self.add_comboboxes(root) def set_style(self, root): # variables created for colors ebg = '#404040' fg = '#FFFFFF' style = ttk.Style() # Be sure to include this or style.map() won't function as expected. style.theme_use('alt') # the following alters the Listbox root.option_add('*TCombobox*Listbox.Background', ebg) root.option_add('*TCombobox*Listbox.Foreground', fg) root.option_add('*TCombobox*Listbox.selectBackground', fg) root.option_add('*TCombobox*Listbox.selectForeground', ebg) # the following alters the Combobox entry field style.map('TCombobox', fieldbackground=[('readonly', ebg)]) style.map('TCombobox', selectbackground=[('readonly', ebg)]) style.map('TCombobox', selectforeground=[('readonly', fg)]) style.map('TCombobox', background=[('readonly', ebg)]) style.map('TCombobox', foreground=[('readonly', fg)]) def add_buttons(self, root): #Adds button self.btn_load = tk.Button(root, height=2, width=35, text="LOAD DATASET") self.btn_load.grid(row=0, column=0, padx=5, pady=5, sticky="w") self.btn_reg = tk.Button(root, height=2, width=35, text="SPLIT DATA FOR FORECASTING") self.btn_reg.grid(row=9, column=0, padx=5, pady=5, sticky="w") self.btn_pred = tk.Button(root, height=2, width=35, text="SPLIT DATA FOR PREDICTION") self.btn_pred.grid(row=12, column=0, padx=5, pady=5, sticky="w") def add_labels(self, root): #Adds labels self.label1 = tk.Label(root, text = "CHOOSE DISTRIBUTION") self.label1.grid(row=1, column=0, padx=5, pady=1, sticky="w") self.label2 = tk.Label(root, text = "YEAR-WISE TIME-SERIES PLOT") self.label2.grid(row=3, column=0, padx=5, pady=1, sticky="w") self.label3 = tk.Label(root, text = "MONTH-WISE TIME-SERIES PLOT") self.label3.grid(row=5, column=0, padx=5, pady=1, sticky="w") self.label4 = tk.Label(root, text = "TECHNICAL INDICATORS") self.label4.grid(row=7, column=0, padx=5, pady=1, sticky="w") self.label5 = tk.Label(root, text = "CHOOSE FORECASTING") self.label5.grid(row=10, column=0, padx=5, pady=1, sticky="w") self.label6 = tk.Label(root, text = "CHOOSE PREDICTION") self.label6.grid(row=13, column=0, padx=5, pady=1, sticky="w") def add_canvas(self, root): #Menambahkan canvas1 widget pada root untuk menampilkan hasil self.figure1 = Figure(figsize=(6.2, 7.6), dpi=100) self.figure1.patch.set_facecolor('#F0F0F0') self.canvas1 = FigureCanvasTkAgg(self.figure1, master=root) self.canvas1.get_tk_widget().grid(row=0, column=1, columnspan=1, rowspan=25, padx=5, pady=5, sticky="n") #Menambahkan canvas2 widget pada root untuk menampilkan hasil self.figure2 = Figure(figsize=(6.2, 7.6), dpi=100) self.figure2.patch.set_facecolor('#F0F0F0') self.canvas2 = FigureCanvasTkAgg(self.figure2, master=root) self.canvas2.get_tk_widget().grid(row=0, column=2, columnspan=1, rowspan=25, padx=5, pady=5, sticky="n") def add_listboxes(self, root): #Creates list widget self.listbox = tk.Listbox(root, selectmode=tk.SINGLE, width=40, fg ="black", bg="#F0F0F0", highlightcolor="black", selectbackground="red",relief="flat", borderwidth=5, highlightthickness=0) self.listbox.grid(row=2, column=0, sticky='n', padx=5, pady=1) # Menyisipkan item ke dalam list widget items = ["Missing Values", "Correlation Coefficient", "Year", "Day", "Month", "Quarter", "Open versus Adj Close versus Year", "Low versus High versus Quarter", "Adj Close versus Volume versus Month", "High versus Volume versus Day", "Distribution of Volume by Year", "Distribution of Volume by Days of Week", "Distribution of Volume by Month", "Distribution of Volume by Quarter", "Year versus Categorized Volume", "Day versus Categorized Volume", "Week versus Categorized Volume", "Month versus Categorized Volume", "Quarter versus Categorized Volume", "Quarter versus High Per Categorized Volume", "Day versus Adj Close Per Categorized Volume", "Categorized Volume", "Correlation Matrix"] for item in items: self.listbox.insert(tk.END, item) self.listbox.config(height=len(items)) def add_comboboxes(self, root): # Create ComboBoxes self.combo_year = ttk.Combobox(root, width=38) self.combo_year["values"] = ["Low and High", "Open and Close", "Adj Close and Close", "Year-Wise Mean EWM Low and High", "Year-Wise Mean EWM Open and Close", "Normalized Year-Wise Data", "Adj Close by Year", "Volume by Year", "Open by Year", "Close by Year", "Low by Year", "High by Year"] self.combo_year.grid(row=4, column=0, padx=5, pady=1, sticky="n") self.combo_month = ttk.Combobox(root, width=38, style='TCombobox') self.combo_month["values"] = ["Quarter-Wise Low and High", "Quarter-Wise Open and Close", "Month-Wise Open and Adj Close", "Month-Wise Mean EWM Low and High", "Month-Wise Mean EWM Open and Close", "Month-Wise Adj Close", "Month-Wise Open", "Month-Wise Close", "Month-Wise Low", "Month-Wise High", "Month-Wise Volume", "Normalized Month-Wise Data", "Adj Close by Month", "Open by Month", "Close by Month", "Low by Month", "High by Month", "Volume by Month"] self.combo_month.grid(row=6, column=0, padx=5, pady=1, sticky="n") self.combo_tech = ttk.Combobox(root, width=38, style='TCombobox') self.combo_tech["values"] = ["Adj Close versus Daily_Returns versus Year", "Volume versus Daily_Returns versus Quarter", "Low versus Daily_Returns versus Month", "High versus Daily_Returns versus Day", "Technical Indicators", "Differences"] self.combo_tech.grid(row=8, column=0, padx=5, pady=1, sticky="n") self.combo_reg = ttk.Combobox(root, width=38, style='TCombobox') self.combo_reg["values"] = ["Linear Regression", "RF Regression", "Decision Trees Regression", "KNN Regression", "AdaBoost Regression", "Gradient Boosting Regression", "MLP Regression", "SVR Regression", "Lasso Regression", "Ridge Regression"] self.combo_reg.grid(row=11, column=0, padx=5, pady=1, sticky="n") self.combo_pred = ttk.Combobox(root, width=38, style='TCombobox') self.combo_pred["values"] = ["Logistic Regression", "Random Forest", "Decision Trees", "K-Nearest Neighbors", "AdaBoost", "Gradient Boosting", "Extreme Gradient Boosting", "Light Gradient Boosting", "Multi-Layer Perceptron", "Support Vector Classifier"] self.combo_pred.grid(row=14, column=0, padx=5, pady=1, sticky="n") #process_data.py import os import pandas as pd class Process_Data: def read_dataset(self, filename): #Reads dataset curr_path = os.getcwd() path = os.path.join(curr_path, filename) df = pd.read_csv(path) return df def preprocess(self): df = self.read_dataset("BTC-USD.csv") #Extracts day, month, week, quarter, and year df['Date'] = pd.to_datetime(df['Date']) df['Day'] = df['Date'].dt.weekday df['Month'] = df['Date'].dt.month df['Year'] = df['Date'].dt.year df['Week'] = df['Date'].dt.isocalendar().week df['Quarter']= df['Date'].dt.quarter #Sets Date column as index df = df.set_index("Date") return df def create_dummy(self, df): #Creates a dummy dataframe for visualization df_dummy=df.copy() #Converts days and months from numerics to meaningful string days = {0:'Sunday',1:'Monday',2:'Tuesday',3:'Wednesday', 4:'Thursday',5: 'Friday',6:'Saturday'} df_dummy['Day'] = df_dummy['Day'].map(days) months={1:'January',2:'February',3:'March',4:'April', 5:'May',6:'June',7:'July',8:'August',9:'September', 10:'October',11:'November',12:'December'} df_dummy['Month']= df_dummy['Month'].map(months) quarters = {1:'Jan-March', 2:'April-June',3:'July-Sept', 4:'Oct-Dec'} df_dummy['Quarter'] = df_dummy['Quarter'].map(quarters) #Categorizes Volume feature # Calculate min, max, and average values of 'Volume' min_volume = df['Volume'].min() max_volume = df['Volume'].max() avg_volume = df['Volume'].mean() print(min_volume, avg_volume, max_volume) #5.914.570 1.484.7039.725 350.967.941.479 labels = ['0-500M', '500M-1000M', '1000M-10000M','10000M-20000M','>20000M'] df_dummy['Cat_Volume'] = pd.cut(df_dummy['Volume'], [0, 500000000, 1000000000, 10000000000, 20000000000, max_volume], labels=labels) return df_dummy def normalize_year_wise_data(self, df): #Normalizes year-wise data cols = list(df.columns) cols.remove("Month") cols.remove("Day") cols.remove("Week") cols.remove("Year") cols.remove("Quarter") year_data_mean = df.resample('y').mean() year_data_ewm = year_data_mean.ewm(span=5).mean() year_norm = (year_data_mean[cols] - year_data_mean[cols].min()) / (year_data_mean[cols].max() - year_data_mean[cols].min()) return year_data_mean, year_data_ewm, year_norm def normalize_month_wise_data(self, df): cols = list(df.columns) cols.remove("Month") cols.remove("Day") cols.remove("Week") cols.remove("Year") cols.remove("Quarter") month_data_mean = df[cols].resample('m').mean() month_data_ewm = month_data_mean.ewm(span=5).mean() month_norm = (month_data_mean - month_data_mean.min()) / (month_data_mean.max() - month_data_mean.min()) return month_data_mean, month_data_ewm, month_norm def compute_daily_returns(self, df): """Compute and return the daily return values.""" # Note: Returned DataFrame must have the same number of rows daily_return = (df / df.shift(1)) - 1 daily_return[0] = 0 return daily_return def calculate_MACD(self, df, nslow=26, nfast=12): emaslow = df.ewm(span=nslow, min_periods=nslow, adjust=True, ignore_na=False).mean() emafast = df.ewm(span=nfast, min_periods=nfast, adjust=True, ignore_na=False).mean() dif = emafast - emaslow MACD = dif.ewm(span=9, min_periods=9, adjust=True, ignore_na=False).mean() return dif, MACD def calculate_RSI(self, df, periods=14): # wilder's RSI delta = df.diff() up, down = delta.copy(), delta.copy() up[up < 0] = 0 down[down > 0] = 0 rUp = up.ewm(com=periods,adjust=False).mean() rDown = down.ewm(com=periods, adjust=False).mean().abs() rsi = 100 - 100 / (1 + rUp / rDown) return rsi def calculate_SMA(self, df, peroids=15): SMA = df.rolling(window=peroids, min_periods=peroids, center=False).mean() return SMA def calculate_BB(self, df, peroids=15): STD = df.rolling(window=peroids,min_periods=peroids, center=False).std() SMA = self.calculate_SMA(df) upper_band = SMA + (2 * STD) lower_band = SMA - (2 * STD) return upper_band, lower_band def calculate_stdev(self, df,periods=5): STDEV = df.rolling(periods).std() return STDEV def technical_indicators(self, df): stock_close = df["Adj Close"] #Computes daily returns df["Daily_Returns"] = self.compute_daily_returns(stock_close) #Calculates Simple Moving Average for Adj Close df['SMA'] = self.calculate_SMA(stock_close) #Calculates Bollinger Bands for Adj Close upper_band, lower_band = self.calculate_BB(stock_close) df['Upper_band'] = upper_band df['Lower_band'] = lower_band #Calculates MACD for Adj Close DIF, MACD = self.calculate_MACD(stock_close) df['DIF'] = DIF df['MACD'] = MACD #Calculates RSI for Adj Close df['RSI'] = self.calculate_RSI(stock_close) #Calculates Standard deviation for Adj Close df['STDEV']= self.calculate_stdev(stock_close) #Plots the difference of Open and Close and the difference of High and Low Open_Close = df.Open - stock_close High_Low = df.High - df.Low df['Open_Close'] = Open_Close df['High_Low'] = High_Low return df def save_result(self, y_test, y_pred, fname): # Convert y_test and y_pred to pandas Series for easier handling y_test_series = pd.Series(y_test) y_pred_series = pd.Series(y_pred) # Calculate y_result_series y_result_series = pd.Series(y_pred - y_test == 0) y_result_series = y_result_series.map({True: 'True', False: 'False'}) # Create a DataFrame to hold y_test, y_pred, and y_result data = pd.DataFrame({'y_test': y_test_series, 'y_pred': y_pred_series, 'result': y_result_series}) # Save the DataFrame to a CSV file data.to_csv(fname, index=False) #helper_plot.py from tkinter import * import seaborn as sns import numpy as np import pandas as pd from pandastable import Table from sklearn.metrics import confusion_matrix, roc_curve, accuracy_score from sklearn.model_selection import learning_curve from process_data import Process_Data from machine_learning import Machine_Learning from regression import Regression class Helper_Plot: def __init__(self): self.obj_data = Process_Data() self.obj_reg = Regression() self.obj_ml = Machine_Learning() def shows_table(self, root, df, width, height, title): frame = Toplevel(root) #new window self.table = Table(frame, dataframe=df, showtoolbar=True, showstatusbar=True) # Sets dimension of Toplevel frame.geometry(f"{width}x{height}") frame.title(title) self.table.show() def plot_missing_values(self, df, figure, canvas): figure.clear() ax = figure.add_subplot(1,1,1) #Plots null values missing = df.isna().sum().reset_index() missing.columns = ['features', 'total_missing'] missing['percent'] = (missing['total_missing'] / len(df)) * 100 missing.index = missing['features'] del missing['features'] missing['total_missing'].plot(kind = 'bar', ax=ax) ax.set_title('Missing Values Count', fontsize = 12) ax.set_facecolor('#F0F0F0') # Set font for tick labels ax.tick_params(axis='both', which='major', labelsize=5) ax.tick_params(axis='both', which='minor', labelsize=5) figure.tight_layout() canvas.draw() def plot_corr_coeffs(self, df, figure, canvas): figure.clear() ax = figure.add_subplot(1,1,1) #correlation coefficient of every column with Adj Close column all_corr = df.corr().abs()['Adj Close'].sort_values(ascending = False) # Filters correlations greater than 0.25 filtered_corr = all_corr[all_corr > 0.25] # Define a custom color palette (replace with your preferred colors) custom_palette = sns.color_palette("Set1", len(filtered_corr)) filtered_corr.plot(kind='barh', ax=ax, color=custom_palette) ax.set_title("Correlation Coefficient of Features with Adj Close (Threshold > 0.25)", fontsize = 10) ax.set_ylabel("Coefficient") ax.set_facecolor('#F0F0F0') # Set font for tick labels ax.tick_params(axis='both', which='major', labelsize=5) ax.tick_params(axis='both', which='minor', labelsize=5) ax.grid(True) figure.tight_layout() canvas.draw() # Defines function to create pie chart and bar plot as subplots def plot_piechart(self, df, var, figure, canvas, title=''): figure.clear() # Pie Chart (top subplot) ax1 = figure.add_subplot(2,1,1) label_list = list(df[var].value_counts().index) colors = sns.color_palette("Set1", len(label_list)) _, _, autopcts = ax1.pie(df[var].value_counts(), autopct="%1.1f%%", colors=colors, startangle=30, labels=label_list, wedgeprops={"linewidth": 2, "edgecolor": "white"}, # Add white edge shadow=True, textprops={'fontsize': 7}) ax1.set_title(title, fontsize=10) # Bar Plot (bottom subplot) ax2 = figure.add_subplot(2,1,2) ax = df[var].value_counts().plot(kind="barh", color=colors, alpha=0.8, ax = ax2) for i, j in enumerate(df[var].value_counts().values): ax.text(.7, i, j, weight="bold", fontsize=7) ax2.set_title(title, fontsize=10) ax2.set_xlabel("Count") ax2.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() def plot_piechart_group(self, df, figure, canvas, title=''): figure.clear() # Pie Chart (top subplot) ax1 = figure.add_subplot(2,1,1) label_list = list(df.value_counts().index) colors = sns.color_palette("Set1", len(label_list)) _, _, autopcts = ax1.pie(df.value_counts(), autopct="%1.1f%%", colors=colors, startangle=30, labels=label_list, wedgeprops={"linewidth": 2, "edgecolor": "white"}, # Add white edge shadow=True, textprops={'fontsize': 7}) ax1.set_title(title, fontsize=10) # Bar Plot (bottom subplot) ax2 = figure.add_subplot(2,1,2) ax = df.plot(kind="barh", color=colors, alpha=0.8, ax = ax2) for i, j in enumerate(df.values): ax.text(.7, i, j, weight="bold", fontsize=7) ax2.set_title(title, fontsize=10) ax2.set_xlabel("Count") ax2.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() def plot_scatter(self, df, x, y, hue, figure, canvas): figure.clear() ax = figure.add_subplot(1,1,1) sns.scatterplot(data=df, x=x, y=y, hue=hue, palette="Set1", ax=ax) ax.set_title(x + " versus " + y + " by " + hue) ax.set_xlabel(x) ax.set_ylabel(y) ax.grid(True) ax.legend(facecolor='#E6E6FA', edgecolor='black') ax.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() #Puts label inside stacked bar def put_label_stacked_bar(self, ax,fontsize): #patches is everything inside of the chart for rect in ax.patches: # Find where everything is located height = rect.get_height() width = rect.get_width() x = rect.get_x() y = rect.get_y() # The height of the bar is the data value and can be used as the label label_text = f'{width:.0f}' # ax.text(x, y, text) label_x = x + width / 2 label_y = y + height / 2 # plots only when height is greater than specified value if width > 0: ax.text(label_x, label_y, label_text, \ ha='center', va='center', \ weight = "bold",fontsize=fontsize) #Plots one variable against another variable def dist_one_vs_another_plot(self, df, cat1, cat2, figure, canvas, title): figure.clear() ax1 = figure.add_subplot(1,1,1) group_by_stat = df.groupby([cat1, cat2]).size() colors = sns.color_palette("Set1", len(df[cat1].unique())) group_by_stat.unstack().plot(kind='barh', stacked=True, ax=ax1,color=colors) ax1.set_title(title, fontsize=12) ax1.set_ylabel('Number of Cases', fontsize=10) ax1.set_xlabel(cat1, fontsize=10) self.put_label_stacked_bar(ax1,7) # Set font for tick labels ax1.tick_params(axis='both', which='major', labelsize=8) ax1.tick_params(axis='both', which='minor', labelsize=8) ax1.legend(facecolor='#E6E6FA', edgecolor='black', fontsize=8) ax1.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() def box_plot(self, df, x, y, hue, figure, canvas, title): figure.clear() ax1 = figure.add_subplot(1,1,1) sns.boxplot(data = df, x = x, y = y, hue = hue, ax=ax1) ax1.set_title(title, fontsize=14) ax1.set_xlabel(x, fontsize=10) ax1.set_ylabel(y, fontsize=10) ax1.set_facecolor('#F0F0F0') ax1.legend(facecolor='#E6E6FA', edgecolor='black') figure.tight_layout() canvas.draw() def plot_corr_mat(self, df, figure, canvas): figure.clear() ax = figure.add_subplot(1,1,1) categorical_columns = df.select_dtypes(include=['object', 'category']).columns df_removed = df.drop(columns=categorical_columns) corrdata = df_removed.corr() annot_kws = {"size": 5} # Filter correlations greater than 0.1 mask = abs(corrdata) > 0.1 filtered_corr = corrdata[mask] # Drops features that don't meet the threshold filtered_corr = filtered_corr.dropna(axis=0, how='all') filtered_corr = filtered_corr.dropna(axis=1, how='all') sns.heatmap(filtered_corr, ax = ax, lw=1, annot=True, cmap="Greens", annot_kws=annot_kws) ax.set_title('Correlation Matrix (Threshold > 0.1)', fontweight="bold", fontsize=10) # Set font for x and y labels ax.set_xlabel('Features', fontweight="bold", fontsize=12) ax.set_ylabel('Features', fontweight="bold", fontsize=12) # Set font for tick labels ax.tick_params(axis='both', which='major', labelsize=5) ax.tick_params(axis='both', which='minor', labelsize=5) figure.tight_layout() canvas.draw() def choose_plot(self, df1, df2, chosen, figure1, canvas1, figure2, canvas2): print(chosen) if chosen == "Day": self.plot_piechart(df2, "Day", figure1, canvas1, "Case Distribution of Day") elif chosen == "Month": self.plot_piechart(df2, "Month", figure2, canvas2, "Case Distribution of Month") elif chosen == "Quarter": self.plot_piechart(df2, "Quarter", figure1, canvas1, "Case Distribution of Quarter") elif chosen == "Year": self.plot_piechart(df2, "Year", figure2, canvas2, "Case Distribution of Year") elif chosen == "Missing Values": self.plot_missing_values(df1, figure1, canvas1) elif chosen == "Correlation Coefficient": self.plot_corr_coeffs(df1, figure2, canvas2) elif chosen == "Open versus Adj Close versus Year": self.plot_scatter(df2, "Open", "Adj Close", "Year", figure1, canvas1) elif chosen == "Low versus High versus Quarter": self.plot_scatter(df2, "Low", "High", "Quarter", figure2, canvas2) elif chosen == "Adj Close versus Volume versus Month": self.plot_scatter(df2, "Adj Close", "Volume", "Month", figure1, canvas1) elif chosen == "High versus Volume versus Day": self.plot_scatter(df2, "High", "Volume", "Day", figure2, canvas2) elif chosen == "Distribution of Volume by Year": self.plot_piechart_group(df2.groupby('Year')['Volume'].sum(), figure1, canvas1, chosen) elif chosen == "Distribution of Volume by Days of Week": self.plot_piechart_group(df2.groupby('Day')['Volume'].sum(), figure2, canvas2, chosen) elif chosen == "Distribution of Volume by Month": self.plot_piechart_group(df2.groupby('Month')['Volume'].sum(), figure1, canvas1, chosen) elif chosen == "Distribution of Volume by Quarter": self.plot_piechart_group(df2.groupby('Quarter')['Volume'].sum(), figure2, canvas2, chosen) elif chosen == "Day versus Categorized Volume": self.dist_one_vs_another_plot(df2, "Day", "Cat_Volume", figure1, canvas1, chosen) elif chosen == "Year versus Categorized Volume": self.dist_one_vs_another_plot(df2, "Year", "Cat_Volume", figure2, canvas2, chosen) elif chosen == "Week versus Categorized Volume": self.dist_one_vs_another_plot(df2, "Week", "Cat_Volume", figure1, canvas1, chosen) elif chosen == "Month versus Categorized Volume": self.dist_one_vs_another_plot(df2, "Month", "Cat_Volume", figure2, canvas2, chosen) elif chosen == "Quarter versus Categorized Volume": self.dist_one_vs_another_plot(df2, "Quarter", "Cat_Volume", figure1, canvas1, chosen) elif chosen == "Quarter versus High Per Categorized Volume": self.box_plot(df2, "Quarter", "High", "Cat_Volume", figure2, canvas2, chosen) elif chosen == "Day versus Adj Close Per Categorized Volume": self.box_plot(df2, "Day", "Adj Close", "Cat_Volume", figure1, canvas1, chosen) elif chosen == "Categorized Volume": self.plot_piechart(df2, "Cat_Volume", figure2, canvas2, "Case Distribution of Categorized Volume") if chosen == "Correlation Matrix": self.plot_corr_mat(df1, figure1, canvas1) def line_plot_year_wise(self, df, feat1, feat2, year1, year2, figure, canvas): figure.clear() ax1 = figure.add_subplot(2, 1, 1) data1 = df[df["Year"]==year1] data2 = df[df["Year"]==year2] # Convert the column and index to NumPy arrays date_index1 = data1.index.to_numpy() date_index2 = data2.index.to_numpy() # Line plot ax1.plot(date_index1, data1[feat1].to_numpy(), color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat1) ax1.plot(date_index1,data1[feat2].to_numpy(), color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat2) ax1.set_xlabel('YEAR') ax1.set_title(feat1 + " and " + feat2 + ' (YEAR = ' + str(year1) + ')', fontsize=12) ax1.legend(facecolor='#E6E6FA', edgecolor='black') ax1.set_facecolor('#F0F0F0') ax1.grid(True) ax2 = figure.add_subplot(2, 1, 2) ax2.plot(date_index2, data2[feat1].to_numpy(), color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat1) ax2.plot(date_index2, data2[feat2].to_numpy(), color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat2) ax2.set_xlabel('YEAR') ax2.set_title(feat1 + " and " + feat2 + ' (YEAR = ' + str(year2) + ')', fontsize=12) ax2.legend(facecolor='#E6E6FA', edgecolor='black') ax2.set_facecolor('#F0F0F0') ax2.grid(True) figure.tight_layout() canvas.draw() def line_plot_norm_data(self, norm_data, figure, canvas, label, title): figure.clear() ax = figure.add_subplot(1, 1, 1) # Convert the column and index to NumPy arrays date_index = norm_data.index.to_numpy() # Iterate over all columns (excluding 'Date') for column in norm_data.columns: if column != 'Date': values = norm_data[column].to_numpy() ax.plot(values, date_index, marker='o', linestyle='-', linewidth=1, markersize=2, label=column) ax.set_ylabel(label) ax.set_title(title, fontsize=12) ax.legend(fontsize=7, facecolor='#E6E6FA', edgecolor='black') ax.set_facecolor('#F0F0F0') ax.grid(True) figure.tight_layout() canvas.draw() def line_plot_data_mean_ewm(self, data_mean, data_ewm, feat1, feat2, figure, canvas, label, title): figure.clear() ax1 = figure.add_subplot(2, 1, 1) # Convert the column and index to NumPy arrays date_index = data_mean.index.to_numpy() # Line plot ax1.plot(date_index, data_mean[feat1].to_numpy(), color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label="Mean") ax1.plot(date_index,data_ewm[feat1].to_numpy(), color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label="EWM") ax1.set_title(title + feat1, fontsize=12) ax1.legend(facecolor='#E6E6FA', edgecolor='black') ax1.set_facecolor('#F0F0F0') ax1.grid(True) ax2 = figure.add_subplot(2, 1, 2) ax2.plot(date_index, data_mean[feat2].to_numpy(), color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label="Mean") ax2.plot(date_index,data_ewm[feat2].to_numpy(), color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label="EWM") ax2.set_xlabel(label) ax2.set_title(title + feat2, fontsize=12) ax2.legend(facecolor='#E6E6FA', edgecolor='black') ax2.grid(True) ax2.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() def box_violin_strip_heat(self, data, filter, feat1, figure1, canvas1, figure2, canvas2, title): figure1.clear() ax1 = figure1.add_subplot(2, 1, 1) sns.boxplot(x = filter, y = feat1, data = data, ax=ax1) ax1.set_title("Box Plot of " + feat1 + " by " + filter, fontsize=12) # Set font for tick labels ax1.tick_params(axis='both', which='major', labelsize=6) ax1.tick_params(axis='both', which='minor', labelsize=6) ax1.grid(True) ax1.set_facecolor('#F0F0F0') ax2 = figure1.add_subplot(2, 1, 2) sns.violinplot(x = filter, y = feat1, data = data, ax=ax2) ax2.set_title("Violin Plot of " + feat1 + " by " + filter, fontsize=12) # Set font for tick labels ax2.tick_params(axis='both', which='major', labelsize=6) ax2.tick_params(axis='both', which='minor', labelsize=6) ax2.grid(True) ax2.set_facecolor('#F0F0F0') figure1.tight_layout() canvas1.draw() figure2.clear() ax3 = figure2.add_subplot(2, 1, 1) sns.stripplot(x = filter, y = feat1, data = data, ax=ax3) ax3.set_title("Strip Plot of " + feat1 + " by " + filter, fontsize=12) # Set font for tick labels ax3.tick_params(axis='both', which='major', labelsize=6) ax3.tick_params(axis='both', which='minor', labelsize=6) ax3.set_facecolor('#F0F0F0') ax3.grid(True) ax4 = figure2.add_subplot(2, 1, 2) sns.swarmplot(x = filter, y = feat1, data = data, ax=ax4) ax4.set_title("Swarm Plot of " + feat1 + " by " + filter, fontsize=12) # Set font for tick labels ax4.tick_params(axis='both', which='major', labelsize=6) ax4.tick_params(axis='both', which='minor', labelsize=6) ax4.grid(True) ax4.set_facecolor('#F0F0F0') figure2.tight_layout() canvas2.draw() def choose_year_wise(self, df, data_mean, data_ewm, data_norm, chosen, figure1, canvas1, figure2, canvas2): if chosen == "Low and High": self.line_plot_year_wise(df, "Low", "High", 2015, 2016, figure1, canvas1) if chosen == "Open and Close": self.line_plot_year_wise(df, "Open", "Close", 2017, 2018, figure2, canvas2) if chosen == "Adj Close and Close": self.line_plot_year_wise(df, "Adj Close", "Close", 2019, 2020, figure1, canvas1) if chosen == "Year-Wise Mean EWM Low and High": self.line_plot_data_mean_ewm(data_mean, data_ewm, "Low", "High", figure2, canvas2, "YEAR", "Year-Wise Mean and EWM of ") if chosen == "Year-Wise Mean EWM Open and Close": self.line_plot_data_mean_ewm(data_mean, data_ewm, "Open", "Close", figure1, canvas1, "YEAR", "Year-Wise Mean and EWM of ") if chosen == "Normalized Year-Wise Data": self.line_plot_norm_data(data_norm, figure1, canvas1, "YEAR", "Normalized Year-Wise Data") if chosen == "Adj Close by Year": self.box_violin_strip_heat(df, "Year", "Adj Close", figure1, canvas1, figure2, canvas2, "Year-Wise Normalized ") if chosen == "Volume by Year": self.box_violin_strip_heat(df, "Year", "Volume", figure1, canvas1, figure2, canvas2, "Year-Wise Normalized ") if chosen == "Open by Year": self.box_violin_strip_heat(df, "Year", "Open", figure1, canvas1, figure2, canvas2, "Year-Wise Normalized ") if chosen == "Close by Year": self.box_violin_strip_heat(df, "Year", "Close", figure1, canvas1, figure2, canvas2, "Year-Wise Normalized ") if chosen == "High by Year": self.box_violin_strip_heat(df, "Year", "High", figure1, canvas1, figure2, canvas2, "Year-Wise Normalized ") if chosen == "Low by Year": self.box_violin_strip_heat(df, "Year", "Low", figure1, canvas1, figure2, canvas2, "Year-Wise Normalized ") def line_plot_month_wise(self, df, feat1, feat2, year, filter, filter1, filter2, figure, canvas): figure.clear() ax1 = figure.add_subplot(2, 1, 1) data1 = df[(df["Year"]==year)&(df[filter]==filter1)] data2 = df[(df["Year"]==year)&(df[filter]==filter2)] # Convert the column and index to NumPy arrays date_index1 = data1.index.to_numpy() date_index2 = data2.index.to_numpy() # Line plot ax1.plot(date_index1, data1[feat1].to_numpy(), color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat1) ax1.plot(date_index1,data1[feat2].to_numpy(), color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat2) ax1.set_xlabel('DATE') ax1.set_title(feat1 + " and " + feat2 + " " + filter + " = " + filter1 + " " + str(year), fontsize=12) ax1.legend(facecolor='#E6E6FA', edgecolor='black') ax1.set_facecolor('#F0F0F0') ax1.grid(True) # Set font for tick labels ax1.tick_params(axis='both', which='major', labelsize=7) ax1.tick_params(axis='both', which='minor', labelsize=7) ax2 = figure.add_subplot(2, 1, 2) ax2.plot(date_index2, data2[feat1].to_numpy(), color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat1) ax2.plot(date_index2,data2[feat2].to_numpy(), color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat2) ax2.set_xlabel('DATE') ax2.set_title(feat1 + " and " + feat2 + " " + filter + " = " + filter2 + " " + str(year), fontsize=12) ax2.legend(facecolor='#E6E6FA', edgecolor='black') ax2.set_facecolor('#F0F0F0') ax2.grid(True) # Set font for tick labels ax2.tick_params(axis='both', which='major', labelsize=7) ax2.tick_params(axis='both', which='minor', labelsize=7) figure.tight_layout() canvas.draw() def color_month(self, month): if month == 1: return 'January','blue' elif month == 2: return 'February','green' elif month == 3: return 'March','orange' elif month == 4: return 'April','yellow' elif month == 5: return 'May','red' elif month == 6: return 'June','violet' elif month == 7: return 'July','purple' elif month == 8: return 'August','black' elif month == 9: return 'September','brown' elif month == 10: return 'October','darkblue' elif month == 11: return 'November','grey' else: return 'December','pink' def line_plot_month(self, month, data, ax): label, color = self.color_month(month) mdata = data[data.index.month == month] date_index = mdata.index.to_numpy() ax.plot(date_index, mdata.to_numpy(), marker='o', linestyle='-', color=color, linewidth=2, markersize=1, label=label) def sns_plot_month(self, monthly_data, feat, title, figure, canvas): figure.clear() ax = figure.add_subplot(1, 1, 1) ax.set_title(title, fontsize=12) ax.set_xlabel('YEAR', fontsize=10) ax.set_ylabel(feat, fontsize=10) for i in range(1,13): self.line_plot_month(i, monthly_data[feat], ax) ax.legend(facecolor='#E6E6FA', edgecolor='black') ax.grid() ax.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() def choose_month_wise(self, df1, df2, data_mean, data_ewm, data_norm, chosen, figure1, canvas1, figure2, canvas2): if chosen == "Quarter-Wise Low and High": self.line_plot_month_wise(df2, "Low", "High", 2018, "Quarter", "Jan-March", "April-June", figure1, canvas1) if chosen == "Quarter-Wise Open and Close": self.line_plot_month_wise(df2, "Open", "Close", 2015, "Quarter", "July-Sept", "Oct-Dec", figure2, canvas2) if chosen == "Month-Wise Open and Adj Close": self.line_plot_month_wise(df2, "Open", "Adj Close", 2016, "Month", "February", "March", figure1, canvas1) self.line_plot_month_wise(df2, "Open", "Adj Close", 2017, "Month", "April", "May", figure2, canvas2) if chosen == "Month-Wise Mean EWM Low and High": self.line_plot_data_mean_ewm(data_mean, data_ewm, "Low", "High", figure1, canvas1, "MONTH", "Month-Wise Mean and EWM of ") if chosen == "Month-Wise Mean EWM Open and Close": self.line_plot_data_mean_ewm(data_mean, data_ewm, "Open", "Close", figure2, canvas2, "MONTH", "Month-Wise Mean and EWM of ") if chosen == "Month-Wise Adj Close": self.sns_plot_month(data_mean, "Adj Close", chosen, figure1, canvas1) if chosen == "Month-Wise Open": self.sns_plot_month(data_mean, "Open", chosen, figure2, canvas2) if chosen == "Month-Wise Close": self.sns_plot_month(data_mean, "Close", chosen, figure1, canvas1) if chosen == "Month-Wise Low": self.sns_plot_month(data_mean, "Low", chosen, figure2, canvas2) if chosen == "Month-Wise High": self.sns_plot_month(data_mean, "High", chosen, figure1, canvas1) if chosen == "Month-Wise Volume": self.sns_plot_month(data_mean, "Volume", chosen, figure2, canvas2) if chosen == "Normalized Month-Wise Data": self.line_plot_norm_data(data_norm, figure1, canvas1, "YEAR", "Normalized Month-Wise Data") if chosen == "Adj Close by Month": self.box_violin_strip_heat(df2, "Month", "Adj Close", figure1, canvas1, figure2, canvas2, "Month-Wise Normalized ") if chosen == "Open by Month": self.box_violin_strip_heat(df2, "Month", "Open", figure1, canvas1, figure2, canvas2, "Month-Wise Normalized ") if chosen == "Close by Month": self.box_violin_strip_heat(df2, "Month", "Close", figure1, canvas1, figure2, canvas2, "Month-Wise Normalized ") if chosen == "Low by Month": self.box_violin_strip_heat(df2, "Month", "Low", figure1, canvas1, figure2, canvas2, "Month-Wise Normalized ") if chosen == "High by Month": self.box_violin_strip_heat(df2, "Month", "High", figure1, canvas1, figure2, canvas2, "Month-Wise Normalized ") if chosen == "Volume by Month": self.box_violin_strip_heat(df2, "Month", "Volume", figure1, canvas1, figure2, canvas2, "Month-Wise Normalized ") def plot_technical_indicators(self, df, figure1, canvas1, figure2, canvas2): figure1.clear() ax1 = figure1.add_subplot(2, 1, 1) df["Adj Close"][:365].plot(title='Simple Moving Average, SMA, and Bands', label='Moving Average', ax=ax1) df['SMA'][:365].plot(label="SMA", ax=ax1) df['Upper_band'][:365].plot(label='upper band', ax=ax1) df['Lower_band'][:365].plot(label='lower band', ax=ax1) ax1.legend(facecolor='#E6E6FA', edgecolor='black') ax1.grid(True) ax1.set_facecolor('#F0F0F0') ax2 = figure1.add_subplot(2, 1, 2) df['DIF'][:365].plot(title='DIF and MACD',label='DIF', ax=ax2) df['MACD'][:365].plot(label='MACD', ax=ax2) ax2.legend(facecolor='#E6E6FA', edgecolor='black') ax2.grid(True) ax2.set_facecolor('#F0F0F0') figure1.tight_layout() canvas1.draw() figure2.clear() ax3 = figure2.add_subplot(2, 1, 1) df['RSI'].plot(title='RSI',label='RSI', color="red", ax=ax3) ax3.grid(True) ax3.set_facecolor('#F0F0F0') ax4 = figure2.add_subplot(2, 1, 2) df['STDEV'].plot(title='STDEV',label='STDEV', ax=ax4) ax4.set_facecolor('#F0F0F0') ax4.grid(True) figure2.tight_layout() canvas2.draw() def plot_difference(self, df, figure, canvas): figure.clear() ax1 = figure.add_subplot(2, 1, 1) df['Open_Close'][:365].plot(title='open-close', label='open_close', color="red", ax=ax1) ax1.grid(True) ax1.set_facecolor('#F0F0F0') ax2 = figure.add_subplot(2, 1, 2) df['High_Low'][:365].plot(title='high-low', label='high-low', ax=ax2) ax2.grid(True) ax2.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() def choose_plot_technical_indicators(self, df, chosen, figure1, canvas1, figure2, canvas2): print(chosen) if chosen == "Adj Close versus Daily_Returns versus Year": self.plot_scatter(df, "Adj Close", "Daily_Returns", "Year", figure1, canvas1) if chosen == "Volume versus Daily_Returns versus Quarter": self.plot_scatter(df, "Volume", "Daily_Returns", "Quarter", figure2, canvas2) if chosen == "Low versus Daily_Returns versus Month": self.plot_scatter(df, "Low", "Daily_Returns", "Month", figure1, canvas1) if chosen == "High versus Daily_Returns versus Day": self.plot_scatter(df, "High", "Daily_Returns", "Day", figure2, canvas2) if chosen == "Technical Indicators": self.plot_technical_indicators(df, figure1, canvas1, figure2, canvas2) if chosen == "Differences": self.plot_difference(df, figure1, canvas1) def scatter_train_test_regression(self, ytrain, ytest, predictions_train, predictions_test, figure, canvas, label): # Visualizes the training set results in a scatter plot figure.clear() ax1 = figure.add_subplot(2, 1, 1) ax1.scatter(x=ytrain, y=predictions_train, color='red', label='Training Data') ax1.set_title('The actual versus predicted (Training set): ' + label, fontweight='bold', fontsize=10) ax1.set_xlabel('Actual Train Set', fontsize=8) ax1.set_ylabel('Predicted Train Set', fontsize=8) ax1.plot([ytrain.min(), ytrain.max()], [ytrain.min(), ytrain.max()], 'b--', linewidth=2, label='Perfect Prediction') ax1.grid(True) ax1.set_facecolor('#F0F0F0') ax1.legend(facecolor='#E6E6FA', edgecolor='black') ax2 = figure.add_subplot(2, 1, 2) ax2.scatter(x=ytest, y=predictions_test, color='red', label='Test Data') ax2.set_title('The actual versus predicted (Test set): ' + label, fontweight='bold', fontsize=10) ax2.set_xlabel('Actual Test Set', fontsize=8) ax2.set_ylabel('Predicted Test Set', fontsize=8) ax2.plot([ytest.min(), ytest.max()], [ytest.min(), ytest.max()], 'b--', linewidth=2, label='Perfect Prediction') ax2.grid(True) ax2.set_facecolor('#F0F0F0') ax2.legend(facecolor='#E6E6FA', edgecolor='black') figure.tight_layout() canvas.draw() def lineplot_train_test_regression(self, ytrain, ytest, yval, yfinal, predictions_train, predictions_test, predictions_val, all_pred, figure, canvas, label): figure.clear() ax1 = figure.add_subplot(4, 1, 1) ax1.plot(ytrain.index.to_numpy(), ytrain.to_numpy(), color="blue", linewidth=2, linestyle="-", label='Actual') ax1.plot(ytrain.index.to_numpy(), predictions_train, color="red", linewidth=2, linestyle="-", label='Predicted') ax1.set_title('Actual and Predicted Training Set: ' + label, fontsize=10) ax1.set_xlabel('Date', fontsize=8) ax1.set_ylabel("Adj Close", fontsize=8) ax1.legend(prop={'size': 8},facecolor='#E6E6FA', edgecolor='black') ax1.grid(True) ax1.set_facecolor('#F0F0F0') # Set font for tick labels ax1.tick_params(axis='both', which='major', labelsize=8) ax1.tick_params(axis='both', which='minor', labelsize=8) ax2 = figure.add_subplot(4, 1, 2) ax2.plot(ytest.index.to_numpy(), ytest.to_numpy(), color="blue", linewidth=2, linestyle="-", label='Actual') ax2.plot(ytest.index.to_numpy(), predictions_test, color="red", linewidth=2, linestyle="-", label='Predicted') ax2.set_title('Actual and Predicted Test Set: ' + label, fontsize=10) ax2.set_xlabel('Date', fontsize=8) ax2.set_ylabel("Adj Close", fontsize=8) ax2.legend(prop={'size': 8}, facecolor='#E6E6FA', edgecolor='black') ax2.grid(True) ax2.set_facecolor('#F0F0F0') # Set font for tick labels ax2.tick_params(axis='both', which='major', labelsize=8) ax2.tick_params(axis='both', which='minor', labelsize=8) ax3 = figure.add_subplot(4, 1, 3) ax3.plot(yval.index.to_numpy(), yval.to_numpy(), color="blue", linewidth=2, linestyle="-", label='Actual') ax3.plot(yval.index.to_numpy(), predictions_val, color="red", linewidth=2, linestyle="-", label='Predicted') ax3.set_title('Actual and Predicted Validation Set (90 days forecasting) ' + label, fontsize=8) ax3.set_xlabel('Date', fontsize=8) ax3.set_ylabel("Adj Close", fontsize=8) ax3.legend(prop={'size': 8}, facecolor='#E6E6FA', edgecolor='black') ax3.grid(True) ax3.set_facecolor('#F0F0F0') # Set font for tick labels ax3.tick_params(axis='both', which='major', labelsize=8) ax3.tick_params(axis='both', which='minor', labelsize=8) ax4 = figure.add_subplot(4, 1, 4) ax4.plot(yfinal.index.to_numpy(), yfinal.to_numpy(), color="blue", linewidth=2, linestyle="-", label='Actual') ax4.plot(yfinal.index.to_numpy(), all_pred, color="red", linewidth=2, linestyle="-", label='Predicted') ax4.set_title('Actual and Predicted All Set ' + label, fontsize=8) ax4.set_xlabel('Date', fontsize=8) ax4.set_ylabel("Adj Close", fontsize=8) ax4.legend(prop={'size': 8}, facecolor='#E6E6FA', edgecolor='black') ax4.grid(True) ax4.set_facecolor('#F0F0F0') # Set font for tick labels ax4.tick_params(axis='both', which='major', labelsize=8) ax4.tick_params(axis='both', which='minor', labelsize=8) figure.tight_layout() canvas.draw() def choose_plot_regression(self, chosen, X_final_reg, X_train_reg, X_test_reg, X_val_reg, y_final_reg, y_train_reg, y_test_reg, y_val_reg, figure1, canvas1, figure2, canvas2): if chosen == "Linear Regression": best_lin_reg = self.obj_reg.linear_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_lin_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "RF Regression": best_rf_reg = self.obj_reg.rf_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_rf_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "Decision Trees Regression": best_dt_reg = self.obj_reg.dt_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_dt_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "Gradient Boosting Regression": best_gb_reg = self.obj_reg.gb_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_gb_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "XGB Regression": best_xgb_reg = self.obj_reg.xgb_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_xgb_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "MLP Regression": best_mlp_reg = self.obj_reg.mlp_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_mlp_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "Lasso Regression": best_lasso_reg = self.obj_reg.lasso_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_lasso_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "Ridge Regression": best_ridge_reg = self.obj_reg.ridge_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_ridge_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "AdaBoost Regression": best_ada_reg = self.obj_reg.adaboost_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_ada_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) if chosen == "KNN Regression": best_knn_reg = self.obj_reg.knn_regression(X_train_reg, y_train_reg) predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_knn_reg, X_final_reg, y_final_reg, X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen) self.scatter_train_test_regression(y_train_reg, y_test_reg, predictions_train, predictions_test, figure1, canvas1, chosen) self.lineplot_train_test_regression(y_train_reg, y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred, figure2, canvas2, chosen) def plot_cm_roc(self, model, X_test, y_test, ypred, name, figure, canvas): figure.clear() #Plots confusion matrix ax1 = figure.add_subplot(2,1,1) cm = confusion_matrix(y_test, ypred, ) sns.heatmap(cm, annot=True, linewidth=3, linecolor='red', fmt='g', cmap="viridis", annot_kws={"size": 14}, ax=ax1) ax1.set_title('Confusion Matrix' + " of " + name, fontsize=12) ax1.set_xlabel('Y predict', fontsize=10) ax1.set_ylabel('Y test', fontsize=10) ax1.xaxis.set_ticklabels(['Daily Returns = 1', 'Daily Returns = 0'], fontsize=10) ax1.yaxis.set_ticklabels(['Daily Returns = 1', 'Daily Returns = 0'], fontsize=10) ax1.set_facecolor('#F0F0F0') #Plots ROC ax2 = figure.add_subplot(2,1,2) Y_pred_prob = model.predict_proba(X_test) Y_pred_prob = Y_pred_prob[:, 1] fpr, tpr, thresholds = roc_curve(y_test, Y_pred_prob) ax2.plot([0,1],[0,1], color='navy', linestyle='--', linewidth=3, label='Random Guess') ax2.plot(fpr,tpr, color='red', linewidth=3, label='ROC Curve') ax2.set_xlabel('False Positive Rate', fontsize=10) ax2.set_ylabel('True Positive Rate', fontsize=10) ax2.set_title('ROC Curve of ' + name , fontsize=12) ax2.grid(True) ax2.legend(facecolor='#E6E6FA', edgecolor='black') ax2.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() #Plots true values versus predicted values diagram and learning curve def plot_real_pred_val_learning_curve(self, model, X_train, y_train, X_test, y_test, ypred, name, figure, canvas): figure.clear() #Plots true values versus predicted values diagram ax1 = figure.add_subplot(2,1,1) acc=accuracy_score(y_test, ypred) ax1.scatter(range(len(ypred)),ypred,color="blue", lw=2,label="Predicted") ax1.scatter(range(len(y_test)), y_test, color="red", label="Actual") ax1.set_title("Predicted Values vs True Values of " + name, fontsize=12) ax1.set_xlabel("Accuracy: " + str(round((acc*100),3)) + "%") ax1.legend(facecolor='#E6E6FA', edgecolor='black') ax1.grid(True, alpha=0.75, lw=1, ls='-.') ax1.set_facecolor('#F0F0F0') #Plots learning curve train_sizes=np.linspace(.1, 1.0, 5) train_sizes, train_scores, test_scores, fit_times, _ = learning_curve(model, X_train, y_train, cv=None, n_jobs=None, train_sizes=train_sizes, return_times=True) train_scores_mean = np.mean(train_scores, axis=1) train_scores_std = np.std(train_scores, axis=1) test_scores_mean = np.mean(test_scores, axis=1) test_scores_std = np.std(test_scores, axis=1) ax2 = figure.add_subplot(2,1,2) ax2.fill_between(train_sizes, train_scores_mean - train_scores_std, train_scores_mean + train_scores_std, alpha=0.1, color="r") ax2.fill_between(train_sizes, test_scores_mean - test_scores_std, test_scores_mean + test_scores_std, alpha=0.1, color="g") ax2.plot(train_sizes, train_scores_mean, 'o-', color="b", label="Training score") ax2.plot(train_sizes, test_scores_mean, 'o-', color="r", label="Cross-validation score") ax2.legend(loc="best", facecolor='#E6E6FA', edgecolor='black') ax2.set_title("Learning curve of " + name, fontsize=12) ax2.set_xlabel("fit_times") ax2.set_ylabel("Score") ax2.grid(True, alpha=0.75, lw=1, ls='-.') ax2.set_facecolor('#F0F0F0') figure.tight_layout() canvas.draw() def choose_plot_ML(self, root, chosen, X_train, X_test, y_train, y_test, figure1, canvas1, figure2, canvas2): if chosen == "Logistic Regression": best_model, y_pred = self.obj_ml.implement_LR(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_lr = self.obj_data.read_dataset("results_LR.csv") self.shows_table(root, df_lr, 450, 750, "Y_test and Y_pred of Logistic Regression") if chosen == "Random Forest": best_model, y_pred = self.obj_ml.implement_RF(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_rf = self.obj_data.read_dataset("results_RF.csv") self.shows_table(root, df_rf, 450, 750, "Y_test and Y_pred of Random Forest") if chosen == "K-Nearest Neighbors": best_model, y_pred = self.obj_ml.implement_KNN(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_knn = self.obj_data.read_dataset("results_KNN.csv") self.shows_table(root, df_knn, 450, 750, "Y_test and Y_pred of KNN") if chosen == "Decision Trees": best_model, y_pred = self.obj_ml.implement_DT(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_dt = self.obj_data.read_dataset("results_DT.csv") self.shows_table(root, df_dt, 450, 750, "Y_test and Y_pred of Decision Trees") if chosen == "Gradient Boosting": best_model, y_pred = self.obj_ml.implement_GB(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_gb = self.obj_data.read_dataset("results_GB.csv") self.shows_table(root, df_gb, 450, 750, "Y_test and Y_pred of Gradient Boosting") if chosen == "Extreme Gradient Boosting": best_model, y_pred = self.obj_ml.implement_XGB(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_xgb = self.obj_data.read_dataset("results_XGB.csv") self.shows_table(root, df_xgb, 450, 750, "Y_test and Y_pred of Extreme Gradient Boosting") if chosen == "Multi-Layer Perceptron": best_model, y_pred = self.obj_ml.implement_MLP(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_mlp = self.obj_data.read_dataset("results_MLP.csv") self.shows_table(root, df_mlp, 450, 750, "Y_test and Y_pred of Multi-Layer Perceptron") if chosen == "Support Vector Classifier": best_model, y_pred = self.obj_ml.implement_SVC(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_svc = self.obj_data.read_dataset("results_SVC.csv") self.shows_table(root, df_svc, 450, 750, "Y_test and Y_pred of Support Vector Classifier") if chosen == "AdaBoost": best_model, y_pred = self.obj_ml.implement_ADA(chosen, X_train, X_test, y_train, y_test) #Plots confusion matrix and ROC self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1) #Plots true values versus predicted values diagram and learning curve self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, X_test, y_test, y_pred, chosen, figure2, canvas2) #Shows table of result df_ada = self.obj_data.read_dataset("results_ADA.csv") self.shows_table(root, df_ada, 450, 750, "Y_test and Y_pred of AdaBoost Classifier") #machine_learning.py import numpy as np from imblearn.over_sampling import SMOTE from sklearn.model_selection import train_test_split, RandomizedSearchCV, GridSearchCV, StratifiedKFold from sklearn.preprocessing import StandardScaler import joblib from sklearn.linear_model import LogisticRegression from sklearn.metrics import confusion_matrix, accuracy_score, recall_score, precision_score from sklearn.metrics import classification_report, f1_score, plot_confusion_matrix from sklearn.ensemble import RandomForestClassifier from sklearn.neighbors import KNeighborsClassifier from sklearn.tree import DecisionTreeClassifier from sklearn.ensemble import AdaBoostClassifier, GradientBoostingClassifier from xgboost import XGBClassifier from sklearn.neural_network import MLPClassifier from sklearn.svm import SVC import os import joblib import pandas as pd from process_data import Process_Data class Machine_Learning: def __init__(self): self.obj_data = Process_Data() def oversampling_splitting(self, df): #Sets target column y = df["Daily_Returns"] #Ensures y is of integer type y = np.array([1 if i>0 else 0 for i in y]).astype(int) #Drops irrelevant column X = df.drop(["Daily_Returns"], axis =1) #Checks null values because of technical indicators print(X.isnull().sum().to_string()) print('Total number of null values: ', X.isnull().sum().sum()) #Fills each null value in every column with mean value cols = list(X.columns) for n in cols: X[n].fillna(X[n].mean(),inplace = True) #Checks again null values print(X.isnull().sum().to_string()) print('Total number of null values: ', X.isnull().sum().sum()) # Check and convert data types X = X.astype(float) y = y.astype(int) sm = SMOTE(random_state=42) X,y = sm.fit_resample(X, y) #Splits the data into training and testing X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 2021, stratify=y) #Use Standard Scaler scaler = StandardScaler() X_train_stand = scaler.fit_transform(X_train) X_test_stand = scaler.transform(X_test) #Saves into pkl files joblib.dump(X_train_stand, 'X_train.pkl') joblib.dump(X_test_stand, 'X_test.pkl') joblib.dump(y_train, 'y_train.pkl') joblib.dump(y_test, 'y_test.pkl') def load_files(self): X_train = joblib.load('X_train.pkl') X_test = joblib.load('X_test.pkl') y_train = joblib.load('y_train.pkl') y_test = joblib.load('y_test.pkl') return X_train, X_test, y_train, y_test def train_model(self, model, X, y): model.fit(X, y) return model def predict_model(self, model, X, proba=False): if ~proba: y_pred = model.predict(X) else: y_pred_proba = model.predict_proba(X) y_pred = np.argmax(y_pred_proba, axis=1) return y_pred def run_model(self, name, model, X_train, X_test, y_train, y_test, proba=False): y_pred = self.predict_model(model, X_test, proba) accuracy = accuracy_score(y_test, y_pred) recall = recall_score(y_test, y_pred, average='weighted') precision = precision_score(y_test, y_pred, average='weighted') f1 = f1_score(y_test, y_pred, average='weighted') print(name) print('accuracy: ', accuracy) print('recall: ', recall) print('precision: ', precision) print('f1: ', f1) print(classification_report(y_test, y_pred)) return y_pred def logistic_regression(self, name, X_train, X_test, y_train, y_test): #Logistic Regression Classifier # Define the parameter grid for the grid search param_grid = { 'C': [0.01, 0.1, 1, 10], 'penalty': ['none', 'l2'], 'solver': ['newton-cg', 'lbfgs', 'liblinear', 'saga'], } # Initialize the Logistic Regression model logreg = LogisticRegression(max_iter=5000, random_state=2021) # Create GridSearchCV with the Logistic Regression model and the parameter grid grid_search = GridSearchCV(logreg, param_grid, cv=3, scoring='accuracy', n_jobs=-1) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best Logistic Regression model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'LR_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for LR:") print(grid_search.best_params_) return best_model def implement_LR(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/LR_Model.pkl" if os.path.exists(file_path): model = joblib.load('LR_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.logistic_regression(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_LR.csv") print("Training Logistic Regression done...") return model, y_pred def random_forest(self, name, X_train, X_test, y_train, y_test): #Random Forest Classifier # Define the parameter grid for the grid search param_grid = { 'n_estimators': [100, 200, 300], 'max_depth': [10, 20, 30, 40, 50], 'min_samples_split': [2, 5, 10], 'min_samples_leaf': [1, 2, 4] } # Initialize the RandomForestClassifier model rf = RandomForestClassifier(random_state=2021) # Create GridSearchCV with the RandomForestClassifier model and the parameter grid grid_search = GridSearchCV(rf, param_grid, cv=3, scoring='accuracy', n_jobs=-1) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best RandomForestClassifier model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'RF_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for RF:") print(grid_search.best_params_) return best_model def implement_RF(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/RF_Model.pkl" if os.path.exists(file_path): model = joblib.load('RF_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.random_forest(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_RF.csv") print("Training Random Forest done...") return model, y_pred def knearest_neigbors(self, name, X_train, X_test, y_train, y_test): #KNN Classifier # Define the parameter grid for the grid search param_grid = { 'n_neighbors': list(range(2, 10)) } # Initialize the KNN Classifier knn = KNeighborsClassifier() # Create GridSearchCV with the KNN model and the parameter grid grid_search = GridSearchCV(knn, param_grid, cv=3, scoring='accuracy', n_jobs=-1) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best KNN model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'KNN_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for KNN:") print(grid_search.best_params_) return best_model def implement_KNN(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/KNN_Model.pkl" if os.path.exists(file_path): model = joblib.load('KNN_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.knearest_neigbors(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_KNN.csv") print("Training KNN done...") return model, y_pred def decision_trees(self, name, X_train, X_test, y_train, y_test): # Initialize the DecisionTreeClassifier model dt_clf = DecisionTreeClassifier(random_state=2021) # Define the parameter grid for the grid search param_grid = { 'max_depth': np.arange(1, 51, 1), 'criterion': ['gini', 'entropy'], 'min_samples_split': [2, 5, 10], 'min_samples_leaf': [1, 2, 4], } # Create GridSearchCV with the DecisionTreeClassifier model and the parameter grid grid_search = GridSearchCV(dt_clf, param_grid, cv=3, scoring='accuracy', n_jobs=-1) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best DecisionTreeClassifier model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'DT_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for DT:") print(grid_search.best_params_) return best_model def implement_DT(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/DT_Model.pkl" if os.path.exists(file_path): model = joblib.load('DT_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.decision_trees(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_DT.csv") print("Training Decision Trees done...") return model, y_pred def gradient_boosting(self, name, X_train, X_test, y_train, y_test): #Gradient Boosting Classifier # Initialize the GradientBoostingClassifier model gbt = GradientBoostingClassifier(random_state=2021) # Define the parameter grid for the grid search param_grid = { 'n_estimators': [100, 200, 300], 'max_depth': [10, 20, 30], 'subsample': [0.6, 0.8, 1.0], 'max_features': [0.2, 0.4, 0.6, 0.8, 1.0], } # Create GridSearchCV with the GradientBoostingClassifier model and the parameter grid grid_search = GridSearchCV(gbt, param_grid, cv=3, scoring='accuracy', n_jobs=-1) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best GradientBoostingClassifier model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'GB_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for GB:") print(grid_search.best_params_) return best_model def implement_GB(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/GB_Model.pkl" if os.path.exists(file_path): model = joblib.load('GB_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.gradient_boosting(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_GB.csv") print("Training Gradient Boosting done...") return model, y_pred def extreme_gradient_boosting(self, name, X_train, X_test, y_train, y_test): # Define the parameter grid for the grid search param_grid = { 'n_estimators': [100, 200, 300], 'max_depth': [10, 20, 30], 'learning_rate': [0.01, 0.1, 0.2], 'subsample': [0.6, 0.8, 1.0], 'colsample_bytree': [0.6, 0.8, 1.0], } # Initialize the XGBoost classifier xgb = XGBClassifier(random_state=2021, use_label_encoder=False, eval_metric='mlogloss') # Create GridSearchCV with the XGBoost classifier and the parameter grid grid_search = GridSearchCV(xgb, param_grid, cv=3, scoring='accuracy', n_jobs=-1) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best XGBoost classifier model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'XGB_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for XGB:") print(grid_search.best_params_) return best_model def implement_XGB(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/XGB_Model.pkl" if os.path.exists(file_path): model = joblib.load('XGB_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.extreme_gradient_boosting(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_XGB.csv") print("Training Extreme Gradient Boosting done...") return model, y_pred def multi_layer_perceptron(self, name, X_train, X_test, y_train, y_test): # Define the parameter grid for the grid search param_grid = { 'hidden_layer_sizes': [(50,), (100,), (50, 50), (100, 50), (100, 100)], 'activation': ['logistic', 'relu'], 'solver': ['adam', 'sgd'], 'alpha': [0.0001, 0.001, 0.01], 'learning_rate': ['constant', 'invscaling', 'adaptive'], } # Initialize the MLP Classifier mlp = MLPClassifier(random_state=2021) # Create GridSearchCV with the MLP Classifier and the parameter grid grid_search = GridSearchCV(mlp, param_grid, cv=3, scoring='accuracy', n_jobs=-1) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best MLP Classifier model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'MLP_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for MLP:") print(grid_search.best_params_) return best_model def implement_MLP(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/MLP_Model.pkl" if os.path.exists(file_path): model = joblib.load('MLP_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.multi_layer_perceptron(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_MLP.csv") print("Training Multi-Layer Perceptron done...") return model, y_pred def support_vector(self, name, X_train, X_test, y_train, y_test): #Support Vector Classifier # Define the parameter grid for the grid search param_grid = { 'C': [0.1, 1, 10], 'kernel': ['linear', 'poly', 'rbf'], 'gamma': ['scale', 'auto', 0.1, 1], } # Initialize the SVC model model_svc = SVC(random_state=2021, probability=True) # Create GridSearchCV with the SVC model and the parameter grid grid_search = GridSearchCV(model_svc, param_grid, cv=3, scoring='accuracy', n_jobs=-1, refit=True) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best MLP Classifier model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'SVC_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for SVC:") print(grid_search.best_params_) return best_model def implement_SVC(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/SVC_Model.pkl" if os.path.exists(file_path): model = joblib.load('SVC_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.support_vector(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_SVC.csv") print("Training Support Vector Classifier done...") return model, y_pred def adaboost_classifier(self, name, X_train, X_test, y_train, y_test): # Define the parameter grid for the grid search param_grid = { 'n_estimators': [50, 100, 150], 'learning_rate': [0.01, 0.1, 0.2], } # Initialize the AdaBoost classifier adaboost = AdaBoostClassifier(random_state=2021) # Create GridSearchCV with the AdaBoost classifier and the parameter grid grid_search = GridSearchCV(adaboost, param_grid, cv=3, scoring='accuracy', n_jobs=-1) # Train and perform grid search grid_search.fit(X_train, y_train) # Get the best AdaBoost Classifier model from the grid search best_model = grid_search.best_estimator_ #Saves model joblib.dump(best_model, 'ADA_Model.pkl') # Print the best hyperparameters found print(f"Best Hyperparameters for AdaBoost:") print(grid_search.best_params_) return best_model def implement_ADA(self, chosen, X_train, X_test, y_train, y_test): file_path = os.getcwd()+"/ADA_Model.pkl" if os.path.exists(file_path): model = joblib.load('ADA_Model.pkl') y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) else: model = self.adaboost_classifier(chosen, X_train, X_test, y_train, y_test) y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) #Saves result into excel file self.obj_data.save_result(y_test, y_pred, "results_ADA.csv") print("Training AdaBoost done...") return model, y_pred #regression.py import pandas as pd import numpy as np from sklearn.preprocessing import MinMaxScaler import joblib from sklearn.metrics import mean_squared_error, mean_absolute_error from sklearn.metrics import roc_auc_score,roc_curve, r2_score, explained_variance_score from sklearn.linear_model import LinearRegression from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor, AdaBoostRegressor from sklearn.model_selection import GridSearchCV from sklearn.tree import DecisionTreeRegressor from xgboost import XGBRegressor from sklearn.neural_network import MLPRegressor from sklearn.linear_model import LassoCV from sklearn.linear_model import RidgeCV from sklearn.neighbors import KNeighborsRegressor class Regression: def splitting_data_regression(self, df): #Sets target column y_final = pd.DataFrame(df["Adj Close"]) X = df.drop(["Adj Close"], axis =1) #Checks null values because of technical indicators print(X.isnull().sum().to_string()) print('Total number of null values: ', X.isnull().sum().sum()) #Fills each null value in every column with mean value cols = list(X.columns) for n in cols: X[n].fillna(X[n].mean(),inplace = True) #Checks again null values print(X.isnull().sum().to_string()) print('Total number of null values: ', X.isnull().sum().sum()) #Normalizes data scaler = MinMaxScaler() X_minmax_data = scaler.fit_transform(X) X_final = pd.DataFrame(columns=X.columns, data=X_minmax_data, index=X.index) print('Shape of features : ', X_final.shape) print('Shape of target : ', y_final.shape) #Shifts target array to predict the n + 1 samples n=90 y_final = y_final.shift(-1) y_val = y_final[-n:-1] y_final = y_final[:-n] #Takes last n rows of data to be validation set X_val = X_final[-n:-1] X_final = X_final[:-n] print("\n -----After process------ \n") print('Shape of features : ', X_final.shape) print('Shape of target : ', y_final.shape) print(y_final.tail().to_string()) y_final=y_final.astype('float64') #Splits data into training and test data at 80% and 20% respectively split_idx=round(0.8*len(X)) print("split_idx=",split_idx) X_train_reg = X_final[:split_idx] y_train_reg = y_final[:split_idx] X_test_reg = X_final[split_idx:] y_test_reg = y_final[split_idx:] #Saves into pkl files joblib.dump(X, 'X_Ori.pkl') joblib.dump(X_final, 'X_final_reg.pkl') joblib.dump(X_train_reg, 'X_train_reg.pkl') joblib.dump(X_test_reg, 'X_test_reg.pkl') joblib.dump(X_val, 'X_val_reg.pkl') joblib.dump(y_final, 'y_final_reg.pkl') joblib.dump(y_train_reg, 'y_train_reg.pkl') joblib.dump(y_test_reg, 'y_test_reg.pkl') joblib.dump(y_val, 'y_val_reg.pkl') def load_regression_files(self): X_Ori = joblib.load('X_Ori.pkl') X_final_reg = joblib.load('X_final_reg.pkl') X_train_reg = joblib.load('X_train_reg.pkl') X_test_reg = joblib.load('X_test_reg.pkl') X_val_reg = joblib.load('X_val_reg.pkl') y_final_reg = joblib.load('y_final_reg.pkl') y_train_reg = joblib.load('y_train_reg.pkl') y_test_reg = joblib.load('y_test_reg.pkl') y_val_reg = joblib.load('y_val_reg.pkl') return X_Ori, X_final_reg, X_train_reg, X_test_reg, X_val_reg, y_final_reg, y_train_reg, y_test_reg, y_val_reg def perform_regression(self, model, X, y, xtrain, ytrain, xtest, ytest, xval, yval, label): model.fit(xtrain, ytrain) predictions_test = model.predict(xtest) predictions_train = model.predict(xtrain) predictions_val = model.predict(xval) # Convert ytest and predictions_test to NumPy arrays ytest_np = ytest.to_numpy().flatten() predictions_test_np = predictions_test.flatten() str_label = 'RMSE using ' + label print(str_label + f': {np.sqrt(mean_squared_error(ytest_np, predictions_test_np))}') print("mean square error: ", mean_squared_error(ytest_np, predictions_test_np)) print("variance or r-squared: ", explained_variance_score(ytest_np, predictions_test_np)) print("mean absolute error (MAE): ", mean_absolute_error(ytest_np, predictions_test_np)) print("R2 (R-squared): ", r2_score(ytest_np, predictions_test_np)) print("Adjusted R2: ", 1 - (1-r2_score(ytest_np, predictions_test_np))*(len(ytest_np)-1)/(len(ytest_np)-xtest.shape[1]-1)) mean_percentage_error = np.mean((ytest_np - predictions_test_np) / ytest_np) * 100 print("Mean Percentage Error (MPE): ", mean_percentage_error) mean_absolute_percentage_error = np.mean(np.abs((ytest_np - predictions_test_np) / ytest_np)) * 100 print("Mean Absolute Percentage Error (MAPE): ", mean_absolute_percentage_error) print('ACTUAL: Avg. ' + f': {ytest_np.mean()}') print('ACTUAL: Median ' + f': {np.median(ytest_np)}') print('PREDICTED: Avg. ' + f': {predictions_test_np.mean()}') print('PREDICTED: Median ' + f': {np.median(predictions_test_np)}') # Evaluation of regression on all dataset all_pred = model.predict(X) print("mean square error (whole dataset): ", mean_squared_error(y, all_pred)) print("variance or r-squared (whole dataset): ", explained_variance_score(y, all_pred)) return predictions_test, predictions_train, predictions_val, all_pred def linear_regression(self, X_train, y_train): #Linear Regression #Creates a Linear Regression model lin_reg = LinearRegression() #Defines the hyperparameter grid to search param_grid = { 'fit_intercept': [True, False], # Try both True and False for fit_intercept 'normalize': [True, False] # Try both True and False for normalize } #Creates GridSearchCV with the Linear Regression model and the hyperparameter grid grid_search = GridSearchCV(lin_reg, param_grid, cv=5, scoring='neg_mean_squared_error') #Fits the GridSearchCV to the training data grid_search.fit(X_train, y_train) #Gets the best Linear Regression model from the grid search best_lin_reg = grid_search.best_estimator_ #Prints the best hyperparameters found print("Best Hyperparameters for Linear Regression:") print(grid_search.best_params_) return best_lin_reg def rf_regression(self, X_train, y_train): #Random Forest Regression # Create a RandomForestRegressor model rf_reg = RandomForestRegressor() # Define the hyperparameter grid to search param_grid = { 'n_estimators': [50, 100, 150], # Number of trees in the forest 'max_depth': [None, 5, 10], # Maximum depth of the tree 'min_samples_split': [2, 5, 10], # Minimum number of samples required to split an internal node 'min_samples_leaf': [1, 2, 4], # Minimum number of samples required to be at a leaf node 'bootstrap': [True, False] # Whether bootstrap samples are used when building trees } # Create GridSearchCV with the RandomForestRegressor model and the hyperparameter grid grid_search = GridSearchCV(rf_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best RandomForestRegressor model from the grid search best_rf_reg = grid_search.best_estimator_ # Print the best hyperparameters found print("Best Hyperparameters for RandomForestRegressor:") print(grid_search.best_params_) return best_rf_reg def dt_regression(self, X_train, y_train): #Decision Tree (DT) regression # Create a DecisionTreeRegressor model dt_reg = DecisionTreeRegressor(random_state=100) # Define the hyperparameter grid to search param_grid = { 'max_depth': [None, 5, 10, 15], # Maximum depth of the tree 'min_samples_split': [2, 5, 10], # Minimum number of samples required to split an internal node 'min_samples_leaf': [1, 2, 4, 6], # Minimum number of samples required to be at a leaf node } # Create GridSearchCV with the DecisionTreeRegressor model and the hyperparameter grid grid_search = GridSearchCV(dt_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best DecisionTreeRegressor model from the grid search best_dt_reg = grid_search.best_estimator_ # Print the best hyperparameters found print("Best Hyperparameters for DecisionTreeRegressor:") print(grid_search.best_params_) return best_dt_reg def gb_regression(self, X_train, y_train): #Gradient Boosting regression # Create the GradientBoostingRegressor model gb_reg = GradientBoostingRegressor() # Define the hyperparameter grid to search param_grid = { 'n_estimators': [50, 100, 150], # Number of boosting stages (trees) to build 'learning_rate': [0.01, 0.1, 0.5], # Step size at each boosting iteration 'max_depth': [3, 5, 7], # Maximum depth of the individual trees 'min_samples_split': [2, 5, 10], # Minimum number of samples required to split an internal node 'min_samples_leaf': [1, 2, 4], # Minimum number of samples required to be at a leaf node } # Create GridSearchCV with the GradientBoostingRegressor model and the hyperparameter grid grid_search = GridSearchCV(gb_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best GradientBoostingRegressor model from the grid search best_gb_reg = grid_search.best_estimator_ # Print the best hyperparameters found print("Best Hyperparameters for GradientBoostingRegressor:") print(grid_search.best_params_) return best_gb_reg def xgb_regression(self, X_train, y_train): #Extreme Gradient Boosting (XGB) # Create the XGBRegressor model xgb_reg = XGBRegressor() # Define the hyperparameter grid to search param_grid = { 'n_estimators': [50, 100, 150], # Number of boosting stages (trees) to build 'learning_rate': [0.01, 0.1, 0.5], # Step size at each boosting iteration 'max_depth': [3, 5, 7], # Maximum depth of the individual trees 'min_child_weight': [1, 2, 4], # Minimum sum of instance weight (hessian) needed in a child 'gamma': [0, 0.1, 0.2], # Minimum loss reduction required to make a further partition on a leaf node 'subsample': [0.8, 1.0], # Subsample ratio of the training instances 'colsample_bytree': [0.8, 1.0] # Subsample ratio of columns when constructing each tree } # Create GridSearchCV with the XGBRegressor model and the hyperparameter grid grid_search = GridSearchCV(xgb_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best XGBRegressor model from the grid search best_xgb_reg = grid_search.best_estimator_ # Print the best hyperparameters found print("Best Hyperparameters for XGBRegressor:") print(grid_search.best_params_) return best_xgb_reg def mlp_regression(self, X_train, y_train): #MLP regression # Create the MLPRegressor model mlp_reg = MLPRegressor() # Define the hyperparameter grid to search param_grid = { 'hidden_layer_sizes': [(50,), (100,), (50, 50), (100, 50)], # Number of neurons in each hidden layer 'activation': ['relu', 'tanh'], # Activation function for the hidden layers 'solver': ['adam', 'sgd'], # Solver for weight optimization 'learning_rate': ['constant', 'invscaling', 'adaptive'], # Learning rate schedule 'learning_rate_init': [0.01, 0.001], # Initial learning rate 'max_iter': [100, 200, 300], # Maximum number of iterations } # Create GridSearchCV with the MLPRegressor model and the hyperparameter grid grid_search = GridSearchCV(mlp_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best MLPRegressor model from the grid search best_mlp_reg = grid_search.best_estimator_ # Print the best hyperparameters found print("Best Hyperparameters for MLPRegressor:") print(grid_search.best_params_) return best_mlp_reg def lasso_regression(self, X_train, y_train): # Create the LassoCV model lasso_reg = LassoCV(n_alphas=1000, max_iter=3000, random_state=0) # Define the hyperparameter grid to search param_grid = { 'normalize': [True, False], # Whether to normalize the features before fitting the model 'fit_intercept': [True, False] # Whether to calculate the intercept for this model } # Create GridSearchCV with the LassoCV model and the hyperparameter grid grid_search = GridSearchCV(lasso_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best LassoCV model from the grid search best_lasso_reg = grid_search.best_estimator_ # Print the best hyperparameters found print("Best Hyperparameters for Lasso Regression:") print(grid_search.best_params_) return best_lasso_reg def ridge_regression(self, X_train, y_train): #Ridge regression ridge_reg = RidgeCV(gcv_mode='auto') # Define the hyperparameter grid to search (optional if you want to include other hyperparameters) param_grid = { 'normalize': [True, False], # Whether to normalize the features before fitting the model 'fit_intercept': [True, False] # Whether to calculate the intercept for this model } # Create GridSearchCV with the RidgeCV model and the hyperparameter grid (optional if you include the param_grid) grid_search = GridSearchCV(ridge_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best RidgeCV model from the grid search best_ridge_reg = grid_search.best_estimator_ # Print the best hyperparameters found (optional if you included the param_grid) print("Best Hyperparameters for Ridge Regression:") print(grid_search.best_params_) return best_ridge_reg def adaboost_regression(self, X_train, y_train): #Adaboost regression # Create the AdaBoostRegressor model ada_reg = AdaBoostRegressor() # Define the hyperparameter grid to search param_grid = { 'n_estimators': [50, 100, 150], # Number of boosting stages (trees) to build 'learning_rate': [0.01, 0.1, 0.5], # Step size at each boosting iteration 'loss': ['linear', 'square', 'exponential'] # Loss function to use when updating weights } # Create GridSearchCV with the AdaBoostRegressor model and the hyperparameter grid grid_search = GridSearchCV(ada_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best AdaBoostRegressor model from the grid search best_ada_reg = grid_search.best_estimator_ # Print the best hyperparameters found print("Best Hyperparameters for AdaBoostRegressor:") print(grid_search.best_params_) return best_ada_reg def knn_regression(self, X_train, y_train): #KNN regression # Create a KNeighborsRegressor model knn_reg = KNeighborsRegressor() # Define the hyperparameter grid to search param_grid = { 'n_neighbors': [3, 5, 7, 9], # Number of neighbors to use for regression 'weights': ['uniform', 'distance'], # Weight function used in prediction 'algorithm': ['auto', 'ball_tree', 'kd_tree', 'brute'] # Algorithm used to compute the nearest neighbors } # Create GridSearchCV with the KNeighborsRegressor model and the hyperparameter grid grid_search = GridSearchCV(knn_reg, param_grid, cv=5, scoring='neg_mean_squared_error') # Fit the GridSearchCV to the training data grid_search.fit(X_train, y_train) # Get the best KNeighborsRegressor model from the grid search best_knn_reg = grid_search.best_estimator_ # Print the best hyperparameters found print("Best Hyperparameters for KNeighborsRegressor:") print(grid_search.best_params_) return best_knn_reg
No comments:
Post a Comment