Tuesday, September 5, 2023

DATA VISUALIZATION, TIME-SERIES FORECASTING, AND PREDICTION USING MACHINE LEARNING WITH TKINTER

 Dataset

Googleplay Book

Amazon

Kobo Store


This is the product of Balige Academy, North Sumatera.


Balige Academy Team

Vivian Siahaan

RIsmon Hasiholan Sianipar

HORASSS!!!

































This "Data Visualization, Time-Series Forecasting, and Prediction using Machine Learning with Tkinter" project is a comprehensive and multifaceted application that leverages data visualization, time-series forecasting, and machine learning techniques to gain insights into bitcoin data and make predictions. This project serves as a valuable tool for financial analysts, traders, and investors seeking to make informed decisions in the stock market.


The project begins with data visualization, where historical bitcoin market data is visually represented using various plots and charts. This provides users with an intuitive understanding of the data's trends, patterns, and fluctuations. Features distribution analysis is conducted to assess the statistical properties of the dataset, helping users identify key characteristics that may impact forecasting and prediction.


One of the project's core functionalities is time-series forecasting. Through a user-friendly interface built with Tkinter, users can select a stock symbol and specify the time horizon for forecasting. The project supports multiple machine learning regressors, such as Linear Regression, Decision Trees, Random Forests, Gradient Boosting, Extreme Gradient Boosting, Multi-Layer Perceptron, Lasso, Ridge, AdaBoost, and KNN, allowing users to choose the most suitable algorithm for their forecasting needs. Time-series forecasting is crucial for making predictions about stock prices, which is essential for investment strategies.


The project employs various machine learning regressors to predict the adjusted closing price of bitcoin stock. By training these models on historical data, users can obtain predictions for future adjusted closing prices. This information is invaluable for traders and investors looking to make buy or sell decisions. The project also incorporates hyperparameter tuning and cross-validation to enhance the accuracy of these predictions.


These models employ metrics such as Mean Absolute Error (MAE), which quantifies the average absolute discrepancy between predicted values and actual values. Lower MAE values signify superior model performance. Additionally, Mean Squared Error (MSE) is used to calculate the average squared differences between predicted and actual values, with lower MSE values indicating better model performance. Root Mean Squared Error (RMSE), derived from MSE, provides insights in the same units as the target variable and is valued for its lower values, denoting superior performance. Lastly, R-squared (R2) evaluates the fraction of variance in the target variable that can be predicted from independent variables, with higher values signifying better model fit. An R2 of 1 implies a perfect model fit.

In addition to close price forecasting, the project extends its capabilities to predict daily returns. By implementing grid search, users can fine-tune the hyperparameters of machine learning models such as Random Forests, Gradient Boosting, Support Vector, Decision Tree, Gradient Boosting, Extreme Gradient Boosting, Multi-Layer Perceptron, and AdaBoost Classifiers. This optimization process aims to maximize the predictive accuracy of daily returns. Accurate daily return predictions are essential for assessing risk and formulating effective trading strategies.


Key metrics in these classifiers encompass Accuracy, which represents the ratio of correctly predicted instances to the total number of instances, Precision, which measures the proportion of true positive predictions among all positive predictions, and Recall (also known as Sensitivity or True Positive Rate), which assesses the proportion of true positive predictions among all actual positive instances. The F1-Score serves as the harmonic mean of Precision and Recall, offering a balanced evaluation, especially when considering the trade-off between false positives and false negatives. The ROC Curve illustrates the trade-off between Recall and False Positive Rate, while the Area Under the ROC Curve (AUC-ROC) summarizes this trade-off. The Confusion Matrix provides a comprehensive view of classifier performance by detailing true positives, true negatives, false positives, and false negatives, facilitating the computation of various metrics like accuracy, precision, and recall. The selection of these metrics hinges on the project's specific objectives and the characteristics of the dataset, ensuring alignment with the intended goals and the ramifications of false positives and false negatives, which hold particular significance in financial contexts where decisions can have profound consequences.


Overall, the "Data Visualization, Time-Series Forecasting, and Prediction using Machine Learning with Tkinter" project serves as a powerful and user-friendly platform for financial data analysis and decision-making. It bridges the gap between complex machine learning techniques and accessible user interfaces, making financial analysis and prediction more accessible to a broader audience. With its comprehensive features, this project empowers users to gain insights from historical data, make informed investment decisions, and develop effective trading strategies in the dynamic world of finance.




#main_program.py
import os
import tkinter as tk
from tkinter import *
from PIL import Image, ImageTk
from design_window import Design_Window
from process_data import Process_Data
from helper_plot import Helper_Plot
from regression import Regression
from machine_learning import Machine_Learning

class Main_Program:
    def __init__(self, root):
        self.initialize()

    def initialize(self):
        self.root = root
        width = 1560
        height = 790
        self.root.geometry(f"{width}x{height}")
        self.root.title("TIME-SERIES ANALYSIS, FORECASTING, AND PREDICTION USING MACHINE LEARNING")

        #Creates necessary objects
        self.obj_window = Design_Window()
        self.obj_data = Process_Data()
        self.obj_plot = Helper_Plot()
        self.obj_reg = Regression()        
        self.obj_ML = Machine_Learning()

        #Places widgets in root
        self.obj_window.add_widgets(self.root)  

        #Reads dataset
        self.df = self.obj_data.preprocess()

        #Create dummy dataset for visualization
        self.df_dummy = self.df.copy()
        self.df_dummy = self.obj_data.create_dummy(self.df_dummy)

        #Binds event
        self.binds_event() 

        #Computes technical indicators
        self.df_final = self.df.copy()
        self.df_final = self.obj_data.technical_indicators(self.df_final)

        #Extracts input and output variables for regression
        self.obj_reg.splitting_data_regression(self.df_final) 

        #Extracts input and output variables for prediction
        self.obj_ML.oversampling_splitting(self.df_final)

        #turns off combo_reg and combo_pred after splitting is done
        self.obj_window.combo_reg['state'] = 'disabled'
        self.obj_window.combo_pred['state'] = 'disabled'

    def binds_event(self):
        #Binds button1 to shows_table() function
        #Shows table if user clicks LOAD DATASET 
        self.obj_window.btn_load.config(command = lambda:self.obj_plot.shows_table(self.root, 
            self.df, 1250, 600, "Bitcoin Dataset"))   

        #Binds listbox to choose_list_widget() function
        self.obj_window.listbox.bind("<<ListboxSelect>>", self.choose_list_widget)

        # Binds combo_year to choose_combo_year()
        self.obj_window.combo_year.bind("<<ComboboxSelected>>", self.choose_combo_year)

        # Binds combo_month to choose_combobox_month()
        self.obj_window.combo_month.bind("<<ComboboxSelected>>", self.choose_combobox_month)

        #Binds btn_reg to split_regression() function 
        self.obj_window.btn_reg.config(command = self.split_regression)  

        #Binds combo_pred to split_prediction() function 
        self.obj_window.btn_pred.config(command = self.split_prediction) 

        # Binds combo_tech to choose_combo_tech()
        self.obj_window.combo_tech.bind("<<ComboboxSelected>>", self.choose_combo_tech)

        # Binds combo_reg to choose_combo_reg()
        self.obj_window.combo_reg.bind("<<ComboboxSelected>>", self.choose_combo_reg)

        # Binds combo_pred to choose_combo_pred()
        self.obj_window.combo_pred.bind("<<ComboboxSelected>>", self.choose_combo_pred)

    def choose_list_widget(self, event):
        chosen = self.obj_window.listbox.get(self.obj_window.listbox.curselection())
        print(chosen)
        self.obj_plot.choose_plot(self.df, self.df_dummy, chosen, 
            self.obj_window.figure1, self.obj_window.canvas1, 
            self.obj_window.figure2, self.obj_window.canvas2)

    def choose_combo_year(self, event):
        chosen = self.obj_window.combo_year.get()
        year_data_mean, year_data_ewm, year_norm = self.obj_data.normalize_year_wise_data(self.df)
        print(year_data_mean)
        self.obj_plot.choose_year_wise(self.df, year_data_mean, year_data_ewm, year_norm, chosen, 
            self.obj_window.figure1, self.obj_window.canvas1, 
            self.obj_window.figure2, self.obj_window.canvas2)
        
    def choose_combobox_month(self, event):
        chosen = self.obj_window.combo_month.get()
        month_data_mean, month_data_ewm, month_norm = self.obj_data.normalize_month_wise_data(self.df)
        self.obj_plot.choose_month_wise(self.df, self.df_dummy, 
            month_data_mean, month_data_ewm, month_norm, chosen, 
            self.obj_window.figure1, self.obj_window.canvas1, 
            self.obj_window.figure2, self.obj_window.canvas2)

    def choose_combo_tech(self, event):
        chosen = self.obj_window.combo_tech.get()
        self.obj_plot.choose_plot_technical_indicators(self.df_final, chosen, 
            self.obj_window.figure1, self.obj_window.canvas1, 
            self.obj_window.figure2, self.obj_window.canvas2)

    def split_regression(self):
        file_path = os.getcwd()+"/X_final_reg.pkl"
        if os.path.exists(file_path):
            self.X_Ori, self.X_final_reg, self.X_train_reg, self.X_test_reg, \
            self.X_val_reg, self.y_final_reg, self.y_train_reg, \
            self.y_test_reg, self.y_val_reg = self.obj_reg.load_regression_files()
        else:
            self.obj_reg.splitting_data_regression(self.df_final)
            self.X_Ori, self.X_final_reg, self.X_train_reg, self.X_test_reg, 
            self.X_val_reg, self.y_final_reg, self.y_train_reg, 
            self.y_test_reg, self.y_val_reg = self.obj_reg.load_regression_files()

        print("Loading regression files done...")

        #turns on combo_reg after splitting is done
        self.obj_window.combo_reg['state'] = 'normal'

        self.obj_window.btn_reg.config(state="disabled")

    def choose_combo_reg(self, event):
        chosen = self.obj_window.combo_reg.get()
        
        self.obj_plot.choose_plot_regression(chosen, self.X_final_reg, 
            self.X_train_reg, self.X_test_reg, self.X_val_reg,
            self.y_final_reg, self.y_train_reg, self.y_test_reg,
            self.y_val_reg,
            self.obj_window.figure1, self.obj_window.canvas1, 
            self.obj_window.figure2, self.obj_window.canvas2)
        
    def split_prediction(self):
        file_path = os.getcwd()+"/X_train.pkl"
        if os.path.exists(file_path):
            self.X_train, self.X_test, self.y_train, self.y_test = self.obj_ML.load_files()
        else:
            self.obj_ML.oversampling_splitting(self.X, self.y)
            self.X_train, self.X_test, self.y_train, self.y_test = self.obj_ML.load_files()

        print("Loading files done...")

        #turns on combo_pred after splitting is done
        self.obj_window.combo_pred['state'] = 'normal'

        self.obj_window.btn_pred.config(state="disabled")

    def choose_combo_pred(self, event):
        chosen = self.obj_window.combo_pred.get()
        self.obj_plot.choose_plot_ML(self.root, chosen, self.X_train, self.X_test, 
            self.y_train, self.y_test, self.obj_window.figure1, 
            self.obj_window.canvas1, self.obj_window.figure2, 
            self.obj_window.canvas2)  
        
if __name__ == "__main__":
    root = tk.Tk()
    app = Main_Program(root)
    root.mainloop()

#design_window.py
import tkinter as tk
from tkinter import ttk
from matplotlib.figure import Figure
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg

class Design_Window:
    def add_widgets(self, root):
        #Set styles
        self.set_style(root)
        #Adds button(s)
        self.add_buttons(root)

        #Adds canvasses
        self.add_canvas(root)

        #Adds labels
        self.add_labels(root)

        #Adds listbox widget
        self.add_listboxes(root)

        #Adds combobox widget
        self.add_comboboxes(root)

    def set_style(self, root):
        # variables created for colors
        ebg = '#404040'
        fg = '#FFFFFF'

        style = ttk.Style()

        # Be sure to include this or style.map() won't function as expected.
        style.theme_use('alt')

        # the following alters the Listbox
        root.option_add('*TCombobox*Listbox.Background', ebg)
        root.option_add('*TCombobox*Listbox.Foreground', fg)
        root.option_add('*TCombobox*Listbox.selectBackground', fg)
        root.option_add('*TCombobox*Listbox.selectForeground', ebg)

        # the following alters the Combobox entry field
        style.map('TCombobox', fieldbackground=[('readonly', ebg)])
        style.map('TCombobox', selectbackground=[('readonly', ebg)])
        style.map('TCombobox', selectforeground=[('readonly', fg)])
        style.map('TCombobox', background=[('readonly', ebg)])
        style.map('TCombobox', foreground=[('readonly', fg)])
        
    def add_buttons(self, root):
        #Adds button
        self.btn_load = tk.Button(root, height=2, width=35, text="LOAD DATASET")
        self.btn_load.grid(row=0, column=0, padx=5, pady=5, sticky="w")

        self.btn_reg = tk.Button(root, height=2, width=35, text="SPLIT DATA FOR FORECASTING")
        self.btn_reg.grid(row=9, column=0, padx=5, pady=5, sticky="w")

        self.btn_pred = tk.Button(root, height=2, width=35, text="SPLIT DATA FOR PREDICTION")
        self.btn_pred.grid(row=12, column=0, padx=5, pady=5, sticky="w")

    def add_labels(self, root):
        #Adds labels
        self.label1 = tk.Label(root, text = "CHOOSE DISTRIBUTION")
        self.label1.grid(row=1, column=0, padx=5, pady=1, sticky="w")

        self.label2 = tk.Label(root, text = "YEAR-WISE TIME-SERIES PLOT")
        self.label2.grid(row=3, column=0, padx=5, pady=1, sticky="w")

        self.label3 = tk.Label(root, text = "MONTH-WISE TIME-SERIES PLOT")
        self.label3.grid(row=5, column=0, padx=5, pady=1, sticky="w")

        self.label4 = tk.Label(root, text = "TECHNICAL INDICATORS")
        self.label4.grid(row=7, column=0, padx=5, pady=1, sticky="w")

        self.label5 = tk.Label(root, text = "CHOOSE FORECASTING")
        self.label5.grid(row=10, column=0, padx=5, pady=1, sticky="w")

        self.label6 = tk.Label(root, text = "CHOOSE PREDICTION")
        self.label6.grid(row=13, column=0, padx=5, pady=1, sticky="w")

    def add_canvas(self, root):
        #Menambahkan canvas1 widget pada root untuk menampilkan hasil
        self.figure1 = Figure(figsize=(6.2, 7.6), dpi=100)
        self.figure1.patch.set_facecolor('#F0F0F0')
        self.canvas1 = FigureCanvasTkAgg(self.figure1, master=root)
        self.canvas1.get_tk_widget().grid(row=0, column=1, columnspan=1, 
            rowspan=25, padx=5, pady=5, sticky="n")

        #Menambahkan canvas2 widget pada root untuk menampilkan hasil
        self.figure2 = Figure(figsize=(6.2, 7.6), dpi=100)
        self.figure2.patch.set_facecolor('#F0F0F0')
        self.canvas2 = FigureCanvasTkAgg(self.figure2, master=root)
        self.canvas2.get_tk_widget().grid(row=0, column=2, columnspan=1, 
            rowspan=25, padx=5, pady=5, sticky="n")

    def add_listboxes(self, root):
        #Creates list widget
        self.listbox = tk.Listbox(root, selectmode=tk.SINGLE, width=40, 
            fg ="black", bg="#F0F0F0", 
            highlightcolor="black", selectbackground="red",relief="flat", 
            borderwidth=5, highlightthickness=0)
        self.listbox.grid(row=2, column=0, sticky='n', padx=5, pady=1)

        # Menyisipkan item ke dalam list widget
        items = ["Missing Values", "Correlation Coefficient", "Year", "Day", 
                 "Month", "Quarter", "Open versus Adj Close versus Year",
                 "Low versus High versus Quarter", "Adj Close versus Volume versus Month", 
                 "High versus Volume versus Day", "Distribution of Volume by Year", 
                 "Distribution of Volume by Days of Week", "Distribution of Volume by Month",
                 "Distribution of Volume by Quarter", "Year versus Categorized Volume",
                 "Day versus Categorized Volume", "Week versus Categorized Volume",
                 "Month versus Categorized Volume", "Quarter versus Categorized Volume",
                 "Quarter versus High Per Categorized Volume",
                 "Day versus Adj Close Per Categorized Volume",
                 "Categorized Volume", "Correlation Matrix"]
        for item in items:
            self.listbox.insert(tk.END, item)

        self.listbox.config(height=len(items)) 
        
    def add_comboboxes(self, root):
        # Create ComboBoxes
        self.combo_year = ttk.Combobox(root, width=38)
        self.combo_year["values"] = ["Low and High", "Open and Close",
            "Adj Close and Close",  "Year-Wise Mean EWM Low and High",  
            "Year-Wise Mean EWM Open and Close",                 
            "Normalized Year-Wise Data", 
            "Adj Close by Year", "Volume by Year",
            "Open by Year", "Close by Year",
            "Low by Year", "High by Year"]
        self.combo_year.grid(row=4, column=0, padx=5, pady=1, sticky="n")

        self.combo_month = ttk.Combobox(root, width=38, style='TCombobox')
        self.combo_month["values"] = ["Quarter-Wise Low and High", "Quarter-Wise Open and Close",
            "Month-Wise Open and Adj Close", "Month-Wise Mean EWM Low and High",
            "Month-Wise Mean EWM Open and Close", "Month-Wise Adj Close",
            "Month-Wise Open", "Month-Wise Close", "Month-Wise Low",
            "Month-Wise High", "Month-Wise Volume", "Normalized Month-Wise Data",
            "Adj Close by Month", "Open by Month", "Close by Month",
            "Low by Month", "High by Month", "Volume by Month"]
        self.combo_month.grid(row=6, column=0, padx=5, pady=1, sticky="n")

        self.combo_tech = ttk.Combobox(root, width=38, style='TCombobox')
        self.combo_tech["values"] = ["Adj Close versus Daily_Returns versus Year", 
            "Volume versus Daily_Returns versus Quarter", 
            "Low versus Daily_Returns versus Month",
            "High versus Daily_Returns versus Day",
            "Technical Indicators", "Differences"]
        self.combo_tech.grid(row=8, column=0, padx=5, pady=1, sticky="n")

        self.combo_reg = ttk.Combobox(root, width=38, style='TCombobox')
        self.combo_reg["values"] = ["Linear Regression", "RF Regression",
            "Decision Trees Regression", "KNN Regression",
            "AdaBoost Regression", "Gradient Boosting Regression",
            "MLP Regression", "SVR Regression", "Lasso Regression", "Ridge Regression"]
        self.combo_reg.grid(row=11, column=0, padx=5, pady=1, sticky="n")

        self.combo_pred = ttk.Combobox(root, width=38, style='TCombobox')
        self.combo_pred["values"] = ["Logistic Regression", "Random Forest",
            "Decision Trees", "K-Nearest Neighbors",
            "AdaBoost", "Gradient Boosting",
            "Extreme Gradient Boosting", "Light Gradient Boosting", 
            "Multi-Layer Perceptron", "Support Vector Classifier"]
        self.combo_pred.grid(row=14, column=0, padx=5, pady=1, sticky="n")


#process_data.py
import os
import pandas as pd 

class Process_Data:
    def read_dataset(self, filename):
        #Reads dataset
        curr_path = os.getcwd()
        path = os.path.join(curr_path, filename) 
        df = pd.read_csv(path)

        return df
    
    def preprocess(self):
        df = self.read_dataset("BTC-USD.csv")

        #Extracts day, month, week, quarter, and year
        df['Date'] = pd.to_datetime(df['Date'])
        df['Day'] = df['Date'].dt.weekday
        df['Month'] = df['Date'].dt.month
        df['Year']  = df['Date'].dt.year
        df['Week'] = df['Date'].dt.isocalendar().week
        df['Quarter']= df['Date'].dt.quarter

        #Sets Date column as index
        df = df.set_index("Date")

        return df  

    def create_dummy(self, df):
        #Creates a dummy dataframe for visualization
        df_dummy=df.copy()

        #Converts days and months from numerics to meaningful string
        days = {0:'Sunday',1:'Monday',2:'Tuesday',3:'Wednesday',
                4:'Thursday',5: 'Friday',6:'Saturday'}
        df_dummy['Day'] = df_dummy['Day'].map(days)

        months={1:'January',2:'February',3:'March',4:'April',
                5:'May',6:'June',7:'July',8:'August',9:'September',
                10:'October',11:'November',12:'December'}
        df_dummy['Month']= df_dummy['Month'].map(months)

        quarters = {1:'Jan-March', 2:'April-June',3:'July-Sept',
                    4:'Oct-Dec'}
        df_dummy['Quarter'] = df_dummy['Quarter'].map(quarters)

        #Categorizes Volume feature
        # Calculate min, max, and average values of 'Volume'
        min_volume = df['Volume'].min()
        max_volume = df['Volume'].max()
        avg_volume = df['Volume'].mean()
        print(min_volume, avg_volume, max_volume)
        #5.914.570 1.484.7039.725 350.967.941.479
        labels = ['0-500M', '500M-1000M', '1000M-10000M','10000M-20000M','>20000M']
        df_dummy['Cat_Volume'] = pd.cut(df_dummy['Volume'],
            [0, 500000000, 1000000000, 10000000000, 20000000000, max_volume], labels=labels)

        return df_dummy

    def normalize_year_wise_data(self, df):
        #Normalizes year-wise data
        cols = list(df.columns)
        cols.remove("Month")
        cols.remove("Day")
        cols.remove("Week")
        cols.remove("Year")
        cols.remove("Quarter")

        year_data_mean = df.resample('y').mean()
        year_data_ewm = year_data_mean.ewm(span=5).mean()        
        year_norm = (year_data_mean[cols] - year_data_mean[cols].min()) / (year_data_mean[cols].max() - year_data_mean[cols].min())

        return year_data_mean, year_data_ewm, year_norm

    def normalize_month_wise_data(self, df):
        cols = list(df.columns)
        cols.remove("Month")
        cols.remove("Day")
        cols.remove("Week")
        cols.remove("Year")
        cols.remove("Quarter")        
        month_data_mean = df[cols].resample('m').mean()
        month_data_ewm = month_data_mean.ewm(span=5).mean()

        month_norm = (month_data_mean - month_data_mean.min()) / (month_data_mean.max() - month_data_mean.min())

        return month_data_mean, month_data_ewm, month_norm

    def compute_daily_returns(self, df):
        """Compute and return the daily return values."""
        # Note: Returned DataFrame must have the same number of rows
        daily_return = (df / df.shift(1)) - 1
        daily_return[0] = 0
        return daily_return

    def calculate_MACD(self, df, nslow=26, nfast=12):
        emaslow = df.ewm(span=nslow, min_periods=nslow, adjust=True, ignore_na=False).mean()
        emafast = df.ewm(span=nfast, min_periods=nfast, adjust=True, ignore_na=False).mean()
        dif = emafast - emaslow
        MACD = dif.ewm(span=9, min_periods=9, adjust=True, ignore_na=False).mean()
        return dif, MACD

    def calculate_RSI(self, df, periods=14):
        # wilder's RSI
        delta = df.diff()
        up, down = delta.copy(), delta.copy()

        up[up < 0] = 0
        down[down > 0] = 0

        rUp = up.ewm(com=periods,adjust=False).mean()
        rDown = down.ewm(com=periods, adjust=False).mean().abs()

        rsi = 100 - 100 / (1 + rUp / rDown)
        return rsi

    def calculate_SMA(self, df, peroids=15):
        SMA = df.rolling(window=peroids, min_periods=peroids, center=False).mean()
        return SMA

    def calculate_BB(self, df, peroids=15):
        STD = df.rolling(window=peroids,min_periods=peroids, center=False).std()
        SMA = self.calculate_SMA(df)
        upper_band = SMA + (2 * STD)
        lower_band = SMA - (2 * STD)
        return upper_band, lower_band

    def calculate_stdev(self, df,periods=5):
        STDEV = df.rolling(periods).std()
        return STDEV

    def technical_indicators(self, df):
        stock_close = df["Adj Close"]

        #Computes daily returns
        df["Daily_Returns"] = self.compute_daily_returns(stock_close)

        #Calculates Simple Moving Average for Adj Close
        df['SMA'] = self.calculate_SMA(stock_close)

        #Calculates Bollinger Bands for Adj Close
        upper_band, lower_band = self.calculate_BB(stock_close)
        df['Upper_band'] = upper_band
        df['Lower_band'] = lower_band

        #Calculates MACD for Adj Close
        DIF, MACD = self.calculate_MACD(stock_close)
        df['DIF'] = DIF
        df['MACD'] = MACD

        #Calculates RSI for Adj Close
        df['RSI'] = self.calculate_RSI(stock_close)

        #Calculates Standard deviation for Adj Close
        df['STDEV']= self.calculate_stdev(stock_close)

        #Plots the difference of Open and Close and the difference of High and Low
        Open_Close = df.Open - stock_close
        High_Low = df.High - df.Low
        df['Open_Close'] = Open_Close
        df['High_Low'] = High_Low

        return df
    
    def save_result(self, y_test, y_pred, fname):
        # Convert y_test and y_pred to pandas Series for easier handling
        y_test_series = pd.Series(y_test)
        y_pred_series = pd.Series(y_pred)
        
        # Calculate y_result_series
        y_result_series = pd.Series(y_pred - y_test == 0)
        y_result_series = y_result_series.map({True: 'True', False: 'False'})

        # Create a DataFrame to hold y_test, y_pred, and y_result
        data = pd.DataFrame({'y_test': y_test_series, 'y_pred': y_pred_series, 'result': y_result_series})

        # Save the DataFrame to a CSV file
        data.to_csv(fname, index=False)

#helper_plot.py
from tkinter import *
import seaborn as sns
import numpy as np 
import pandas as pd
from pandastable import Table
from sklearn.metrics import confusion_matrix, roc_curve, accuracy_score
from sklearn.model_selection import learning_curve
from process_data import Process_Data
from machine_learning import Machine_Learning
from regression import Regression

class Helper_Plot:
    def __init__(self):
        self.obj_data = Process_Data()
        self.obj_reg = Regression()
        self.obj_ml = Machine_Learning()

    def shows_table(self, root, df, width, height, title):
       frame = Toplevel(root) #new window
       self.table = Table(frame, dataframe=df, showtoolbar=True, showstatusbar=True)
       
       # Sets dimension of Toplevel
       frame.geometry(f"{width}x{height}")
       frame.title(title)
       self.table.show()

    def plot_missing_values(self, df, figure, canvas):
        figure.clear()
        ax = figure.add_subplot(1,1,1) 
        #Plots null values
        missing = df.isna().sum().reset_index()
        missing.columns = ['features', 'total_missing']
        missing['percent'] = (missing['total_missing'] / len(df)) * 100
        missing.index = missing['features']
        del missing['features']
        missing['total_missing'].plot(kind = 'bar', ax=ax)
        ax.set_title('Missing Values Count', fontsize = 12)
        ax.set_facecolor('#F0F0F0')
        
        # Set font for tick labels
        ax.tick_params(axis='both', which='major', labelsize=5)
        ax.tick_params(axis='both', which='minor', labelsize=5)        
        figure.tight_layout()
        canvas.draw()

    def plot_corr_coeffs(self, df, figure, canvas):
        figure.clear()
        ax = figure.add_subplot(1,1,1) 
        
        #correlation coefficient of every column with Adj Close column
        all_corr = df.corr().abs()['Adj Close'].sort_values(ascending = False)

        # Filters correlations greater than 0.25
        filtered_corr = all_corr[all_corr > 0.25]
    
        # Define a custom color palette (replace with your preferred colors)
        custom_palette = sns.color_palette("Set1", len(filtered_corr))
        filtered_corr.plot(kind='barh', ax=ax, color=custom_palette)
        ax.set_title("Correlation Coefficient of Features with Adj Close (Threshold > 0.25)", fontsize = 10)
        ax.set_ylabel("Coefficient")
        ax.set_facecolor('#F0F0F0')
        
        # Set font for tick labels
        ax.tick_params(axis='both', which='major', labelsize=5)
        ax.tick_params(axis='both', which='minor', labelsize=5)
        
        ax.grid(True)
        figure.tight_layout()
        canvas.draw()

    # Defines function to create pie chart and bar plot as subplots   
    def plot_piechart(self, df, var, figure, canvas, title=''):
        figure.clear()

        # Pie Chart (top subplot)
        ax1 = figure.add_subplot(2,1,1)        
        label_list = list(df[var].value_counts().index)
        colors = sns.color_palette("Set1", len(label_list))  
        _, _, autopcts = ax1.pie(df[var].value_counts(), autopct="%1.1f%%", colors=colors,
            startangle=30, labels=label_list,
            wedgeprops={"linewidth": 2, "edgecolor": "white"},  # Add white edge
            shadow=True, textprops={'fontsize': 7})
        ax1.set_title(title, fontsize=10)

        # Bar Plot (bottom subplot)
        ax2 = figure.add_subplot(2,1,2)
        ax = df[var].value_counts().plot(kind="barh", color=colors, alpha=0.8, ax = ax2) 
        for i, j in enumerate(df[var].value_counts().values):
            ax.text(.7, i, j, weight="bold", fontsize=7)

        ax2.set_title(title, fontsize=10)
        ax2.set_xlabel("Count")
        ax2.set_facecolor('#F0F0F0')
        figure.tight_layout()
        canvas.draw()

    def plot_piechart_group(self, df, figure, canvas, title=''):
        figure.clear()

        # Pie Chart (top subplot)
        ax1 = figure.add_subplot(2,1,1)        
        label_list = list(df.value_counts().index)
        colors = sns.color_palette("Set1", len(label_list))  
        _, _, autopcts = ax1.pie(df.value_counts(), autopct="%1.1f%%", colors=colors,
            startangle=30, labels=label_list,
            wedgeprops={"linewidth": 2, "edgecolor": "white"},  # Add white edge
            shadow=True, textprops={'fontsize': 7})
        ax1.set_title(title, fontsize=10)

        # Bar Plot (bottom subplot)
        ax2 = figure.add_subplot(2,1,2)
        ax = df.plot(kind="barh", color=colors, alpha=0.8, ax = ax2) 
        for i, j in enumerate(df.values):
            ax.text(.7, i, j, weight="bold", fontsize=7)

        ax2.set_title(title, fontsize=10)
        ax2.set_xlabel("Count")
        ax2.set_facecolor('#F0F0F0')
        figure.tight_layout()
        canvas.draw()

    def plot_scatter(self, df, x, y, hue, figure, canvas):
        figure.clear()  
        ax = figure.add_subplot(1,1,1)    
        sns.scatterplot(data=df, x=x, y=y, hue=hue, palette="Set1", ax=ax)
        ax.set_title(x + " versus " + y + " by " + hue)
        ax.set_xlabel(x)
        ax.set_ylabel(y)
        ax.grid(True)
        ax.legend(facecolor='#E6E6FA', edgecolor='black')
        ax.set_facecolor('#F0F0F0')
        figure.tight_layout()
        canvas.draw()

    #Puts label inside stacked bar
    def put_label_stacked_bar(self, ax,fontsize):
        #patches is everything inside of the chart
        for rect in ax.patches:
            # Find where everything is located
            height = rect.get_height()
            width = rect.get_width()
            x = rect.get_x()
            y = rect.get_y()
    
            # The height of the bar is the data value and can be used as the label
            label_text = f'{width:.0f}'  
    
            # ax.text(x, y, text)
            label_x = x + width / 2
            label_y = y + height / 2

            # plots only when height is greater than specified value
            if width > 0:
                ax.text(label_x, label_y, label_text, \
                    ha='center', va='center', \
                    weight = "bold",fontsize=fontsize)
    
    #Plots one variable against another variable
    def dist_one_vs_another_plot(self, df, cat1, cat2, figure, canvas, title):
        figure.clear()
        ax1 = figure.add_subplot(1,1,1)

        group_by_stat = df.groupby([cat1, cat2]).size()
        colors = sns.color_palette("Set1", len(df[cat1].unique()))
        group_by_stat.unstack().plot(kind='barh', stacked=True, ax=ax1,color=colors)
        ax1.set_title(title, fontsize=12)
        ax1.set_ylabel('Number of Cases', fontsize=10)
        ax1.set_xlabel(cat1, fontsize=10)
        self.put_label_stacked_bar(ax1,7)
        # Set font for tick labels
        ax1.tick_params(axis='both', which='major', labelsize=8)
        ax1.tick_params(axis='both', which='minor', labelsize=8)    
        ax1.legend(facecolor='#E6E6FA', edgecolor='black', fontsize=8) 
        ax1.set_facecolor('#F0F0F0')
        figure.tight_layout()
        canvas.draw()

    def box_plot(self, df, x, y, hue, figure, canvas, title):
        figure.clear()
        ax1 = figure.add_subplot(1,1,1)

        sns.boxplot(data = df, x = x, y = y, hue = hue, ax=ax1)
        ax1.set_title(title, fontsize=14)
        ax1.set_xlabel(x, fontsize=10)
        ax1.set_ylabel(y, fontsize=10)
        ax1.set_facecolor('#F0F0F0')
        ax1.legend(facecolor='#E6E6FA', edgecolor='black')
        figure.tight_layout()
        canvas.draw()

    def plot_corr_mat(self, df, figure, canvas):
        figure.clear()    
        ax = figure.add_subplot(1,1,1)  
        categorical_columns = df.select_dtypes(include=['object', 'category']).columns 
        df_removed = df.drop(columns=categorical_columns) 
        corrdata = df_removed.corr()

        annot_kws = {"size": 5}
        # Filter correlations greater than 0.1
        mask = abs(corrdata) > 0.1
        filtered_corr = corrdata[mask]
        
        # Drops features that don't meet the threshold
        filtered_corr = filtered_corr.dropna(axis=0, how='all')
        filtered_corr = filtered_corr.dropna(axis=1, how='all')
               
        sns.heatmap(filtered_corr, ax = ax, lw=1, annot=True, cmap="Greens", annot_kws=annot_kws)
        ax.set_title('Correlation Matrix (Threshold > 0.1)', fontweight="bold", fontsize=10)

        # Set font for x and y labels
        ax.set_xlabel('Features', fontweight="bold", fontsize=12)
        ax.set_ylabel('Features', fontweight="bold", fontsize=12)

        # Set font for tick labels
        ax.tick_params(axis='both', which='major', labelsize=5)
        ax.tick_params(axis='both', which='minor', labelsize=5)

        figure.tight_layout()
        canvas.draw()

    def choose_plot(self, df1, df2, chosen, figure1, canvas1, figure2, canvas2):
        print(chosen)
        if chosen == "Day":
            self.plot_piechart(df2, "Day", figure1, canvas1, "Case Distribution of Day")

        elif chosen == "Month":
            self.plot_piechart(df2, "Month", figure2, canvas2, "Case Distribution of Month")

        elif chosen == "Quarter":
            self.plot_piechart(df2, "Quarter", figure1, canvas1, "Case Distribution of Quarter")            

        elif chosen == "Year":
            self.plot_piechart(df2, "Year", figure2, canvas2, "Case Distribution of Year")              

        elif chosen == "Missing Values":
            self.plot_missing_values(df1, figure1, canvas1)

        elif chosen == "Correlation Coefficient":
            self.plot_corr_coeffs(df1, figure2, canvas2)

        elif chosen == "Open versus Adj Close versus Year":
            self.plot_scatter(df2, "Open", "Adj Close", "Year", figure1, canvas1)

        elif chosen == "Low versus High versus Quarter":
            self.plot_scatter(df2, "Low", "High", "Quarter", figure2, canvas2)       

        elif chosen == "Adj Close versus Volume versus Month":
            self.plot_scatter(df2, "Adj Close", "Volume", "Month", figure1, canvas1) 

        elif chosen == "High versus Volume versus Day":
            self.plot_scatter(df2, "High", "Volume", "Day", figure2, canvas2) 

        elif chosen == "Distribution of Volume by Year":
            self.plot_piechart_group(df2.groupby('Year')['Volume'].sum(), figure1, canvas1, chosen) 

        elif chosen == "Distribution of Volume by Days of Week":
            self.plot_piechart_group(df2.groupby('Day')['Volume'].sum(), figure2, canvas2, chosen) 

        elif chosen == "Distribution of Volume by Month":
            self.plot_piechart_group(df2.groupby('Month')['Volume'].sum(), figure1, canvas1, chosen)

        elif chosen == "Distribution of Volume by Quarter":
            self.plot_piechart_group(df2.groupby('Quarter')['Volume'].sum(), figure2, canvas2, chosen)            

        elif chosen == "Day versus Categorized Volume":
            self.dist_one_vs_another_plot(df2, "Day", "Cat_Volume", figure1, canvas1, chosen)              

        elif chosen == "Year versus Categorized Volume":
            self.dist_one_vs_another_plot(df2, "Year", "Cat_Volume", figure2, canvas2, chosen) 

        elif chosen == "Week versus Categorized Volume":
            self.dist_one_vs_another_plot(df2, "Week", "Cat_Volume", figure1, canvas1, chosen)   

        elif chosen == "Month versus Categorized Volume":
            self.dist_one_vs_another_plot(df2, "Month", "Cat_Volume", figure2, canvas2, chosen)              

        elif chosen == "Quarter versus Categorized Volume":
            self.dist_one_vs_another_plot(df2, "Quarter", "Cat_Volume", figure1, canvas1, chosen) 

        elif chosen == "Quarter versus High Per Categorized Volume":
            self.box_plot(df2, "Quarter", "High", "Cat_Volume", figure2, canvas2, chosen)

        elif chosen == "Day versus Adj Close Per Categorized Volume":
            self.box_plot(df2, "Day", "Adj Close", "Cat_Volume", figure1, canvas1, chosen)

        elif chosen == "Categorized Volume":
            self.plot_piechart(df2, "Cat_Volume", figure2, canvas2, "Case Distribution of Categorized Volume")   

        if chosen == "Correlation Matrix":
            self.plot_corr_mat(df1, figure1, canvas1)

    def line_plot_year_wise(self, df, feat1, feat2, year1, year2, figure, canvas):
        figure.clear()    
        ax1 = figure.add_subplot(2, 1, 1)  
        data1 = df[df["Year"]==year1]
        data2 = df[df["Year"]==year2]
        # Convert the column and index to NumPy arrays
        date_index1 = data1.index.to_numpy()
        date_index2 = data2.index.to_numpy()

        # Line plot
        ax1.plot(date_index1, data1[feat1].to_numpy(), 
            color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat1)
        ax1.plot(date_index1,data1[feat2].to_numpy(), 
            color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat2)
        ax1.set_xlabel('YEAR')
        ax1.set_title(feat1 + " and " + feat2 + ' (YEAR = ' + str(year1) + ')', fontsize=12)
        ax1.legend(facecolor='#E6E6FA', edgecolor='black')
        ax1.set_facecolor('#F0F0F0')
        ax1.grid(True)

        ax2 = figure.add_subplot(2, 1, 2) 
        ax2.plot(date_index2, data2[feat1].to_numpy(), 
            color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat1)
        ax2.plot(date_index2, data2[feat2].to_numpy(), 
            color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat2)
        ax2.set_xlabel('YEAR')
        ax2.set_title(feat1 + " and " + feat2 + ' (YEAR = ' + str(year2) + ')', fontsize=12)
        ax2.legend(facecolor='#E6E6FA', edgecolor='black')
        ax2.set_facecolor('#F0F0F0')
        ax2.grid(True)

        figure.tight_layout()
        canvas.draw()

    def line_plot_norm_data(self, norm_data, figure, canvas, label, title):
        figure.clear()    
        ax = figure.add_subplot(1, 1, 1)  
        
        # Convert the column and index to NumPy arrays
        date_index = norm_data.index.to_numpy()

        # Iterate over all columns (excluding 'Date')
        for column in norm_data.columns:
            if column != 'Date':
                values = norm_data[column].to_numpy()
                ax.plot(values, date_index, marker='o', linestyle='-', 
                        linewidth=1, markersize=2, label=column)

        ax.set_ylabel(label)
        ax.set_title(title, fontsize=12)
        ax.legend(fontsize=7, facecolor='#E6E6FA', edgecolor='black')
        ax.set_facecolor('#F0F0F0')
        ax.grid(True)

        figure.tight_layout()
        canvas.draw()

    def line_plot_data_mean_ewm(self, data_mean, data_ewm, feat1, feat2, figure, canvas, label, title):
        figure.clear()    
        ax1 = figure.add_subplot(2, 1, 1)  

        # Convert the column and index to NumPy arrays
        date_index = data_mean.index.to_numpy()

        # Line plot
        ax1.plot(date_index, data_mean[feat1].to_numpy(), 
            color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label="Mean")
        ax1.plot(date_index,data_ewm[feat1].to_numpy(), 
            color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label="EWM")
        ax1.set_title(title + feat1, fontsize=12)
        ax1.legend(facecolor='#E6E6FA', edgecolor='black')
        ax1.set_facecolor('#F0F0F0')
        ax1.grid(True)

        ax2 = figure.add_subplot(2, 1, 2) 
        ax2.plot(date_index, data_mean[feat2].to_numpy(), 
            color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label="Mean")
        ax2.plot(date_index,data_ewm[feat2].to_numpy(), 
            color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label="EWM")
        ax2.set_xlabel(label)
        ax2.set_title(title + feat2, fontsize=12)
        ax2.legend(facecolor='#E6E6FA', edgecolor='black')
        ax2.grid(True)
        ax2.set_facecolor('#F0F0F0')

        figure.tight_layout()
        canvas.draw()

    def box_violin_strip_heat(self, data, filter, feat1, figure1, canvas1, figure2, canvas2, title):
        figure1.clear()    
        ax1 = figure1.add_subplot(2, 1, 1)
        sns.boxplot(x = filter, y = feat1, data = data, ax=ax1)       
        ax1.set_title("Box Plot of " + feat1 + " by " + filter, fontsize=12)       
        # Set font for tick labels
        ax1.tick_params(axis='both', which='major', labelsize=6)
        ax1.tick_params(axis='both', which='minor', labelsize=6)
        ax1.grid(True)
        ax1.set_facecolor('#F0F0F0')

        ax2 = figure1.add_subplot(2, 1, 2)
        sns.violinplot(x = filter, y = feat1, data = data, ax=ax2)       
        ax2.set_title("Violin Plot of " + feat1 + " by " + filter, fontsize=12)  
        # Set font for tick labels
        ax2.tick_params(axis='both', which='major', labelsize=6)
        ax2.tick_params(axis='both', which='minor', labelsize=6)  
        ax2.grid(True)     
        ax2.set_facecolor('#F0F0F0')
        figure1.tight_layout()
        canvas1.draw()

        figure2.clear()    
        ax3 = figure2.add_subplot(2, 1, 1)
        sns.stripplot(x = filter, y = feat1, data = data, ax=ax3)       
        ax3.set_title("Strip Plot of " + feat1 + " by " + filter, fontsize=12)  
        # Set font for tick labels
        ax3.tick_params(axis='both', which='major', labelsize=6)
        ax3.tick_params(axis='both', which='minor', labelsize=6)
        ax3.set_facecolor('#F0F0F0')
        ax3.grid(True)

        ax4 = figure2.add_subplot(2, 1, 2)
        sns.swarmplot(x = filter, y = feat1, data = data, ax=ax4)
        ax4.set_title("Swarm Plot of " + feat1 + " by " + filter, fontsize=12)
        # Set font for tick labels
        ax4.tick_params(axis='both', which='major', labelsize=6)
        ax4.tick_params(axis='both', which='minor', labelsize=6) 
        ax4.grid(True)
        ax4.set_facecolor('#F0F0F0')
        figure2.tight_layout()
        canvas2.draw()

    def choose_year_wise(self, df, data_mean, data_ewm, data_norm, chosen, figure1, canvas1, figure2, canvas2): 
        if chosen == "Low and High":
            self.line_plot_year_wise(df, "Low", "High", 2015, 2016, figure1, canvas1)  

        if chosen == "Open and Close":
            self.line_plot_year_wise(df, "Open", "Close", 2017, 2018, figure2, canvas2)  

        if chosen == "Adj Close and Close":
            self.line_plot_year_wise(df, "Adj Close", "Close", 2019, 2020, figure1, canvas1) 

        if chosen == "Year-Wise Mean EWM Low and High":
            self.line_plot_data_mean_ewm(data_mean, data_ewm, "Low", "High", 
                figure2, canvas2, "YEAR", "Year-Wise Mean and EWM of ") 

        if chosen == "Year-Wise Mean EWM Open and Close":
            self.line_plot_data_mean_ewm(data_mean, data_ewm, "Open", "Close", 
                figure1, canvas1, "YEAR", "Year-Wise Mean and EWM of ")

        if chosen == "Normalized Year-Wise Data":
            self.line_plot_norm_data(data_norm, figure1, canvas1, "YEAR", "Normalized Year-Wise Data") 

        if chosen == "Adj Close by Year":
            self.box_violin_strip_heat(df, "Year", "Adj Close", figure1, canvas1, figure2, canvas2, 
                "Year-Wise Normalized ")

        if chosen == "Volume by Year":
            self.box_violin_strip_heat(df, "Year", "Volume", figure1, canvas1, figure2, canvas2, 
                "Year-Wise Normalized ")     

        if chosen == "Open by Year":
            self.box_violin_strip_heat(df, "Year", "Open", figure1, canvas1, figure2, canvas2, 
                "Year-Wise Normalized ")   
            
        if chosen == "Close by Year":
            self.box_violin_strip_heat(df, "Year", "Close", figure1, canvas1, figure2, canvas2, 
                "Year-Wise Normalized ")   

        if chosen == "High by Year":
            self.box_violin_strip_heat(df, "Year", "High", figure1, canvas1, figure2, canvas2, 
                "Year-Wise Normalized ")   
            
        if chosen == "Low by Year":
            self.box_violin_strip_heat(df, "Year", "Low", figure1, canvas1, figure2, canvas2, 
                "Year-Wise Normalized ")   

    def line_plot_month_wise(self, df, feat1, feat2, year, filter, filter1, filter2, figure, canvas):
        figure.clear()    
        ax1 = figure.add_subplot(2, 1, 1)  

        data1 = df[(df["Year"]==year)&(df[filter]==filter1)]
        data2 = df[(df["Year"]==year)&(df[filter]==filter2)]
        
        # Convert the column and index to NumPy arrays
        date_index1 = data1.index.to_numpy()
        date_index2 = data2.index.to_numpy()

        # Line plot
        ax1.plot(date_index1, data1[feat1].to_numpy(), 
            color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat1)
        ax1.plot(date_index1,data1[feat2].to_numpy(), 
            color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat2)
        ax1.set_xlabel('DATE')
        ax1.set_title(feat1 + " and " + feat2 + " " + filter + " = " + filter1 + " " + str(year), fontsize=12)
        ax1.legend(facecolor='#E6E6FA', edgecolor='black')
        ax1.set_facecolor('#F0F0F0')
        ax1.grid(True)

        # Set font for tick labels
        ax1.tick_params(axis='both', which='major', labelsize=7)
        ax1.tick_params(axis='both', which='minor', labelsize=7)

        ax2 = figure.add_subplot(2, 1, 2) 
        ax2.plot(date_index2, data2[feat1].to_numpy(), 
            color="red", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat1)
        ax2.plot(date_index2,data2[feat2].to_numpy(), 
            color="blue", marker='o', linestyle='-', linewidth=2, markersize=1, label=feat2)
        ax2.set_xlabel('DATE')
        ax2.set_title(feat1 + " and " + feat2 + " " + filter + " = " + filter2 + " " + str(year), fontsize=12)
        ax2.legend(facecolor='#E6E6FA', edgecolor='black')
        ax2.set_facecolor('#F0F0F0')
        ax2.grid(True)

        # Set font for tick labels
        ax2.tick_params(axis='both', which='major', labelsize=7)
        ax2.tick_params(axis='both', which='minor', labelsize=7)

        figure.tight_layout()
        canvas.draw()

    def color_month(self, month):
        if month == 1:
            return 'January','blue'
        elif month == 2:
            return 'February','green'
        elif month == 3:
            return 'March','orange'
        elif month == 4:
            return 'April','yellow'
        elif month == 5:
            return 'May','red'
        elif month == 6:
            return 'June','violet'
        elif month == 7:
            return 'July','purple'
        elif month == 8:
            return 'August','black'
        elif month == 9:
            return 'September','brown'
        elif month == 10:
            return 'October','darkblue'
        elif month == 11:
            return 'November','grey'
        else:
            return 'December','pink'

    def line_plot_month(self, month, data, ax):
        label, color = self.color_month(month)
        mdata = data[data.index.month == month]       
        date_index = mdata.index.to_numpy()

        ax.plot(date_index, mdata.to_numpy(), 
            marker='o', linestyle='-', 
            color=color, linewidth=2, markersize=1, label=label)

    def sns_plot_month(self, monthly_data, feat, title, figure, canvas):
        figure.clear()    
        ax = figure.add_subplot(1, 1, 1)         
        ax.set_title(title, fontsize=12)
        ax.set_xlabel('YEAR', fontsize=10)
        ax.set_ylabel(feat, fontsize=10)

        for i in range(1,13):
            self.line_plot_month(i, monthly_data[feat], ax)

        ax.legend(facecolor='#E6E6FA', edgecolor='black')
        ax.grid()
        ax.set_facecolor('#F0F0F0')
        figure.tight_layout()
        canvas.draw()

    def choose_month_wise(self, df1, df2, data_mean, data_ewm, 
            data_norm, chosen, figure1, canvas1, figure2, canvas2):  
        if chosen == "Quarter-Wise Low and High":
            self.line_plot_month_wise(df2, "Low", "High", 2018, "Quarter", "Jan-March", "April-June", figure1, canvas1)

        if chosen == "Quarter-Wise Open and Close":
            self.line_plot_month_wise(df2, "Open", "Close", 2015, "Quarter", "July-Sept", "Oct-Dec", figure2, canvas2)

        if chosen == "Month-Wise Open and Adj Close":
            self.line_plot_month_wise(df2, "Open", "Adj Close", 2016, "Month", "February", "March", figure1, canvas1)
            self.line_plot_month_wise(df2, "Open", "Adj Close", 2017, "Month", "April", "May", figure2, canvas2)

        if chosen == "Month-Wise Mean EWM Low and High":
            self.line_plot_data_mean_ewm(data_mean, data_ewm, "Low", "High", 
                figure1, canvas1, "MONTH", "Month-Wise Mean and EWM of ") 

        if chosen == "Month-Wise Mean EWM Open and Close":
            self.line_plot_data_mean_ewm(data_mean, data_ewm, "Open", "Close", 
                figure2, canvas2, "MONTH", "Month-Wise Mean and EWM of ")

        if chosen == "Month-Wise Adj Close":
            self.sns_plot_month(data_mean, "Adj Close", chosen, figure1, canvas1)

        if chosen == "Month-Wise Open":
            self.sns_plot_month(data_mean, "Open", chosen, figure2, canvas2)

        if chosen == "Month-Wise Close":
            self.sns_plot_month(data_mean, "Close", chosen, figure1, canvas1)

        if chosen == "Month-Wise Low":
            self.sns_plot_month(data_mean, "Low", chosen, figure2, canvas2)

        if chosen == "Month-Wise High":
            self.sns_plot_month(data_mean, "High", chosen, figure1, canvas1)

        if chosen == "Month-Wise Volume":
            self.sns_plot_month(data_mean, "Volume", chosen, figure2, canvas2)

        if chosen == "Normalized Month-Wise Data":
            self.line_plot_norm_data(data_norm, figure1, canvas1, "YEAR", "Normalized Month-Wise Data")

        if chosen == "Adj Close by Month":
            self.box_violin_strip_heat(df2, "Month", "Adj Close", figure1, canvas1, figure2, canvas2, 
                "Month-Wise Normalized ")

        if chosen == "Open by Month":
            self.box_violin_strip_heat(df2, "Month", "Open", figure1, canvas1, figure2, canvas2, 
                "Month-Wise Normalized ")

        if chosen == "Close by Month":
            self.box_violin_strip_heat(df2, "Month", "Close", figure1, canvas1, figure2, canvas2, 
                "Month-Wise Normalized ")

        if chosen == "Low by Month":
            self.box_violin_strip_heat(df2, "Month", "Low", figure1, canvas1, figure2, canvas2, 
                "Month-Wise Normalized ")

        if chosen == "High by Month":
            self.box_violin_strip_heat(df2, "Month", "High", figure1, canvas1, figure2, canvas2, 
                "Month-Wise Normalized ")

        if chosen == "Volume by Month":
            self.box_violin_strip_heat(df2, "Month", "Volume", figure1, canvas1, figure2, canvas2, 
                "Month-Wise Normalized ")

    def plot_technical_indicators(self, df, figure1, canvas1, figure2, canvas2):
        figure1.clear()    
        ax1 = figure1.add_subplot(2, 1, 1)  

        df["Adj Close"][:365].plot(title='Simple Moving Average, SMA, and Bands', label='Moving Average', ax=ax1)         
        df['SMA'][:365].plot(label="SMA", ax=ax1)
        df['Upper_band'][:365].plot(label='upper band', ax=ax1)
        df['Lower_band'][:365].plot(label='lower band', ax=ax1)
        ax1.legend(facecolor='#E6E6FA', edgecolor='black')
        ax1.grid(True)
        ax1.set_facecolor('#F0F0F0')

        ax2 = figure1.add_subplot(2, 1, 2)  
        df['DIF'][:365].plot(title='DIF and MACD',label='DIF', ax=ax2)
        df['MACD'][:365].plot(label='MACD', ax=ax2)
        ax2.legend(facecolor='#E6E6FA', edgecolor='black')
        ax2.grid(True)
        ax2.set_facecolor('#F0F0F0')

        figure1.tight_layout()
        canvas1.draw()

        figure2.clear()    
        ax3 = figure2.add_subplot(2, 1, 1)  

        df['RSI'].plot(title='RSI',label='RSI', color="red", ax=ax3)
        ax3.grid(True)
        ax3.set_facecolor('#F0F0F0')

        ax4 = figure2.add_subplot(2, 1, 2)
        df['STDEV'].plot(title='STDEV',label='STDEV', ax=ax4)
        ax4.set_facecolor('#F0F0F0')
        ax4.grid(True)
        figure2.tight_layout()
        canvas2.draw()

    def plot_difference(self, df, figure, canvas):
        figure.clear()    
        ax1 = figure.add_subplot(2, 1, 1)  

        df['Open_Close'][:365].plot(title='open-close', label='open_close', color="red", ax=ax1)
        ax1.grid(True)
        ax1.set_facecolor('#F0F0F0')

        ax2 = figure.add_subplot(2, 1, 2)  
        df['High_Low'][:365].plot(title='high-low', label='high-low', ax=ax2)
        ax2.grid(True)
        ax2.set_facecolor('#F0F0F0')

        figure.tight_layout()
        canvas.draw()

    def choose_plot_technical_indicators(self, df, chosen, figure1, canvas1, figure2, canvas2):
        print(chosen)
        if chosen == "Adj Close versus Daily_Returns versus Year":
            self.plot_scatter(df, "Adj Close", "Daily_Returns", "Year", figure1, canvas1)

        if chosen == "Volume versus Daily_Returns versus Quarter":
            self.plot_scatter(df, "Volume", "Daily_Returns", "Quarter", figure2, canvas2)

        if chosen == "Low versus Daily_Returns versus Month":
            self.plot_scatter(df, "Low", "Daily_Returns", "Month", figure1, canvas1)

        if chosen == "High versus Daily_Returns versus Day":
            self.plot_scatter(df, "High", "Daily_Returns", "Day", figure2, canvas2)

        if chosen == "Technical Indicators":
            self.plot_technical_indicators(df, figure1, canvas1, figure2, canvas2)

        if chosen == "Differences":
            self.plot_difference(df, figure1, canvas1)

    def scatter_train_test_regression(self, ytrain, ytest, predictions_train, predictions_test, figure, canvas, label):
        # Visualizes the training set results in a scatter plot
        figure.clear()    
        ax1 = figure.add_subplot(2, 1, 1)  
        ax1.scatter(x=ytrain, y=predictions_train, color='red', label='Training Data')
        ax1.set_title('The actual versus predicted (Training set): ' + label, fontweight='bold', fontsize=10)
        ax1.set_xlabel('Actual Train Set', fontsize=8)
        ax1.set_ylabel('Predicted Train Set', fontsize=8)
        ax1.plot([ytrain.min(), ytrain.max()], [ytrain.min(), ytrain.max()], 'b--', linewidth=2, label='Perfect Prediction')
        ax1.grid(True)
        ax1.set_facecolor('#F0F0F0')
        ax1.legend(facecolor='#E6E6FA', edgecolor='black')

        ax2 = figure.add_subplot(2, 1, 2)  
        ax2.scatter(x=ytest, y=predictions_test, color='red', label='Test Data')
        ax2.set_title('The actual versus predicted (Test set): ' + label, fontweight='bold', fontsize=10)
        ax2.set_xlabel('Actual Test Set', fontsize=8)
        ax2.set_ylabel('Predicted Test Set', fontsize=8)
        ax2.plot([ytest.min(), ytest.max()], [ytest.min(), ytest.max()], 'b--', linewidth=2, label='Perfect Prediction')
        ax2.grid(True)
        ax2.set_facecolor('#F0F0F0')
        ax2.legend(facecolor='#E6E6FA', edgecolor='black')
        
        figure.tight_layout()
        canvas.draw()

    def lineplot_train_test_regression(self, ytrain, ytest, yval, yfinal,
            predictions_train, predictions_test, predictions_val, all_pred, figure, canvas, label):
        figure.clear()    
        ax1 = figure.add_subplot(4, 1, 1)    
        ax1.plot(ytrain.index.to_numpy(), ytrain.to_numpy(), color="blue", linewidth=2, linestyle="-", label='Actual')
        ax1.plot(ytrain.index.to_numpy(), predictions_train, color="red", linewidth=2, linestyle="-", label='Predicted')
        ax1.set_title('Actual and Predicted Training Set: ' + label, fontsize=10)
        ax1.set_xlabel('Date', fontsize=8)
        ax1.set_ylabel("Adj Close", fontsize=8)
        ax1.legend(prop={'size': 8},facecolor='#E6E6FA', edgecolor='black') 
        ax1.grid(True)   
        ax1.set_facecolor('#F0F0F0')
        # Set font for tick labels
        ax1.tick_params(axis='both', which='major', labelsize=8)
        ax1.tick_params(axis='both', which='minor', labelsize=8)

        ax2 = figure.add_subplot(4, 1, 2)    
        ax2.plot(ytest.index.to_numpy(), ytest.to_numpy(), color="blue", linewidth=2, linestyle="-", label='Actual')
        ax2.plot(ytest.index.to_numpy(), predictions_test, color="red", linewidth=2, linestyle="-", label='Predicted')
        ax2.set_title('Actual and Predicted Test Set: ' + label, fontsize=10)
        ax2.set_xlabel('Date', fontsize=8)
        ax2.set_ylabel("Adj Close", fontsize=8)
        ax2.legend(prop={'size': 8}, facecolor='#E6E6FA', edgecolor='black') 
        ax2.grid(True) 
        ax2.set_facecolor('#F0F0F0')
        # Set font for tick labels
        ax2.tick_params(axis='both', which='major', labelsize=8)
        ax2.tick_params(axis='both', which='minor', labelsize=8)

        ax3 = figure.add_subplot(4, 1, 3)    
        ax3.plot(yval.index.to_numpy(), yval.to_numpy(), color="blue", linewidth=2, linestyle="-", label='Actual')
        ax3.plot(yval.index.to_numpy(), predictions_val, color="red", linewidth=2, linestyle="-", label='Predicted')
        ax3.set_title('Actual and Predicted Validation Set (90 days forecasting) ' + label, fontsize=8)
        ax3.set_xlabel('Date', fontsize=8)
        ax3.set_ylabel("Adj Close", fontsize=8)
        ax3.legend(prop={'size': 8}, facecolor='#E6E6FA', edgecolor='black') 
        ax3.grid(True) 
        ax3.set_facecolor('#F0F0F0')
        # Set font for tick labels
        ax3.tick_params(axis='both', which='major', labelsize=8)
        ax3.tick_params(axis='both', which='minor', labelsize=8)

        ax4 = figure.add_subplot(4, 1, 4)    
        ax4.plot(yfinal.index.to_numpy(), yfinal.to_numpy(), color="blue", linewidth=2, linestyle="-", label='Actual')
        ax4.plot(yfinal.index.to_numpy(), all_pred, color="red", linewidth=2, linestyle="-", label='Predicted')
        ax4.set_title('Actual and Predicted All Set ' + label, fontsize=8)
        ax4.set_xlabel('Date', fontsize=8)
        ax4.set_ylabel("Adj Close", fontsize=8)
        ax4.legend(prop={'size': 8}, facecolor='#E6E6FA', edgecolor='black') 
        ax4.grid(True) 
        ax4.set_facecolor('#F0F0F0')
        # Set font for tick labels
        ax4.tick_params(axis='both', which='major', labelsize=8)
        ax4.tick_params(axis='both', which='minor', labelsize=8)

        figure.tight_layout()
        canvas.draw()
    
    def choose_plot_regression(self, chosen, X_final_reg, X_train_reg, 
            X_test_reg, X_val_reg, y_final_reg, y_train_reg, y_test_reg, y_val_reg, 
            figure1, canvas1, figure2, canvas2):  
        if chosen == "Linear Regression":
            best_lin_reg = self.obj_reg.linear_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_lin_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)

        if chosen == "RF Regression":
            best_rf_reg = self.obj_reg.rf_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_rf_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)
            
        if chosen == "Decision Trees Regression":
            best_dt_reg = self.obj_reg.dt_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_dt_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)

        if chosen == "Gradient Boosting Regression":
            best_gb_reg = self.obj_reg.gb_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_gb_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)

        if chosen == "XGB Regression":
            best_xgb_reg = self.obj_reg.xgb_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_xgb_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)

        if chosen == "MLP Regression":
            best_mlp_reg = self.obj_reg.mlp_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_mlp_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)

        if chosen == "Lasso Regression":
            best_lasso_reg = self.obj_reg.lasso_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_lasso_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)

        if chosen == "Ridge Regression":
            best_ridge_reg = self.obj_reg.ridge_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_ridge_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)

        if chosen == "AdaBoost Regression":
            best_ada_reg = self.obj_reg.adaboost_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_ada_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)

        if chosen == "KNN Regression":
            best_knn_reg = self.obj_reg.knn_regression(X_train_reg, y_train_reg)
            predictions_test, predictions_train, predictions_val, all_pred = self.obj_reg.perform_regression(best_knn_reg, X_final_reg, y_final_reg, 
                X_train_reg, y_train_reg, X_test_reg, y_test_reg, X_val_reg, y_val_reg, chosen)

            self.scatter_train_test_regression(y_train_reg, y_test_reg, 
                predictions_train, predictions_test, figure1, canvas1, chosen)

            self.lineplot_train_test_regression(y_train_reg, 
                y_test_reg, y_val_reg, y_final_reg, predictions_train, predictions_test, predictions_val, all_pred,
                figure2, canvas2, chosen)                                  

    def plot_cm_roc(self, model, X_test, y_test, ypred, name, figure, canvas):
        figure.clear()    
        
        #Plots confusion matrix
        ax1 = figure.add_subplot(2,1,1)  
        cm = confusion_matrix(y_test, ypred, )
        sns.heatmap(cm, annot=True, linewidth=3, linecolor='red', fmt='g', cmap="viridis", annot_kws={"size": 14}, ax=ax1)
        ax1.set_title('Confusion Matrix' + " of " + name, fontsize=12)
        ax1.set_xlabel('Y predict', fontsize=10)
        ax1.set_ylabel('Y test', fontsize=10)
        ax1.xaxis.set_ticklabels(['Daily Returns = 1', 'Daily Returns = 0'], fontsize=10)
        ax1.yaxis.set_ticklabels(['Daily Returns = 1', 'Daily Returns = 0'], fontsize=10)
        ax1.set_facecolor('#F0F0F0')
        
        #Plots ROC
        ax2 = figure.add_subplot(2,1,2)
        Y_pred_prob = model.predict_proba(X_test)
        Y_pred_prob = Y_pred_prob[:, 1]

        fpr, tpr, thresholds = roc_curve(y_test, Y_pred_prob)
        ax2.plot([0,1],[0,1], color='navy', linestyle='--', linewidth=3, label='Random Guess')
        ax2.plot(fpr,tpr, color='red', linewidth=3, label='ROC Curve')
        ax2.set_xlabel('False Positive Rate', fontsize=10)
        ax2.set_ylabel('True Positive Rate', fontsize=10)
        ax2.set_title('ROC Curve of ' + name , fontsize=12)
        ax2.grid(True)
        ax2.legend(facecolor='#E6E6FA', edgecolor='black')
        ax2.set_facecolor('#F0F0F0')

        figure.tight_layout()
        canvas.draw()   

    #Plots true values versus predicted values diagram and learning curve
    def plot_real_pred_val_learning_curve(self, model, X_train, y_train, X_test, y_test, ypred, name, figure, canvas):
        figure.clear()    
        
        #Plots true values versus predicted values diagram
        ax1 = figure.add_subplot(2,1,1)  
        acc=accuracy_score(y_test, ypred)
        ax1.scatter(range(len(ypred)),ypred,color="blue", lw=2,label="Predicted")
        ax1.scatter(range(len(y_test)), 
            y_test, color="red", label="Actual")
        ax1.set_title("Predicted Values vs True Values of " + name, fontsize=12)
        ax1.set_xlabel("Accuracy: " + str(round((acc*100),3)) + "%")
        ax1.legend(facecolor='#E6E6FA', edgecolor='black')
        ax1.grid(True, alpha=0.75, lw=1, ls='-.')
        ax1.set_facecolor('#F0F0F0')

        #Plots learning curve
        train_sizes=np.linspace(.1, 1.0, 5)
        train_sizes, train_scores, test_scores, fit_times, _ = learning_curve(model, 
            X_train, y_train, cv=None, n_jobs=None, train_sizes=train_sizes, return_times=True)
        train_scores_mean = np.mean(train_scores, axis=1)
        train_scores_std = np.std(train_scores, axis=1)
        test_scores_mean = np.mean(test_scores, axis=1)
        test_scores_std = np.std(test_scores, axis=1)

        ax2 = figure.add_subplot(2,1,2)
        ax2.fill_between(train_sizes, train_scores_mean - train_scores_std,
            train_scores_mean + train_scores_std, alpha=0.1, color="r")
        ax2.fill_between(train_sizes, test_scores_mean - test_scores_std,
            test_scores_mean + test_scores_std, alpha=0.1, color="g")
        ax2.plot(train_sizes, train_scores_mean, 'o-', 
            color="b", label="Training score")
        ax2.plot(train_sizes, test_scores_mean, 'o-', 
            color="r", label="Cross-validation score")
        ax2.legend(loc="best", facecolor='#E6E6FA', edgecolor='black')
        ax2.set_title("Learning curve of " + name, fontsize=12)
        ax2.set_xlabel("fit_times")
        ax2.set_ylabel("Score")
        ax2.grid(True, alpha=0.75, lw=1, ls='-.')
        ax2.set_facecolor('#F0F0F0')

        figure.tight_layout()
        canvas.draw()  

    def choose_plot_ML(self, root, chosen, X_train, X_test, y_train, y_test, figure1, canvas1, figure2, canvas2):  
        if chosen == "Logistic Regression":
            best_model, y_pred = self.obj_ml.implement_LR(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)

            #Shows table of result
            df_lr = self.obj_data.read_dataset("results_LR.csv")
            self.shows_table(root, df_lr, 450, 750, "Y_test and Y_pred of Logistic Regression")

        if chosen == "Random Forest":
            best_model, y_pred = self.obj_ml.implement_RF(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)
            
            #Shows table of result
            df_rf = self.obj_data.read_dataset("results_RF.csv")
            self.shows_table(root, df_rf, 450, 750, "Y_test and Y_pred of Random Forest")   

        if chosen == "K-Nearest Neighbors":
            best_model, y_pred = self.obj_ml.implement_KNN(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)
            
            #Shows table of result
            df_knn = self.obj_data.read_dataset("results_KNN.csv")
            self.shows_table(root, df_knn, 450, 750, "Y_test and Y_pred of KNN")              

        if chosen == "Decision Trees":
            best_model, y_pred = self.obj_ml.implement_DT(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)
            
            #Shows table of result
            df_dt = self.obj_data.read_dataset("results_DT.csv")
            self.shows_table(root, df_dt, 450, 750, "Y_test and Y_pred of Decision Trees")  

        if chosen == "Gradient Boosting":
            best_model, y_pred = self.obj_ml.implement_GB(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)
            
            #Shows table of result
            df_gb = self.obj_data.read_dataset("results_GB.csv")
            self.shows_table(root, df_gb, 450, 750, "Y_test and Y_pred of Gradient Boosting") 

        if chosen == "Extreme Gradient Boosting":
            best_model, y_pred = self.obj_ml.implement_XGB(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)
            
            #Shows table of result
            df_xgb = self.obj_data.read_dataset("results_XGB.csv")
            self.shows_table(root, df_xgb, 450, 750, "Y_test and Y_pred of Extreme Gradient Boosting") 

        if chosen == "Multi-Layer Perceptron":
            best_model, y_pred = self.obj_ml.implement_MLP(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)
            
            #Shows table of result
            df_mlp = self.obj_data.read_dataset("results_MLP.csv")
            self.shows_table(root, df_mlp, 450, 750, "Y_test and Y_pred of Multi-Layer Perceptron") 

        if chosen == "Support Vector Classifier":
            best_model, y_pred = self.obj_ml.implement_SVC(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)
            
            #Shows table of result
            df_svc = self.obj_data.read_dataset("results_SVC.csv")
            self.shows_table(root, df_svc, 450, 750, "Y_test and Y_pred of Support Vector Classifier")

        if chosen == "AdaBoost":
            best_model, y_pred = self.obj_ml.implement_ADA(chosen, X_train, X_test, y_train, y_test)

            #Plots confusion matrix and ROC
            self.plot_cm_roc(best_model, X_test, y_test, y_pred, chosen, figure1, canvas1)

            #Plots true values versus predicted values diagram and learning curve
            self.plot_real_pred_val_learning_curve(best_model, X_train, y_train, 
                X_test, y_test, y_pred, chosen, figure2, canvas2)
            
            #Shows table of result
            df_ada = self.obj_data.read_dataset("results_ADA.csv")
            self.shows_table(root, df_ada, 450, 750, "Y_test and Y_pred of AdaBoost Classifier")


#machine_learning.py
import numpy as np 
from imblearn.over_sampling import SMOTE
from sklearn.model_selection import train_test_split, RandomizedSearchCV, GridSearchCV, StratifiedKFold
from sklearn.preprocessing import StandardScaler
import joblib
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix, accuracy_score, recall_score, precision_score
from sklearn.metrics import classification_report, f1_score, plot_confusion_matrix
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import AdaBoostClassifier, GradientBoostingClassifier
from xgboost import XGBClassifier
from sklearn.neural_network import MLPClassifier
from sklearn.svm import SVC
import os
import joblib
import pandas as pd 
from process_data import Process_Data

class Machine_Learning:
    def __init__(self):
        self.obj_data = Process_Data()

    def oversampling_splitting(self, df):
        #Sets target column
        y = df["Daily_Returns"]

        #Ensures y is of integer type
        y = np.array([1 if i>0 else 0 for i in y]).astype(int)

        #Drops irrelevant column
        X = df.drop(["Daily_Returns"], axis =1)

        #Checks null values because of technical indicators
        print(X.isnull().sum().to_string())
        print('Total number of null values: ', X.isnull().sum().sum())

        #Fills each null value in every column with mean value
        cols = list(X.columns)
        for n in cols:
            X[n].fillna(X[n].mean(),inplace = True)

        #Checks again null values
        print(X.isnull().sum().to_string())
        print('Total number of null values: ', X.isnull().sum().sum())

        # Check and convert data types
        X = X.astype(float)
        y = y.astype(int)

        sm = SMOTE(random_state=42)
        X,y = sm.fit_resample(X, y)

        #Splits the data into training and testing
        X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 2021, stratify=y)   

        #Use Standard Scaler
        scaler = StandardScaler()
        X_train_stand = scaler.fit_transform(X_train)
        X_test_stand = scaler.transform(X_test)    
    
        #Saves into pkl files
        joblib.dump(X_train_stand, 'X_train.pkl')
        joblib.dump(X_test_stand, 'X_test.pkl')
        joblib.dump(y_train, 'y_train.pkl')
        joblib.dump(y_test, 'y_test.pkl')  

    def load_files(self):
        X_train = joblib.load('X_train.pkl')
        X_test = joblib.load('X_test.pkl')
        y_train = joblib.load('y_train.pkl')
        y_test = joblib.load('y_test.pkl')
    
        return X_train, X_test, y_train, y_test

    def train_model(self, model, X, y):
        model.fit(X, y)
        return model

    def predict_model(self, model, X, proba=False):
        if ~proba:
            y_pred = model.predict(X)
        else:
            y_pred_proba = model.predict_proba(X)
            y_pred = np.argmax(y_pred_proba, axis=1)

        return y_pred

    def run_model(self, name, model, X_train, X_test, y_train, y_test, proba=False):   
        y_pred = self.predict_model(model, X_test, proba)
    
        accuracy = accuracy_score(y_test, y_pred)
        recall = recall_score(y_test, y_pred, average='weighted')
        precision = precision_score(y_test, y_pred, average='weighted')
        f1 = f1_score(y_test, y_pred, average='weighted')
    
        print(name)
        print('accuracy: ', accuracy)
        print('recall: ', recall)
        print('precision: ', precision)
        print('f1: ', f1)
        print(classification_report(y_test, y_pred)) 

        return y_pred

    def logistic_regression(self, name, X_train, X_test, y_train, y_test):
        #Logistic Regression Classifier
        # Define the parameter grid for the grid search
        param_grid = {
            'C': [0.01, 0.1, 1, 10],
            'penalty': ['none', 'l2'],
            'solver': ['newton-cg', 'lbfgs', 'liblinear', 'saga'],
        }

        # Initialize the Logistic Regression model
        logreg = LogisticRegression(max_iter=5000, random_state=2021)
    
        # Create GridSearchCV with the Logistic Regression model and the parameter grid
        grid_search = GridSearchCV(logreg, param_grid, cv=3, scoring='accuracy', n_jobs=-1)
    
        # Train and perform grid search
        grid_search.fit(X_train, y_train)
    
        # Get the best Logistic Regression model from the grid search
        best_model = grid_search.best_estimator_

        #Saves model
        joblib.dump(best_model, 'LR_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for LR:")
        print(grid_search.best_params_)        

        return best_model

    def implement_LR(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/LR_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('LR_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.logistic_regression(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_LR.csv")

        print("Training Logistic Regression done...")
        return model, y_pred

    def random_forest(self, name, X_train, X_test, y_train, y_test):
        #Random Forest Classifier    
        # Define the parameter grid for the grid search
        param_grid = {
            'n_estimators': [100, 200, 300],
            'max_depth': [10, 20, 30, 40, 50],
            'min_samples_split': [2, 5, 10],
            'min_samples_leaf': [1, 2, 4]
        }

        # Initialize the RandomForestClassifier model
        rf = RandomForestClassifier(random_state=2021)
    
        # Create GridSearchCV with the RandomForestClassifier model and the parameter grid
        grid_search = GridSearchCV(rf, param_grid, cv=3, scoring='accuracy', n_jobs=-1)
    
        # Train and perform grid search
        grid_search.fit(X_train, y_train)
    
        # Get the best RandomForestClassifier model from the grid search
        best_model = grid_search.best_estimator_
    
        #Saves model
        joblib.dump(best_model, 'RF_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for RF:")
        print(grid_search.best_params_)        

        return best_model

    def implement_RF(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/RF_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('RF_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.random_forest(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_RF.csv")

        print("Training Random Forest done...")
        return model, y_pred

    def knearest_neigbors(self, name, X_train, X_test, y_train, y_test):
        #KNN Classifier
        # Define the parameter grid for the grid search
        param_grid = {
            'n_neighbors': list(range(2, 10))
        }

        # Initialize the KNN Classifier
        knn = KNeighborsClassifier()
    
        # Create GridSearchCV with the KNN model and the parameter grid
        grid_search = GridSearchCV(knn, param_grid, cv=3, scoring='accuracy', n_jobs=-1)
    
        # Train and perform grid search
        grid_search.fit(X_train, y_train)
    
        # Get the best KNN model from the grid search
        best_model = grid_search.best_estimator_
    
        #Saves model
        joblib.dump(best_model, 'KNN_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for KNN:")
        print(grid_search.best_params_)        

        return best_model

    def implement_KNN(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/KNN_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('KNN_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.knearest_neigbors(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_KNN.csv")

        print("Training KNN done...")
        return model, y_pred

    def decision_trees(self, name, X_train, X_test, y_train, y_test):
        # Initialize the DecisionTreeClassifier model
        dt_clf = DecisionTreeClassifier(random_state=2021)
    
        # Define the parameter grid for the grid search
        param_grid = {
            'max_depth': np.arange(1, 51, 1),
            'criterion': ['gini', 'entropy'],
            'min_samples_split': [2, 5, 10],
            'min_samples_leaf': [1, 2, 4],
        }
    
        # Create GridSearchCV with the DecisionTreeClassifier model and the parameter grid
        grid_search = GridSearchCV(dt_clf, param_grid, cv=3, scoring='accuracy', n_jobs=-1)
    
        # Train and perform grid search
        grid_search.fit(X_train, y_train)
    
        # Get the best DecisionTreeClassifier model from the grid search
        best_model = grid_search.best_estimator_
    
        #Saves model
        joblib.dump(best_model, 'DT_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for DT:")
        print(grid_search.best_params_)        

        return best_model

    def implement_DT(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/DT_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('DT_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.decision_trees(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_DT.csv")

        print("Training Decision Trees done...")
        return model, y_pred

    def gradient_boosting(self, name, X_train, X_test, y_train, y_test):
        #Gradient Boosting Classifier      
        # Initialize the GradientBoostingClassifier model
        gbt = GradientBoostingClassifier(random_state=2021)

        # Define the parameter grid for the grid search
        param_grid = {
            'n_estimators': [100, 200, 300],
            'max_depth': [10, 20, 30],
            'subsample': [0.6, 0.8, 1.0],
            'max_features': [0.2, 0.4, 0.6, 0.8, 1.0],
        }
    
        # Create GridSearchCV with the GradientBoostingClassifier model and the parameter grid
        grid_search = GridSearchCV(gbt, param_grid, cv=3, scoring='accuracy', n_jobs=-1)

        # Train and perform grid search
        grid_search.fit(X_train, y_train)

        # Get the best GradientBoostingClassifier model from the grid search
        best_model = grid_search.best_estimator_
    
        #Saves model
        joblib.dump(best_model, 'GB_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for GB:")
        print(grid_search.best_params_)        

        return best_model

    def implement_GB(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/GB_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('GB_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.gradient_boosting(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_GB.csv")

        print("Training Gradient Boosting done...")
        return model, y_pred

    def extreme_gradient_boosting(self, name, X_train, X_test, y_train, y_test):
        # Define the parameter grid for the grid search
        param_grid = {
            'n_estimators': [100, 200, 300],
            'max_depth': [10, 20, 30],
            'learning_rate': [0.01, 0.1, 0.2],
            'subsample': [0.6, 0.8, 1.0],
            'colsample_bytree': [0.6, 0.8, 1.0],
        }

        # Initialize the XGBoost classifier
        xgb = XGBClassifier(random_state=2021, use_label_encoder=False, eval_metric='mlogloss')

        # Create GridSearchCV with the XGBoost classifier and the parameter grid
        grid_search = GridSearchCV(xgb, param_grid, cv=3, scoring='accuracy', n_jobs=-1)

        # Train and perform grid search
        grid_search.fit(X_train, y_train)

        # Get the best XGBoost classifier model from the grid search
        best_model = grid_search.best_estimator_
    
        #Saves model
        joblib.dump(best_model, 'XGB_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for XGB:")
        print(grid_search.best_params_)        

        return best_model

    def implement_XGB(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/XGB_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('XGB_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.extreme_gradient_boosting(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_XGB.csv")

        print("Training Extreme Gradient Boosting done...")
        return model, y_pred

    def multi_layer_perceptron(self, name, X_train, X_test, y_train, y_test):
        # Define the parameter grid for the grid search
        param_grid = {
            'hidden_layer_sizes': [(50,), (100,), (50, 50), (100, 50), (100, 100)],
            'activation': ['logistic', 'relu'],
            'solver': ['adam', 'sgd'],
            'alpha': [0.0001, 0.001, 0.01],
            'learning_rate': ['constant', 'invscaling', 'adaptive'],
        }

        # Initialize the MLP Classifier
        mlp = MLPClassifier(random_state=2021)

        # Create GridSearchCV with the MLP Classifier and the parameter grid
        grid_search = GridSearchCV(mlp, param_grid, cv=3, scoring='accuracy', n_jobs=-1)

        # Train and perform grid search
        grid_search.fit(X_train, y_train)

        # Get the best MLP Classifier model from the grid search
        best_model = grid_search.best_estimator_
    
        #Saves model
        joblib.dump(best_model, 'MLP_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for MLP:")
        print(grid_search.best_params_)        

        return best_model

    def implement_MLP(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/MLP_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('MLP_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.multi_layer_perceptron(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_MLP.csv")

        print("Training Multi-Layer Perceptron done...")
        return model, y_pred

    def support_vector(self, name, X_train, X_test, y_train, y_test):
        #Support Vector Classifier
        # Define the parameter grid for the grid search
        param_grid = {
            'C': [0.1, 1, 10],
            'kernel': ['linear', 'poly', 'rbf'],
            'gamma': ['scale', 'auto', 0.1, 1],
        }

        # Initialize the SVC model
        model_svc = SVC(random_state=2021, probability=True)

        # Create GridSearchCV with the SVC model and the parameter grid
        grid_search = GridSearchCV(model_svc, param_grid, cv=3, scoring='accuracy', n_jobs=-1, refit=True)
    
        # Train and perform grid search
        grid_search.fit(X_train, y_train)

        # Get the best MLP Classifier model from the grid search
        best_model = grid_search.best_estimator_
    
        #Saves model
        joblib.dump(best_model, 'SVC_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for SVC:")
        print(grid_search.best_params_)        

        return best_model

    def implement_SVC(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/SVC_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('SVC_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.support_vector(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_SVC.csv")

        print("Training Support Vector Classifier done...")
        return model, y_pred

    def adaboost_classifier(self, name, X_train, X_test, y_train, y_test):
        # Define the parameter grid for the grid search
        param_grid = {
            'n_estimators': [50, 100, 150],
            'learning_rate': [0.01, 0.1, 0.2],
        }

        # Initialize the AdaBoost classifier
        adaboost = AdaBoostClassifier(random_state=2021)

        # Create GridSearchCV with the AdaBoost classifier and the parameter grid
        grid_search = GridSearchCV(adaboost, param_grid, cv=3, scoring='accuracy', n_jobs=-1)

    
        # Train and perform grid search
        grid_search.fit(X_train, y_train)

        # Get the best AdaBoost Classifier model from the grid search
        best_model = grid_search.best_estimator_
    
        #Saves model
        joblib.dump(best_model, 'ADA_Model.pkl')    
    
        # Print the best hyperparameters found
        print(f"Best Hyperparameters for AdaBoost:")
        print(grid_search.best_params_)        

        return best_model

    def implement_ADA(self, chosen, X_train, X_test, y_train, y_test):
        file_path = os.getcwd()+"/ADA_Model.pkl"
        if os.path.exists(file_path):
            model = joblib.load('ADA_Model.pkl')
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True) 
        else:
            model = self.adaboost_classifier(chosen, X_train, X_test, y_train, y_test)
            y_pred = self.run_model(chosen, model, X_train, X_test, y_train, y_test, proba=True)

        #Saves result into excel file
        self.obj_data.save_result(y_test, y_pred, "results_ADA.csv")

        print("Training AdaBoost done...")
        return model, y_pred


#regression.py
import pandas as pd
import numpy as np 
from sklearn.preprocessing import MinMaxScaler
import joblib
from sklearn.metrics import mean_squared_error, mean_absolute_error
from sklearn.metrics import roc_auc_score,roc_curve, r2_score, explained_variance_score
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor, AdaBoostRegressor
from sklearn.model_selection import GridSearchCV
from sklearn.tree import DecisionTreeRegressor
from xgboost import XGBRegressor
from sklearn.neural_network import MLPRegressor
from sklearn.linear_model import LassoCV
from sklearn.linear_model import RidgeCV
from sklearn.neighbors import KNeighborsRegressor

class Regression:
    def splitting_data_regression(self, df): 
        #Sets target column
        y_final = pd.DataFrame(df["Adj Close"])
        X = df.drop(["Adj Close"], axis =1)

        #Checks null values because of technical indicators
        print(X.isnull().sum().to_string())
        print('Total number of null values: ', X.isnull().sum().sum())

        #Fills each null value in every column with mean value
        cols = list(X.columns)
        for n in cols:
            X[n].fillna(X[n].mean(),inplace = True)

        #Checks again null values
        print(X.isnull().sum().to_string())
        print('Total number of null values: ', X.isnull().sum().sum())

        #Normalizes data
        scaler = MinMaxScaler()
        X_minmax_data = scaler.fit_transform(X)
        X_final = pd.DataFrame(columns=X.columns, data=X_minmax_data, index=X.index)
        print('Shape of features : ', X_final.shape)
        print('Shape of target : ', y_final.shape)

        #Shifts target array to predict the n + 1 samples
        n=90
        y_final = y_final.shift(-1)
        y_val = y_final[-n:-1]
        y_final = y_final[:-n]

        #Takes last n rows of data to be validation set
        X_val = X_final[-n:-1]
        X_final = X_final[:-n]

        print("\n -----After process------ \n")
        print('Shape of features : ', X_final.shape)
        print('Shape of target : ', y_final.shape)
        print(y_final.tail().to_string())

        y_final=y_final.astype('float64')

        #Splits data into training and test data at 80% and 20% respectively
        split_idx=round(0.8*len(X))
        print("split_idx=",split_idx)
        X_train_reg = X_final[:split_idx]
        y_train_reg = y_final[:split_idx]
        X_test_reg = X_final[split_idx:]
        y_test_reg = y_final[split_idx:]   

        #Saves into pkl files
        joblib.dump(X, 'X_Ori.pkl')
        joblib.dump(X_final, 'X_final_reg.pkl')
        joblib.dump(X_train_reg, 'X_train_reg.pkl')
        joblib.dump(X_test_reg, 'X_test_reg.pkl')
        joblib.dump(X_val, 'X_val_reg.pkl')
        joblib.dump(y_final, 'y_final_reg.pkl')
        joblib.dump(y_train_reg, 'y_train_reg.pkl')
        joblib.dump(y_test_reg, 'y_test_reg.pkl') 
        joblib.dump(y_val, 'y_val_reg.pkl')

    def load_regression_files(self):
        X_Ori = joblib.load('X_Ori.pkl')
        X_final_reg = joblib.load('X_final_reg.pkl')
        X_train_reg = joblib.load('X_train_reg.pkl')
        X_test_reg = joblib.load('X_test_reg.pkl')
        X_val_reg = joblib.load('X_val_reg.pkl')
        y_final_reg = joblib.load('y_final_reg.pkl')
        y_train_reg = joblib.load('y_train_reg.pkl')
        y_test_reg = joblib.load('y_test_reg.pkl')  
        y_val_reg = joblib.load('y_val_reg.pkl') 

        return X_Ori, X_final_reg, X_train_reg, X_test_reg, X_val_reg, y_final_reg, y_train_reg, y_test_reg, y_val_reg 

    def perform_regression(self, model, X, y, xtrain, ytrain, xtest, ytest, xval, yval, label):
        model.fit(xtrain, ytrain)
        predictions_test = model.predict(xtest)
        predictions_train = model.predict(xtrain)
        predictions_val = model.predict(xval)

        # Convert ytest and predictions_test to NumPy arrays
        ytest_np = ytest.to_numpy().flatten()
        predictions_test_np = predictions_test.flatten()

        str_label = 'RMSE using ' + label
        print(str_label + f': {np.sqrt(mean_squared_error(ytest_np, predictions_test_np))}')    
        print("mean square error: ", mean_squared_error(ytest_np, predictions_test_np))
        print("variance or r-squared: ", explained_variance_score(ytest_np, predictions_test_np))
        print("mean absolute error (MAE): ", mean_absolute_error(ytest_np, predictions_test_np))
        print("R2 (R-squared): ", r2_score(ytest_np, predictions_test_np))
        print("Adjusted R2: ", 1 - (1-r2_score(ytest_np, predictions_test_np))*(len(ytest_np)-1)/(len(ytest_np)-xtest.shape[1]-1))
    
        mean_percentage_error = np.mean((ytest_np - predictions_test_np) / ytest_np) * 100
        print("Mean Percentage Error (MPE): ", mean_percentage_error)
    
        mean_absolute_percentage_error = np.mean(np.abs((ytest_np - predictions_test_np) / ytest_np)) * 100
        print("Mean Absolute Percentage Error (MAPE): ", mean_absolute_percentage_error)
    
        print('ACTUAL: Avg. ' + f': {ytest_np.mean()}')
        print('ACTUAL: Median ' + f': {np.median(ytest_np)}')    
        print('PREDICTED: Avg. ' + f': {predictions_test_np.mean()}')
        print('PREDICTED: Median ' + f': {np.median(predictions_test_np)}')  

        # Evaluation of regression on all dataset
        all_pred = model.predict(X)
        print("mean square error (whole dataset): ", mean_squared_error(y, all_pred))
        print("variance or r-squared (whole dataset): ", explained_variance_score(y, all_pred))

        return predictions_test, predictions_train, predictions_val, all_pred

    def linear_regression(self, X_train, y_train):
        #Linear Regression    
        #Creates a Linear Regression model
        lin_reg = LinearRegression()

        #Defines the hyperparameter grid to search
        param_grid = {
            'fit_intercept': [True, False],   # Try both True and False for fit_intercept
            'normalize': [True, False]        # Try both True and False for normalize
        }

        #Creates GridSearchCV with the Linear Regression model and the hyperparameter grid
        grid_search = GridSearchCV(lin_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        #Fits the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        #Gets the best Linear Regression model from the grid search
        best_lin_reg = grid_search.best_estimator_

        #Prints the best hyperparameters found
        print("Best Hyperparameters for Linear Regression:")
        print(grid_search.best_params_)

        return best_lin_reg

    def rf_regression(self, X_train, y_train):
        #Random Forest Regression    
        # Create a RandomForestRegressor model
        rf_reg = RandomForestRegressor()

        # Define the hyperparameter grid to search
        param_grid = {
            'n_estimators': [50, 100, 150],           # Number of trees in the forest
            'max_depth': [None, 5, 10],                # Maximum depth of the tree
            'min_samples_split': [2, 5, 10],           # Minimum number of samples required to split an internal node
            'min_samples_leaf': [1, 2, 4],             # Minimum number of samples required to be at a leaf node
            'bootstrap': [True, False]                 # Whether bootstrap samples are used when building trees
        }

        # Create GridSearchCV with the RandomForestRegressor model and the hyperparameter grid
        grid_search = GridSearchCV(rf_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best RandomForestRegressor model from the grid search
        best_rf_reg = grid_search.best_estimator_

        # Print the best hyperparameters found
        print("Best Hyperparameters for RandomForestRegressor:")
        print(grid_search.best_params_)

        return best_rf_reg
    
    def dt_regression(self, X_train, y_train):
        #Decision Tree (DT) regression
        # Create a DecisionTreeRegressor model
        dt_reg = DecisionTreeRegressor(random_state=100)

        # Define the hyperparameter grid to search
        param_grid = {
            'max_depth': [None, 5, 10, 15],          # Maximum depth of the tree
            'min_samples_split': [2, 5, 10],         # Minimum number of samples required to split an internal node
            'min_samples_leaf': [1, 2, 4, 6],        # Minimum number of samples required to be at a leaf node
        }

        # Create GridSearchCV with the DecisionTreeRegressor model and the hyperparameter grid
        grid_search = GridSearchCV(dt_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best DecisionTreeRegressor model from the grid search
        best_dt_reg = grid_search.best_estimator_

        # Print the best hyperparameters found
        print("Best Hyperparameters for DecisionTreeRegressor:")
        print(grid_search.best_params_)

        return best_dt_reg

    def gb_regression(self, X_train, y_train):
        #Gradient Boosting regression
        # Create the GradientBoostingRegressor model
        gb_reg = GradientBoostingRegressor()

        # Define the hyperparameter grid to search
        param_grid = {
            'n_estimators': [50, 100, 150],                # Number of boosting stages (trees) to build
            'learning_rate': [0.01, 0.1, 0.5],             # Step size at each boosting iteration
            'max_depth': [3, 5, 7],                        # Maximum depth of the individual trees
            'min_samples_split': [2, 5, 10],               # Minimum number of samples required to split an internal node
            'min_samples_leaf': [1, 2, 4],                 # Minimum number of samples required to be at a leaf node
        }

        # Create GridSearchCV with the GradientBoostingRegressor model and the hyperparameter grid
        grid_search = GridSearchCV(gb_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best GradientBoostingRegressor model from the grid search
        best_gb_reg = grid_search.best_estimator_

        # Print the best hyperparameters found
        print("Best Hyperparameters for GradientBoostingRegressor:")
        print(grid_search.best_params_)

        return best_gb_reg
    
    def xgb_regression(self, X_train, y_train):
        #Extreme Gradient Boosting (XGB) 
        # Create the XGBRegressor model
        xgb_reg = XGBRegressor()

        # Define the hyperparameter grid to search
        param_grid = {
            'n_estimators': [50, 100, 150],             # Number of boosting stages (trees) to build
            'learning_rate': [0.01, 0.1, 0.5],          # Step size at each boosting iteration
            'max_depth': [3, 5, 7],                     # Maximum depth of the individual trees
            'min_child_weight': [1, 2, 4],              # Minimum sum of instance weight (hessian) needed in a child
            'gamma': [0, 0.1, 0.2],                     # Minimum loss reduction required to make a further partition on a leaf node
            'subsample': [0.8, 1.0],                    # Subsample ratio of the training instances
            'colsample_bytree': [0.8, 1.0]              # Subsample ratio of columns when constructing each tree
        }

        # Create GridSearchCV with the XGBRegressor model and the hyperparameter grid
        grid_search = GridSearchCV(xgb_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best XGBRegressor model from the grid search
        best_xgb_reg = grid_search.best_estimator_

        # Print the best hyperparameters found
        print("Best Hyperparameters for XGBRegressor:")
        print(grid_search.best_params_)

        return best_xgb_reg
    
    def mlp_regression(self, X_train, y_train):
        #MLP regression
        # Create the MLPRegressor model
        mlp_reg = MLPRegressor()

        # Define the hyperparameter grid to search
        param_grid = {
            'hidden_layer_sizes': [(50,), (100,), (50, 50), (100, 50)],   # Number of neurons in each hidden layer
            'activation': ['relu', 'tanh'],                                # Activation function for the hidden layers
            'solver': ['adam', 'sgd'],                                     # Solver for weight optimization
            'learning_rate': ['constant', 'invscaling', 'adaptive'],      # Learning rate schedule
            'learning_rate_init': [0.01, 0.001],                           # Initial learning rate
            'max_iter': [100, 200, 300],                                   # Maximum number of iterations
        }

        # Create GridSearchCV with the MLPRegressor model and the hyperparameter grid
        grid_search = GridSearchCV(mlp_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best MLPRegressor model from the grid search
        best_mlp_reg = grid_search.best_estimator_

        # Print the best hyperparameters found
        print("Best Hyperparameters for MLPRegressor:")
        print(grid_search.best_params_)

        return best_mlp_reg

    def lasso_regression(self, X_train, y_train):
        # Create the LassoCV model
        lasso_reg = LassoCV(n_alphas=1000, max_iter=3000, random_state=0)

        # Define the hyperparameter grid to search
        param_grid = {
            'normalize': [True, False],        # Whether to normalize the features before fitting the model
            'fit_intercept': [True, False]     # Whether to calculate the intercept for this model
        }

        # Create GridSearchCV with the LassoCV model and the hyperparameter grid
        grid_search = GridSearchCV(lasso_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best LassoCV model from the grid search
        best_lasso_reg = grid_search.best_estimator_

        # Print the best hyperparameters found
        print("Best Hyperparameters for Lasso Regression:")
        print(grid_search.best_params_)

        return best_lasso_reg
    
    def ridge_regression(self, X_train, y_train):
        #Ridge regression
        ridge_reg = RidgeCV(gcv_mode='auto')

        # Define the hyperparameter grid to search (optional if you want to include other hyperparameters)
        param_grid = {
            'normalize': [True, False],        # Whether to normalize the features before fitting the model
            'fit_intercept': [True, False]     # Whether to calculate the intercept for this model
        }

        # Create GridSearchCV with the RidgeCV model and the hyperparameter grid (optional if you include the param_grid)
        grid_search = GridSearchCV(ridge_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best RidgeCV model from the grid search
        best_ridge_reg = grid_search.best_estimator_

        # Print the best hyperparameters found (optional if you included the param_grid)
        print("Best Hyperparameters for Ridge Regression:")
        print(grid_search.best_params_)

        return best_ridge_reg

    def adaboost_regression(self, X_train, y_train):
        #Adaboost regression
        # Create the AdaBoostRegressor model
        ada_reg = AdaBoostRegressor()

        # Define the hyperparameter grid to search
        param_grid = {
            'n_estimators': [50, 100, 150],          # Number of boosting stages (trees) to build
            'learning_rate': [0.01, 0.1, 0.5],       # Step size at each boosting iteration
            'loss': ['linear', 'square', 'exponential']  # Loss function to use when updating weights
        }

        # Create GridSearchCV with the AdaBoostRegressor model and the hyperparameter grid
        grid_search = GridSearchCV(ada_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best AdaBoostRegressor model from the grid search
        best_ada_reg = grid_search.best_estimator_

        # Print the best hyperparameters found
        print("Best Hyperparameters for AdaBoostRegressor:")
        print(grid_search.best_params_)

        return best_ada_reg

    def knn_regression(self, X_train, y_train):
        #KNN regression
        # Create a KNeighborsRegressor model
        knn_reg = KNeighborsRegressor()

        # Define the hyperparameter grid to search
        param_grid = {
            'n_neighbors': [3, 5, 7, 9],             # Number of neighbors to use for regression
            'weights': ['uniform', 'distance'],       # Weight function used in prediction
            'algorithm': ['auto', 'ball_tree', 'kd_tree', 'brute']  # Algorithm used to compute the nearest neighbors
        }

        # Create GridSearchCV with the KNeighborsRegressor model and the hyperparameter grid
        grid_search = GridSearchCV(knn_reg, param_grid, cv=5, scoring='neg_mean_squared_error')

        # Fit the GridSearchCV to the training data
        grid_search.fit(X_train, y_train)

        # Get the best KNeighborsRegressor model from the grid search
        best_knn_reg = grid_search.best_estimator_

        # Print the best hyperparameters found
        print("Best Hyperparameters for KNeighborsRegressor:")
        print(grid_search.best_params_)

        return best_knn_reg






No comments:

Post a Comment