Credit_Card_Fraud_Detection.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/aakanshadalmia/Credit-Card-Fraud-Detection/blob/main/Credit_Card_Fraud_Detection.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fb8ec695"
      },
      "source": [
        "# Credit Card Fraud Detection\n",
        "-------------------------\n",
        "\n",
        "**Problem Statement:** The challenge is for credit card companies to recognize fraudulent credit card transactions so that customers are not charged for items that they did not purchase.\n",
        " \n",
        "**Goal:** This project is aimed at finding a classification model with the best performance for an imbalanced dataset.\n",
        "\n",
        "**Evaluation Metrics Used:** F1 score"
      ],
      "id": "fb8ec695"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cf57e3ae"
      },
      "source": [
        "## About the Dataset\n",
        "---------\n",
        "\n",
        "The dataset contains transactions made by credit cards in September 2013 by European cardholders.\n",
        "\n",
        "This dataset presents transactions that occurred in two days, where we have 492 frauds out of 284,807 transactions. The dataset is **highly unbalanced, the positive class (frauds) account for 0.172% of all transactions.**\n",
        "\n",
        "It contains only numerical input variables which are the result of a PCA transformation. Due to confidentiality issues, the original features and more background information about the data has been omitted.\n",
        "\n",
        "Features V1, V2, … V28 are the principal components obtained with PCA, the only features which have not been transformed with PCA are 'Time' and 'Amount'. Feature 'Time' contains the seconds elapsed between each transaction and the first transaction in the dataset. The feature 'Amount' is the transaction Amount, this feature can be used for example-dependant cost-sensitive learning. \n",
        "\n",
        "**Feature 'Class' is the response variable and it takes value 1 in case of fraud and 0 otherwise.**\n",
        "\n",
        "_Given the class imbalance ratio, Area Under the Precision-Recall Curve (AUPRC) has been used for evaluation instead of using a Confusion matrix to determine accuracy for the unbalanced classification._"
      ],
      "id": "cf57e3ae"
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "f08f002c"
      },
      "outputs": [],
      "source": [
        "#Import all required libraries\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn import metrics\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.tree import DecisionTreeClassifier, plot_tree\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from xgboost import XGBClassifier\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.decomposition import PCA"
      ],
      "id": "f08f002c"
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m5WH6CVuQxAs",
        "outputId": "15b3bad4-31e0-4a43-95bd-29df2f3aa2e5"
      },
      "id": "m5WH6CVuQxAs",
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "80e7cabf"
      },
      "outputs": [],
      "source": [
        "#Reading data from csv file into a dataframe\n",
        "\n",
        "data = pd.read_csv(\"/content/drive/MyDrive/creditcard.csv\")"
      ],
      "id": "80e7cabf"
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 299
        },
        "id": "0a775c90",
        "outputId": "01c2470e-e63b-4233-d7e6-87d90adda70b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Time        V1        V2        V3        V4        V5        V6        V7  \\\n",
              "0   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
              "1   0.0  1.191857  0.266151  0.166480  0.448154  0.060018 -0.082361 -0.078803   \n",
              "2   1.0 -1.358354 -1.340163  1.773209  0.379780 -0.503198  1.800499  0.791461   \n",
              "3   1.0 -0.966272 -0.185226  1.792993 -0.863291 -0.010309  1.247203  0.237609   \n",
              "4   2.0 -1.158233  0.877737  1.548718  0.403034 -0.407193  0.095921  0.592941   \n",
              "\n",
              "         V8        V9  ...       V21       V22       V23       V24       V25  \\\n",
              "0  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n",
              "1  0.085102 -0.255425  ... -0.225775 -0.638672  0.101288 -0.339846  0.167170   \n",
              "2  0.247676 -1.514654  ...  0.247998  0.771679  0.909412 -0.689281 -0.327642   \n",
              "3  0.377436 -1.387024  ... -0.108300  0.005274 -0.190321 -1.175575  0.647376   \n",
              "4 -0.270533  0.817739  ... -0.009431  0.798278 -0.137458  0.141267 -0.206010   \n",
              "\n",
              "        V26       V27       V28  Amount  Class  \n",
              "0 -0.189115  0.133558 -0.021053  149.62      0  \n",
              "1  0.125895 -0.008983  0.014724    2.69      0  \n",
              "2 -0.139097 -0.055353 -0.059752  378.66      0  \n",
              "3 -0.221929  0.062723  0.061458  123.50      0  \n",
              "4  0.502292  0.219422  0.215153   69.99      0  \n",
              "\n",
              "[5 rows x 31 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d408629c-4363-4a6b-b1e7-4e52d6fea763\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Time</th>\n",
              "      <th>V1</th>\n",
              "      <th>V2</th>\n",
              "      <th>V3</th>\n",
              "      <th>V4</th>\n",
              "      <th>V5</th>\n",
              "      <th>V6</th>\n",
              "      <th>V7</th>\n",
              "      <th>V8</th>\n",
              "      <th>V9</th>\n",
              "      <th>...</th>\n",
              "      <th>V21</th>\n",
              "      <th>V22</th>\n",
              "      <th>V23</th>\n",
              "      <th>V24</th>\n",
              "      <th>V25</th>\n",
              "      <th>V26</th>\n",
              "      <th>V27</th>\n",
              "      <th>V28</th>\n",
              "      <th>Amount</th>\n",
              "      <th>Class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.0</td>\n",
              "      <td>-1.359807</td>\n",
              "      <td>-0.072781</td>\n",
              "      <td>2.536347</td>\n",
              "      <td>1.378155</td>\n",
              "      <td>-0.338321</td>\n",
              "      <td>0.462388</td>\n",
              "      <td>0.239599</td>\n",
              "      <td>0.098698</td>\n",
              "      <td>0.363787</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.018307</td>\n",
              "      <td>0.277838</td>\n",
              "      <td>-0.110474</td>\n",
              "      <td>0.066928</td>\n",
              "      <td>0.128539</td>\n",
              "      <td>-0.189115</td>\n",
              "      <td>0.133558</td>\n",
              "      <td>-0.021053</td>\n",
              "      <td>149.62</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.0</td>\n",
              "      <td>1.191857</td>\n",
              "      <td>0.266151</td>\n",
              "      <td>0.166480</td>\n",
              "      <td>0.448154</td>\n",
              "      <td>0.060018</td>\n",
              "      <td>-0.082361</td>\n",
              "      <td>-0.078803</td>\n",
              "      <td>0.085102</td>\n",
              "      <td>-0.255425</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.225775</td>\n",
              "      <td>-0.638672</td>\n",
              "      <td>0.101288</td>\n",
              "      <td>-0.339846</td>\n",
              "      <td>0.167170</td>\n",
              "      <td>0.125895</td>\n",
              "      <td>-0.008983</td>\n",
              "      <td>0.014724</td>\n",
              "      <td>2.69</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1.0</td>\n",
              "      <td>-1.358354</td>\n",
              "      <td>-1.340163</td>\n",
              "      <td>1.773209</td>\n",
              "      <td>0.379780</td>\n",
              "      <td>-0.503198</td>\n",
              "      <td>1.800499</td>\n",
              "      <td>0.791461</td>\n",
              "      <td>0.247676</td>\n",
              "      <td>-1.514654</td>\n",
              "      <td>...</td>\n",
              "      <td>0.247998</td>\n",
              "      <td>0.771679</td>\n",
              "      <td>0.909412</td>\n",
              "      <td>-0.689281</td>\n",
              "      <td>-0.327642</td>\n",
              "      <td>-0.139097</td>\n",
              "      <td>-0.055353</td>\n",
              "      <td>-0.059752</td>\n",
              "      <td>378.66</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1.0</td>\n",
              "      <td>-0.966272</td>\n",
              "      <td>-0.185226</td>\n",
              "      <td>1.792993</td>\n",
              "      <td>-0.863291</td>\n",
              "      <td>-0.010309</td>\n",
              "      <td>1.247203</td>\n",
              "      <td>0.237609</td>\n",
              "      <td>0.377436</td>\n",
              "      <td>-1.387024</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.108300</td>\n",
              "      <td>0.005274</td>\n",
              "      <td>-0.190321</td>\n",
              "      <td>-1.175575</td>\n",
              "      <td>0.647376</td>\n",
              "      <td>-0.221929</td>\n",
              "      <td>0.062723</td>\n",
              "      <td>0.061458</td>\n",
              "      <td>123.50</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2.0</td>\n",
              "      <td>-1.158233</td>\n",
              "      <td>0.877737</td>\n",
              "      <td>1.548718</td>\n",
              "      <td>0.403034</td>\n",
              "      <td>-0.407193</td>\n",
              "      <td>0.095921</td>\n",
              "      <td>0.592941</td>\n",
              "      <td>-0.270533</td>\n",
              "      <td>0.817739</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.009431</td>\n",
              "      <td>0.798278</td>\n",
              "      <td>-0.137458</td>\n",
              "      <td>0.141267</td>\n",
              "      <td>-0.206010</td>\n",
              "      <td>0.502292</td>\n",
              "      <td>0.219422</td>\n",
              "      <td>0.215153</td>\n",
              "      <td>69.99</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 31 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d408629c-4363-4a6b-b1e7-4e52d6fea763')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-d408629c-4363-4a6b-b1e7-4e52d6fea763 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-d408629c-4363-4a6b-b1e7-4e52d6fea763');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "data.head()"
      ],
      "id": "0a775c90"
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "i4rPEJzZQcTP"
      },
      "outputs": [],
      "source": [
        "data.isna().sum()\n",
        "data = data.fillna(0)"
      ],
      "id": "i4rPEJzZQcTP"
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "522b3c2b",
        "outputId": "362da698-61c6-498d-e155-9a08be527192"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Total number of transactions: 284807\n",
            "Legitimate Transactions: 284315\n",
            "Fraud Transactions: 492\n",
            "% of Fraud Transactions: 0.1727\n"
          ]
        }
      ],
      "source": [
        "#Estimating the imbalance in the dataset\n",
        "\n",
        "print(\"Total number of transactions:\", len(data))\n",
        "normal = len(data[data.Class == 0])\n",
        "fraud = len(data[data.Class == 1])\n",
        "print(\"Legitimate Transactions:\", normal)\n",
        "print(\"Fraud Transactions:\", fraud)\n",
        "print(\"% of Fraud Transactions:\", round(fraud / len(data) * 100,4))"
      ],
      "id": "522b3c2b"
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "74d39776"
      },
      "outputs": [],
      "source": [
        "#Defining the training and testing datasets\n",
        "\n",
        "X = data.drop('Class',axis = 1)\n",
        "Y = data.Class.values\n",
        "\n",
        "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.3, random_state = 1)"
      ],
      "id": "74d39776"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vPvZogk3Anqu"
      },
      "source": [
        "# Principal Component Analysis"
      ],
      "id": "vPvZogk3Anqu"
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.preprocessing import StandardScaler, MinMaxScaler\n",
        "\n",
        "scaler = MinMaxScaler()\n",
        "scaler.fit(X_train)\n",
        "X_train_scaled = scaler.transform(X_train)\n",
        "X_test_scaled = scaler.transform(X_test)"
      ],
      "metadata": {
        "id": "iFI6TxnbhKPf"
      },
      "id": "iFI6TxnbhKPf",
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "E4JYhserABTp"
      },
      "outputs": [],
      "source": [
        "pca = PCA(n_components = 10).fit(X_train_scaled)"
      ],
      "id": "E4JYhserABTp"
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kl99I240AA4p",
        "outputId": "7cb36548-3400-4255-b2bd-11b3614c3868"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(199364, 10)"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "X_train_pca = pca.transform(X_train_scaled)\n",
        "X_train_pca.shape"
      ],
      "id": "kl99I240AA4p"
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "41Soq3IcAAr7"
      },
      "outputs": [],
      "source": [
        "X_test_pca = pca.transform(X_test_scaled)"
      ],
      "id": "41Soq3IcAAr7"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4a1ab62e"
      },
      "source": [
        "## K-Nearest Neighbours"
      ],
      "id": "4a1ab62e"
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "2e8f39d7"
      },
      "outputs": [],
      "source": [
        "#Model Building\n",
        "\n",
        "kNN = KNeighborsClassifier(n_neighbors = 5, n_jobs = -1, leaf_size = 600)\n",
        "kNN.fit(X_train_pca, Y_train)\n",
        "\n",
        "#Applying model to test dataset\n",
        "kNN_Y_pred = kNN.predict(X_test_pca)"
      ],
      "id": "2e8f39d7"
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "8ab71d06",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3c9fa038-012c-4000-86cd-0312000b6b8f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for kNN model is: 0.7239819004524887\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "\n",
        "kNN_f1 = metrics.f1_score(Y_test,kNN_Y_pred)\n",
        "print(\"F1 score for kNN model is:\",kNN_f1)"
      ],
      "id": "8ab71d06"
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "813c5ea8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "db93ee2b-34cf-43c7-eb22-4aa2e1dda5b8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fitting 3 folds for each of 8 candidates, totalling 24 fits\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=3, estimator=KNeighborsClassifier(), n_jobs=-1,\n",
              "             param_grid={'metric': ['euclidean', 'manhattan'],\n",
              "                         'n_neighbors': [3, 6, 9, 12]},\n",
              "             scoring='f1', verbose=10)"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "#Hyperparameter Tuning\n",
        "grid_params = {'n_neighbors' : list(range(3,15,3)),\n",
        "               'metric' : ['euclidean','manhattan']}\n",
        "\n",
        "gridsearch_kNN = GridSearchCV(KNeighborsClassifier(), grid_params, cv = 3, scoring = 'f1', n_jobs = -1, verbose = 10)\n",
        "gridsearch_kNN.fit(X_train_pca, Y_train)\n",
        "gridsearch_kNN"
      ],
      "id": "813c5ea8"
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "jEZlzIxKCmTX",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d54daeea-9e75-4fd9-f0a0-d0599c02d9cc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Best possible Cross Validation score: 0.7674746856128912\n"
          ]
        }
      ],
      "source": [
        "print(\"Best possible Cross Validation score:\",gridsearch_kNN.best_score_)"
      ],
      "id": "jEZlzIxKCmTX"
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "K2vdnbgqDABc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3c82c867-2862-4f04-ac73-d8e880c92521"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'metric': 'euclidean', 'n_neighbors': 3}"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "gridsearch_kNN.best_params_"
      ],
      "id": "K2vdnbgqDABc"
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "oWtSK_9aC44W"
      },
      "outputs": [],
      "source": [
        "kNN_tuned = KNeighborsClassifier(gridsearch_kNN.best_params_['n_neighbors'], metric = gridsearch_kNN.best_params_['metric'])\n",
        "kNN_tuned.fit(X_train_pca,Y_train)\n",
        "kNN_tuned_Y_pred = kNN_tuned.predict(X_test_pca)"
      ],
      "id": "oWtSK_9aC44W"
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "xdOpLEpUE9tf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1f84c3eb-79f6-4ef1-8779-652a4de2c159"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for kNN model after hyperparameter tuning is: 0.7454545454545455\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "\n",
        "kNN_best_f1 = metrics.f1_score(Y_test,kNN_tuned_Y_pred)\n",
        "print(\"F1 score for kNN model after hyperparameter tuning is:\",kNN_best_f1)"
      ],
      "id": "xdOpLEpUE9tf"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4711db28"
      },
      "source": [
        "## Logistic Regression"
      ],
      "id": "4711db28"
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "9b789501",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0ed651b8-a884-4861-af7e-e7e2ca1695ff"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 2 concurrent workers.\n",
            "[Parallel(n_jobs=-1)]: Done   1 out of   1 | elapsed:    1.4s finished\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(n_jobs=-1, verbose=2)"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "#Model Building\n",
        "\n",
        "log_reg = LogisticRegression(verbose = 2, n_jobs = -1)\n",
        "log_reg.fit(X_train_pca, Y_train)"
      ],
      "id": "9b789501"
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "2d73b791"
      },
      "outputs": [],
      "source": [
        "#Applying model to test dataset\n",
        "\n",
        "LR_Y_pred = log_reg.predict(X_test_pca)"
      ],
      "id": "2d73b791"
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "03885c81",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4d337864-57a9-46db-fd88-26c8c523b239"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for Logistic Regression model is: 0.5427135678391961\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "\n",
        "logr_f1 = metrics.f1_score(Y_test,LR_Y_pred)\n",
        "print(\"F1 score for Logistic Regression model is:\",logr_f1)"
      ],
      "id": "03885c81"
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "B0Ss-pWoQ6r7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "823c54d6-27d7-447b-b5ef-54b63ffba0e6"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=3, estimator=LogisticRegression(max_iter=500), n_jobs=-1,\n",
              "             param_grid={'class_weight': [{0: 1000, 1: 100}, {0: 1000, 1: 10},\n",
              "                                          {0: 1000, 1: 1.0}, {0: 500, 1: 1.0},\n",
              "                                          {0: 400, 1: 1.0}, {0: 300, 1: 1.0},\n",
              "                                          {0: 200, 1: 1.0}, {0: 1.0, 1: 10},\n",
              "                                          {0: 1.0, 1: 99}, {0: 1.0, 1: 100},\n",
              "                                          {0: 1.0, 1: 150}, {0: 1.0, 1: 200},\n",
              "                                          {0: 1.0, 1: 300}, {0: 1.0, 1: 400},\n",
              "                                          {0: 1.0, 1: 500}, {0: 1.0, 1: 1000},\n",
              "                                          {0: 10, 1: 1000},\n",
              "                                          {0: 100, 1: 1000}]},\n",
              "             scoring='f1')"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ],
      "source": [
        "#Hyperparameter Tuning\n",
        "\n",
        "w = [{0:1000,1:100},{0:1000,1:10}, {0:1000,1:1.0}, \n",
        "     {0:500,1:1.0}, {0:400,1:1.0}, {0:300,1:1.0}, {0:200,1:1.0}, \n",
        "     {0:1.0,1:10}, {0:1.0,1:99}, {0:1.0,1:100}, {0:1.0,1:150}, \n",
        "     {0:1.0,1:200}, {0:1.0,1:300},{0:1.0,1:400},{0:1.0,1:500}, \n",
        "     {0:1.0,1:1000}, {0:10,1:1000},{0:100,1:1000} ]\n",
        "crange = np.arange(0.5, 20.0, 0.5)\n",
        "param_grid = {\"class_weight\": w}\n",
        "\n",
        "gridsearch_logr = GridSearchCV(LogisticRegression(max_iter = 500), param_grid, cv = 3, scoring = 'f1', n_jobs = -1)\n",
        "gridsearch_logr.fit(X_train_pca,Y_train)"
      ],
      "id": "B0Ss-pWoQ6r7"
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "yWcVNZ0vQ7Km",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c408b9de-a928-442d-8370-1266c603b283"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cross validation score after tuning:  0.7529077322624754\n",
            "Final parameters after tuning {'class_weight': {0: 100, 1: 1000}}\n"
          ]
        }
      ],
      "source": [
        "print(\"Cross validation score after tuning: \",gridsearch_logr.best_score_)\n",
        "print(\"Final parameters after tuning\",gridsearch_logr.best_params_)"
      ],
      "id": "yWcVNZ0vQ7Km"
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "OgcKKqAZV2Uu"
      },
      "outputs": [],
      "source": [
        "logr_tuned = LogisticRegression(class_weight = gridsearch_logr.best_params_['class_weight'], max_iter = 500)\n",
        "logr_tuned.fit(X_train_pca,Y_train)\n",
        "\n",
        "logr_tuned_pred = logr_tuned.predict(X_test_pca)"
      ],
      "id": "OgcKKqAZV2Uu"
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "33NCTjvAWSYJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a154b68b-d8cf-4934-9086-375ec22f26c0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for Logistic Regression after Hyperparameter tuning is: 0.7081712062256809\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "\n",
        "logr_best_f1 = metrics.f1_score(Y_test,logr_tuned_pred)\n",
        "print(\"F1 score for Logistic Regression after Hyperparameter tuning is:\",logr_best_f1)"
      ],
      "id": "33NCTjvAWSYJ"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "81c3d3c1"
      },
      "source": [
        "## Decision Tree"
      ],
      "id": "81c3d3c1"
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "9d3e48cc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "647f6744-b380-4377-8183-34edb3437fec"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DecisionTreeClassifier(criterion='entropy')"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ],
      "source": [
        "#Model Building\n",
        "\n",
        "DT = DecisionTreeClassifier(criterion = 'entropy')\n",
        "DT.fit(X_train_pca,Y_train)"
      ],
      "id": "9d3e48cc"
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "b09c7969"
      },
      "outputs": [],
      "source": [
        "#Applying model to test dataset\n",
        "\n",
        "DT_Y_pred = DT.predict(X_test_pca)"
      ],
      "id": "b09c7969"
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "193f3860",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4fe50359-fe4b-42c3-a23d-5132477bc6a9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for Decision Tree model is: 0.6415094339622641\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "\n",
        "dt_f1 = metrics.f1_score(Y_test, DT_Y_pred)\n",
        "print(\"F1 score for Decision Tree model is:\",dt_f1)"
      ],
      "id": "193f3860"
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "1a26c850",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "34c5cac5-0188-4cdb-9907-52886a8096e4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=3, estimator=DecisionTreeClassifier(), n_jobs=-1,\n",
              "             param_grid={'criterion': ['gini', 'entropy'],\n",
              "                         'max_depth': [2, 3, 4, 5, 6, 7, 8, 9],\n",
              "                         'min_samples_leaf': [5, 10, 15, 20]},\n",
              "             scoring='f1')"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "#Hyperparameter Tuning\n",
        "\n",
        "criteria = ['gini', 'entropy']\n",
        "max_depth = list(range(2,10))\n",
        "min_samples_leaf = [5,10,15,20]\n",
        "\n",
        "param_grid = {\"criterion\" : criteria, \"max_depth\" : max_depth, \"min_samples_leaf\" : min_samples_leaf}\n",
        "\n",
        "gridsearch_dt = GridSearchCV(DecisionTreeClassifier(), param_grid, cv = 3, scoring = 'f1', n_jobs = -1)\n",
        "gridsearch_dt.fit(X_train_pca,Y_train)"
      ],
      "id": "1a26c850"
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "Kw29H4eleTzg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "53841a26-1760-4a80-9627-2cb9cac83256"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cross validation score after tuning:  0.7831620269976435\n",
            "Final parameters after tuning {'criterion': 'entropy', 'max_depth': 4, 'min_samples_leaf': 10}\n"
          ]
        }
      ],
      "source": [
        "print(\"Cross validation score after tuning: \",gridsearch_dt.best_score_)\n",
        "print(\"Final parameters after tuning\",gridsearch_dt.best_params_)"
      ],
      "id": "Kw29H4eleTzg"
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "8-sKEbR6eUS1"
      },
      "outputs": [],
      "source": [
        "dt_tuned = DecisionTreeClassifier(criterion = gridsearch_dt.best_params_['criterion'],\n",
        "                                  max_depth = gridsearch_dt.best_params_['max_depth'],\n",
        "                                  min_samples_leaf = gridsearch_dt.best_params_['min_samples_leaf'])\n",
        "dt_tuned.fit(X_train_pca,Y_train)\n",
        "\n",
        "dt_tuned_pred = dt_tuned.predict(X_test_pca)"
      ],
      "id": "8-sKEbR6eUS1"
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "3q0IHNsheeM0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "243d2359-9af0-42f1-9ea8-bd3003861a9d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for Decision Tree model is: 0.7272727272727273\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "\n",
        "dt_best_f1 = metrics.f1_score(Y_test, dt_tuned_pred)\n",
        "print(\"F1 score for Decision Tree model is:\",dt_best_f1)"
      ],
      "id": "3q0IHNsheeM0"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ceb7920c"
      },
      "source": [
        "## Support Vector Machine"
      ],
      "id": "ceb7920c"
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fd0cd6ee",
        "outputId": "4cb0abce-a4e7-4190-caed-3426c6f5f30a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVC(max_iter=10000)"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "#Model Building\n",
        "svm = SVC(max_iter=10000)\n",
        "svm.fit(X_train_pca, Y_train)"
      ],
      "id": "fd0cd6ee"
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "32f2ee2a"
      },
      "outputs": [],
      "source": [
        "#Applying model to test dataset\n",
        "svm_Y_pred = svm.predict(X_test_pca)"
      ],
      "id": "32f2ee2a"
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "123feea2",
        "outputId": "3ba78f80-c31e-4317-ec31-e56ce509117f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for SVM model is: 0.7368421052631579\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "svm_f1 = metrics.f1_score(Y_test,svm_Y_pred)\n",
        "print(\"F1 score for SVM model is:\",svm_f1)"
      ],
      "id": "123feea2"
    },
    {
      "cell_type": "code",
      "source": [
        "#Hyperparameter Tuning\n",
        "kernel = ['poly', 'rbf', 'linear']\n",
        "\n",
        "param_grid = {\"kernel\" : kernel}\n",
        "\n",
        "gridsearch_svc = GridSearchCV(SVC(max_iter=1000000), param_grid, cv = 3, scoring = 'f1', n_jobs = 1, verbose=2)\n",
        "gridsearch_svc.fit(X_train_pca,Y_train)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xpjkuv7HVc_g",
        "outputId": "c749d8b6-1d86-4780-aae8-eb62aeb86e11"
      },
      "id": "xpjkuv7HVc_g",
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fitting 3 folds for each of 3 candidates, totalling 9 fits\n",
            "[CV] END ........................................kernel=poly; total time=   8.5s\n",
            "[CV] END ........................................kernel=poly; total time=   8.0s\n",
            "[CV] END ........................................kernel=poly; total time=   6.1s\n",
            "[CV] END .........................................kernel=rbf; total time=   6.8s\n",
            "[CV] END .........................................kernel=rbf; total time=   6.8s\n",
            "[CV] END .........................................kernel=rbf; total time=   6.3s\n",
            "[CV] END ......................................kernel=linear; total time=   2.6s\n",
            "[CV] END ......................................kernel=linear; total time=   2.5s\n",
            "[CV] END ......................................kernel=linear; total time=   2.4s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=3, estimator=SVC(max_iter=1000000), n_jobs=1,\n",
              "             param_grid={'kernel': ['poly', 'rbf', 'linear']}, scoring='f1',\n",
              "             verbose=2)"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Cross validation score after tuning: \",gridsearch_svc.best_score_)\n",
        "print(\"Final parameters after tuning\",gridsearch_svc.best_params_)"
      ],
      "metadata": {
        "id": "lYPK8JQ6bPjf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0fdd2217-897b-446e-830e-8ca232fada87"
      },
      "id": "lYPK8JQ6bPjf",
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cross validation score after tuning:  0.7955404541216899\n",
            "Final parameters after tuning {'kernel': 'rbf'}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "svc_tuned = SVC(kernel = gridsearch_svc.best_params_['kernel'])\n",
        "svc_tuned.fit(X_train_pca,Y_train)\n",
        "svc_tuned_pred = svc_tuned.predict(X_test_pca)"
      ],
      "metadata": {
        "id": "h4mTqJ41bPpH"
      },
      "id": "h4mTqJ41bPpH",
      "execution_count": 39,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#Model Evaluation\n",
        "\n",
        "svm_best_f1 = metrics.f1_score(Y_test,svc_tuned_pred)\n",
        "print(\"F1 score for SVM after hyperparameter tuning is: \",svm_best_f1)"
      ],
      "metadata": {
        "id": "gK8jTDHJbYZo",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "67a8ca6c-e78b-46a8-8b79-8dbaa6c2c16c"
      },
      "id": "gK8jTDHJbYZo",
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for SVM after hyperparameter tuning is:  0.7368421052631579\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "12ae74ce"
      },
      "source": [
        "## Random Forest"
      ],
      "id": "12ae74ce"
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "992cac9b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f0b3503b-c462-4d1c-dd50-7c11168adfee"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RandomForestClassifier(n_jobs=-1)"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ],
      "source": [
        "#Model Building\n",
        "\n",
        "rf = RandomForestClassifier(n_jobs = -1)\n",
        "rf.fit(X_train_pca, Y_train)"
      ],
      "id": "992cac9b"
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "id": "871abb7e"
      },
      "outputs": [],
      "source": [
        "#Applying model to test dataset\n",
        "\n",
        "rf_Y_pred = rf.predict(X_test_pca)"
      ],
      "id": "871abb7e"
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "id": "9170363b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a7f527fb-00ef-42fc-d193-c40c45c07168"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for Random Forest model is: 0.7391304347826088\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "\n",
        "rf_f1 = metrics.f1_score(Y_test, rf_Y_pred)\n",
        "print(\"F1 score for Random Forest model is:\",rf_f1)"
      ],
      "id": "9170363b"
    },
    {
      "cell_type": "code",
      "source": [
        "#Hyperparameter tuning \n",
        "\n",
        "criteria = ['gini','entropy']\n",
        "max_depth = [2,10,20]\n",
        "min_samples_leaf = [5,20]\n",
        "\n",
        "param_grid = {\"criterion\" : criteria, \"max_depth\" : max_depth, \"min_samples_leaf\" : min_samples_leaf}\n",
        "\n",
        "grid_search_rf = GridSearchCV(RandomForestClassifier(),param_grid, cv = 3, verbose = 2, n_jobs = -1, scoring = 'f1')\n",
        "grid_search_rf.fit(X_train_pca,Y_train)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gGJ_2l1C1HAQ",
        "outputId": "4d09a53f-66c9-4648-8879-590a53d877cd"
      },
      "id": "gGJ_2l1C1HAQ",
      "execution_count": 70,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fitting 3 folds for each of 12 candidates, totalling 36 fits\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=3, estimator=RandomForestClassifier(), n_jobs=-1,\n",
              "             param_grid={'criterion': ['gini', 'entropy'],\n",
              "                         'max_depth': [2, 10, 20],\n",
              "                         'min_samples_leaf': [5, 20]},\n",
              "             scoring='f1', verbose=2)"
            ]
          },
          "metadata": {},
          "execution_count": 70
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Cross validation score after tuning: \",grid_search_rf.best_score_)\n",
        "print(\"Final parameters after tuning\",grid_search_rf.best_params_)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0TNkWJC22Bad",
        "outputId": "1bf0e1ee-031a-4473-c3f5-df0570053a9c"
      },
      "id": "0TNkWJC22Bad",
      "execution_count": 72,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cross validation score after tuning:  0.7809101107155051\n",
            "Final parameters after tuning {'criterion': 'entropy', 'max_depth': 10, 'min_samples_leaf': 5}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Fitting Random Forest with the best parameters and making a prediction on test data set\n",
        "\n",
        "rf_tuned = RandomForestClassifier(criterion = grid_search_rf.best_params_['criterion'],\n",
        "                                  max_depth = grid_search_rf.best_params_['max_depth'],\n",
        "                                  min_samples_leaf = grid_search_rf.best_params_['min_samples_leaf'])\n",
        "rf_tuned.fit(X_train_pca,Y_train)\n",
        "\n",
        "rf_tuned_pred = rf_tuned.predict(X_test_pca)"
      ],
      "metadata": {
        "id": "PLzOFtW22SdS"
      },
      "id": "PLzOFtW22SdS",
      "execution_count": 73,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "rf_best_f1 = metrics.f1_score(Y_test, rf_tuned_pred)\n",
        "print(\"F1 score for Random Forest after hyperparameter tuning is: \",rf_best_f1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lcvbYmQu2nlJ",
        "outputId": "aaf609cf-e601-4775-c6a7-1faae1e6d379"
      },
      "id": "lcvbYmQu2nlJ",
      "execution_count": 74,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for Random Forest after hyperparameter tuning is:  0.7456140350877194\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0f35170f"
      },
      "source": [
        "## XGBoost"
      ],
      "id": "0f35170f"
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {
        "id": "5d56b9fd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e3930dbe-85c3-4600-cdd4-30d784c54cc7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "XGBClassifier()"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ],
      "source": [
        "#Model Building\n",
        "\n",
        "xgb =XGBClassifier()\n",
        "xgb.fit(X_train_pca, Y_train)"
      ],
      "id": "5d56b9fd"
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {
        "id": "aea942d5"
      },
      "outputs": [],
      "source": [
        "#Applying model to test dataset\n",
        "\n",
        "xgb_Y_pred = rf.predict(X_test_pca)"
      ],
      "id": "aea942d5"
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "id": "32375768",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "16db4444-c350-42c2-aacb-35f0dbcbcdb0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for XGBoost model is: 0.7391304347826088\n"
          ]
        }
      ],
      "source": [
        "#Model Evaluation\n",
        "\n",
        "xgb_f1 = metrics.f1_score(Y_test, xgb_Y_pred)\n",
        "print(\"F1 score for XGBoost model is:\",xgb_f1)"
      ],
      "id": "32375768"
    },
    {
      "cell_type": "code",
      "source": [
        "#Hyperparameter Tuning\n",
        "\n",
        "param_grid = {'min_child_weight': [1, 5, 10],\n",
        "              'gamma': [0.5, 1, 1.5, 2, 5],\n",
        "              'max_depth': [3, 4, 5]\n",
        "              }\n",
        "\n",
        "grid_search_xgb = GridSearchCV(XGBClassifier(), param_grid, n_jobs = -1, scoring = \"f1\", cv = 3, verbose = 2)\n",
        "grid_search_xgb.fit(X_train_pca,Y_train)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m8DndmFtC0o7",
        "outputId": "16ffaf82-1d3f-4625-88dd-320badffd649"
      },
      "id": "m8DndmFtC0o7",
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fitting 3 folds for each of 45 candidates, totalling 135 fits\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=3, estimator=XGBClassifier(), n_jobs=-1,\n",
              "             param_grid={'gamma': [0.5, 1, 1.5, 2, 5], 'max_depth': [3, 4, 5],\n",
              "                         'min_child_weight': [1, 5, 10]},\n",
              "             scoring='f1', verbose=2)"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Cross validation score after tuning: \",grid_search_xgb.best_score_)\n",
        "print(\"Final parameters after tuning\",grid_search_xgb.best_params_)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SNCW_TXkC08m",
        "outputId": "b70ed29f-5f84-4b26-b8dd-a448951cde99"
      },
      "id": "SNCW_TXkC08m",
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cross validation score after tuning:  0.8082571789231805\n",
            "Final parameters after tuning {'gamma': 0.5, 'max_depth': 5, 'min_child_weight': 5}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Fitting Random Forest with the best parameters and making a prediction on test data set\n",
        "\n",
        "xgb_tuned = XGBClassifier(min_child_weight = grid_search_xgb.best_params_['min_child_weight'],\n",
        "                                  max_depth = grid_search_xgb.best_params_['max_depth'],\n",
        "                                  gamma = grid_search_xgb.best_params_['gamma'])\n",
        "xgb_tuned.fit(X_train_pca,Y_train)\n",
        "\n",
        "xgb_tuned_pred = xgb_tuned.predict(X_test_pca)"
      ],
      "metadata": {
        "id": "V5408HJBC1Ip"
      },
      "id": "V5408HJBC1Ip",
      "execution_count": 54,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "xgb_best_f1 = metrics.f1_score(Y_test, xgb_tuned_pred)\n",
        "print(\"F1 score for XGBoost after hyperparameter tuning is: \",xgb_best_f1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XTa5ydbfC1UN",
        "outputId": "e78ca018-191d-4046-8013-2ea3dd6b5a24"
      },
      "id": "XTa5ydbfC1UN",
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "F1 score for XGBoost after hyperparameter tuning is:  0.748936170212766\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "91b46124"
      },
      "source": [
        "# Comparison of Model Performance"
      ],
      "id": "91b46124"
    },
    {
      "cell_type": "code",
      "execution_count": 75,
      "metadata": {
        "id": "e91b3bb8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 237
        },
        "outputId": "a7fcfc85-d98a-4078-d984-bcf64129fd6c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                    Model  F1 score before Tuning  F1 score after Tuning\n",
              "0                 XGBoost                0.739130               0.748936\n",
              "1           Random Forest                0.739130               0.745614\n",
              "2                     KNN                0.723982               0.745455\n",
              "3  Support Vector Machine                0.736842               0.736842\n",
              "4           Decision Tree                0.641509               0.727273\n",
              "5     Logistic Regression                0.542714               0.708171"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d7b97902-c899-4e7e-9095-991f862bddfb\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Model</th>\n",
              "      <th>F1 score before Tuning</th>\n",
              "      <th>F1 score after Tuning</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>XGBoost</td>\n",
              "      <td>0.739130</td>\n",
              "      <td>0.748936</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Random Forest</td>\n",
              "      <td>0.739130</td>\n",
              "      <td>0.745614</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>KNN</td>\n",
              "      <td>0.723982</td>\n",
              "      <td>0.745455</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Support Vector Machine</td>\n",
              "      <td>0.736842</td>\n",
              "      <td>0.736842</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Decision Tree</td>\n",
              "      <td>0.641509</td>\n",
              "      <td>0.727273</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>Logistic Regression</td>\n",
              "      <td>0.542714</td>\n",
              "      <td>0.708171</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d7b97902-c899-4e7e-9095-991f862bddfb')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-d7b97902-c899-4e7e-9095-991f862bddfb button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-d7b97902-c899-4e7e-9095-991f862bddfb');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ],
      "source": [
        "df_perf = pd.DataFrame(data = [['KNN',kNN_f1,kNN_best_f1],\n",
        "                               ['Logistic Regression',logr_f1,logr_best_f1],\n",
        "                               ['Decision Tree',dt_f1,dt_best_f1],\n",
        "                               ['Support Vector Machine',svm_f1,svm_best_f1],\n",
        "                               ['Random Forest',rf_f1,rf_best_f1],\n",
        "                               ['XGBoost',xgb_f1,xgb_best_f1]\n",
        "                               ], columns = ['Model','F1 score before Tuning','F1 score after Tuning'])\n",
        "                               \n",
        "df = df_perf.sort_values(by = 'F1 score after Tuning',ascending = False).reset_index(drop = True)\n",
        "df"
      ],
      "id": "e91b3bb8"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "19498448",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 378
        },
        "outputId": "b8ffdb6b-947b-4921-f63f-80f3e1a243a4"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAFpCAYAAACS4uOlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3gVVfrHPy8phC7NgkBClyKGIoiooChFAbuSsPZd7IrrWsEfLoq66io2VCyrroCy7lpQFFFEQXERlBUp0qQqSCf0lPf3x5kbbhq5SW4y9968n+fJkztnzsy8Z8p3zrznnPeIqmIYhmHELlX8NsAwDMMoX0zoDcMwYhwTesMwjBjHhN4wDCPGMaE3DMOIcUzoDcMwYhwT+nJCRIaKyKd+2xFARKqJyBQR2Ski/6rA484UkT+GmFdFpGUF2OTLufATEflYRK7w246KRESOEpGvRCRDRP7utz1+EvFCLyLpIjJPRHaLyG/eDXuK33YVh6pOUNW+ftsRxEXAUUB9Vb04/0oRud8T2lvzpd/qpd9fQXYWivfC2O/dB1tE5D8ickwpd3fYcxGLqOoAVX29NNuKyGoROTPcNlUAw4AtQG1VvT3/ShF5TUQOevdU4O9Sb91Nnu4cEJHXKtjusBPRQi8ifwbGAg/hHsymwDjgXD/tKg4RiffbhkJIBpapatZh8iwDLs+XdoWXHgncpKo1gdbAEcCTJd2BiMQR2rkoavtIvLYRizj80plkYLEeflToo6paM+jvbS/9V+BB4NVyt7IYwnLPqWpE/gF1gN3AxYfJUxX3IvjV+xsLVPXW9QbWA3cCvwO/AecBZ+OEaxtwb9C+7gfeAd4GMoDvgROC1t8NrPTWLQbOD1p3JfA1Tni24m6QK4HZ3nrx1v0O7AIWAh2CyvkGsBlYA4wEqgTtdzbwOLAd+AUYcJjz0RaYCewAFgGDvfS/AgeBTO+cXlPItvcDbwJLgPZeWnuvrG8C9wfl/ROwwjuHHwCNgtadBSwFdgLPAl8Cfwxaf7V3jO3ANCA5aJ0CLYso28x8+7kR+Mn7fRww3bPnZ+CSoHyvAc8DU4E93nXKcy5wFZ6R3vn/3bsedbztUzy7rgHWAl/lu947gFXAyV76Om8fVwTZcA7wg3ft1+U7l4H9X+HtfwswImh9HHAvh+69+UCT4sp9uPNHCe4r4J9ADrDPO193euknAd945f8f0DvfscZ452gf0NIr4w3Acq8cDwAtvH3sAiYDid72DYAPvX1vA2bhPROF2Hcy8B3ufvsOODnoumd613o3cGYh274GPFiMDj0IvFZMnpa4+3ynd/3eDlrXPugabcLTHELTrruAjd41qMIhDdrqna96IetpuIQ53H9AfyALiD9MntHAt8CRQEPvpnkg6GRlAf8HJODEaTMwEajlXYB9QDM9JHSZuM/6BOAv3gOQ4K2/GGjknfBLcaJxTNCDkwXcDMQD1cgr9P1wD+gRONFvG7TtG8D7nk0puJfQNUH7zfRsjwOu924KKeRcJODE914gETgD90C1CSrfm4c5l/fjBP1e4G9e2qPAPQQJvbffLUBn72Z9Bvgq6AHNCDqHt3nnJSAw53o2tvXO00jgmyAbQhJ67zgzcA9ADZx4XuXts5NnX7ugh3kn0NO7dkn5zwXu5bMCaA7UBP4D/NNbl+LZ9YZ3rMC1zfKOGYcTg7XAc9456eudh5pB9+Lx3vE74h748/Lt/yVv3ycAB4C23vo7cBWDNrh75wSgfnHlLub8XUmI95WXfzVBQgkcixObs70yneUtNww61lrcMxbv3QuKu89re+kHgM+9c14HV6G4wtv+YeAFb7sE4NTCbAPq4V5Ul3nHSfOW6wdd+yKFvLj1Xp5QhH4SMIJD99cpXnotXAXzdi+9FtC9BNr1N+9+qgbc6uVv7KW9CEwKWU/9FPNiTt5QYGMxeVYCZwct9wNWB52sfUBc0EnXwIn20uZz6IG7H/g2aF0V7yKdWsSxFwDnBj04a/Otv5JDQn8GTsBPIqhmgnvIDhL0cALXAjOD9rEiaF11rwxHF2LPqbi3f/D+J3FIoO8nNKFvintIE7z/Tcgr9K/gPncD29XEiUYKzu0TfA4FVzMJCMzHBH1NeOd4L16tnuKFfi+ulrcBmIB7QC4FZuXL+yIwyvv9GvBGYWUNWv4cuCFouY1XpngOCXHzfNd2edDy8V6eo4LStgKpRZRlLPCk9zuw/8ZB6+cCQ7zfP+PdZ/n2cdhyF3H+goU+pPvKW7+avEJ/F96LMChtGoeEeiYwOt96BXrme/buClr+OzDW+z0a91Io9F4I2uYyYG6+tDnAlUHXvjih3+/dUzuALYXkCUXo3wDGB19DLz0N+KGIbYrTroNAUtD6JUCfoOVjAvfo4WwL/EWyj34r0KAY/1Qj3Od2gDVeWu4+VDXb+73P+78paP0+nFAFWBf4oao5OJFqBCAil4vIAhHZISI7gA64mmWBbfOjqjNwbozngN9FZLyI1Pa2TyikDMcGLW8M2s9e72ewzQEaAes8u4vaV7Go6lpc7fYhnJjlL1eec66qu3HX6tiADUHrlLznJRl4KugcbsO9DEK18RZVPUJVj1XVoaq62dtn98A+vf0OBY4O2q7Ia1NYmbzf8bh2oaL2kf8+QlULvbdEpLuIfCEim0VkJ3Adee8dCLrOuBda4Bo3wYlCfkIp9+EI9b4qjGTg4nzHPgUnPgEKO+f5z09Rz+JjuHvwUxFZJSJ3F2FH/usGJb/nH/fuqSNUNf81CZU7cffxXBFZJCJXe+lFXTsoXrs2q+r+oOVk4N2g870EyCbvPVokkSz0c3Cfd+cdJs+vuBMQoKmXVlqaBH54DUiNgV9FJBn3aX0T7rPwCOAn3MUNoIfbsao+rapdgHa4xsQ7cJ/amYWUYUMpbP8VaJKv4au0+3oD97n5RhHHybVXRGrgXAkbcF9AwedQgpdxD/+1QQ/WEapaTVW/KYWNwfv8Mt8+a6rq9UF5Dntt8pcJd96yyCtExe3jcEzEtWU0UdU6OLeEHH6TXNbhfNmFpRdX7nCRv+zrcDX64GPXUNVHDrNN6AdTzVDV21W1OTAY+LOI9Ckka/7rBqW/50uNqm5U1T+paiPcF/k4r5vwOpxrqjCK067CzvmAfOc8SVVDKmvECr2q7sT5158TkfNEpLqIJIjIABF51Ms2CRgpIg1FpIGX/80yHLaLiFzgfUUMx71ovsX5QxXn40dErsLV6ENCRE70anUJON/+fiDH+9qYDIwRkVreC+XPpSzDf3E1wTu989QbGAS8VYp9vY3zM08uZN0k4CoRSRWRqria/39VdTXwEdA+6BzeQt4a5gvAPSLSHkBE6ohIWbs3fgi0FpHLvHIneOe7bQn2MQm4TUSaiUhNr0xvayl65RRBLWCbqu4XkW5Aegm2fRl4QERaeT1YOopIfcJT7lDZRF7BehMYJCL9RCRORJJEpLeINA7HwURkoIi09CoKO3E115xCsk7FnYN0EYn3uka2w52bstoQLyJJOPdqoIyFehdE5OKgsm/HaUWOZ8cxIjJcRKp6z3h3L19JtesFnE4ke8dsKCIh9z6MWKEHUNW/44RvJE5k1+Fq1e95WR4E5gE/4hqsvvfSSsv7ON9noIHnAlXNVNXFOB/iHNxNfzyuR0Go1MZ9EWzHfaJtxX2egmvA3YPruTEbV/srcZcuVT2IE/YBuC+FccDlqrq0FPvap6qfqeq+QtZ9BtwH/BtXg28BDPHWbcE1Wj+CK2Mrgs6Tqr6La2B6S0R24b6KBpTUvnz2ZOBeSkNwNaKNHGrECpVXcQ27X+Ea4Pfjrku4uAEYLSIZuAe6sBdoUTzh5f8U1zvlFaBamModKg/jRGmHiPzFc+edi2u4DzyXdxA+PWkFfIbrLTMHGKeqX+TPpKpbgYG4r8+tOBfKQO8+LCsjce6ku4E/eL9HFpH3ROC/IrIb9+V2q6qu8q7RWbjnciOux9Hp3jYl1a6nvH1/6t1H3wLdD5M/D+I59is94gYEtVTVP/hti2EYRjiJ6Bq9YRiGUXZM6A3DMGIcc90YhmHEOFajNwzDiHFM6A3DMGIcE3rDMIwYx4TeMAwjxjGhNwzDiHFM6A3DMGIcE3rDMIwYx4TeMAwjxjGhNwzDiHFM6A3DMGIcE3rDMIwYx4TeMAwjxjGhNwzDiHFM6A3DMGIcE3rDMIwYx4TeMAwjxjGhNwzDiHHi/TYgPw0aNNCUlBS/zTAMw4gq5s+fv0VVGxa2LuKEPiUlhXnz5vlthmEYRlQhImuKWmeuG8MwjBjHhN4wDCPGMaE3DMOIcUISehHpLyI/i8gKEbm7kPVPisgC72+ZiOwIWpcdtO6DcBpvGIZhFE+xjbEiEgc8B5wFrAe+E5EPVHVxII+q3haU/2agU9Au9qlqavhMNgzDMEpCKDX6bsAKVV2lqgeBt4BzD5M/DZgUDuMMwzAqAxMWTiBlbApV/lqFlLEpTFg4Iaz7D0XojwXWBS2v99IKICLJQDNgRlBykojME5FvReS8UltqGIYRg0xYOIFhU4axZucaFGXNzjUMmzIsrGIf7n70Q4B3VDU7KC1ZVTeISHNghogsVNWVwRuJyDBgGEDTpk3DbJJhGEbFkJ2Tze6Du8k4mEHGgQx2HdhFxkHv/4GMPL8D695Z/A77svbl2c/ezL2M+HwEQ48fGha7QhH6DUCToOXGXlphDAFuDE5Q1Q3e/1UiMhPnv1+ZL894YDxA165dNRTD8zNh4QRGfD6CtTvX0rROU8b0GRO2k2QYRsVR0c9ydk52iYS50DTv957MPSEdM75KPLWr1qZWYq0CIh9g7c61YStjKEL/HdBKRJrhBH4IkJ4/k4gcB9QF5gSl1QX2quoBEWkA9AQeDYfhwQQ+ffZm7gXI/fQBTOwNI4oI9VnOyskKXZgPZLDrYOHCnHEwI/dYxZEYl0itxFpOoKu6/0fWOJIW9VpQO/FQWq3EWnl+B/IH/64aVxURASBlbAprdhYc1Nq0Tvi8G6JafAVaRM4GxgJxwKuqOkZERgPzVPUDL8/9QJKq3h203cnAi0AOrj1grKq+crhjde3aVUsaAqGoE5VcJ5nVw1eXaF+GYVQ8WTlZrN25lh6v9OD3Pb8XWJ9QJYGmdZrmCvn+rP0h7TcpPql44S1CjPP/rhpfNdzFBgq+3ACqJ1Rn/KDxJaqoish8Ve1a2LqQfPSqOhWYmi/t//It31/Idt8Ax4dsaSkp6hMnnJ8+hmGUjX2Z+/hlxy+s2LaCldtWuv/bV7Jy+0pW71hNVk5Wkdtm5mTS7dhuhxXmwmrTCXEJFVjC0hEQ8/J0V4VUo69Iwlmjb1CtAb/f8XvuJ5JhGOXLjv07WLnNiXdA0AO/N2TkbdqrU7UOLeq1oGW9lrSo24IWdVswYsYINu3ZVGC/9nVePGWu0Uc6Y/qMKfDpIwhb9m2h12u9eGbAM5xw9Ak+WmgYsYGqsmnPpgI18oCob923NU/+o2ocRct6LenTvA8t6h4S9Zb1WlKvWr0ClbCkhKRC3Rhj+oypkPLFKjEh9IV9+jxw+gPsz9rPvTPupfP4zlzb5VoeOP0B6lev77O1hhHZZOdks27XugI18pXbV7Jy28o8PUuqSBWa1mlKi7otuKjdRa5m7tXSm9dtTs3EmiU69tDjh/L1bBi/agTZNdYSt6cpVzSP/R50EybAiBGwdi00bQpjxsDQMBY5Jlw3h2P7vu2MmjmKcd+No05SHR48/UGGdRlGXJW4sB3DMKKN/Vn7+WX7L4W6WFbvWE1mTmZu3qpxVWlWt1me2nhA0FOOSCExLjFsdk2YAMOGwd6gjjDVq8P48eEVvkgiXGU+nOsm5oU+wMJNC7nlk1uYuXomJxx1Ak8PeJrTkk8L+3EMI1LYdWBXXhfLtpWs2O5Eff2u9SiHnv1aibWcgNdrQcu67n9A1I+tfSxVpOyBblVh/37Yt+/Q3969eZcvvxy2bCm4bd268OCDZTYhIhk5ErZvL5ienAyrV4e+HxN6D1XlncXvcPunt7Nu1zrSOqTx6FmP0rh243I5nmGUlRuen5DHjTGs+RjGXe+qearK5r2bC3WxrNi2gi178yrmkTWOzFMjb163BSm1W9IoqQXVtAH790uRApx/ubR5jNARgZyckuQ3oc/D3sy9/G323/jb138jrkocI04dwZ97/Jmk+KRyPa5hlIQbnp/A8xuGQULQN312PEfmdKJq9YP8nrWSA7r70DoVauY0pebBFlTb14Kqe1uSsKsFsqMFuq0FBzNqFRDfkghJMAkJzr1Qrdqhv+KWQ8lzwQXw228Fj9e4McyfXzpbI50uXWD9+oLpVqMPE79s/4XbP72dd5e+S4u6LXiy35MMbD3QumMaFc7O/TtZtnUZP2/9Off/5B/fhbjMgpmz42FlX9jWEra3gG0tqLKzJdUOpFCjatUSi2tpBDmunJq4zEfvMB99OTB95XRu/eRWlmxZQv+W/RnbbyxtGrSpUBuM2CczO5NV21cdEvMtP+f+Du47XkWq0OyIZqzcthIKq3OosOqKnDwCnBD544JCprx7oEQi4SizCX0IZGZn8uzcZ7n/y/vZl7mP4ScNZ+RpI6ldtXaF22JEL6rKxt0bCxXzVdtXkR0U2LVh9Ya0adCG1vVau//1W9Omfhta1GtBYlwi8X9JIbtWwYGAcbuTyXpsdQWWyogGTOhLwKbdm7jn83v4x4J/cHTNo3n0zEcZ2nFoWHodGLHD7oO7WbZ1WQExX7Z1GRkHM3LzJcUn0bp+61wRD/5ft1rdwx6j/ZAJLG4xDBKDvukzq3P9seNzG2QNI4AJfSmYu2EuN398M3M3zKVH4x48M+AZujTq4rdZRgWSlZPF6h2rC4j5z1t/5teMX3PzCULyEcmFinmTOk1KVUn47DM46yw44Q8T+OnownvdGEYwJvSlJEdzeH3B69z9+d1s3rOZP3b+I2POGEPDGg39Ns0IE4EuioWJ+cptK/MMHKqbVJc2DdoUEPOW9VpSLaFa2GzKyIDjj4ekJPjhB+eDN4ziMKEvIzv372T0l6N5eu7T1EiowejTR3PDiTcQXyUmIkhUCvZm7mXFthUFxHzZ1mXs2L8jN19iXCIt67UsIOZtGrShQfUGFWLr9dfDiy/C119Djx4VckgjBjChDxNLNi/h1k9uZfqq6bRv2J6nBzzNGc3O8NusSkNxMw9l52SzdufaAt0Ul21dViBkdePajQsV8+Q6yb6Gx/j8czjzTLj9dnj8cd/MMKIQE/owoqq8//P73DbtNlbvWM1F7S7i8bMeJ/mIZL9Ni2kKm5whMS6R/i36Ex8Xz7Kty1i+dTkHsg/krq9dtXahYt6qXitqJNbwoxiHJeCyqVoVFiwwl41RMkzoy4F9mfv4+5y/89Csh1CUu3vezZ097wyrr9Zw5GgOxz5xLBt3byx0fWFi3rp+a46qcVRUDX4LuGxmz4aTT/bbGiPaqBRC79cgi7U713LH9DuYvGgyyXWSeaLfE5x/3PlRJTCRyN7MvXy26jOm/DyFD5d/WKTIC0LOqFKO448gzGVjlJWYF/pIGDY9c/VMbvn4Fhb+vpA+zfrwVP+naH9k+4o5eIywYdcGPlz2IVOWTeHzXz5nf9Z+aiXWon/L/sz4ZUaBSS0gNmYeMpeNEQ5iXuhTUmBNwQGEJQ4KVFaycrJ4Yd4L3PfFfWQcyODmbjczqvcojkg6ouKMiCJUle9/+54py6YwZdkUvv/tewCaHdGMQa0HMajNIE5LPo3EuMSwTaAciZjLxggHMS/0Vaq4WNf5KWmYz3CxZe8WRs4Yyfj542lQvQEP93mYqzpdZaNrcW0bn//yea5L5teMXxGEHk16OHFvPYh2DdsV6voqrtdNNGIuGyNcxLzQR0qNPj/f//Y9N398M9+s+4aujbryzIBnOKnxSf4Z5BO/ZfyW65L5bNVn7MvaR83EmvRr0Y9BrQdxdquzK+UgNHPZGOEk9icHH1O4j36Mz/MJdz6mM7Ovms3EhRO5Y/od9HilB1eccAWPnPkIR9c82l/jyhFVZcHGBbkumXm/uhd3cp1krul0DYPaDKJXci+qxlf12VJ/uesu13lg9mwTeaN8iYkaPbgG2XvvdQ9O1arwyiuRFdo040AGY2aN4Yk5T5AUn8SoXqO4ufvNYZ1v00/2Z+1nxi8zcl0y63etRxC6N+6e65LpcGQH643kMWMG9OkDf/4z/P3vfltjxAIx77oJZtQoeOABN2NLo0ZhNCxMLN+6nOHThjN1+VTa1G/DU/2fol/Lfn6bVSo27t7IR8s+YsqyKUxfNZ29mXupkVCDvi36Mqj1IM5pfQ5H1jjSbzMjDnPZGOVBpRL6pUuhbVt48kkYPjyMhoWZj5Z9xPBpw1mxbQXntjmXJ/o9QfO6zf0267CoKv/b9L9cf/vcDXMBaFK7SW4vmd4pvW1KxmK44QZ44QWYNQt69vTbGiNWqFRCD9C5M8THw9y5YTKqnDiQdYCx347lga8eICsni7+c/BfuOeWeiBqevz9rP1/88gVTlk3hw2Ufsm7XOgC6Hdst1yXT8aiO5pIJEXPZGOVFpRP6xx+HO+6AZcugVaswGVaObNi1gbs+u4sJCyfQuHZjHj/rcS5pf4lv4rlp9yY+Wu65ZFZOZ0/mHqonVOes5mflumRiuTG5vNi927lsEhKcy6Z6db8tMmKJSif069e7MAh//Svcd1+YDKsAZq+dzc0f38yCjQvoldyLpwc8TcejOpb7cVWVhb8vZMrPU3JdMorSuHZjBrYayKA2gzij2Rnmkikj5rIxypNKJ/QAvXrBpk2wZIkbOBUtZOdk8/L3LzNixgi279/O9V2vZ/Tpo6lXrV5Yj3Mg6wBfrvkyV9zX7HQDEbo26prrkkk9OtVcMmEi4LK57TZ44gm/rTFikUop9C++CNddB99/D506hcGwCmbbvm383xf/x/PznqduUl3GnDGGP3b+Y5lipW/es5mpy6cyZdkUpq2cxu6Du6kWX40zm5+Z65JpVCsCuypFOeayMSqCSin0W7fC0Ue7njePPRYGw3zix00/csvHt/Dlmi9JPTqVZwY8wylNTwlpW1Vl0eZFuX3b56ybg6I0qtUo1yXTp1kfC61cztx4Izz/vLlsjPKlUgo9wMCB8L//ufAIVaI4zIyq8q/F/+L2T29n/a71pB+fzqNnPsrMNTMLxH65uN3FfLn6y9xeMr/s+AVwo3QDLpnOx3Q2l0wFYS4bo6KotEI/caIbHfvll3DaaWHZpa/sObiHR2Y/wmPfPIaqomieyavjJI74KvEcyD5AUnwSfZr1YVDrQQxsPZBjax/ro+WVE3PZGBVJzMe6KYrBg93DNXFibAh9jcQaPHDGA1zV6So6jOvAvqx9edZnazbVqlRj8sWTObP5mVRPMGXxk7vucl+Ts2aZyBv+EsUOjeKpWdOJ/b/+BQcP+m1N+Ghetzn7s/YXum5P5h4GtxlsIu8zX3wB48a5NiLzyxt+E9NCD5CeDtu2wfTpflsSXprWaVqidKPi2L0brr7aDdZ78EG/rTGMSiD0/fpB3brOfRNLjOkzpkCtvXpCdcb08Tk2s8HddzuXzauvmsvGiAxiXugTE+Hii+H992HPHr+tCR9Djx/K+EHjSa6TjCAk10mOiWn1op0vvoDnnoNbb4VTQusFaxjlTki9bkSkP/AUEAe8rKqP5Fv/JHC6t1gdOFJVj/DWXQGM9NY9qKqvH+5Y4ex1E2DmTDj9dJg0CYYMCeuuDSOXQC+b+HjXrddq80ZFUqZeNyISBzwHnAWsB74TkQ9UdXEgj6reFpT/ZqCT97seMAroCigw39t2exnKU2JOPRWOPda5b0zojfIi4LL56isTeSOyCMV10w1YoaqrVPUg8BZw7mHypwGTvN/9gOmqus0T9+lA/7IYXBri4pzAf/KJa5g1jHBjLhsjkglF6I8F1gUtr/fSCiAiyUAzYEZJty1v0tIgMxP+/W8/jm7EMrt3wzXXQMuW/s9TbBiFEe7G2CHAO6qaXZKNRGSYiMwTkXmbN28Os0mOzp2hdevY631j+M/dd8Pq1dbLxohcQhH6DUCToOXGXlphDOGQ2ybkbVV1vKp2VdWuDRs2DMGkkiPi+tR/+SVsKMp6wyghM2c6l80tt7i2IMOIREIR+u+AViLSTEQScWL+Qf5MInIcUBeYE5Q8DegrInVFpC7Q10vzhbQ0UIW33/bLAiOWCAyMatkSHnrIb2sMo2iKFXpVzQJuwgn0EmCyqi4SkdEiMjgo6xDgLQ3qr6mq24AHcC+L74DRXpovtG4NXbqY+8YID/fcYy4bIzqI6eiVhfHEE3D77fDzz074DaM0BMZm3HorjB3rtzWGcfh+9DE/MjY/l17q/PWTJhWf1zAKY88e57Jp0cJcNkZ0UOmE/thj3XyyEyc6f71hlJRAL5t//MNcNkZ0UOmEHlzvm2XL3HyyhlESZs6EZ5+1XjZGdFEphf7CC92sP+a+MUpCsMvGBkYZ0USlFPp69aB/fyf02SUa2mVUZoJdNjVq+G2NYYROpRR6cO6bX39107wZRnF8+aVz2dx8s7lsjOij0gr9oEGuVmbuG6M4rJeNEe1UWqGvUQPOPTf25pM1ws8998CqVW5glLlsjGik0go9OPfN9u0wzbegDEak8+WX8MwzrpfNaaf5bY1hlI5KLfR9+0L9+ua+MQrHXDZGrFDsDFOxTEICXHQR/POfLkBVzZp+W2REEgGXzZdfmsvGiG4qdY0enPtm7174oEA8TqMyE3DZ3HyzuWyM6KfSC/0pp0Djxua+MQ4R7LJ5+GG/rTGMslPphb5KlUPzyW7d6rc1RiRgvWyMWKPSCz04901WFrzzjt+WGH5jLhsjFjGhB1JT4bjjbEKSyk7AZdO8ublsjNjChB4Xnz4tzYVDWLfOb2sMv7j3XnPZGLGJCb2HzSdbufnqK3j6aeey6dXLb2sMI7xUuqkED0e3bs5Xb3HqKxd79sAJJ7gX/Y8/Wm3eiE5sKsEQSUuDH36ApUv9tsSoSO69F1auNJeNEbuY0Adh88lWPgIum5tuMpeNEbuY0AfRqBGcfrrNJ1tZCO5l88gjfltjGOWHCX0+0tNhxQqYP99vS4zyZsQI5/fJ5d8AACAASURBVLJ55RVz2RixjQl9Pi64wAU7sz71sc2sWYdcNr17+22NYZQvJvT5qFsXzj4b3nrL5pONVfbsgauugpQUGxhlVA5M6AshPR1++8011BmxR8Bl8+qrFpraqByY0BfCwIFOAMx9E3uYy8aojJjQF0L16nDeeS7I2YEDfltjhIu9e81lY1ROTOiLID0dduyw+WRjieCBUeayMSoTJvRFcOaZbj5Zc9/EBgGXzY03msvGqHyY0BdBQgJccombYjAjw29rjLKwd68bGJWSYgOjjMqJCf1hSE+Hffvg/ff9tsQoCyNGuEFw5rIxKism9Ifh5JOhSROLfRPNzJoFTz1lLhujcmNCfxiqVHERLadNg82b/bbGKCnmsjEMhwl9MaSnuxGyNp9s9BFw2bzyirlsjMqNCX0xdOwIbdua+ybaCLhsbrjBRSQ1jMqMCX0xiLha/axZsHat39YYoRBw2SQnw9/+5rc1huE/JvQhkJbm/r/1lr92GKFhvWwMIy8m9CHQogV0727um2hg9mxz2RhGfkISehHpLyI/i8gKEbm7iDyXiMhiEVkkIhOD0rNFZIH390G4DK9o0tJgwQJYvNhvS4yiCMSyMZeNYeSlWKEXkTjgOWAA0A5IE5F2+fK0Au4Beqpqe2B40Op9qprq/Q0On+kVyyWXuO6WVquPXEaONJeNYRRGKDX6bsAKVV2lqgeBt4Bz8+X5E/Ccqm4HUNXfw2um/xxzDJxxhhN6m0828pg9G8aONZeNYRRGKEJ/LLAuaHm9lxZMa6C1iHwtIt+KSP+gdUkiMs9LP6+M9vpKWpqLfvjdd35bYgRjvWwM4/CEqzE2HmgF9AbSgJdE5AhvXbKqdgXSgbEi0iL/xiIyzHsZzNscwUNQL7gAEhMtomWkMXIkLF9uA6MMoyhCEfoNQJOg5cZeWjDrgQ9UNVNVfwGW4YQfVd3g/V8FzAQ65T+Aqo5X1a6q2rVhw4YlLkRFccQRcM458PbbNp9spPD1185lc/31zrVmGEZBQhH674BWItJMRBKBIUD+3jPv4WrziEgDnCtnlYjUFZGqQek9gajut5KWBhs3wsyZfltiBPeyefRRv60xjMilWKFX1SzgJmAasASYrKqLRGS0iAR60UwDtorIYuAL4A5V3Qq0BeaJyP+89EdUNaqF3uaTjRzMZWMYoSEaYV1IunbtqvPmzfPbjMNyxRUuRv2mTVC1qt/WVE6+/hpOPRWuuw7GjfPbGsPwHxGZ77WHFsBGxpaCtDTYuRM+/thvSyoXEya4kMNVqrjY8vXqWS8bwwgFE/pS0KcPNGxo7puKZMIEGDYM1qxx4xiysmD3bjfVo2EYh8eEvhQE5pOdMgV27fLbmsrBiBGu8TWYAwdcumEYh8eEvpSkpcH+/TafbEVRVIhoCx1tGMVjQl9KevRw3frMfVMxNG1asnTDMA5hQl9KAvPJTp9u88lWBNdfXzCtenUYM6bibTGMaMOEvgwE5pP917/8tiS2yc6Gd9+FGjWgcWM361dyMowfD0OH+m2dYUQ+JvRl4PjjoX17c9+UN88+C//9L7z4IqxbBzk5sHq1ibxhhIoJfRlJT3eDd9as8duS2GT1atezZsAAd64Nwyg5JvRlxOaTLT9U3chXgOefdy4bwzBKjgl9GWnWDE46ydw35cGECTBtGjz8sPPJG4ZROkzow0B6Ovz4Iyxa5LclscPmzTB8uHuJ3nCD39YYRnRjQh8GbD7Z8DN8uBt1/PLLEBfntzWGEd2Y0IeBo45y8W8mTrT5ZMPBRx+5c3nvva5Xk2EYZcOEPkykp8Mvv7hugEbpychwg6PatYN77vHbGsOIDUzow8T557vY9NYoWzbuvRfWr3cuG4v1bxjhwYQ+TNSp4+aTnTzZhdA1Ss4338Bzz8FNN7lYQoZhhAcT+jCSnu5mnfriC78tiT4OHIA//hGaNLH4NYYRbkzow8jZZ0Pt2ua+KQ0PPQRLlsALL0CtWn5bYxixhQl9GKlWzfnq//MfF6veCI2ffnKDooYOdaEODMMILyb0YSY93fX/njrVb0uig+xs57KpXRuefNJvawwjNjGhDzNnnAFHHmnum1B57jnXJfWpp9w8vIZhhB8T+jATHw+XXgoffmjzyRbHmjWuO6VFpjSM8sWEvhxIS3O9SN59129LIhdVuPZa99siUxpG+WJCXw6cdBKkpJj75nBYZErDqDhM6MsBEeeK+Pxz16/eyItFpjSMisWEvpxIS7P5ZIvCIlMaRsViQl9OdOjg5pQ1901epk61yJSGUdGY0Jcj6ekwZ46Lamm4yJTXXWeRKQ2jojGhL0eGDHH/bT5Zh0WmNAx/MKEvR1JS4OSTzX0DFpnSMPzEhL6cSU93sVwWLvTbEv8IRKZs3NgiUxqGH5jQlzMXX+x6llTm+WQffthFpnzxRYtMaRh+YEJfzhx5JJx5ZuWdT3bRIheC2CJTGoZ/mNBXAOnpLq7LnDl+W1KxZGfDNddYZErD8BsT+grgvPMgKanyuW8sMqVhRAYm9BVA7dowcCC8/XblmU/WIlMaRuRgQl9BpKe7GC+ff+63JeWPqhsYBRaZ0jAiARP6CmLAAKhTp3K4byZOhE8+cY2wFpnSMPwnJKEXkf4i8rOIrBCRu4vIc4mILBaRRSIyMSj9ChFZ7v1dES7Do42kJLjgAjef7L59fltTfmzeDLfe6iJT3nij39YYhgEhCL2IxAHPAQOAdkCaiLTLl6cVcA/QU1XbA8O99HrAKKA70A0YJSJ1w1qCKCI93cV7+egjvy0pPywypWFEHqHU6LsBK1R1laoeBN4Czs2X50/Ac6q6HUBVf/fS+wHTVXWbt2460D88pkcfp58ORx0Vu+4bi0xpGJFJKEJ/LLAuaHm9lxZMa6C1iHwtIt+KSP8SbFtpiItz88l+9BHs2OG3NeHFIlMaRuQSrsbYeKAV0BtIA14SkSNC3VhEhonIPBGZt3nz5jCZFJmkp8fmfLIjRlhkSsOIVEIR+g1Ak6Dlxl5aMOuBD1Q1U1V/AZbhhD+UbVHV8araVVW7NozxkTXdukHz5rHlvpkzB5591iJTGkakEorQfwe0EpFmIpIIDAE+yJfnPVxtHhFpgHPlrAKmAX1FpK7XCNvXS6u0iLhpBj//HDZu9NuasnPggAtzYJEpDSNyKVboVTULuAkn0EuAyaq6SERGi8hgL9s0YKuILAa+AO5Q1a2qug14APey+A4Y7aVVatLTIScHJk/225KyY5EpDSPyEY2wkIpdu3bVefPm+W1GuZOa6vrWf/ut35aUnkWLoFMnF4p5wgS/rTGMyo2IzFfVroWts5GxPpGW5gJ+rVrltyWlIzvbTSZSuzaMHeu3NYZhHA4Tep8IzCcbrY2y48a5rxGLTGkYkY8JvU8kJ8Mpp0TnhCRr1ri+8haZ0jCiAxN6H0lLg8WLo2s+WYtMaRjRhwm9jwTmk504sfi8kYJFpjSM6MOE3kcaNoS+fZ2fPifHb2uKxyJTGkZ0YkLvM+npsHZtdMwne9ttFpnSMKIRE3qfOfdc158+0t03H3/s+spbZErDiD5M6H2mVi0YPNiNks3M9NuawsnIgGuvhbZtLTKlYUQjJvQRQHo6bNkSufPJBiJTvvKKRaY0jGjEhD4C6N8fjjgiMt03FpnSMKIfE/oIoGpVuPBCF6N+716/rTnEgQMuzIFFpjSM6MaEPkJIT4fduyNrPtlHHnEDuiwypWFENyb0EUKvXnDMMZHjvlm0yNXi09NdqAPDMKIXE/oIITCf7NSpsH27v7ZYZErDiC1M6COI9HQ4eBD+8x9/7QhEphw71iJTGkYsYEIfQXTtCi1a+Bu6OBCZsn9/GDrUPzsMwwgfJvQRhIir1c+YAb/9VvHHV4Xrr3e/X3jBIlMaRqxgQh9hpKU5wX377Yo/9sSJLtSBRaY0jNjChD7CaNvWzSdb0e4bi0xpGLGLCX0Ekp4Oc+fCihUVd0yLTGkYsUu83wYYBRkyBO6809Xq77uv/I8XiEw5apRFpjQcmZmZrF+/nv379/ttipGPpKQkGjduTEJCQsjbiEbYhKVdu3bVefPm+W2G7/TqBb//7kamlmejaEYGdOgANWrADz9Y0DLD8csvv1CrVi3q16+PWKt8xKCqbN26lYyMDJo1a5ZnnYjMV9WuhW1nrpsIJS0Nli6F//2vfI8zYgSsW2eRKY287N+/30Q+AhER6tevX+IvLRP6COWiiyA+vnxDIgQiU954o0WmNApiIh+ZlOa6mNBHKA0aQL9+8NZb5TOfbHBkyoceCv/+DcOIHEzoI5i0NOdW+frr8O/bIlMakU5cXBypqamccMIJdO7cmW+++abYbZ5++mnatm3L0DAO6542bRqpqamkpqZSs2ZN2rRpQ2pqKpdffnmJ93XyySeHza4SoaoR9delSxc1HBkZqtWqqV53XXj3+9NPqgkJqunp4d2vETssXry4RPnffFM1OVlVxP1/882y21CjRo3c35988omedtppxW7Tpk0bXbduXcjHyMzMLJFNvXr10u+++65E25QHhV0fYJ4WoatWo49gatZ0k4f/61/hm0/WIlMa4WbCBBg2zMVJUnX/hw1z6eFi165d1K1bN3f5scce48QTT6Rjx46MGjUKgOuuu45Vq1YxYMAAnnzySbZt28Z5551Hx44dOemkk/jxxx8BuP/++7nsssvo2bMnl112GZs3b+bCCy/kxBNP5MQTT+TrED+hU1JS2LJlCwDz5s2jd+/eufu/+uqr6d27N82bN+fpp5/O3aZmzZoAzJw5k969e3PRRRdx3HHHMXToUNTrATl16lSOO+44unTpwi233MLAgQPLdvKwfvQRT1qa89NPnw5nn132/QUiU/7znxaZ0giN4cNhwYKi13/7rWvzCWbvXrjmGnjppcK3SU0tvqKxb98+UlNT2b9/P7/99hszZswA4NNPP2X58uXMnTsXVWXw4MF89dVXvPDCC3zyySd88cUXNGjQgJtvvplOnTrx3nvvMWPGDC6//HIWeAVZvHgxs2fPplq1aqSnp3PbbbdxyimnsHbtWvr168eSJUtCPT2FsnTpUr744gsyMjJo06YN119/fYF+7z/88AOLFi2iUaNG9OzZk6+//pquXbty7bXX8tVXX9GsWTPS0tLKZEcAE/oIp39/qFvX9b4pq9CvXWuRKY3wk1/ki0sPlWrVquUK85w5c7j88sv56aef+PTTT/n000/p1KkTALt372b58uWcdtppebafPXs2//73vwE444wz2Lp1K7t27QJg8ODBVKtWDYDPPvuMxYsX5263a9cudu/enVv7Lg3nnHMOVatWpWrVqhx55JFs2rSJxo0b58nTrVu33LTU1FRWr15NzZo1ad68eW4f+bS0NMaPH19qOwKY0Ec4iYmuq+XEia6WVL166fajCtdd535bZEqjJBRX805Jce6a/CQnw8yZ4bGhR48ebNmyhc2bN6Oq3HPPPVx77bWl3l+NGjVyf+fk5PDtt9+SlJRUon3Ex8eT43WJy9+vvWrQoJS4uDiysrIKbB9KnnBhPvooIC0N9uyBKVNKv49JkywypVE+jBlTsAJSvXp4J5RfunQp2dnZ1K9fn379+vHqq6+ye/duADZs2MDvv/9eYJtTTz2VCV5DwcyZM2nQoAG1a9cukK9v374888wzucsLDuenCiIlJYX58+cD5H45lJU2bdqwatUqVq9eDcDbYQpja0IfBZx2GjRqVPrBU1u2uMiU3btbZEoj/AwdCuPHuwqEiPs/fnzZ3YMBH31qaiqXXnopr7/+OnFxcfTt25f09HR69OjB8ccfz0UXXURGRkaB7e+//37mz59Px44dufvuu3n99dcLPc7TTz/NvHnz6NixI+3ateOFF14Iyb5Ro0Zx66230rVrV+LCFAmwWrVqjBs3jv79+9OlSxdq1apFnTp1yrxfi3UTJdx+OzzzDGzcCPXqlWzbP/wBJk92sWwsaJkRCkuWLKFt27Z+m1EpCbQPqCo33ngjrVq14rbbbsuTp7DrY7FuYoC0NNfFsqTzyQYiU957r4m8YUQDL730EqmpqbRv356dO3eWqS0igNXoowRVaNPGhSzwepkVi0WmNEqL1egjG6vRxyiB+WRnzoQNG0LbZuRIi0xpGIYJfVQRmE928uTi886Z43z6FpnSMIyQhF5E+ovIzyKyQkTuLmT9lSKyWUQWeH9/DFqXHZT+QTiNr2y0aQOdOxff++bgQYtMaRjGIYodMCUiccBzwFnAeuA7EflAVRfny/q2qt5UyC72qWpq2U01wLlv/vIXWL4cWrUqPM/DD7vIlB99ZJEpDcMIrUbfDVihqqtU9SDwFnBu+ZplFMWllzp//aRJha9fvNgNVElPD09sHMPwk/feew8RYenSpblpmzdvpnv37nTq1IlZs2Yxbty4Mh3j/PPPJzU1lZYtW1KnTp3cvvuhhEUO5oUXXuCNN94oky3lRShCfyywLmh5vZeWnwtF5EcReUdEmgSlJ4nIPBH5VkTOK4uxhnPHnHaac9/k7zBlkSkNv5iwcAIpY1Oo8tcqpIxNYcLC8ISunDRpEqeccgqTgmo2n3/+Occffzw//PADTZo0KbHQq2pu6AKAd999lwULFvDyyy9z6qmnsmDBAhYsWFDi2PHXXXddqWLUVwThaoydAqSoakdgOhA8BC3Z6/KTDowVkRb5NxaRYd7LYN7mzZvDZFLskp4OP//sukwG8/zzrhF27FiLTGlUHBMWTmDYlGGs2bkGRVmzcw3Dpgwrs9jv3r2b2bNn88orr/DWW28BLjzBnXfeyfvvv09qaip33XUXK1euJDU1lTvuuAMoPITx6tWradOmDZdffjkdOnRg3bp1RR4X4LXXXuOmmw55ogcOHMhML3BPzZo1GTFiBCeccAInnXQSmzZtAtxI3McffxyA3r17c9ddd9GtWzdat27NrFmzANi7dy+XXHIJ7dq14/zzz6d79+5URHfyUIKabQCCa+iNvbRcVHVr0OLLwKNB6zZ4/1eJyEygE7Ay3/bjgfHg+tGHbn7l5MIL4aabnPumc2eXZpEpjfJi+CfDWbCx6Pgv367/lgPZeUNV7s3cyzXvX8NL8wuPU5x6dCpj+x/+s/P999+nf//+tG7dmvr16zN//ny6dOnC6NGjmTdvHs8++yyrV69m0aJFufFpigph3LRpU5YvX87rr7/OSSedVMIzkJc9e/Zw0kknMWbMGO68805eeuklRo4cWSBfVlYWc+fOZerUqfz1r3/ls88+Y9y4cdStW5fFixfz008/kZpaMc2XodTovwNaiUgzEUkEhgB5es+IyDFBi4OBJV56XRGp6v1uAPQE8jfiGiWkfn03n+ykSW4+2UBkSlWLTGlUPPlFvrj0UJk0aRJDhgwBYMiQIXncN0URHMK4c+fOLF26lOXLlwOQnJxcZpEHSExMzJ0MpEuXLrkByPJzwQUXFMgze/bs3DJ16NCBjh07ltmeUCi2Rq+qWSJyEzANiANeVdVFIjIaN3XVB8AtIjIYyAK2AVd6m7cFXhSRHNxL5ZFCeusYpSA9HT78EGbNcgOoPv4YnnrKIlMa4ae4mnfK2BTW7CwYpzi5TjIzr5xZqmNu27aNGTNmsHDhQkSE7OxsRITHHnvssNsVFcJ49erVeUITF0dwCGLIG4Y4ISEB8WpThwsvHAhDXN4hiEMhJB+9qk5V1daq2kJVx3hp/+eJPKp6j6q2V9UTVPV0VV3qpX+jqsd76cer6ivlV5TKxeDBkJAAAwY4V01iopugxDAqmjF9xlA9IW+c4uoJ1RnTp/Rxit955x0uu+wy1qxZw+rVq1m3bh3NmjXL9XUHqFWrVp7IlaGGMC6OlJQUFixYQE5ODuvWrWPu3LmlLkswPXv2ZLI34nHx4sUsXLgwLPstDpt4JEp57z3nttm3zy0fPOjcN1WqmI/eqFiGHu9uuBGfj2DtzrU0rdOUMX3G5KaXhkmTJnHXXXflSbvwwguZNGkS3bt3z02rX78+PXv2pEOHDgwYMIDHHnuMJUuW0MMbDl6zZk3efPPNEocR7tmzJ82aNaNdu3a0bduWzoHGsDJyww03cMUVV9CuXTuOO+442rdvH5YwxMVhQc2ilMPN6lOEy9AwQsaCmpUP2dnZZGZmkpSUxMqVKznzzDP5+eefSUxMLNF+ShrUzGr0UcratSVLNwzDf/bu3cvpp59OZmYmqsq4ceNKLPKlwYQ+SmnatPAafdOmFW+LYRihUatWrQrpN58fi14ZpVTEPJ1G5SbS3LqGozTXxYQ+SimveToNAyApKYmtW7ea2EcYqsrWrVtJSkoq0XbWGGsYRgEyMzNZv359nv7jRmSQlJRE48aNSUhIyJNujbGGYZSIhIQEmjVr5rcZRpgw141hGEaMY0JvGIYR45jQG4ZhxDgR1xgrIpuBQnqIh0wDYEuYzIkWKluZK1t5wcpcWShLmZNVtdCZKCJO6MuKiMwrquU5VqlsZa5s5QUrc2WhvMpsrhvDMIwYx4TeMAwjxolFoR/vtwE+UNnKXNnKC1bmykK5lDnmfPSGYRhGXmKxRm8YhmEEYUJvGIYR40SV0IvI0X7bYBhG2ZHA7NpGhRA1Qi8ijYERInKl37YY5YM9/JUDEakLnOH9Pl1EwjMhawQjInVEpLr3O7mi7/Voil65G1gGnCAiQ1V1gt8GlTciIqqqIhKvqlleWpyqZvttW3mgXs8AEbkOqIUbJXhvrJYX8lzjVKA6sFlVl/ttVzlTDegrIiNxGtTfZ3vKFRFJBLoBHUSkBlAfGAHsrSgbIr5GLyJNRKShqu4A/gH8BPQQkT/4bFq54wnAIOB5EXlRRBJVNVtESjalfRQhIjcClwKfAunA3f5aVL541/g8XLe6i4DHReRsn80qF0SkCoCq/grsBDoBP6rqnuD1sYaqHgR+xd3P1wITVHVvRT7HEX1iRaQrLu7NNBEZAvRW1VeARUArEbncVwPLGRE5Abgf+BhX25svIlVjVey9B70NMBDoAywA/iYi1Xw1rBwRkSOBPwGn4yoxRwP/jbXr63255Hi/2+EqbUOAeBF5BEBVc2KpHS6fe2Yx8DXuWT5HRJpU5JdqxPejF5EPgbOBPwJDgZ+BBFzgn2OAKar6b/8sLB9EpAMwHFirqqO9tNeAE4CTVPWAj+aFhYDbImg5Hvgnzm2zG7hSVfd7tfyNMXqdjwYeAFbiXnBXqOpKEekFLFPV33w1MMyIyE3ATcApwFacS+NGXIXuJ+BE4D5V3eebkWFGRG4D6qrq/4lIJ5yOZeO+Vk8E4lR1TnnaEJE1ehHpJSJjAVR1IO4teJ6q9sHVBH4DTgUuB272/F6xSG2gvVcDQlWvBJYD/xMPP40rC8EiLyL9vAa5Y4GncLX5VzyR/wNwA/CDf9aGj8A1E5GaXnvLRmA7cA3wZ0/kTweeA+r4aGrY8dyQVwG9VHULkIJrdxsLNALuAV6NQZG/CJjsJS0AJgIKzATewLl1yteOSKzRe7WcpcBLqnqHlzYL2K2qA7zlRrga/a5YaLwKapQ7wUvaiROAp3BfMe+q6lIv7wmq+j+fTA0rInILcAnwDs5/2R/3Ev8r8BXQFrhGVRf5ZmSYEZHBOGGvClwNNMWdg5a4Ml8D3KGqH/pmZBgo5IttINAO2AwkA1cAXwB/VdU1IlJPVbf5Y2348VyOz+IaXmsCpwHnAKOBjcDJwE8VoV8RKfQAInIM8F/gP6o63Ev7AshS1bN8NS7MiEgVzz/ZH3gcmA30AsYAHwGPAeuByaq6OP8DFC14tdlgX20z4HlV7S8iDwIdgQtUNUtEjsV93uLVemMC70X+HHAXTtw7AtcBGcCZuF4oy1V1VrReZyjwxdYQyMSV7Unv/z+AH4EncI2TH/lla7go7HqJyD+A5sAuDlVcqnhf5xWHqkbEH64x6jtcw9RJXlp9XG32iaB8PwDv+W1vmMpcH4j3ftfFCXwfb7kdsAnnt22B+8Rr7rfNZSxvtaDfvXE1nKdxn+wfBtbjet008tveMJW5CXCG97s5TuDGB61/AJgBHO+3reVU/tu8azsTuCVwv3vrzgHmASl+2xmGckrQ73TgeuAyb/nkwP0MnAV8ANSqUPv8PkFe4ROAYTjf+yxgBfAIrjHyOJwb446g/Ml+2xyGMtcA7gguC64hMiVo+SLgWe93bb9tLmN5zwEmBZXrE+8cfAxsCMp3Ja7m08Bvm8NQZgH64hrQqwNJnrB/AvQLyvc33NdrTb9tDkeZg35fAkzHtQW+hOsyG/AipANzYu0Fh/s6m+u91D4Fpget+zPOR1/hZfbddeM1PAX8VpfjHn7F+e7uxnWlTMfVhv5PVR/0ydSw4vUwqYN7+K8AHsb581qqaj8vz0XAhcAfcF2uc3wyt0yISD/ci/tqnK/yelybw79EpD3u2sfhvtYG43rbLPTL3nDidRmtBbyNu77TcPd1beBTVZ3u5WulUd7WlM9dUx9XSYsDeuBGwg5U1UwRaYnz09dU1Q2+GRxmRCQBmAA8papfe2nvAdtU9WoR+T/gX6q6pKJt863XTVCPkXpAjqruAt4EsoCjgDqqeiGutjMcGAf8xw9bw43nk89S1a04F01r4DpVvRHYLSIzRGQUMAr4p6pmR7HI9wUmAR+o6g+40a4dcL2JEtU1sg7F1eKXAWmxIPJBg4NycP7pt3CNzb1x7TDbgfO8lyC4r9ioJkjkr8O1Q3QCXge6qmo/T+SvxTW07492kS+k11s2cBA3FiLA3cABAFUd7YfIg78hEOJxD8AROP80qrpDRF7C9ToYLCI1VPV94EMR+US9MADRTGDgiIicgXuhvYU7F+eKyA2qeqE3UrI6cJOqfhmtjXIicg6uQfl1oL6IXKSq74jIPtwD8KOITFXV/bhGuajH+1LL9q5xV5xb8ldVfU1EMnGf74rrUngHsBYOiWS0IyI9gQuAC1U1wzsHVstm/QAAGqhJREFULUSkG67v/FXAEI3ycSD5vl564xpbtwAvA/8WkQ2q+i3ua6aV1wV8r2/X2Sc/VgNgNc51MQh4O9/6usDNuGHh5wbOpx+2llP5z8P56gZ4y/G4vuPP4wQwwW8bw1DGRFy3sp7e8rW4NogLvOVzgc9woyMT/bY3TGU+BngIV3k5A9eF7iVc75LuXp404Euc776K3zaHocwB96/gvs6fxHWgOD8oz6Pe32tAW79tDnP5b/Ce5WeBVUBPoB+wEHgV545s77edvtToVXWL1396Ns53+7246JS7caPEtorIVFzNZ463TazUeOrgGp4HAb+JSHecCDzt+fjOw/WrXumjmWVGVQ+KyKOqmuktvygiOTh3Bar6H3HBni7H9co46Ke9YWIX0Bn3FbMHuFhdN8lrgIkikqaqk7xa/3aNUndcgHxfmomquk1E7gP2AyeKyGZVna2qd3r5c4PzxQIi0gLXeeB8Vf1FRE4F3se95PvhvuayNAJcVL42xnr+209wPqz/4PzV8bhPoI3AMFXN8M3AMBE0GKoO7mU2E1frScQJQj9cl9E/i0gDdaMGY4bAOAHv959w3c0+9USvpqru9tfCshMQMXGhaJ8F2gP3AZ+pc+NcjWtwP0/Lebh7ReNV2s7AfaE/DXyLC3MAME1Vv/LyRaULMoCInIjzNqxXN56lHu5a/0kPBWa7Faihqg/5aGoBfA2BoKqf4gYGZQK3Al1xXbKG4MLTxpLI98f1LgHnp9yBG/l7PXA+zoddNdZEHnKDVQUaJ18Cvgd6iUitGBF58UT+VNw9fAOuYfkcvIY5VX0VGIkbDRvViEhjEanl/b4Q15h+F/B37+8UnKumBu46J0F0f5WLyADcGIg/AX8RkWR1o3jrkHdC7+pAQx9MPDx++468a382LqDRUX7bUk7lOwNXgz+5kHXn43y4g/22s4xlDO4/Hcehr8W4IvIc4bfNYS7/QOB/HBrwFuhS+QTQpKhzFW1/uJfXFOBob/ky4MGg9T1wo7iTcb3JGvptcxjK3N97ftt4y//BdQNO8pY/xQ2CGourxERcO0RETDyiqlM9//QnItJFo9x3GUBcqFnBDRB6AvhWRIbihrqvVdVRuAdlpKp+EK2ftvl6INyKc1vsFZGRqrpbvMlSVFUDedXNLxATeJ/w1wF/VNXvRCRBXY+Ta3Bdhu8RkdvU62kSjdcYcsdDjAbuVNWN3lfaHqC111X2oKrOERdxtrr61JUwnIhITdxYluWq+rP3TJ+I+zK7TER+UNW+InKul/acRuB4iIiJXqmuG+WpsSDyQf1rc9Q1Pn2Ac03NwIngF0Bn7yYaGs0iD3n6T5+Bc7t9gJtF6BPPB5/tNUBGrcgVw37cQ17TWw7EGa+CC+fwokZ/d8LjgBeBv6vq5yKSjKu8fIxrdHxJXBTSP+FCW8SCSy5BnWvxWeA7EXkZF7LhPlU9B9fDqLuIdFPV91V1ciSKPESQ0ANo7Phr1RO9x0TkUrzBMcDVqnovrsvV0biwBvsg+gXQq9HcCDyjqh+q6rU4l9RUzxcfS70tAqGGE7x2lb24HmQdRCRFXZtED+AV3DWOhUijK3GNrG29sk0ANqnqPlU9Hxdq90ycC+siVV3nn6llx3t+7xORc7zr9zGuLXEH8G8AVf0G95Jv4JuhIRJRQh8LeCLfF3gGF7/kZg59+q3yPn8nAw+om1ItKgn6agmwAdcI1UNEagOo6g248RLvFJI/Kgl6kQ/GjQv4h9dF9k3geODvIvKot+41Vd3so7lhwXO9ZeJCdTTFBdj7VFUfDuRR1XvUhRS/VFUX+2RqWBA3lePfcO2G6wBUdS5utO+3wP0iUl1EzgJa4UKqRzS+x7qJFYIEoBpu9OO7uHgm44BBqrrBGyuQChxQ1enR6q7J55PvhQux+xuu4vAGznXzD3VhLRCRozW2Qg0PwAUnG4ob4HYxrovsf3G12qbAQo2hbpRyKJR2Au6ezsC5bn710gP3f1Te0wGCXtp/UNX/BqX3xblem+PGfvTCdbW8JBpebBHRGBsLeDf5WTjf9EHcqDjBBXL6VdykC1WB9wNujGh9IIJE/hZcY/KPuEkzxuJqfa8ASSIyTlUzol3kRaQVLgZPoHtsW1yYjuP4//bOPVzzsV7jnxszTJud85mkJNthI6Ky2SrlfLhKTiORs8GUHLcOKuRQM5VUU5oOIokKSZlsYjShISWnmhJhHGvTDGNm7v3H/bzjtVKY9Y7f+r3zfK5rXbPWe5jredf7ru/veb6H+05QP45M+e5k+4pmVjlvKcF8QUevZhQZ9T+eeCX8qfOZaOtnuotliLvZLztNBJLOIAOOk8iw43dIPWZcG4I8MDTaK/vhi0jRnk8q8muSQbAPlPs2Ice7tze9zkG+xhV5Vj//VWQHuzxpp9yIFKreSMSsfkh8Mhtf9yBf80LE/GU2cFrX7cuTHd4G5eergGkkX7tgE2vt4Wt+TqvswN9H+Xc48APSN7/Qy7W2efia30paQ/cDruq6fW2yw1+dFGWP6bz+ptf8Ur5qjn6QKCxNrvYL2r6RTPVeCGwk6Wpy1D3a9oTmVjo4JK1K5KIXKW11T5SvR53WyZtI3/imjkrlbrYfb27FvcE5fZ1C0lGbK45BOKeUKcAKkt5GtE02s/2I7Vn/9D9sAe5E+0zzniNpO0kd4cGZyhTwDDLc+Gn3R6F9C9IhdSlwr6SdSrrqNmLYPgX4I6WuWV5/a6iBfi7pKi4u7kyzjiQ6LtvY/hspxu1PxLx2tn1pywuS95MJwNeQD/5j5IL2va7HjCA7H2i5do0iV9HhTuB2ivKkpG+W228nhfbxwM9sTy7PbeX7XC7gne+3Iempm4mJxh6SVoE5wX5Bp2++tQ0FA5hIzG4eJu/3lmSYESd9szsZkmqlVHotxs4FXYWnTUhR7nO2L5O0K8nN7+o+ydUOLK5JOpQUlC8nE4HjSHCfQP4wdnUxMW8ripftFeSP+mukc2gkUdw8nBi232P7A2WAZlVH1Kq1hcgBBfa1ScriMdsTSiFyb+B64HLb9zS41J5RTmJrkYvZ/cB5RFV0GLnIrVPuv5oE+Xfb/m0jix0ktRg7F5Qgvw3ZrQ8HzpY0w3FMmkl6x7dte7DXc8XIdgMeI4F9P4qRte2Rkt5DCs/n2r6rsQX3jiWJV8B7iBPSpkSnZp1y+0HABZK+YXsfcqSnT4L8YcAJwD1Eq+Y/bf9U0ixykZspaXyfpGuGE7XRrchrfT05tV1LWimHk26q+4Czbf+pmWUOnrqjnwskrURytofZnlS6EHYHTrX9o7Kz/z/bP2l0oT1C0odIimJ/27eVFrs9SQCcSPwEnmlyjb2i67T2RtJd8Qky5bknCQrjbJ8saVmiYfOrBpfbUxRRtkOIofffyCTsEkSGd5ZisHFXH6Vr5lBOcScTjaLFiILu0mQA8KIm19YL6o5+7phK8ngCsH2WpNWAL0t6l+0Lob2yrJJWB56xfa/i77m97TdJWrQMfK1RXvPCZEjoEjI12HpKkF/A9g3KOP9XiY7NPqSzatHyuIeAh5pb6eDpuqgtACxFZDpWBFa2PVXSIaTT5CpJW9q+usHlzhM6p9aSersRWM/2DpJWIGmcWxpeYk+oO/oXQdcfxDLE/el+SWOIDdz5jsDTpqSPfATR7Pm/Jtc8tyjys0cRHY/pZDPwU3Jxe5QU8N9CRuBPBRZzHwmUwRwxOju941uRne3xti9oeGk9Y0C6ZpjTH78yOcHcDlxk+w+KBd4Z5LTaalmDF0LR7znZ9sim19JraqB/kSgj70eTI+2viSvSMWT0fybpw30PCZJntDFX3XVBW5gU495FdnSvIIMi33AMF3Yh/eMfaXC5PacMtd1n+5byc7du0QVkLuLcRhfZAwYE+YOJjeV0Is0xifTG30bMcFr3OZ5bJC1OCs77umsqth+o7ZUvAkkbAB8iwmQTeNYlaBRwJdnt7kEKdZsRS7lWMSDNNIL0yK9GrNKetH1sCfKjgI+RoNA3lJz8iUS2ApiTxpHtq0iOvnFLuF7QFeT3It0lnyXDT18nbYXHkvrL1qUeM7/wN9J503cnl7qjfx4UfXEB021PU5zs1yc794OItPAUSa/vtBIqin7fIheBVrZgwRyrv3VtHyFpXdKFcCeZdP0rsYr7uO3fNLjMniJpeVJ4vcP2wSVn7a6AqOf7vm1IWouM8n+6FFcPIzHgrHL/xsTMfEsy+fxQPxZe/xXqM1/bDnVHPwBFd/tnJGVxuaTleLbr4kDSSzul9BZ/TdKK5ak3AVu2PMi/l0gNfxagBPMxRMdmD5Kv36ufgnzhCVJQ3loZeJvd2c3Dc9smWxzkFyTv4+uA0eViNoPMBgDgTHXfQoYAb5nfgjzMmYTuO2qg70LS64iuxWdJi9ktpAh3BzELmUkMQ95HlPtOKYXZBWw/09ZilcJCpH3wo6UIt3DZvd5Kgv1S5ITT6olXeI6e/IYlL/9aspM9ETi0FGBbG9QHUj6fs2xfCvyK6DHt4/j3Pi3pR5JeVz7X69HyqebKP1JTN4US6L4JLGp7x3Lb2kTG4JjSlbA3kQBYCbiwDJK08ij/fOuWdAJR7/uYI+PQMRS5kRzj+2a3o5i1f5EU1bck/gGTSAvlvsCn+mUOooOk0cTz9WlyOrvY9jhJZ5Fp0NcARzr6LpU+ovbRFxz9jk8An5F0rO3TiCnwIcQ5aAkSDH7cXZFve5AvAW8JEuR+TaYEt5P0M2IJdySR6G19kO/qolmEFCH3d2zxtiAj/o+SE93w8n3fUFon303My58uXWS7SJpme1R5zCsct6xKn1FTN4VyvO0IV71d0vfItOu6xFhiPBmBbz1dQf5w4CNk9PsnJLjdQuRazyP5+kPbmpLqIOnfJK1YgvxaRHJ4KrBeKb5dQ7qnjiSSy+MdNc7W0klPdd9ETmvrlZ8nkNrEB0snFaTFstKH1B19oQzHLGD7dkW4awzRpe6Y/X5B8Qdtu8lzZ1e7JtmxvwU4APiL7UnAJEnnAssC0/pkGOrVwMclTSIaPaNIOmpN0kZ4HfA7Muk6rO272gEntteStNu9kr4I7C3padu3SrqNXPS+B+08nVZeHDVHzxxPzFnl+45l2utJsJ9E9C4ea3SRg6SknmaQGsRUxfLwWFKIXAbYtrTcvR+4xH3gddqNpE+TNNyxtj+vTDmPJrMCi5Dfw0m2WylD22FAkB9N7A6HkUG+J8mcx4HkBLc9sPX8NBQ1vzLfBvqune3mZPd6me2nyn2dYL826Rs/wDEeaCWK2fFBRKFvESIxfDrpHNqI5ODvkbQnscXb3vafm1pvr+h6j5cm+ekVSHA7ykW3pbzHSxMRupvbWlwfSNd7vjvxON2KDERdQwrOywM32/5DU2usvHzMl6mbrkD+DuBLwHs7QR6ek8a5TdKOtv/e3GoHR3mNpxPziIdIsP8BYKJh8hngE6XPeh1gz34I8jBnsnUn0jk1qlzM7gPGShpJJiHfYXtM93MaWm7PUDxu9wMWsD2diO3NIAXoRUnHWKtdsCovjfkq0EtaxvbDJZAvRo7yo2xf1xX85TC7PK21+VpFo+UiYP3SG98Rr9oM+AUxW9iHFJyXIzMDfRHkYY6swUcoQR7A9lckmUz6ziDF91bzPKeQB4g2z/6SDrL9ZdvjFQ2jnUhL6ZNNrLXSDPNNoC8f8lGKWcQU209ImkqKUZBui9nAayU9aPsJaP0O7xEiSLYh8AdiGjG8BP3dgeOB77vPBJy6WBa4xvYvSk3iafKWflXSdcQ4pbWTzPAPOfndgafIYNuFpfHmHeUh42x/SdL5tmuQn8+Yn9orZ5FUxfRSmIOkMt5dgt8zinjZGOCV/+w/aRPOVOsmwDhJh5SAMLOkaaaRdrrWpqUG8jwthc8AGwDYnl5OaW+WtLXtO9oe5OE5rbJHkvbQEcDnJe3r+CJcAbxV0r7lKa0T3KsMnr7f0Zed3CK2Hy9FuRWBVctQ1EcknQecL+lx4A1EsOu+JtfcS2zfpIz0X1l2f2fDHE2fv5PhoNb3T3elpd5JJJYXIHK7B0q6jJxeViFDbwc0t9LeI2lDYBsy4Xs0sQE8trQDf0mxt7wBWn9Crcwlfd11U3Z4byQdF3cB/wV8mOi2HAXcavs0RXlyWeB+2zf2S+dFN4oC55Wkte5x4ExgZNt3tZJW6Qx0SdoOOIkEu7OAH9o+QdJXy8NXAcba/nEzq+0NpcayLvGqnUhOZ8sCbyaa+Zsp7lCfAw62fU5ji60MCfo60EOmIoGvEVuwUba/rYzAr0M6UR60fUyTa3y5KMH+BpKy2tKZBG41kr4BPG57tKSxJMCvTYL93rb/2PXYV7po+LSVcmI5k8hVCLiDiOvNUtRHl7I9RtL+5ML2Ldu/b27FlaFA36Zuunbl04FbSZfBtpJusn0ncJOks0lnwlr9EPReiJLGWQeYVX4H/cDFJAe9AEnXnAisTFyC/qgYtQ+3/W0y8t9aShfVhcCath+QtAOZeei0Sk4HdlA8f3cENu90G1Xmb/pyR981KLMzyV0eQQaFRpMj7/tJP/nmwBXujzH/+RJJyxIJ6ZOAu0kq40Db50p6MznNHWD72gaX2RMkrQfcTE4q55XbrietorfZvqwMSo0Afjc/bF4qL46+DPQAkrYBTgGOtj2h5OuXIxK0uwBLkl1f6wPA/EQJ7JvavqTrto7K6N5AJ7VxBanPfNj2ZU2sdV7QVWv5IJn03ZWYeb+CpCM/A4xzH6iNVnpH3wR6SSsAu9keW34+mejU3ExaDPcgevPXkgDwpO2JDS23MheUttAjiKb6UyRN8xfSMvhN4Au2f17SU9NJyub2fiuuK5Z/PyW1idW7bt+ByBr0TddYpTf0U6Bfk+Ro/1ryl0cQpcIlgUvJ7mdJIrtbB0ZaShl8mwmMJbvYFYg42w7kArB1Z9itnylpnGuAI2x/q+n1VIY2/VSMvYuo9I2XdL/to8v04xO271bkWr9N0jc10LeQkn6bWYqPh0t6FREpOwe4mtRftiAj/n2NIzO8FXCDoqk/vuk1VYYu/TQZu5rjZ3oasJykU21PLkF+FyLkdYqrWl+r6J52LRpEs0oKB9v32P4CES27CXiQCJXNFzjmKG8Arm96LZWhTatTN13dNWuQ9Mz4MgC1NjnOP2j7GEV+90HbV/VbvrafKfMOO9r+rqQtgTfYPvNfPH7BciGo73Gl0kWrUzclyO9Eui1+CYwso/CflPQp4ip0iu0Tup/T1HorLw3bT0laQ9KfSMF1/xd4yuzyvPoeVypdtH1HvzjPtppNJDnaLxJFxjMkrUus4SY3uMzKXKBnZaNXJzn4ZWyvU+77B0ewJtdaqQx12p6jn0WkeKeUP/bfAucCB0s61PZvapBvHyX1MlvSpsSkfHdggqSJklYo6ZlXQUxiGl1spdICWhXoO4U5SSsWZb4nSK/8RZJGlF3en4lV3jYlV19pGSUl9zZyUvuk7am2RwO3ABdI2gP436LAWalUXoBW5ehLANga+Chwd+m+OIHY4k2WdA4ZqNmbmCIP1CevtIfFieronKE224dJOo6oNB5u+46mFleptIlW5ei7umsOAKYCOxNrtK3JsMwwYDKwGHA2sLP7yBqvn+nqoFqOzD5MK7LDFwPb2Z7Q9dhFSqG2dtdUKi+CIb+jH/DHPAv4he1rSxHu9JKr3bGoE3bGw8cSHZsa5FtCCfI7AIcDljQR+DK5kF8gaS/bV5THPtV5TmMLrlRaxJDN0Ut6JcwJAJ0UzBPA5pIO6yrCPQos3/XUh8hO/tcv32org0XSa4DTiY78meS9Pol0VR0OXCxpqSJHXKlUXgJDckdf9EwmSzrL9pgS7IfZfrjoi59fRMxuIrrbozvPrfrb7aLrxLYEcE/nAi3pz0R87m22z5N0te1Hm1xrpdJWhuTuyPbTwEjgeEkHl9ueKcF+MhErWxnYmMjQXt3YYitzRdcpbUT597fEuHwUgGOMch/wH+X+qQOeV6lUXiRDuhjbpb19vGNy3BlxXx14E3BeJ7VT87Xto3RQHQBMIW2yBt4CLAp8h+To32e7arlUKoNgSO7oOxTRpq2AU8sA1CxJ/018T6d2gnsN8u2jDEMdD5xPtOM3BjYAvk5SijsCR9UgX6kMniG9o+9QdvaXA98nMrQn2L642VVV5hZJKwEXANcX0blFgc1I3/xxth8p0rsz62mtUhk8Q3pH36Hs7LcD9iQ5+YtVaHhplbljGnAdsLukTWw/WVonVwXWA3CxwqtBvlIZPEOy6+b5sH1j0Tl5su7y2kXXMNSGwErArcQ34F6iMDoW+D2wClCN2iuVHtOaQF/4e9MLqLx0SpDfCvgK2cl/DPgc8HNgKSJcdiOwn+3J9UJeqfSWVgX6WnxtJ4qf78HAXrYnStqZpOLuB04GHgPeWv6tVCo9phU5+kr7GFA/2RlYmxRcsf0Dkr45rtx/ARl++2hxlapUKj2kVTv6Snso6ZqNgTUce8cZwOqSdrL9Q6JK+XZg4TLxPA6Y3dGxqVQqvaMG+so8oezolwCOkDSb5OSPAo6StBuwOjFrnwZg+5HGFlup9Dk10Fd6TpGqeEbS1cCHKSmaojY6i1g+nmP7kvL4WnytVOYhNUdfGTTF8evVkhYsrk/XSFrW9gzgGqJKeXgRpBsL3AZsWDpxanG9UpnH1EBfGRQlsF8JbASMKK5PdwPflbR0CfbXkQ6bo4mOzXnAnaQgW6lU5jGtkECoDE0krQZcBoyxfU4xg5ld7htDZIb3IoNQhwAf79j/dT+2UqnMW2qgr8w1kvYF1rd9ZDEEWY/4ud5LLgD/A6xPcvLH2f5+eV7NyVcqLyO1GFsZDFOA/SW9E9iNaMuvQ3x7t7d9kKRlgIVsP9AJ8DXIVyovLzVHXxkMNwIXEt2afyeG7JsTK8BFJA23/bDtB6AWXSuVpqipm8qgkbSk7ce6ft6CSBvs2gnylUqlOeqOvjJoOkFe0jBJ25LhqFNrkK9UhgY10Fd6gqRhpMvmg8CJtn/U8JIqlUqhpm4qPaME+6VsP1g7ayqVoUMN9JVKpdLn1NRNpVKp9Dk10FcqlUqfUwN9pVKp9Dk10FcqlUqfUwN9pVKp9Dk10FcqlUqfUwN9pVKp9Dn/D9IQyiFP4i5GAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "ax = plt.subplot()\n",
        "\n",
        "plt.title(\"Comparison of Model Performance in terms of F1 score\\n\")\n",
        "\n",
        "p1 = plt.plot(df_perf['Model'], df_perf['F1 score before Tuning'], marker = \"o\", c = 'b', label = \"Before Tuning\")\n",
        "plt.plot(df_perf['Model'], df_perf['F1 score after Tuning'], marker = \"o\", c = 'g', label = \"After Tuning\")\n",
        "\n",
        "plt.figure(figsize=(400,400))\n",
        "plt.setp(ax.get_xticklabels(),rotation = 45, ha = 'right')\n",
        "ax.legend()\n",
        "plt.grid()\n",
        "\n",
        "plt.show()"
      ],
      "id": "19498448"
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.11"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}