Gridworld.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The agent-environment interaction\n",
    "\n",
    "In this exercise, you will implement the interaction of a reinforecment learning agent with its environment. We will use the gridworld environment from the second lecture. You will find a description of the environment below, along with two pieces of relevant material from the lectures: the agent-environment interface and the Q-learning algorithm.\n",
    "\n",
    "1. Create an agent that chooses actions randomly with this environment. \n",
    "\n",
    "2. Create an agent that uses Q-learning. You can use initial Q values of 0, a stochasticity parameter for the $\\epsilon$-greedy policy function $\\epsilon=0.05$, and a learning rate $\\alpha = 0.1$. But feel free to experiment with other settings of these three parameters.\n",
    "\n",
    "3. Plot the mean total reward obtained by the two agents through the episodes. This is called a **learning curve**. Run enough episodes for the Q-learning agent to converge to a near-optimal policy. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The agent-environment interface\n",
    "\n",
    "<img src=\"img/agent-environment.png\" style=\"width: 500px;\" align=\"left\"/> \n",
    "\n",
    "<br><br><br>\n",
    "\n",
    "The interaction of the agent with its environments starts at decision stage $t=0$ with the observation of the current state $s_0$. (Notice that there is no reward at this initial stage.) The agent then chooses an action to execute at decision stage $t=1$. The environment responds by changing its state to $s_1$ and returning the numerical reward signal $r_1$. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The environment: Navigation in a gridworld\n",
    "\n",
    "<img src=\"img/gold.png\" style=\"width: 250px;\" align=\"left\"/>\n",
    "\n",
    "The agent has four possible actions in each state (grid square): west, north, south, and east. The actions are unreliable. They move the agent in the intended direction with probability 0.8, and with probability 0.2, they move the agent in a random other direction. It the direction of movement is blocked, the agent remains in the same grid square. The initial state of the agent is one of the five grid squares at the bottom, selected randomly. The grid squares with the gold and the bomb are **terminal states**. If the agent finds itself in one of these squares, the episode ends. Then a new episode begins with the agent at the initial state.\n",
    "\n",
    "You will use a reinforcement learning algorithm to compute the best policy for finding the gold with as few steps as possible while avoiding the bomb. For this, we will use the following reward function: $-1$ for each navigation action, an additional $+10$ for finding the gold, and an additional $-10$ for hitting the bomb. For example, the immediate reward for transitioning into the square with the gold is $-1 + 10 = +9$. Do not use discounting (that is, set $\\gamma=1$)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Q-learning\n",
    "\n",
    "![title](img/q.png)\n",
    "From Sutton & Barto (1998), Reinforcement Learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import operator\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classes for the Enviroment and the Agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The GridWorld class contains the environment\n",
    "- The dimensions of the environment are defined\n",
    "- Locations of all rewards are stored\n",
    "- Functions for different methods written\n",
    "    - `get_available_actions` returns possible actions\n",
    "    - `agent_on_map` prints out current location of the agent on the grid (used for debugging)\n",
    "    - `get_reward` returns the reward for an input position\n",
    "    - `make_step` moves the agent in a specified direction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "class GridWorld:\n",
    "    ## Initialise starting data\n",
    "    def __init__(self):\n",
    "        # Set information about the gridworld\n",
    "        self.height = 5\n",
    "        self.width = 5\n",
    "        self.grid = np.zeros(( self.height, self.width)) - 1\n",
    "        \n",
    "        # Set random start location for the agent\n",
    "        self.current_location = ( 4, np.random.randint(0,5))\n",
    "        \n",
    "        # Set locations for the bomb and the gold\n",
    "        self.bomb_location = (1,3)\n",
    "        self.gold_location = (0,3)\n",
    "        self.terminal_states = [ self.bomb_location, self.gold_location]\n",
    "        \n",
    "        # Set grid rewards for special cells\n",
    "        self.grid[ self.bomb_location[0], self.bomb_location[1]] = -10\n",
    "        self.grid[ self.gold_location[0], self.gold_location[1]] = 10\n",
    "        \n",
    "        # Set available actions\n",
    "        self.actions = ['UP', 'DOWN', 'LEFT', 'RIGHT']\n",
    "    \n",
    "        \n",
    "    ## Put methods here:\n",
    "    def get_available_actions(self):\n",
    "        \"\"\"Returns possible actions\"\"\"\n",
    "        return self.actions\n",
    "    \n",
    "    def agent_on_map(self):\n",
    "        \"\"\"Prints out current location of the agent on the grid (used for debugging)\"\"\"\n",
    "        grid = np.zeros(( self.height, self.width))\n",
    "        grid[ self.current_location[0], self.current_location[1]] = 1\n",
    "        return grid\n",
    "    \n",
    "    def get_reward(self, new_location):\n",
    "        \"\"\"Returns the reward for an input position\"\"\"\n",
    "        return self.grid[ new_location[0], new_location[1]]\n",
    "        \n",
    "    \n",
    "    def make_step(self, action):\n",
    "        \"\"\"Moves the agent in the specified direction. If agent is at a border, agent stays still\n",
    "        but takes negative reward. Function returns the reward for the move.\"\"\"\n",
    "        # Store previous location\n",
    "        last_location = self.current_location\n",
    "        \n",
    "        # UP\n",
    "        if action == 'UP':\n",
    "            # If agent is at the top, stay still, collect reward\n",
    "            if last_location[0] == 0:\n",
    "                reward = self.get_reward(last_location)\n",
    "            else:\n",
    "                self.current_location = ( self.current_location[0] - 1, self.current_location[1])\n",
    "                reward = self.get_reward(self.current_location)\n",
    "        \n",
    "        # DOWN\n",
    "        elif action == 'DOWN':\n",
    "            # If agent is at bottom, stay still, collect reward\n",
    "            if last_location[0] == self.height - 1:\n",
    "                reward = self.get_reward(last_location)\n",
    "            else:\n",
    "                self.current_location = ( self.current_location[0] + 1, self.current_location[1])\n",
    "                reward = self.get_reward(self.current_location)\n",
    "            \n",
    "        # LEFT\n",
    "        elif action == 'LEFT':\n",
    "            # If agent is at the left, stay still, collect reward\n",
    "            if last_location[1] == 0:\n",
    "                reward = self.get_reward(last_location)\n",
    "            else:\n",
    "                self.current_location = ( self.current_location[0], self.current_location[1] - 1)\n",
    "                reward = self.get_reward(self.current_location)\n",
    "\n",
    "        # RIGHT\n",
    "        elif action == 'RIGHT':\n",
    "            # If agent is at the right, stay still, collect reward\n",
    "            if last_location[1] == self.width - 1:\n",
    "                reward = self.get_reward(last_location)\n",
    "            else:\n",
    "                self.current_location = ( self.current_location[0], self.current_location[1] + 1)\n",
    "                reward = self.get_reward(self.current_location)\n",
    "                \n",
    "        return reward\n",
    "    \n",
    "    def check_state(self):\n",
    "        \"\"\"Check if the agent is in a terminal state (gold or bomb), if so return 'TERMINAL'\"\"\"\n",
    "        if self.current_location in self.terminal_states:\n",
    "            return 'TERMINAL'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "class RandomAgent():        \n",
    "    # Choose a random action\n",
    "    def choose_action(self, available_actions):\n",
    "        \"\"\"Returns a random choice of the available actions\"\"\"\n",
    "        return np.random.choice(available_actions)   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Q_Agent():\n",
    "    # Intialise\n",
    "    def __init__(self, environment, epsilon=0.05, alpha=0.1, gamma=1):\n",
    "        self.environment = environment\n",
    "        self.q_table = dict() # Store all Q-values in dictionary of dictionaries \n",
    "        for x in range(environment.height): # Loop through all possible grid spaces, create sub-dictionary for each\n",
    "            for y in range(environment.width):\n",
    "                self.q_table[(x,y)] = {'UP':0, 'DOWN':0, 'LEFT':0, 'RIGHT':0} # Populate sub-dictionary with zero values for possible moves\n",
    "\n",
    "        self.epsilon = epsilon\n",
    "        self.alpha = alpha\n",
    "        self.gamma = gamma\n",
    "        \n",
    "    def choose_action(self, available_actions):\n",
    "        \"\"\"Returns the optimal action from Q-Value table. If multiple optimal actions, chooses random choice.\n",
    "        Will make an exploratory random action dependent on epsilon.\"\"\"\n",
    "        if np.random.uniform(0,1) < self.epsilon:\n",
    "            action = available_actions[np.random.randint(0, len(available_actions))]\n",
    "        else:\n",
    "            q_values_of_state = self.q_table[self.environment.current_location]\n",
    "            maxValue = max(q_values_of_state.values())\n",
    "            action = np.random.choice([k for k, v in q_values_of_state.items() if v == maxValue])\n",
    "        \n",
    "        return action\n",
    "    \n",
    "    def learn(self, old_state, reward, new_state, action):\n",
    "        \"\"\"Updates the Q-value table using Q-learning\"\"\"\n",
    "        q_values_of_state = self.q_table[new_state]\n",
    "        max_q_value_in_new_state = max(q_values_of_state.values())\n",
    "        current_q_value = self.q_table[old_state][action]\n",
    "        \n",
    "        self.q_table[old_state][action] = (1 - self.alpha) * current_q_value + self.alpha * (reward + self.gamma * max_q_value_in_new_state)\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def play(environment, agent, trials=500, max_steps_per_episode=1000, learn=False):\n",
    "    \"\"\"The play function runs iterations and updates Q-values if desired.\"\"\"\n",
    "    reward_per_episode = [] # Initialise performance log\n",
    "    \n",
    "    for trial in range(trials): # Run trials\n",
    "        cumulative_reward = 0 # Initialise values of each game\n",
    "        step = 0\n",
    "        game_over = False\n",
    "        while step < max_steps_per_episode and game_over != True: # Run until max steps or until game is finished\n",
    "            old_state = environment.current_location\n",
    "            action = agent.choose_action(environment.actions) \n",
    "            reward = environment.make_step(action)\n",
    "            new_state = environment.current_location\n",
    "            \n",
    "            if learn == True: # Update Q-values if learning is specified\n",
    "                agent.learn(old_state, reward, new_state, action)\n",
    "                \n",
    "            cumulative_reward += reward\n",
    "            step += 1\n",
    "            \n",
    "            if environment.check_state() == 'TERMINAL': # If game is in terminal state, game over and start next trial\n",
    "                environment.__init__()\n",
    "                game_over = True     \n",
    "                \n",
    "        reward_per_episode.append(cumulative_reward) # Append reward for current trial to performance log\n",
    "        \n",
    "    return reward_per_episode # Return performance log"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run Random Agent\n",
    "\n",
    "- Random agent moves randomly and does not learn from it's actions.\n",
    "- This gives a base performance to compare the Q-Learning agent to"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current position of the agent = (4, 1)\n",
      "[[0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0.]]\n",
      "Available_actions = ['UP', 'DOWN', 'LEFT', 'RIGHT']\n",
      "Randomly chosen action = RIGHT\n",
      "Reward obtained = -1.0\n",
      "Current position of the agent = (4, 2)\n",
      "[[0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "env = GridWorld()\n",
    "agent = RandomAgent()\n",
    "\n",
    "print(\"Current position of the agent =\", env.current_location)\n",
    "print(env.agent_on_map())\n",
    "available_actions = env.get_available_actions()\n",
    "print(\"Available_actions =\", available_actions)\n",
    "chosen_action = agent.choose_action(available_actions)\n",
    "print(\"Randomly chosen action =\", chosen_action)\n",
    "reward = env.make_step(chosen_action)\n",
    "print(\"Reward obtained =\", reward)\n",
    "print(\"Current position of the agent =\", env.current_location)\n",
    "print(env.agent_on_map())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Here the random agent is ran for 500 trials\n",
    "- Performance is obviously inconsistent and not optimal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10da07048>]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD8CAYAAACVZ8iyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXe8HVW593/PLqemN9IbhBIQohwi\nTUGkW7DBhVtQr1wU5V7bvQKiqCiKoqhcEcUrgr4gvSggvdeQkJCEkISQQkICCek5bbf1/jGzZtas\nWVP2nr3PPmef5/v5JGfv2VPWzKy1nvWU9SwSQoBhGIYZ3KTqXQCGYRim/rAwYBiGYVgYMAzDMCwM\nGIZhGLAwYBiGYcDCgGEYhgELA4ZhGAYsDBiGYRiwMGAYhmEAZOpdgLiMGTNGTJ8+vd7FYBiGGTAs\nWLDgXSHE2Dj7DhhhMH36dMyfP7/exWAYhhkwENG6uPuymYhhGIZhYcAwDMOwMGAYhmHAwoBhGIYB\nCwOGYRgGLAwYhmEYsDBgGIZhUEdhQEQnE9EKIlpFRBfWqxwMwzBBPLVyC9Zt7ax3MfqEuggDIkoD\nuBrAKQBmAziLiGbXoywMwzQ+H/jZY7jmiTfKPu7s6+bhhF8+VdWyPLlyCwrFkm/7c6veRXeuWNVr\nlUO9NIO5AFYJIVYLIXIAbgZwWp3KwtSJb976Cv78/Np6F4MZBKzf1o2fPrAcALBuaydOuPJJvLun\nN/SYYkkAAHIFf8ddKU+u3ILPXjcPv9UE0+bdPfjn/3sRX715YdWuVS71EgaTAKxXvm+wt3kgonOJ\naD4Rzd+yZUufFY6xKJUEvnv3Ury2aVdVzrd5dw+EEM73O17egEvuebUq5+5LXlq7DZfdt6ysY7py\nBezuydeoREw5XPvUary+eQ/+sWRT6H67uqv/vt7Z1QMAWKuZnnrzlsB56vX69XP1EgZk2CZ8G4S4\nVgjRIYToGDs2Vq4lxsakhpbLxp3d+MsL63DODZXnhFr45nbc9OKbWPNuJ+Ze9ij++MyaxOWqN6f/\n7nn84ek16MoVYh9z2I8ewXu+/1ANS9XY5AolXP6P5dhVBYFasgckqZSpG3KpxrV82L1cirzX7skX\n7b/V00LKpV7CYAOAKcr3yQA21qksDcedL2/APhf/A+u3dSU6T6kK9fKTv30O375rCbbstlTyfyx9\nO/lJDezszuNPz65BqeQbU9SMTTt7Yu/bWUdbcCNw98K38Lsn38CVD60s+1i9Tsh6rXfIOjttzaAp\nU71uUtjSQL9yd77+9aNewuAlALOIaAYRNQE4E8Df6lSWAceCddsx/cL7sGrzHuPvf3vFkquvb96d\n6Do9BauCpqpQS1qzaQC1Ub0B4OcPrsAP/r4MT6zcXJPzA9boTTUtbNoRXxiE0dlbwIOvxhOST6zY\njG2duapcdyCRt3vw3grs90WhCQOpGRhkwYbtXXj+ja0AXGHQ1pQu+5pByKLockjVCPLFEh5fsRnb\n+/g910UYCCEKAM4H8CCA1wDcKoToV8bj065+tmKTxk/ufw3n/rl26bbvWfQWAODZVe8af3cqnNEa\nF58uezQbNYLSKZYEPvCzxxyhBLiNuSaqN6wGBADrtibThsK4/B/Lcd6NLzvfN+3srsp5L7nnVXzx\nLwuwbGO4b6YnX8Tn/vQSPnvdvKpcd2BSvuZX1DSDoiMM/PX66J8+jrP+8AIARRhkw4XBts4c3vO9\nB7Hwze2RZSkFtE1VM1i/rQuf/9NL+PcbXoo8XzWp2zwDIcT9Qoh9hRB7CyEuq1c5gnhl/Q788N7y\nnISS3z+1Gg8te6fKJXLJF60albaHNrt78ti8yzBKTSYLHJt4ukxh0FsoYv22bnzr9lfcc/ValX1X\nd3w7ezm02A129RZ/TPgtL72Jy/+xPPE1Nmz3dv5vl2EmCkM6FaOiW3K2wFv+dnUc+gOJJAMbXRiI\nALu9jhQGrRGawYurt2J3bwG/ezI8dLVQLGHdNqt++jUDVxjIerXi7WSafbnwDOQBSNEeZWdsYfDR\n/30Gc3/8KHryRVx055LQTiVXKMWOZZYVNI4s6C0U0Z2z/snIiELRbYR7eq2GlcQ2atIqevJFfPuu\nJXhji2UyM5nOLrhjSWRDjYNuVnhnd7AwEELE0oJ29eQdM0Rnb7iglCGO+aLAxXctGZTmokrQzURS\nOKS1F9pbcOumEMIZuLQ1ha8BRnYDiXJX/ezBFfj9k6vtY7y/qcJgoy0MCn3o/wIGmTDY01vArx5Z\n6XSIVz680vMStu7pxW8ee71sJ+StL62v6mitVBIQQuC6Z9Zg5Tv+0UFBq8zSNPLY8s3467w38apt\nbjD14Z+65lkccMkDscohzUR6ozHxkauewQGXPIADLnkAH7nqaU85AWB3TzKN4KFX38bB338IC9Z5\nVfH7l2zCTS++iadft0xmlZihnlq5BQ/FsNnrI8muEKF6w3NrcfD3Hwp14s9bsw0Hf/8hvLR2G4Bo\nJ3NeiRC78cU38fOHVgTuu2DdNty3ODh0Mu491woh+q6jKyqDEiGE4zPQO+S177rvqrdQckKB9fp/\n3TNrPO9Vnifqlp5aqYaNmqOJAGDTDksD1TWaWjOohMG1T63Grx55HWf8/nlcdv8yXPXo67h9wQbn\n9wvvXIKfP7QS89cF2/6WbdyF6Rfeh+fecO3137pjMU7+1dOJypYvlnDji+tw7+KNOOCSB/CDvy/D\npfcuwyeufhalksADS9/G9Avvw4btXU4lyaa9r69Zi3ogw5B+6VuWoJh+4X2RHWc5PgN1RL7RYD7Z\nEzHqjeLFNVaHGWWXreQ6Z183D+f+ZUHkfnqnoGtY1z+7xnmujy63HNmr3w1OZSAdldu7rPcQFqr6\n7Kp3sWTDTs+2MBn96Wuex1duetmzTQiBv857E925Yux7rgUX3L4Y+33HHZB05Qq45aU3YwkI0y53\nLNiAnV3BdVkdlPTkS4Fmop1KcENPvugI34ISVrejK4dL712GsxW/jXsWgVN//TSO/uljxnKUlMLr\nTUqtSxt3sjCoOVm79SxavwP/74U3AXjDxmT4Yz4kRn/eGqsBP5AwRPLqx1fh4795xvl+zRNv4OK7\nluL8mxait1DC9c+tBWB1yOu2deEG+/vqLZ2O+UXvnMoNX9y6xzUzlEoCx/38Cdz04pvONllBTUKl\nXPbE1Ax68kU8scIfESTfkx5Noj+DdVu7MOfSh/Doa++UPfpUzQQmfA1YM3n9+XlrudnNu3odQZ3X\nyqtqnXrnHybILr5rCa582BtWmSkzzGvBuu246M4luPiuJc42o6+pTIQQeHjZO4GdV6kk8Mgy933c\nMn89csWSY8783j2v4oI7lmCeLfBNBFXB1zbtwjdvewUX3rk48Fi1E97TWwg0E6lmup58yREiqrlT\nHquaYlOKmWjZpl3YsL0bp/z6afznXxd6ggLUxyOv/IenVqPjRw+jx64nLdkUNipRag+9+nZV5gzF\nYVAJgyEtftufqp5Jm2yYSSNrd0r5YnRH8/VbFuHap7y26sUbduDYKx7HFQ+uwOINO50GYnIWjWzL\nArCElLQPN2dSzkglo1Xmchu22lm+sGYrVr/biR/83Q3qkprBa5t2+e6jXNSOTu80VOF74R2L8bk/\nvYQ173bi3/74Im6bb01UdzpXZd9coWR8bju68vjCDfPxxMryZnNKc9sLq7fizGuf980Y1keSumbg\nmgsEsmnrS6FU8jznnFL+TlsYjBnSDMB1skuEEHhl/Q5736LjaJa8ua2rLE1ICtSXFe3q5RgRMFE8\ns+pd/Mef5+NXj5jnAPzpubU458/z8XfNbPWyrYG/aZtcKrGRy/DL7V05XP34Kvz3ba/49lHPu2V3\nL5bZM+r197nHIwyKjuBW35n8rAoIKZNVofPapl34+ysbcepVrsXANDi57P7X8O6enNPWpoxs80Sp\nnfuXBb5BQK0YVMLAZOPdoaiX8kVv7wp2zJk6pSDuWvgWfny/N4rligdXYO1Wr20SgKehjx1qdQ4H\nThwOwKrAWzutkUh3vhg4stE7iyjU2OY/Pm2F0R46baSzrVsZuf74/uX49SOvl3V+ld1KQ8sXvR2k\nKpBfWmt1EIViCU+//i7+53ZrxCdNYGqemB/dt8yT42VIs1fYb9sT/B578kV89eaFjuMZcE1d9yza\niBdWb/M5ndXH3ZpNO+Xe3plDV67gaFACQMauJ7mi8HRGasciO/8RttD/zeOrPLboO15+C6dd/Swe\nfPVt9OSKjjlJ8tjyzfin3z8feI86st6s29aFKaNaAQAPLXsH59/0Mnb15LGjK+doK9s6c1j61s5Y\n2pWMNnvuja3Y1ZPHvYs34jPXPIfHbQ1vw3brnuRgpcl+NlKQq6PicpHacHtTBlc8uAK3L9iAHV05\nzyhf1cZOveppR/j4zqUc050vGjUDWf9U05GMdAp7VN25IrYqDv9coeTp9Hf35NGUSWGvYS0ezQCw\n3nNfMKiEgSla49WNO3HlQysghHBedFiUhhzxSWGQ1BHWnSuiJ1/E20pHfvbh03DmYVNwycesRK5b\ndvc4FalHqaSZtC4MvFFEshE8vnwz7l74lu/aclLZ+m1djo1bFZi68PxlwMgPcCeVBaGaifLFkkdl\nlkKpWBLOc1A7zfsWb/I9dwB4RbOhTxje4vmuPx+Vx5dvxj2LNuK//uomBpM24/HDrPOofpBiydup\nj2jLOmais6+bh589sMIRFkK4Hd7L67bjJ8qAQBVmUjNQfT2/sgVuT76IzXa00rw125x3pfPqxl0o\nlURkMrWrH1+FpW/tdMon+7I7X34L9y7ehNvmb8CcSx/GCVdaGTo/e908fPR/n3Ec8zp/nfcmnrPn\nuTTbnfjSt3bi4O8/hPNvWoj567bjca0TIyLs6S0471YOhHqVwcA1T7yB/77tlcCIN725SeGlhn9a\n9/Gk8z1I45BtVwp1n2Zg/65ro9Y2v/0/rCf45G+f9Qw8b1uwAUf8xPUt7OzOozWbxrihzT5tb3kf\nhZgOSmHw7IXH4cn/ORaTRrTiwVffwVWPrcIbW/Y4Lzps5p+008oKotazR18rf27Bhu3d2P+7D3hi\n2Ee0N+HyTx+MvccOQTpFeHdPzmkEqmYg1Vw5onpb0wxkI/j89S/ha7cs8l174Zs7cO/ijY4mNLQl\n4xGEXYYw0CD7pckEp6KaXPJF4TEVycb4jVsXOdtVreUrN72saGTucc2aA33iiFbP97AoKHl59X5l\n488VrfKot/qFG17CvYqZY3irKwze2tGNzbt7nBFiSTETXf/cWlz3rDt5Ue20pbBVn8UdL2/AI8ve\nwf7ffQBPrrDMXOu2doWaJS+4YzH2/c4/jL8JIbBldy+ueHAFvqskBdT9HTJc+S07kkX+lfVyzbud\nuGuhG2xx0Z1L8M//96J9v9Y23Z8jQ4zVDlxGygDuRET5/rtzJfz0geW4fcEGX3Re0JuUQkOfJbxx\nZw9um78eb27tcu5NpygE5q3Zhv2/+wCef2MrOhUzXU++5GgE6rM3zYB2NMKQgWFUh76rO4+WbApj\nhzX7fhs7tLlP0qwMKmGwp7eIicNbMGlEK6aNbsfw1qzzW7HkjpS3GcxEX7t5Id7/40d8nZLakL9w\nw/zQiUjrt3XhOTuCRLL4rR2+/aSjO50ijG5vchzbgNVgHK3E3iZH5brPwD/Zxvv9h/cuw/k3LXRG\nIpNHtmF7Zw5XPfo6pl94H7oMmpRuqpAENVbZIe/p1TUDtyz/9/RqTL/wPtyzyJ2xrI+Oso7ZpYQF\n67bhjS17nBGpZOIIr2ZAIHT86BFjueT1t3qEgSyfbR5wUiAU8cQKr/9hRFsW3Tnr9z09BeQKJU+I\nYSZtbloezcC+R/09nWPPXpcRVKu3mNOOSG6zI+JMnVGuWDJmnd3WmfMIS330PMJuG1ttR+kpv34K\nX7/Fb48H/Ll/JCaH/F2Khio7Wyn4V7/r3mcc/0FPvoi/vGA57U1zAf7n9sU46w8vIMiiWxLAM3aW\n0BfXbHU0NcDqC4omzcBwMvkUg2TBSTHWQ3A1gxbfb3d9+cjIpHrVYFAJg87eAtoVu7K01QJW5yPV\nOFUzkA3x7kUb8c6uXp+5oqTVgKcNKWgPu+wRXPnQCnzgZ4/7Gr5JeKgho6Pam3DLfDfbt6oZSGkg\nhYFuJtKvFRQXL23XU0a2YndvwXFY7en176/6U75x6yJ8LSL/ujtL2m1ouULJU7Yb7CgcFT36SD73\nXKGET1/zPD78iyd9obQThns1g1yx6In6OOryxxwhYzIByM7UsQvbnZVpVvOI1ib05IvoLRSRK5bQ\nWygpk4+EYybS8fgM7PchO76z5k7x7Csdy/rM5yBMo9begisMxgxp8vw2bVSb83mHJuSlYHPNkyW7\nzAW853sPevYN6rf18hCshHMnzN4LI9qyjpYpB2GvbXJHz74oLKfKuxf707NrnTk1el1w7yvnCHU9\n4dx//XUh7rSFU1Mm5TUT5dx2ViiWMH/tNut9G7KKSqFRCNBAVhjmCuns7M6jxTYT6URNeqsWg0sY\n5LzCQNUMVLV0m9IwjvuFa3sEXPVadhS6MJAOT5Utu3tx1WOrjGUyZb7MKpX2M4dO9vz24NK3HSer\nvHaLrSLrqr9eOXcEJImTI6LJI9s823sMZiLVrHLny2/hbns0rz8HSSaGZqAi38mi9d4oF9mHqp13\nc8ZrGhjV7u3s9Ib71o5ux5Qn359aDD16xIkzN5hoRrZn0ZUrOEKrt1DyOJizAf4KVTOQz6RQKiGd\nIvzkUwdj+mj3HTjCKWZoYXeuiKdWbsEXrndz2uQKJcf3oT+vKYow0AclphBKwJqYpQYD7OzOB5pH\n9PrTlStg484ezJkyAplUCnnHJGjtp7ZB/Z5Nph41GOj3T602lqG1KeP4R377z+/DlWcc4vldCtqm\ndMobWlpwhUFnrojP/O55fPuuJcZ3IeuN7ufQAxrC6OwtIpO2LAE61UyUF8agEgZ7egueF6Q6PeUI\nY2hzJtRnsNG2ecqOIunEEFOys6zSq3xo/3Ge3+atdWOx5aWDKotfCzGPMKWtdNJI78g6ShioBGWT\nNJuJRGB67K8fPwsA8Ien13i2F7VRO+AfDeqNz1R+mdzPlBZjd0/BGuk7ESMCl/59GV5cs9W37/DW\nJpSEqympZiLLZ2BuWr0Gn0GhKBxB0qqMAneXOYGuO1/EOTfMd4IBAGtkLDs83WwzTRE867d7I2yk\ntrJ1Tw53vuz6CnQhvmF7V6Bm0Jkr4rt3L3XWEJYa1t5jhyCbJlczsIX2ZkWz1X0ksi4TCLfOX4+b\n573pmLLCaG1yQ7HTafIJRElzxhIGsjPe1pn3O3I37TY66mXZVM07nSJ85UP7RJZP0pUrIE2EZi0Q\ngyhY66k2faN/9BO6eoseNUy1w73+zm4Mac5g3/FDsdLg7GnJptCTL+Gt7V5hkDTnv1EzUDqSsNm/\nUZFMujDQQ9Yk0rmrmxFMHWaQMAgKtZXmIT2aSM8XIwlKClbURusAfD6D42fvhcOmj3Q0J5OAenNb\nF/74zBosN9jRf/HwSjzy2juYPqYdgGVWVJ2/KtLEuNn25/QWSo42IIT3HWbT5HRuXgdywbkn+Z5V\nwV7ucoum93X8la69Wl84ZYqiCa7f5g4UepVR8dbOXnzjVtdXoAuDXd0Fo5Y3c0w75q3Z5plIJsN4\n9xk3BBnlmUjUSXh6fVJ9CN+yte9vn7q/77o6bdmMG4pNFLg2gYx0Gj2kCVs7c8YklamU+Z3I+1eF\nQSYVfC0T3fkiUinydfxt2XRVJn3GYdBpBqqZSM3GubungKEtGbQ1pY0jMunYecvRDGwHcsLQ0nci\nzERhfiPZPgpFgYMmDfP9rguDoJTL0nw0ss0rDHryRd/MzyCtKarj0mPtb3lpvXG/1gD7qIwnz6nR\nRNoob0hzBr/55/e5xxje47qtXfjhvcscp6vOKxt2OvcSNqFrWIs7IRAAcoWi40gsCeF5buOVkFfV\nzOA6qoVRGJRLVAJCXVNSI8DU1CS7ewrO+9L9ULpjt6Tk+pEcf8A4zLAFqorUUKaMakU2lfJ1+Go0\nT75YctbtmL92m1OXVZ/Bo69Fx9+3NKWdY8M66HyxhM7eIka3+232EgI5kWaAOxiTj0StLxmtY//D\n2R149JvH+M45d/ooAJagThmEVVTG1Goy6ITBUEUYqLP5ewslNGVSgWqkHAlusScyVctMZEohodqb\nw1P3uhFNU0a24bj9xzlOR8DfcIMm2+zoyqElm0J7s/fee/IlHL3PGHz2iGnONlnhVa2kWBKRGRtV\nlm/a7SxOrhOUO15eV3Usqs9pn3FDrOOVxrPLMJM8zupk8t2GCQN5HSkM3tjSiaW2qVHA61TdS4kQ\nUYWm66AUjjktar5GGCazmIqsD7LDacmmcO9/Ho3DZ47ylGtXd96xg+/U/Ey+tQFKwhBFQz6tDbCc\n0UObM2jOpJFJk88X45npWyg5wRhPrdziqcvDbCEmtbJJWkixSms25YZiG0be6vU6ewsYNcRvs5ek\nyPv+pObpmokUYZBOeTr2EW1Z7D12iO+cFyraTZrIF3hQbsqRJAwaYVAqCezuyWOYYmdUTTDduSKa\n0imjJBbCbaxygkxQNFE1UE0MYRpiSQDrtnbi7V09SKcI133uMFxw8n7K7wL/9scXne9BC79s78xj\niN1IVbrz1jNRK7V0Nqsmh3LNGWGpmoNGQvKYnCf6x/39kW9Yoy418kKuqhbWWZiQjTwsn5JcP2GL\n4mB1Rq9CeMItz/3gTOezyZxmmYlglz+BZhAzPfhEW1NpyaRx0KThvvkZu2zNQI5aPWXV3nXRoBmk\nyK+1SUbaNvlMKuVL1QG49vF8Ubgze+ENX00pfqgUBTvrAaApk3a09zDNIFcoOYPFIMcvERlDg93I\nNLeM2bRX8MjUMnecdyRmjXOFgvq+U6nqLrFZLoNGGHTmCigJV70HvJOSunIFNGVSaDG8DLUDkh1F\nPiCaqBp4fAahE6cEjrniCezuKThRO2plKhSFZwap1Axeu/Rkz3l2dOfQ1pTxpQPozlkRDmp5ZBjq\n7l5/Go+4hMWQBzleZcesjn5NJjr1nVa6bKGbrTK4nPJ5m7Jl7un15hE68cDxeMw2EZgEZ6EknPec\nTjASjLuYujRbydG7nuPqnV09KJYEZk8c5vtN98PkCyXHdCoJc3pKYZDNpJAvCp/fQHbEhZJ33kbB\nEbTuIG5PTwHNmXSoVpovuAnn0iGaQa+tGbQ1ZbDk+yfixNl7+fZJkff+Zf0yWQcyqZTnWiNsE+yh\n00bi4MkjlPK4dTNtEFb6RNJaMmgcyNKROaxVMROpmkG+iKaMWTPoyZWckUmPE1paHTORCXWkE+Yz\n8EYvpOxjlY5by4q5aWcPmg33uKPLWmDFpBlkNc1Amk5U+26QZjB2aLNnwpwkaEYoEHy/8rrqPUc9\ne2kHL18YRL9TWXVM/iXTspTyGQZFo6SVOQphZA2OV0lszcDWBFrs960LoEXrd6BQKqE5k0J7c8Zj\nKtIjkn7w92U+YZCi4E5XRutkU4TtXTlfttH25gy2duas6CzlvuRApSjcyKvufBEj2rKhz6y3UHTW\nMwgXBkV05ooY0mw5bE2DkhSRRxhIM6SpHuodvRr51JQhZR9vsEhDagZE9H0ieouIFtn/TlV+u4iI\nVhHRCiI6qVZlUJEdw9AAzSBftCYKtRhstt15dzairAzSkZk0msiEx0wU4jNQJ0PJEZx67FbN2Zsr\nlHxOYsCyvQ5pzhjtvLqZSHbGenSQCdV/oRKmSARFTkhhsEEJgQwSBvMu/jAmDG9xVqoq1wlXjtkr\n7qI90hcgBbQ+a5cChIE6Mh8zpMnpwE30BDiQP/XeSR7TxER7cp4MY1Sv0d6UxsvrtlsCKkU+Qapr\nBrogsO4FxnYEuEEKmTRh8YaduPNlb84sGeCRL7pO+D8+swZXP24lDSyWhKeONGdSoQnickrkmt5B\nq8hJd/L6prxWKc1MJIWkSRhl027HPrQl45mRLtuobrZKGXwG5cxVSEqtxdAvhRBz7H/3AwARzQZw\nJoADAZwM4LdEVHOXuewYgsxEgDV6M1XinnxRmTovZ43amkGtzUQhmsGqzW4IbMoRBu4BpjBQ6Qg/\no2Oy00Fs68zhkCkjjPeeSXsrqPQZqM7VoM6z3e5I9t3L6zhLohmoo2IpDM47dm/PvuOGtmB4a1Yx\nE3kbVJiN2bpGtDBwNIOYK6vJQYgUHrppTT5iXcCpE+n2GtbiiTTTCdIM9tlriKeu72Xnv5GjUvW3\n900biTe27EGhJJBJkU+QRjmpAWsAEzQCH9VuPYcgc+AQO4ghXywZBwaFkvAMjywzkfXMTBO2evPu\nbHeTGUaGp8r5IlIYmPJaEXnfm2sm8t9HJu2aifQBmCoM1OeUTnnb2p1fPtIYgVQr6qGTnAbgZiFE\nrxBiDYBVAObW+qLSmRhkJgKsxmFKo6umgJCNIV8INxOddOBeTirqcvFEE4V4kNU0BY7PQNUMDCmc\nZcX82WcOwUVKJMMXj5lpHHXqjVY6zfQZxSZkR3Lk3mM823tDOtugeRUmZ26xJDBxeAsuONkfb96c\nTTvaoNqh/fZf3oeTD5oQeH0gnmYgNbYozeDY/cYCgB2plnLMSr5Feuz71uvTaEW7mjC8FU3pFOZM\nGWG81v1LNnlCLyUyegew6sd7p47ElFGtmGxPMlQ1g8kjW7Gt00qMmEqRL7opaHKhChF8k6ckrgPZ\n/J5bsmmkKLhOFTUTWXMm5c7EN1wzVyx5Qkt1ITVzzBBMGtHqDJzkSDxr8N0EaQbqgFAOgNRR/8g2\n7+Q4ud0SGIoDmcjjI9x7zBDsNcyfq6hW1FoYnE9Ei4noOiKSifInAVCDzDfY22rGVY++jovvtlZ3\n8moG3v2aMiljaJ9qJnKSmSmRIyYmDG/FKQeNr6i8QZqBnm5B7ZDlSEYd+ehmIsBKo2C6zpDmjHHE\n7BcGljBU/RFBDmRpYhjWmsWE4S3OjFdTfhdJoDAw2OaLiuNVpyWTckxa7YowOPU9E5zQxCCiHOK3\nfekIJztZWMTRiLYsrv+8O84Z2pJ1NAld4ASZidTR7n7jh2Da6Dbc/ZWjjNd7cc02oz+hOZNy/AKZ\nNOGgScPx9LeOc5yaaeW9TxgsHtnzAAAgAElEQVTe6tTxjMlMFMNJTSE+A5luJCiRX3MmhWw6hVyx\nZIykK5S8GW+bs66ZyDSQ82oGKZ9mIO322wPMRKqGICedyfp03TNrcMHtiz0mP5niI6OYiUYEaAbp\nFHnanP5I0hEabLVJJAyI6BEiWmr4dxqAawDsDWAOgE0AfiEPM5zK2KMS0blENJ+I5m/ZUt6qVSpX\nPrzSmTwztMU86QywXpLRTKQkrZI48wwChEGKKHIheTUdgF4OiaoZ6GpwpzbJRT92W6ffeauaTFQt\noiVjnumoCwh31qwyiSxgtNii2KSfueA43P1lqxMLW14ySBHa01vwjSaLQgSOMNWRqW4mMilz6nmi\nonIOmz7KqcRhZiJdsA1ryThOR7+ZyNr3mH296UfG2Sadc46egf8+cT/c8sUjQstmoimTgnyNpueV\nMZiQrDKl8JH3eLWoqKVBARlaau5apLksyFTXlEmhKZ1CvmBuV8VSyRPlpZqJgjQDJ5rIYJNP2SN4\nmZ1VzrWRz6RFs+nnCiUMbcmiOZPCmnc7ccv89Z6+QWrezRk3IMOnGdj3nkmRR3Dq/UVQ3a4ViYSB\nEOJ4IcRBhn/3CCHeEUIUhRAlAH+AawraAEBNzzgZwEb93Pb5rxVCdAghOsaOHZukqA6qA1kfVTal\ngzUDfYKMtUCIMCYxA6wGEfUyLz3tIF8iOiA4mmiYlotFnbAmRxHZCDOROjLyzHQOKKs+guvsLWLJ\nhp24/rk1zrag6Ba1kqv22ijNYNml/piCrlzR19gLIZpBk/IMdbu3FKLSjJcib0OM4weQgjNs3Wld\nGAxtyThmpV7N9i4vf9bcKVjwneMx057BO6qtCQu+czwu/sgBFaclaM6knMlLpmgVNZpITaGcSRE+\ne+R0fOcjBzjbfv5Q9BKMBH+EkkQOxoJ8Bk3plB12WjIGT/g0g0zKEe6mtturLF+ZTpOvPqftzlgK\n6SGOZiAn5nnPmStak1PVtqgK9sNnjsa/vH8qfnnGnFiagbwHwF9f+loY1MxVTUQThBByNZBPAlhq\nf/4bgJuI6EoAEwHMAuCPxasBQ5oznsagawZNAT6Dnrw5y+b7fvSwL/Wvc25b6ocxflgLfn76Ifj7\nKxs9ttggzUBX2dURubwXGbYGmOPk1ZFRUJplFb3R5oolfOw3zwSWw3Mt+1nLxijPFbRqF2Cp4kEp\ne/W2USwGawbqzE19VrMUBiNas9iyu9d+V+7vsezikXv4tRyPmUjTDKRQIyKMHtLsdhTZlMdvUAnN\nmbRzPtOMVll3iLymSFmH9UlpURAR1m71p/0G3NnDQTNrmzIpO3zWbCayVpwrefYXZWgGOvpEL91M\npJ5TagbS/yPDptU+YEhLBl89/j0A3IyvYQ5keQ/y/CpRloVqU0ufwc+IaAkRLQbwIQBfBwAhxKsA\nbgWwDMADAL4ihIgXIJ2Q4drI2qcZaNFED37tgwC866GqBAkCee6oqBXZz+p1NMhnIEeypqgJk5nI\nhFrx48Q0N6XJsckGyTY1X4v3WOtZSlOafB6V+AwAf+MoChE4AlXtrW12A5enliGvE+xOznTNqIZY\nySBd1Qx0AaqXQX6XzzCK/fYaGvhbczblCgNDnZTbCF5hoDqdy4EI+NT7JvkSHwKuZm5ydANWncyk\nbJ+B4XeTZiC/mwZy+aIrPEzvVE9e52gGijCWWJPOrFn5U5X036o5VvVPDW/N4rDpI3HYDHddcXmP\ngKt9yHLp5eurBHWSmmkGQoh/C/ntMgCX1eraQeh2zKjQUqnSduUKZS87l47hMwh62epx6j4HTRyO\n4/Yfh/3HD8Pxyhqv1jH+SWcmVAEVtS/gNRO1N2WMjly54peObEjy2VmTeSh05B32yHzCoCR8Tjen\n3Mq+ukb13Y/NxtGzxmDNu514aqXZF9WWNScslITnjLLQlUlLGFgDiKBoIokUoHEnITVnU2hvShvN\nVs1pVxiYOva0opXomgHgzeEVBwJh//HDMP87J2DuZY84OYQAt00FReE1pdNosmcnB2sGXp+B60A2\nC04ZSKAGWUhhrIdOO5qBfdN6tI/MYTbFIwxcc2ybMi8gm07hti8dabhHr2YgrxU2EOoL6jfdrQ74\nIgn00FLNgSxj8vf0FkJTE5hIpSjS5mdSW33nIe/njx480RgNkzH4DEyoI80ozUWer2O6NbI58UD/\nFH0gOL5dCl/12WXTqQgHcnCZ9MZSKAVrBh4zkS0M5NFDmjP42CETnc7B9GbbmuONyIGw5+g9s2Um\nCtAMtLoiBWhULnt5bSLy2aYlzdmU2/GYNAMpDOAVnI75KJZRzK2r6q3oAlxqBkG+NsdMVDD7DPLF\nkkfINmdSvuVfdXRhsPJHpyhl9k5EkybFrNOevAMzaSZSfQZq1F57jAmOWWUGMuAK2zIVsKozqIWB\n3u/ooaWtWcvW2tlbKDvthKUZhD/eODZBtQN0OgzDYa5ZwXtNvXKWaybKpgnvnToSy394Mk4+0Bwq\nK4XBETNHG6+lRlxl06nQaJ2w0ZEuXPOFEoL6YXVfKeB1QeOuP+DPvBm11KB6qvM/NMu4j37OIc0Z\ndNmRaX4zkXffuJqB44wk7zKuKs2ZtDIpMUwz8D6jdEh9MyEFMIUIA1kf9VX41FF7Nm05kE2mJN3E\n2JwNn2cAAN12BJxpcKYGNrQ1pX05orxauuWDaM6kcPgMN4mfmtY9zmx33WeQMVyrHgwqYaA3BJOZ\nSA33JCK0N6Wxp6d8YRAnmkhNxBUHeTZTh+n4DDLe34a0BM++LceB3JJNB1Z0mQbhZ585GPO+/WFf\nmVQTW0s25cuZpKI/sse+eQxmT7DWatBHz72FYqAjUvUZOMJA38c+1vT8j9x7tH+jgnquEW1Z/McH\nZvj20U8rO528vWaytyx+ExgQrRmozscgYWDZ4aVm4D+foxno0Sz2M4xrvpCvQt1fbwPyGrpmIOul\nnGeQLwnjzF49+KA1q4aWmp+VHKyYOtsUkRNO2u4x8ciOWhmMkaXRNaVTOHKfMbj3P48GoGsG0ZZ3\nPZooHfD8+5pBIQxmjrXC9C756GzPdp8wsM1Eiy45Aff/1wcAWGrtnl7/PIMoUqlon0G5moGsLKbG\nGTTy03ObqJ2LaZQ479sfxk3nvN/5rjaGIDX8svtfs35vSmOcMmNSllO38ZajGUwZ1eaYLvTn1Vso\nBdqzM0pDczs77z6OZqBs+9Ixe+PJ/znWMY0FopyLyPxO9AmJUvgWSsJnKtM7AilAozQDec5UmJko\n4/oMsoY6J4VikLA8dFrEs7CJoxlIdLOrjLhpSltmoqdWbjEmOdQ1gyHN2dDQUsCdN2J0IKcIw1ub\n7HO5bUUftQPeaCIAOGDCMBB5Z6Hra4KY0B3ITl1lYVB7WrNpHH/AOByiTePXG3CTbTsc0daE2ROt\n0Wh7cxp7evM+tTYKfXahiTgjLq/PwLXt6siORx+J6cJAFQCmjmbcsBaPc0zdJ0gNDzpfJZqB/kjU\n/Dh6Y+nJB2sGcnsmRcpz8x7v+AyUTnvs0GZMG91uPG9zJoVv2etFqOci5z8v+vBB1od8oeQzE+lV\nRZqJghKruee0hUEKvjWBpaBoUoVBqGZg3j6kOYNfnzkntByAWlcNpiYAh0we7nzWQ7X3G29FQxVK\nAovW7wAA45KjuhBtb047709PgSEvLVPImDpbSxhYz80jxGRHrbyYZZt2Yd3WLs/8mWmjvBNHo8yL\ngN+BHBRN1NcMCmFg9UXmiqBi6hyHNFsRNOVmqjb5DPYf7w3/iycMvDbLoOPc8E/NRqsJA/UenfC5\nkCgrtfOIsofqZid5noJHGKQ9aah19HsjcvPj6GainnwpZLKc2/kF2b4dYaW8W9n25X2rs0eX/uAk\nfPnYfexyeQoZ+k4kWcVMFDQDWSJ/TmImUpeeDAstdcwUAcJSnj+KjKOlqOewtn3xmJm45/yjne16\n7qEDbFPg2nc7Q1OI60J0aEsG5x1jJSrUn5UzybFgzVkw1RX1uanaf9bQQa/b2uVMOtPLLUliJpLP\n+Nj9xvb5hDNgkAgDoeRAVzFNOtPR87nHxRRNpDdEvUynvmc8rlbW8AW8nY6zv+FeVCftTz71Hudz\nmDAgIvzoEwfhvv862rOPWu44ZiKJPuo02aebM6mI0FL/zbUpmoFant5CKbDRqJ1f0IjLNPrXs7+q\nE4bU+uKRBTCHxOpmIvl8csWSz9zhMxOJmD4DxUykT2667vOH4eJTD8D4YS3OczK9Q2eeQYCwlOeP\nwtHAlF1NCRQBf2jpsftaGQbeO3UE9rbNusYZxVrdaW/O4Bsn7oe1l3/EV/+cSY75Ymg9kRqV6sdw\nTTj+56+2odmaMIjnQJYmKF0zsH6//vNzserHpxqPrSWDRBiE29klJofq0JYKhYFhJKJXLDWKA7By\n3nzkYG8uGPJoBnIE4f4uK5aqdp81d6rzWb8nvcH86+HTsM84r8biyb2uVPwoYaB3up987yScNXcq\nvnmiuxRnlKlJnuL2Lx2B/znJOk42MN0P05svBnZSqs3XHflq+yjCWT49+YylnXl4mzl9CWkamykM\nMtBMVBROpya3+eYZxPUZSM1AMXdIJo1oxX98cCZI0VJNzz/o+aQ8wiC0GPZ55L7KcQHCQB/97zd+\nKF655ET802FTnNh8U+ZS3degmkGD2nNvoRRYT9JEzjtWr+fWH/9xani2Pjs7XoSe1Ay85qJ6zzMY\nFCudlYR5Aove95tWxGpvyjhLG2ZSFHu+QTpFPruo7kPQR4NRlYEM+zWlU8gXi8ERSdop41RWj5lI\nEWDlLhLTkk17tBQgeqQrn0nH9FHosNfgbc26s0IzKYJ0K4ZpBm64njfdgmkfFbmvTH9tWgxIPxeB\nzJ2lbiaSDuRiyYlwac2mkS8WfI5wN5ooShtzBwhBZQXcum6KuNGjiYisAVQmQPgFkTE4ot0oN+91\nZ08c5vgGrPtIOVqsLGOctuYRBgF1vTdEM0il3MwEqrbiRFIZqqvahtQZyqcb8oyZkG1A1wzClrjt\nCwaFZlASwqwZaNtMjWmIkmkybFT7o08c5Du3Pms5SDNwjwk8vae8HjXcbuVB0U76KZvjzDr2+Azc\nz82ZVEVpGDzXj6kZqLQ2uaNf9ZnliqXANL9uigWKZRPXry8HBrO0hXkkHjMRmTtLv2bgmol68kVn\nHot13WSaQToktBRw655ZM/B24llD3HucfkpWb9NcBV0jveSjs3HHee7sXG8KlviVTA2d9iV6k+lP\nCsG+pXTKjcJSNQN5LtMgyxNUoQjrK04/JFaZnXkGjrnIfYf1ZFAIAwFzLhm9MzA1JnXkESYMDp85\nyvM9nSJnzQNJlM8gavRliuM2mYnCzhm2UpZEfS5q5606c8P41PsmeVIhq4Qt2wgE+Qzs1acI+O+T\n9vP8FtSAVBNckM/Ao6kJ7/U/dvBE/PrMOY5zMgwKKLeONFvkiwJduQJam9LOcT5hENNnIDsWihAG\n8hkYV7PTHOympHaxHMim0FJnMqT3+JZs2hOy6p3paz5/kObultH7W1YxEwXVATWaSF2MSu5talbN\nAZpBXLKaZpDqJ5rBoDATBfkM9I5kpCEBnGqHDVtYXe9000To1RK46aOjoORkQZgnm1nnDMqdpB8R\nZ6KZqhnoqS9aI6KBAODKM4LDEKMaj+keZQeWThHOPmI6Jg5vxTl/nu8rq4qaYsG1ZXv3MWoGSsM8\nbc4kz5oRKh4zEZk7sCAHcr5YQneuhNZs2vEd6GWJO8/ATX8MjB/e6iTDO2of76Q5efoWw/l07cQU\n6hgnP5ErU9zjnGR3Mc2Dajl0Dp48HC+s3ubZpq5PordBWddzET6DIc0Z/Pz0Q3CEMtFQ7m4aZHnW\nAYkxONJx/ESaRlZvzWBQCIMgn4EuiU15RdSJPEEzHAHDiyS/zVPvuMo1E7n7+RubLgv+cHYHSkLg\n/iWbPNvjJKdTy6U7JSup/J7jIzQDMhRPCmF536qGFRhaqtjBXfOaZkYwLm3o/R4c+62MZAN8BmHz\nDHryRbRkU878lUo1A8dMlLI6tYXfPQFEfne2rIsthjquzzNwNQN1tB5dOV2h4t8Wp95JgjruMYZU\n3u3NwWYiVzMIDjSQ5fOvK2KbiQzHeHwGMRMJeo7X5hmknOde9qmqyuAwE8XUDEwVXo0zD3PmmToN\n3Y5fLc1AvZQbK++91gmz98JJB473awYxKq/6HPQFdcp1IutUohm0KpoB4O3EA0MG0+o+fscmYI65\nj7vAiHeeQYDPQOtJMorPYFdPXjMTmY+Nm5vIFZRW9JQuJGXYpEkY69FEumNTPX8Y6RAzUZAw2Nfg\nkwmSvyatVj2v/rO6mFLQOYMGE/IeonwGUQ5+E040kZJkUP1bLwa1ZhBnxl9czcBUqXTNQI8mSmkV\nLqoumEa4U0a1Ye3WLic3v45ewcodyeiNOI7PIIxon4F/W6uWjiJbhmZg7WO+lqmj188XVEc0WWA2\nE2njStmZfe5PLwEA5k4fhe2UD71OVGfTlPZ3wCakBmISxnoOIvkMPBqYcv4UmZcOddfnIGVbuDC4\n47wjsb3TG7pNRE5Ek6mcAPC7fz00chJnVnEg6zm6nPIFPDj3XP4bVcsV1icEIYVJVhPCPAO5DxDC\nHAcex2GjagZhJhJfpRLAKQdZWT7lT/okrLJDS0nu52771Psm4U+fPwz/oswt8ByjfS9HXTehp7co\nl6jGE0czUJ3akT4DUhqZtqtpUlxcp74+/yPeDGTvPi1N6cBZ5ZNs4R6V0kQ1E4WRK3hXm1PRR/Sm\nXDmpgM8q5mg3+52FrIk81bAWuOkaatmHt2YxXZlhbV3Xe4xML5MrhvgMIoS9STMoKqlpKtEMXM3L\n+9zr7TMYJMLAPANZ3fb+GaP8O8Abbho2qvap+RCYOXYI1l7+EWcVqshcRRFvw+nTtIb5of3Ghai7\negNJKAwCRlhxiZ5n4N/WpuUmUrWTwElnyqxaNy+/to/yzOQoPm5Yo64ZxPMZeO+9JZPyjcYlt593\nBP5wdkek6UA3EwUhwyZNddDthLzmIT19sySwrjl//YKj3EGI6RLZdEoZWBkGd9omNSlfUNsKem6O\nmcjwmzphrhKfAZGVNtu9B3Md6GsGiZko3Gcwoi2LW754hPFY1WYeFiOvv0jTiCIoqZpzjkjNwHAP\nERVIP8TUiMohuWZQfmipOgMZ8AqD4DQDqi1ZCgbvvnF8BkHo0UTG47Q6oNu8rcV55Cjcu++E4a2Y\nMDx67WHZGcU1E0WtZxC0j1czMF/DNEPeXY61vHpnnctvZpXRbEHpqL37K3WgXM3A3myKJipqubYq\n4bJPHIT3TvUmzkzYNBMzSISBMI4M4khitbKE2bv1ymYaUUR1xFGjQNOvcWctS8pdz1anEhup9/gK\nhEHWqxm0NPk7eh03tJQCn6vRZxCzQXqylgYco/sM9E62K1cITT4YB3dxm8rNROoayFZZrL9qR+hJ\nmBiw2o3cxeNAlsuxljmCNr2HTDrlCIM47069ZrBpy3x9eY+mQZ03JXtl7eH0jim+MtRbM0jUsono\ndCJ6lYhKRNSh/XYREa0iohVEdJKy/WR72yoiujDJ9eMSN2tpFGGRMHFeZJSqHHWGOLOofee0f/7h\nJw7CcxceV9ZIxqQFVGIj9R4fpR35tzmTzgyaQaQwIDfeX99V1dRko69IM0A8n4E+GOjOlwLNRHHR\nR/VBuJpBcDtwfQYp+xhVGLj73/Dvc3HanIm+85jMREGJ6qII8hm0aP6jsGPiBBoEDRSO2Hs0Prz/\nOHzvY7N9v501V+nIq9CBJx0QVIukmsFSAJ8C8Ht1IxHNBnAmgAMBTATwCBHta/98NYATAGwA8BIR\n/U0IsSxhOSII8hkES3+Voc0Z7O4thGsGscxE4S87MrTUpN3ErD/ZFPmSaoXx6DeP8eXHB5LPMzA5\nbVVMjVNPYd0Sx4GsjHblCHHWXt7ok6DFTsrFMhP5t0f5DHpyRXdUWGE/ELe8MrTUZKrUE6Vd8rHZ\nuOCOxZg+2nXQqu9l7oxRmDtjFO5ZtNFzHlOYrJOormzNwCAMUuSLLAs7RhVA5T7flmwaf/zcYb7t\n3/3o7MBFhCqFNF9NvUgkDIQQrwHGBnwagJuFEL0A1hDRKgBz7d9WCSFW28fdbO9bU2EQ6DOI+fDH\nDmvG7i2FsiadmdZvDeoITVFCxv0q0G7O6JiCW+dvwJF7jwk/ucbeY805eZKaiSrJ0+4ubmN9l2sU\nFEsiJOeMW85hLVn85QtzcfAkr41WHambTBzlYCpH0Epnku58MXAVt7i4C/SE75crBvsM9EWTjtpn\nDJ654DjPPnGei+uH9msGVXEgK2uU6+saAP7BUqX5jsKohV2/0aOJJgFYr3zfYG8L2m6EiM4lovlE\nNH/Lli0VFyZwnkHMhz/OzlkS1pH5IkuF/7Oen8V/jngmH5UoNbVj+iisvfwjxvC9SkhqJtI7vbu+\nfKQvs6lO1l6TwOMUjohS0TNxfmDWWE86amufKpmJyGxFD5qBLDlr7tTA2dFxcYRBxH6ffp/VzORa\nASquIAwuQ5znIp+C+opNc0PiYKrXmRRh9BB/Ujnn+loZ0ylSNK/KO9p7vnKU55y1ot4+g0jNgIge\nATDe8NPFQoh7gg4zbBMwC5/AeiyEuBbAtQDQ0dFR5lpj6nkC7O0p5zqhx48daq3rG7augc9MZNgn\nykQSVRcq8RlUG1Uz2H/8UCx/ezdmjRsSO2OjLlBnTxzmWTfZBBGhLZv2zDCVI92g5GxqbqK4ZQHi\nN3Z92cs4M5DVc6/5yakgIty3xDK1VDoqlMdFaQb/dNhUnNExJVY5TZTTT6nPRt5zpWaiuTNGob0p\njcdXbEE2ncLPTz8Ef3l+Hd47xb8uc5CfISxRXRwOmTICM8e0Y/W7nTXpsN2cUFU/dVlECgMhxPEV\nnHcDgCnK98kApIExaHvNSDIDGQA+sM8Y/P2VjRgfEuoXpzEn9hmEhMf2FdJef8pB43HVWe/F7p4C\nRrZlY49sVYF4RsdkNKVTsTqaMUObfXmSgOAc/o4JKOTcpuiuuI8zVmip7xjyfdaTw5VLytEMonv0\nSjRP5zpl1DN118odyO7x0o+dTacwZkgzvn7CvqHHqOWQwiBpHy6fbk3aWxW0l2pQq9DSvwG4iYiu\nhOVAngVgHqzbnkVEMwC8BcvJ/M81KoND0AzkuC/29I7JOGTKCAgI/PSB5cZ9fCMGU7bDqEiaiPZi\nNBP1cf2Zai8AfuDEYcimUxhlyPQahtrpffvUAwJn7+rceM77PRkqJUGagb7+tAmPmcj+W2k0UaXt\nWF1Qphx+cfoheOr1Le4Ao2K9GZgwvAVHzBwd2MmWWz6zmag8YUCKkJQhrlGh2Wp7PmjSMHzh6Bm4\na+FbVpkSNhQnIq0GDa6+IsAlkTAgok8C+F8AYwHcR0SLhBAnCSFeJaJbYTmGCwC+IoQo2secD+BB\nAGkA1wkhXk10BzEInIEc1yRAhP3GD8WqzXsSlSNq0lktfAbV5qh9xuDGc94fOGM7ClMmzDgdTVAk\nVKBmEMdMFBJmGYU+z6DiaCCSf8s7wacPnYxPHzoZf3lhHYBEsgCZdAp/Pffw0H3K0wz8ZqJKHcjp\nFDmRUFHahbzusJYM7v3PDwCo3sIx8vmatPvH//vYRAvY1ztBnSRpNNFdAO4K+O0yAJcZtt8P4P4k\n1y2XkjB3mo69NeZ5opxgI9qy2GEvkanmRpcEjWziOi/7g88AsARCpZhWz0pyD1FmorCGlmjSmUcz\nqLxBJ55n4PgMkoiDaCo1E+0zbgj222to+Q5kqRkQIScs/1CUZiAfofou3GVBk2oG1l/TYGHGGL9T\nfiAyaGYgJ/EZSKIcwPd85SjMW7PNM7tQJTI3UURxTD/X2+lULmqDDlrysRyCzER6uKQJUwcet1NX\n99J9BqPam7CtMxd47BEz3YFC0EpncZHvv7aioHIH8mlzJuG0OYEBgyHX85uJorQLYwRSyFrG5VBu\n7qpycCx9tX6JEQwKYZAka6lKNmL/aaPbMW108CghaW4iU3n7i4oZF09qacMkpXJJOglOJ8icEC7I\n3do1ZkgTHv76MXjvDx827rnyR6cYE8BVKtSlb6TWHUk59awaVVI2lbTHgRxXM3C3yfWca6kZJEWe\nMWjp2r5icAiDwBnI5Z3HpBnsPbYdb2zpjHd8xAWj6qvp8HpPVCkX1bEbJ749iNHtTdgaMvqOu0aE\njqnTeOWSE52FSCS+aCKl8wpLW6IHESTVjvpKMyynrVSjv/RoBqV4moGpHknNIGknXu48lHKQ5WbN\noA+IyloaF5PN8s4vH4V39/TGOj5KE4muaJU7PPsL1dIMHv7GMdjdEzzvI65af9Q+o/Gxgyfi23ct\nAWAWHvpkNQvyfIqT79+E3LXSTsZJqVLR0eVfJw5BiewquZ4nmihCsza1Z2eVuyp14kkcxUHIM9ZZ\nFgyO9QziroEcRdZQGYe3ZgNTN/iuV4GD2Pu7f9sAUww8wivtCIPyb2JUe1OoSU7OlN4rYkLbjecc\njjOVhYFiRxN5NAPvMeV0GDKtQqXLibqjyv7pQK4U13zmmomaMuW3H5mXKmkaCfl8azL46idteFAI\nAyHMI4NqaAblkCLg9/92aOjv4cc3gGZgyAdUC9V7xph2/OL0Q/DrM+eUdVzli9v4HeNx6M4XAQDt\nlQoD+2+tR5XlvKJqjMLVaCK3I463MJJ6dT0Jn6Rj2siynrkzD6WG7a3WAj2KhjcTBaUvtraVKQwS\nVoQUEU46cK/A3yuaZzDAVIOMx2dA9t/aXOvTh06Ova876Sze/t7ZxIq5JmDgEUR3zhIGMk13xdS4\nHymnE6xGfykHaukUoRjShr3XNZmJbGGgHXz7eUeWVR7HgVyTaCL2GfQJUsU02jHLHFYlHfFEHR7t\nQPbvMNCEQZzUw/Uk9kRE9TP5O6qvfngWjtt/XOR5evKWmai9uVIzkfU3TjqKJJQXWpocj5moJMsQ\nYSZKyWPd/RwHcuJJZ7hGNXIAABaNSURBVHIGcqLTGHG7IdYMakqYZtDXfVDUeqtxF6rxnjNpqfqW\nJBO9+oJqpaMIS+2g0pNPphmErchVTSqdgZz0eqoDOe5gSTW3SNNS4nkG9imjnNiV4Ah01gxqi9QM\nTCO+9qYMJgxvwYWn7N8nZanEQRx1/EDzGZjK25/mSsQ2E8EjDSq+h25HGCTUDGo+zyD+vlUJLVVC\ndZPM0A/yGZSLk6iuJpoBea5RLxregRw2qkinCM9f9OGKZkhWQiWdvUpj+AwaUTOoPMSzO7FmAPu6\n/SeaqBoqt+pALsaM5DGlo6jGLHegtvMMPmmvNXH4TH8Km76k4YWBCPMZ9BFRk6viTpAyr8nQj3rS\nGPR7zaCC52llXq3sevLdJ/UZ1BpT3bv3P4/GdZ/rMOyb/HpO1tI04XQ7EEAuMlVOGeXgI7kGXbvQ\n0sNnjsbayz9S9xxHDW8mcicf1bkgShmuDQgvrcSMNNBmIPenjl/FHfmVfywheadceTRRX/kM/NsO\nmjQcwHBDiaqhGVh/00Q494Mz8YWjZ8RYHMqOTlO2yWOSvp9aagb9hYYXBqUavcRzjp6BUw+eUNYx\nctR54oGmheMqm4HcH4RcIxFXuHonnSUXcsmjiWpLX+cmku8hkyIQUaw5PqbrOppBlXwGSeca9WcG\ngTCIF4lQLt/56Oyyj4n2GZT/+0AzE/V3Ymct1fZLGiveknBt6XpoBtXYN/gctq2/nPkNpjT1VTIT\nORPfWDMYuLj2+Oq8xI5pI7Hv+KEVHRs9qaz8aIlGVlvrQfxoIvUzgRKOzSsV6u5R/ceBXA0zkbxc\nORM9zbmJbNNRlTSDRh58DQJhUF2fQbkzF1WShpYaI6JYGFSVynITAak6BSj0VcbLvs5NVJFm4ARq\nuNtczSBZeWo5A7m/0PDCwJ2BXD/iOicriaOmho8H61viL25DyufKO8C/n3801myNlwLdXA6LhstN\nZNfrslLAhPgMkoeW1jBRXT8hUVdCRKcT0atEVCKiDmX7dCLqJqJF9r/fKb8dSkRLiGgVEV1FNQ4v\nqeVC1uWSNGspawa1p5JlL72Tzsrrlt8zeTg+fsjEso4xlaPWSc7KEgZVuF6yFeDcY2Q0UbUmnfWH\nfqRWJB1XLgXwKQBPGX57Qwgxx/73JWX7NQDOBTDL/ndywjKEUqqyzyAJlXT23t+DHWRMdYhtJvJ8\n9qej6CuO2mcMjt1vLL5bQUBDOZTTmVYjY4O8XlmagUEeVkszkOeuxXoG/YVEZiIhxGtAWREYEwAM\nE0I8b3//M4BPAPhHknJElBFA/wjBjJxUFnOGZTnnZMqjkk6DqH6O/JZsGtd/fm7Nr9P3i9tYf5MO\ndqrmM7D/NnLARi0tzjOIaCERPUlEH7C3TQKwQdlng72tZoRmLe1jomcYR/1u0AwauHLWg9iPk7wf\n+8Ngo5aUZbqvogM5ag2DKNhnEJ9IzYCIHgFgmiV1sRDinoDDNgGYKoTYSkSHAribiA6E2ZwYaOwk\nonNhmZQwderUoN1C6U8zkKOIElimXxt5pFIPYk86Ux3IVD8zUV9R3qSzaoSWlj+iH9piLU969hHT\nnG2OMKmOlaihB1+RwkAIcXy5JxVC9ALotT8vIKI3AOwLSxNQVxyZDGBjyHmuBXAtAHR0dFTkIavV\nDORqIosWHVpqmGcwEKTcAKKiRHVUnQ6wUaiOA9n6W45m0NqUxpqfnOrZJmcMJ/WvOxGBDRy9V5Nb\nI6KxRJS2P8+E5SheLYTYBGA3ER1uRxGdDSBIu6gKpf4QWxpB3Ilx3O/XjmP2HQsgvolDCybqN6tV\n9QeqMfBKVaAZAFJLU6KJ7Ebj9AMVImqYqK6/kMiBTESfBPC/AMYCuI+IFgkhTgLwQQCXElEBQBHA\nl4QQ2+zDzgNwPYBWWI7jmjmPVfqzZiBJOg+BqZzf/euh2Ly7p6J0FJYDuVYlG3hUxWfgrGeQbLwq\nO++kMro/ZD+uNUmjie4CcJdh+x0A7gg4Zj6Ag5JctxxK/SiaKIqkoadM5bQ2pTFtdPwUwqR9a+RO\nolyqmZuoWjb6UkKV7YefOAg/vv81NGca1040eGYgD4C2Gi0MBsBNDBL8PoP6laXe3HHekRjaksF3\n7lpqb6mfmSjoPEnNd2d0TMEZHVOSnaSf0/DCwJ1n0P9ba1RqiYGg3QwW9HQUla501ggcOm2k9cF+\nJNUJLbX+JjUTyfMk1QwGA42r89j0pxnIUQwWzaAhZnF6NIPGDy2NRRUj96qmGaSqoxkMBgaRZlDn\ngsQg6XoHA4WFl5yAUqnepageqmbAVCdwz51nkEwayLKwZhDN4NEM6ujg22fcEABAe8TShlFlbJQO\nZ2hLFsPbsvUuRiJ8Kawb49VUherkJrL+JnYgD2LzXbk0vmbQD2Yg/+wzB+OfDpuCKaPaQvdrkL5+\nUOCdZ+DGttc6e+hAoBoDr2qtUCYP59cSTcMLA2mOqKe9vb05gw/ak5pMDGvNYsvu3sjzNIpmYKI5\nk8J7JvkXV++v6PMMGvjVOFz3uQ5MjxN+W4Vn4ZqJkgoDFtJxaXxhUKM1kKvJbV88Ao+v2IyWbPg6\nuI1siljxo1PqXYSy0F9FIwtqyXH77xVrv+o4kK2/1dIM2GcQTcMLA0l/bqzTx7Tj82NmRO7XKNFE\njQD7DAyQ508iUlXSDKTJKmE2ikHBIHAg199nUC0a4R4aBe88AxrU8wwcqhpaav1NGobsrgSXsECD\ngEEgDKy/jTCoZs2g/6BrBoxLNZ6HrOtJBQs79uPT8MJAOD4DbrFMbSByJzdxLavu4jYyBXXFZbH/\nss8gmoYXBgNhPQNm4OHRDEBuCGN9itOvqE5oqfU3abuVcx7YZxBNwwuDgTQDuRzGDGmudxEYG6LG\nTm0cm6rmJrI1g2qFliYuUePT8NFE/WEGci146lvHIl/kKl4v/Inq6leWfkMVq2O15hmwzyA+DS8M\nGlUzaItIbcHUFl72srZUa57BMbPGYubYdpx/3D5VKFVj0/A9yg3PrwXAjZWpLqR947QH1aVa8wyG\nt2Xx2DePrUKJGp+G9xncv+RtABz+x1QXfzoKrmDVpFqaAROfhtcMJAM5mujBr30QizfsqHcxGAXS\nPg/c2lVFqvgQZKhutZa9ZKJJpBkQ0RVEtJyIFhPRXUQ0QvntIiJaRUQriOgkZfvJ9rZVRHRhkuuX\nw0AeYOw3fihOb/Al9wYaXp8BL25TbRwzUcJ5Bkx8kpqJHgZwkBDiYAArAVwEAEQ0G8CZAA4EcDKA\n3xJRmojSAK4GcAqA2QDOsvetOdxYmWriMRMp2zlqpTpUbT0DJjaJhIEQ4iEhRMH++gKAyfbn0wDc\nLIToFUKsAbAKwFz73yohxGohRA7Azfa+NYdtukyt4KplU0U52JxJI52ixDOQmfhU02fw7wBusT9P\ngiUcJBvsbQCwXtv+/qATEtG5AM4FgKlTpyYq3ED2GTD9m0abw9If+Myhk7Hf+KFozoSndWeqR6Qw\nIKJHAIw3/HSxEOIee5+LARQA3CgPM+wvYNZEAscTQohrAVwLAB0dHYnGHQPZZ8D0b9RookE9/6OK\nbWxke1PoglBM9YmsuUKI48N+J6LPAvgogA8L12C6AYDq8ZwMYKP9OWh7TWjNptGdL/Lojakpw1uz\n+Pap++OE2aZxE8P0f5JGE50M4AIAHxdCdCk//Q3AmUTUTEQzAMwCMA/ASwBmEdEMImqC5WT+W5Iy\nRNGSTdllreVVmMGMrFvnfnBvzBgTY1lIhumHJNVpfwOgGcDDtpr8ghDiS0KIV4noVgDLYJmPviKE\nKAIAEZ0P4EEAaQDXCSFeTViGUKylJPPsM2BqBgcnMI1AImEghAhM+CGEuAzAZYbt9wO4P8l1y6HV\nXleY2ytTK7hqMY1Aw6ejaLaFQZETmjM1ggcaTCPQ8MJA+gx6C8U6l4RhGKb/0vjCwI5T7smX6lwS\nplHhSDWmEWh4YXDotJEArNA/hqkFbCZiGoGGnyHz9RP2xQmz98JBk4bXuyhMg8KygGkEGl4zSKcI\nh0wZEb0jw1QKSwOmAWh4YcAwtYZ9BkwjwMKAYRLCPgOLEw7YCwAwaURrnUvCVELD+wwYptawLLA4\n5wMzcEbHFAxv42CNgQhrBgyTEE5HYUFELAgGMCwMGCYhLAqYRoCFAcMkhBUDphFgYcAwCeFoIqYR\nYGHAMElhWcA0ACwMGCYhbCZiGgEWBgyTEJYFTCPAwoBhEsKhpUwjwMKAYRLCooBpBBIJAyK6goiW\nE9FiIrqLiEbY26cTUTcRLbL//U455lAiWkJEq4joKuJhFTPA4RrMNAJJNYOHARwkhDgYwEoAFym/\nvSGEmGP/+5Ky/RoA5wKYZf87OWEZGKaucGgp0wgkEgZCiIeEEAX76wsAJoftT0QTAAwTQjwvhBAA\n/gzgE0nKwDD1hjUDphGops/g3wH8Q/k+g4gWEtGTRPQBe9skABuUfTbY2xiGYZg6Epm1lIgeATDe\n8NPFQoh77H0uBlAAcKP92yYAU4UQW4noUAB3E9GBMPvaRMi1z4VlUsLUqVOjisowdYE1A6YRiBQG\nQojjw34nos8C+CiAD9umHwghegH02p8XENEbAPaFpQmopqTJADaGXPtaANcCQEdHR6DQYJh6wj4D\nphFIGk10MoALAHxcCNGlbB9LRGn780xYjuLVQohNAHYT0eF2FNHZAO5JUgaGqTesGTCNQNLFbX4D\noBnAw3aE6At25NAHAVxKRAUARQBfEkJss485D8D1AFph+Rj+oZ+UYQYSLAuYRiCRMBBC7BOw/Q4A\ndwT8Nh/AQUmuyzD9CZ4qwzQCPAOZYRLCooBpBFgYMExCWDFgGgEWBgyTEDYTMY0ACwOGYRiGhQHD\nMAzDwoBhGIYBCwOGYRgGLAwYhmEYsDBgGIZhwMKAYRiGAQsDhmEYBiwMGIZhGLAwYBiGYcDCgGEY\nhgELA4ZhGAYsDBiGYRiwMGAYhmHAwoBhGIYBCwOGYRgGVRAGRPRDIlpMRIuI6CEimmhvJyK6iohW\n2b+/Tznms0T0uv3vs0nLwDAMwySjGprBFUKIg4UQcwDcC+ASe/spAGbZ/84FcA0AENEoAN8D8H4A\ncwF8j4hGVqEcDMMwTIUkFgZCiF3K13YAwv58GoA/C4sXAIwgogkATgLwsBBimxBiO4CHAZyctBwM\nwzBM5WSqcRIiugzA2QB2AviQvXkSgPXKbhvsbUHbTec9F5ZWgalTp1ajqAzDMIyBWJoBET1CREsN\n/04DACHExUKIKQBuBHC+PMxwKhGy3b9RiGuFEB1CiI6xY8fGKSrDMAxTAbE0AyHE8THPdxOA+2D5\nBDYAmKL8NhnARnv7sdr2J2Ken2EYhqkB1YgmmqV8/TiA5fbnvwE4244qOhzATiHEJgAPAjiRiEba\njuMT7W0MwzBMnaiGz+ByItoPQAnAOgBfsrffD+BUAKsAdAH4PAAIIbYR0Q8BvGTvd6kQYlsVysEw\nDMNUSGJhIIT4dMB2AeArAb9dB+C6pNdmGIZhqgPPQGYYhmFYGDAMwzAsDBiGYRiwMGAYhmHAwoBh\nGIYBCwOGYRgGLAwYhmEYsDBgGIZhwMKAYRiGAQsDhmEYBiwMGIZhGLAwYBiGYcDCgGEYhgELA4Zh\nGAYsDBiGYRiwMGAYhmHAwoBhGIYBCwOGYRgGCYUBEf2QiBYT0SIieoiIJtrbjyWinfb2RUR0iXLM\nyUS0gohWEdGFSW+AYRiGSU5SzeAKIcTBQog5AO4FcIny29NCiDn2v0sBgIjSAK4GcAqA2QDOIqLZ\nCcvAMAzDJCSRMBBC7FK+tgMQEYfMBbBKCLFaCJEDcDOA05KUgWEYhklOYp8BEV1GROsB/Au8msER\nRPQKEf2DiA60t00CsF7ZZ4O9jWEYhqkjkcKAiB4hoqWGf6cBgBDiYiHEFAA3AjjfPuxlANOEEIcA\n+F8Ad8vTGS4RqE0Q0blENJ+I5m/ZsqWc+2IYhmHKIBO1gxDi+JjnugnAfQC+p5qPhBD3E9FviWgM\nLE1ginLMZAAbQ659LYBrAaCjoyPKBMUwDMNUSNJoolnK148DWG5vH09EZH+ea19nK4CXAMwiohlE\n1ATgTAB/S1IGhmEYJjmRmkEElxPRfgBKANYB+JK9/TMAziOiAoBuAGcKIQSAAhGdD+BBAGkA1wkh\nXk1YBoapC3/+97nY2Z2vdzEYpiqQ1Uf3fzo6OsT8+fPrXQyGYZgBAxEtEEJ0xNmXZyAzDMMwLAwY\nhmEYFgYMwzAMWBgwDMMwYGHAMAzDgIUBwzAMAxYGDMMwDFgYMAzDMBhAk86IaAusWc6VMAbAu1Us\nzkCA73lwwPc8OKj0nqcJIcbG2XHACIMkENH8uLPwGgW+58EB3/PgoC/umc1EDMMwDAsDhmEYZvAI\ng2vrXYA6wPc8OOB7HhzU/J4Hhc+AYRiGCWewaAYMwzBMCA0tDIjoZCJaQUSriOjCepenWhDRdUS0\nmYiWKttGEdHDRPS6/XekvZ2I6Cr7GSwmovfVr+SVQ0RTiOhxInqNiF4loq/a2xv2vomohYjmEdEr\n9j3/wN4+g4hetO/5FnvVQBBRs/19lf379HqWPwlElCaihUR0r/29oe+ZiNYS0RIiWkRE8+1tfVq3\nG1YYEFEawNUATgEwG8BZRDS7vqWqGtcDOFnbdiGAR4UQswA8an8HrPufZf87F8A1fVTGalMA8E0h\nxAEADgfwFft9NvJ99wI4TghxCIA5AE4mosMB/BTAL+173g7gC/b+XwCwXQixD4Bf2vsNVL4K4DXl\n+2C45w8JIeYoIaR9W7eFEA35D8ARAB5Uvl8E4KJ6l6uK9zcdwFLl+woAE+zPEwCssD//HsBZpv0G\n8j8A9wA4YbDcN4A2AC8DeD+syUcZe7tTz2EtJ3uE/Tlj70f1LnsF9zoZVud3HIB7AdAguOe1AMZo\n2/q0bjesZgBgEoD1yvcN9rZGZS8hxCYAsP+Os7c33HOwTQHvBfAiGvy+bXPJIgCbATwM4A0AO4QQ\nBXsX9b6ce7Z/3wlgdN+WuCr8CsC3YK2tDlj30Oj3LAA8REQLiOhce1uf1u1M0hP0Y8iwbTCGTjXU\ncyCiIQDuAPA1IcQuItPtWbsatg24+xZCFAHMIaIRAO4CcIBpN/vvgL9nIvoogM1CiAVEdKzcbNi1\nYe7Z5ighxEYiGgfgYSJaHrJvTe65kTWDDQCmKN8nA9hYp7L0Be8Q0QQAsP9utrc3zHMgoiwsQXCj\nEOJOe3PD3zcACCF2AHgClr9kBBHJgZx6X849278PB7Ctb0uamKMAfJyI1gK4GZap6Fdo7HuGEGKj\n/XczLKE/F31ctxtZGLwEYJYdhdAE4EwAf6tzmWrJ3wB81v78WVg2dbn9bDsC4XAAO6XqOZAgSwX4\nI4DXhBBXKj817H0T0VhbIwARtQI4HpZT9XEAn7F30+9ZPovPAHhM2EblgYIQ4iIhxGQhxHRYbfYx\nIcS/oIHvmYjaiWio/AzgRABL0dd1u96Okxo7ZU4FsBKWnfXiepenivf1VwCbAORhjRK+AMtO+iiA\n1+2/o+x9CVZU1RsAlgDoqHf5K7zno2GpwosBLLL/ndrI9w3gYAAL7XteCuASe/tMAPMArAJwG4Bm\ne3uL/X2V/fvMet9Dwvs/FsC9jX7P9r29Yv97VfZVfV23eQYywzAM09BmIoZhGCYmLAwYhmEYFgYM\nwzAMCwOGYRgGLAwYhmEYsDBgGIZhwMKAYRiGAQsDhmEYBsD/B9DhKBetWZmOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d7b6eb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Initialize environment and agent\n",
    "environment = GridWorld()\n",
    "random_agent = RandomAgent()\n",
    "\n",
    "reward_per_episode = play(environment, random_agent, trials=500)\n",
    "\n",
    "# Simple learning curve\n",
    "plt.plot(reward_per_episode)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Q-Agent\n",
    "\n",
    "- Here the Q-Learning agent is ran for 500 trials again\n",
    "- Performance is plotted\n",
    "- Performance increases greatly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10dace2b0>]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD9CAYAAAC/fMwDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcHHWZ+PHP0z33TObIMTnmSCY3\nIQeEySWHJAQIl0FBDYsEMW50N6i4ugqiiCL+WF1FWY81At5yqSwBo4GEKOEMCSQhJxnINbnPmRxz\n9/f3Rx1T3VPdPZOeI+l+3q9XXumurq6uqql66qnn+60qMcaglFIq+QV6egaUUkp1Dw34SimVIjTg\nK6VUitCAr5RSKUIDvlJKpQgN+EoplSI6JeCLyKMickBE1nuG9RaRF0Rkq/1/UWf8llJKqdPTWRn+\nr4FZEcPuBJYZY0YAy+z3Simleoh01oVXIjIEeM4YM9Z+vwW41BizV0QGAv8wxozqlB9TSinVYV1Z\nw+9vjNkLYP9f3IW/pZRSKo60np4BEZkPzAfIzc29YPTo0T08R0opdXZZvXr1IWNMv3jjdWXA3y8i\nAz0lnQN+IxljFgILASorK82qVau6cJaUUir5iMiO9ozXlSWdRcCt9utbgWe68LeUUkrF0VndMh8D\nXgNGiUi1iMwDHgAuF5GtwOX2e6WUUj2kU0o6xpibonx0WWdMXymlVOL0SlullEoRGvCVUipFaMBX\nSqkUoQFfKaVSRI9feHUmW7JhH+NKChhUmM3yzQcY1i+P8j45YeO89t5hinLT2binlhMNzUwfVcz6\n3TWcaGjmxgtKEREAnlmzmw8M68uLm/eTnZFGWkAY3CeHcwcVALD7WB1b9tVS3CuL+qYWKof0Pq15\nbgkZ/rR6FzdMLCUtGP14vr+2nje3H2Fo3zxe2Lifj00q5Z9bDvLhiSVkpgXZdeQU7+4/zmXn9D+t\n+fDT3BLiV69s53h9E71zM7j1A0MQEdbvrqGhOcQFg6376z27dg+TK3rzf2/vpqJvLlecOwCAFzbu\np76phZAxbDt0kmvGDWRE/15tfmfboZNsP3yS6aOK2XH4JH95azd5mWl86qIKggGJO5+rdxzh1arD\nfHxSGS9uPsAlI/vx1KpqWkIhJpQV+q6T+qYWFq3Zw40XlBKwf+Pg8QYeW7mTYED4aGUpK949xOSK\n3mzZd5zi/Ewam0Pu33nR2j1cOKwPffIy20zbGMNTq6u5bHQxSzft56MXlBEICFUHjnPgeAMfGNYX\ngOP1TSzZsJ8bJpbwjy0HqW9qoTg/i7rGFgYUZDG8OM93eU81NvPc2r18tLKUl7Yeoqwom6H92o67\n3V6vhTkZABw52UBF3zzW7DrK9FHF7D5Wx5L1+xhYmM1Nk8s5erKR37++g7LeOQQDQl1jCx+tbN0n\nojnZ0Mzid/aG7T9ez62zto9/bD7I5WP6s3zLAT4ysTTmNL12HD7JewdPMGN0+N/xmTW7+eDIfhw8\n3sCB4w2kBYTczDTGlhTEnWZTS4hfvbKNopwM9tXUU94nh7zMNC47pz/VR0/xp9XVXDNuIEW5Gazc\ndoSrxw1s9/x2Jg34URw92chnfreaCaUFPDZ/Krf9+k3Kemez4isz3HGMMdz0y9cjvrnBfZWVHuS6\nCYOoOnCcLzy+hkEFWeypqQ8be/sD1wBw7UMrOHqqqc3wjvr96zv45qINNDaHuGXakKjjzf/tKtZW\n13DB4CJW7zjKi5v3s7a6hqoDJ/j6tWOY/dNXOHKykfe+e3W7gmR7vL3rGPcv3uS+n1BWyPnlRVz7\nPy8D1jIfqK3nc4+9Hfa9rfdfRUNziH/9bfhFeRv31LJwbmWb3/nZ8iqeWbuHdd+8gp+8WMVTq6sB\nGD2wFxePiHsxIl98Yi07j5xiza5jLNt8gAllhazddQyAgQVZvgH/5/94jx8v20pOZpBrxw8CrADy\nwxfeBeD7S7YAMLw4j11HTtHQHHKX+b2DJ/j8Y29zxZj+vsuzasdRvvKnde773Mw0rh0/iJk/fMmd\nBlgHjbufXk9JYTbzf7eKppbw+2RF26aefHMX9z67kTGD8rn10ZVRx73iRy/RaM93pI9MLOFEfTPP\nb9wPwIzRxTy/YR8/sJffMaw4lwsGx05m7l+8iT++sZPy3jlMGdon7LOTDc3c/kfP9vFn679RA3q5\nyVM80//7H4QMvP/dq92D86ETDXzh8TVMKC1g/Z5aWkKt6649++Kq7Uf57uLNYcPSAsLab17Bb1/b\nwcKX3mfH4VPsPlbHym1HWPX1mfT1Obh3NS3pRPHm9iMA1NQ18fZOa2evrWsOG+e9gydiTmPlNmsa\nb9j/RwZ7L2+wB+tgcjp2HD4FQF1TS8zx1lbXALB6x1EA3j90EoDX3j8MwJGTjYCVpXaW6qPWvD0x\nfyrQuo69/NbRhj217nx6rdpx1Hc9VR+to7E5xLrqGlZuP8KFw/sQEHhzW9vfi1RT18TOI9Z8bt53\nHIC1u45R3CuTeRdVUFPX5Pu9Y6es9bXPM/+HTjSSHhQKc9LdYVUHTrjBHmBvTR2rtx+1p+E/7bd3\nhi/7Uftv42hobnGXG2DhS++1CfaxrLT/Dpv21rrD/NZrtGAP1r6x62gdTm5QfbTOnR+vU42xt0uA\nXfb6j9wngKjrv73La4zBieWHPevxRL21b6+trgkL9gB17ZhnZ9v2ag4Z3t55zP1s99E6dtvrZO+x\n6LGgK6VUhv/gC+/yp9XVvHLnjDafXfXjFVwysi93XXUO0BoIy3rnsMreIUf2t05zb3nkDdbuOsYt\n0wbH/L0Ne6ygGivQXP7DfzLM5/S54q7F7us+uRn84z8v5SM/e5XPfHAYJ+qbuPfZjRT3yuS5z1/E\n3EdWcuW5A/ji5SM5Xm/tEN9dvJm8zHT+ZUo5YAXwj/7vq/zgY+cxwufU/ri9wW/YU8uf7IwY4N/+\nsJpNe2upbwqREVEimjK0N7+bNwWAi7/3IruOWBvzgPwsln/5Uq740T/ZX9PA/EuG8sSqXfTJtUoB\nE8oKqeiby8ptR5l3UevONexrixlf2jZLu/6nr/iuuyMnG3nv4Anue24T/3z3IBeP6Mvv5k1h9zFr\nPj72i9cAuGXqYGrrmnnoxSr+b80eln/5UpZvPsB/PLmGV++6jLmPvMG14wfx4NJ33fUAuNMBKCnK\npldWGqcaWzjnG3/nE1PLeXP7UesMcOUuGlusYHiqsYVP/2YVL717kMaWEP3zMxlfWsgLduYb6ZLv\nLXeD1eGTDUz41vOMLy3gd/Om8NCyre4Zgtcf3tjJt5/b6L6/4eevsn53a7BevuWg7299+jdv8vCt\nkwiFDFc/tIKjpxp5/osfdBMT7wF4zD1LCBlDRjDA0ws+wPDitqUzLxHYc6yOSUN688a2I9zw81d9\nxztyspGbH36dDXtqWXDpcH7wwhYy04L85d8/wP8s28rid/a56/Kzv1/NOQPzeexfpzDlu8v4xS0X\nMLAg23e6JxuaeXvnUW55ZCUhYzjV2MIXLhvBE2/u4uYp5fzmtR0smD7MPdMCeGf3Me54fA0L51bS\nKyt6KBz/rSUYA4GA0Ngc4o6ZI3hqVTUHjzewYPpw/rhyB5lpQd/vrtx22A3yu4/VkZVu7UPX/eRl\n0gJCQIS50wbz9WvHxFy/nSWlMvwfL9sathM7jDFs2lvLL/75vjvMySSO1zezx/6Ok/Ss2HqI2vpm\nnnizOmw6pUXhG2NTi8EY42b4frYeOMHfN+yLOd+HTzbyhzd2svXACb781Frufdba2Q8cb+CFjfvZ\nvO84P162FYDa+tYM6GtPv+O+fvW9Q7x38CRrdx1jX23s7OKv6/a4r9/eeYz6JmsHnDasD/MurmDe\nxRVMHdaHFVsPcfhEA8frm9xgD7Cvtp5/vnuQXUfqaGwJ8ZPlVRw83sDmfcfpk5tBVnqQyUN68+b2\nI+ytaf1ei50ROQbkZ1FS2LpOh/XL5TvXj+X+D4/lmQUXAtbZ0z/ftQLciq2HaAmZsGkW5aRz4wWl\nTK6wygg7j5ziwPF6frK8itr6Zpas38dbO4/xwN82u8H+/PLCNn/LQYXZ5GdZmXpdUwu/XLGNNbuO\nsXzLQTdAgdU28nJV67CinAymVPiXMD77wWHkZbYGmvcOnqSmrokVWw8RCpmwYH/JyNZS1OZ9x8My\nWm+wj2XppgOcaGjm0Anrb7G/toEn3tzJoRNWpusE/hmji2lqCdHQHOJ4QzMvbj5AKNQ2gy4tyuaO\nmSMIBoSt+49TU9fEpIi2p+HFeXzn+rF8/Rorkdqy7zivVB3m2Kkm7l+8ifqmEDV1TbxTXcPi9fvI\nzggPnJv21vKLl96noTnEH97YGbZ9e9XWNbltPM5ZxI+XbWVfbT0/eOFdDp1o4FvPbuRUYwtXnmuV\n5H7/+k5q65t5eMU23zOPr19zDv955ShuvKCU5pBxz3B+tNSKI9kZQX716jb21za4Z4WRVm4/wm47\nm99XW0+6J2lqDhlCxvDsuj2nfUbfUSkV8B3LNoVnW95Tx1eqDlFzqsk9rVuz65h7ynu8vpl6T6nk\n0InWckdaQCjvHd6gGwgI1Ufr2FtTT07EhjwkovE3np8tr/Id7s0cH3l5G+/uDy8zLdu0n1+/so1n\n11pB/MjJxjYlAUdWeoCbJpdFzRC/MHMEX501mq/OGs3nZwwHrDOJR1/e3mbc/3hyDYAbaB0ldiCd\nVNGbmromvvPcpjbfdQwszApbb0P65PKJqYO5ecpgxpcW0K9XJq+/H34wfXLVrrBgePc1YyjMyWDS\nkNYHrj2yYpu7/r/01FqAsKD9vRvGt6mvlhZm+2aBO4+cCjsord9dQ31TyB2WHgy0CYKOr1w5qs02\n44jM7O+9rnMywP/622Yefnmb+37hS9brnIwg2+1y4AM3jAtb78s3H+Sp1bvaTGvG6GLumDmSOZPK\n3O+OHBB+JrCvpp5PTB3Mpy6sICBWRwgvZ/u491mr3em2C4e0+Z1H7PltaA7xy5feb/M5wNNv7+bl\nqkNxG1j79crkezdOAKx9Hawz6BMN1sHe+bv9+6XD+PTFQ1kwfTjfnj3Wd1o3TymPWoZzvLXzGIdO\nNFBSmE1LyLhlQsf00cXsr23g8Tfbrt+ukJIBf95vVvGWpy6621NrvPnhN7j9sbdo8Rxxt9n17eP1\nTRw95R8s++dntQnqoZBh/W6rrDPL7mniKO6VFXMeI6dVW9/sO94/PMH5vuc2uvPqmPebVdz77EaW\nbLAODEdONrq1y4q+uQAUZFuZa31TqE2ABitLBhjraRQbZ5de/vxWNQ8ubVt2cDKmGyaW4O1o4QS4\nC4dbjXGxzm4mlBbyuctGuO+9JTQRYWJ5IS9vDT843fUX66zmsx8cFvY7kytaG/8efnkbi9buwU8w\nIAwvznPLT0P75lLcK5PzywvplZXu+x3vOnPaRpxhdU0tnDso313uyZ7gHwgI+dn+0/xJxAG+om8u\nY0vyfcf1mjttML2y0hhY4L99/e51qwERrAz90IkGKvrmMmustX0OKsiib27rwS47Pchr7x/mq39+\np820rhhjfafEczZUUpjNqP69uHiE1XNowfTh7rIW5WTw3sGTBAPCxyvLALjxAqt3jRM4b5na+jce\nV1LAyP55bmb90rsH3UbhSM9v3M+66homlBZw51XRb7FeUphNQXY6FX1z3baUA8frOdVgba/Oeiuy\neyKBddCO3C8uGdnP7R3lKOsdflb46YsqaGoJIQJzJpX5zs9Nk63h3352Y7dk+SlVw/c6EVarDT8d\ne//gSXeH96qtb3YbMx198zI4dKKRPnkZZKaHB+mWkHFLSB+eWMJf3t7tflacb+1U00f1Y0JZIT9a\nutX97L3vXs0Df9vEL1dsC5teVnrALa+8/Y3L+e/nt/CHN3a6n88YXcyDHz+PCd96HoDKwUWsimjs\nPHKqNcMvLcpm26GTTB/Vj/9bYwVAb2AsyE6npq6J+64f6/Y8cWSmBfnVJydx26/fbLOeCnPS3R34\nA8P6svFbs5j/u1Ws2HrIzXYHFmRz24VD+NUr29t8/7LRxfz4pvPJzQgiInxowqA244B18HAOZPMv\nGeoGspsml3PnVaPDdvzeuRls/PaVjLlnCQAhAxNKC9wADVYJ6fWvWbd/KrL//mW9c/jNpyYDVlnM\n8fc7LmbWj1bY66w3T3v+tt5hdY0tpAUDvPSV6QCcaGhm7DeXuOM5Zw1FOeltGin/9xMT+ezv3wKs\nA9xzn7sYY9en/+PJNSzZsJ+PnF/C/7thXFgN+duzx/KFx9/mmTV7+Po15zB32hAy0gKsqz7Gh37S\n2h7y189dDGIlF0ERvnnduWSnBwkExO0OOW1YH17c3Hpn8+c+dxHDi/PI8mzr3jOc0qJslnzxEvwU\n5WZw+GQjRTkZPHDDOL5x3RjyMtPcHkjfv3E8ffIyw3rFNLWEOHqykcnfXeY7Tb/f+OwHh5EWEL7z\nV+vsce09V7B8ywHueGKNe3B6/ouXcKqxhS89uYadR05xstGKB33yrL97KCL4PvmZae7rU43NZKYF\nwxLFRbdfSH1TyG03cpbhP64YiTFWz6rM9ADfXbyZq8cNYPE7VqIzdWgf/vPKUXx/yRZq6prcLq9d\nJSUzfIC5j65klV2q2R3RYr77WJ0bAL1ONDRzzUMvhw0bYGcEBdnpZKW1DfjVR+vIy0xjWkT3sv75\n1veyM4JtsrFgQNys22tk/14MLMhiZP88inIzGFQYnlGU984J+97MMW27D67efpQ77SzYOcsYV1ro\nfu7deS8dZdWNi6JshEPsM4RIgz1lij55GWRnBN0AcV5Z629FnsW481CUTV5mWtz+2t7l957KR+tv\nnpMRnt8U57eu90EFWQwqbH3vHPC9B/58T4Y/tG/rb4zzKSM4JSRvDx2A3Ihl7pVpfX7OwHx7HoPu\n90qL2pZ7RKy+4bn2suRnp0dtMATrb5eRZu3mYyO6LRbkpFOQnU56MEDA3uaccZ1tstJTCgPrb54V\nkdg47R3pQaFfjK6GvT3rVETC2i8AhvZruz2lBwMdCoLOWZh3Oy7ISXcz9FJPqa0g21rHWw+ccA86\nFfbfNT3GNSw5GWkEA+Lu+wBlRTnkZ7fNn3My0si1l3OqHQO8JcecjDSG2vuRX6+mzpayGT7Afz+/\nhcfnT4ta0y7KSSc/O93t6uhnQH4263fXkpkWJDM9fCNpMVaGX1KYTVowwE//ZSIL/mhlbM7GnhYI\ntAlEAJ++eChg9eV/Zs0e3tldQ25GGl+4foS7U0Y2LOZH1Jg/MXUwzS0hrpswiOfW7eWtHUdZ5snW\nnOzSGMMvbrnA3SEf/WQlaYEAE0oLGdm/l7uhRvIeqO65dgx/X7+PlduPhC1Pth0c7ps9likVvcN6\n4QSjBPSSQv+eGLHGG+o5+JQURi+X/c9N57v9/HvnZLDo9gvZdugkvbLSwgLnRytLOdXYwg0XtF7Q\n4w34GWkB7r1uDAdPNHDOwHx+POc8nlpVzctVhwgGhGH98rhv9rlMHx3+ZE8R4Ycfm+D2GXcaKUcP\nyOfV9w5T3juHm6cO5qLhfd2/s5+czGDY96PxruJAQPj5zRPZfayOfr1i9wF/+NZKXtx8gOERPcgi\ngzRASaF1YBpYkO32a/fjJDlFuf5lrMgExpHmM82H51ay+1gdBdnp3PHEGne4sw9cNKIv8y6qcLe3\nQYXZ3Hf9WC4aHl6GuXlKOQePN/DXd/YCsGD6MHplpbm922Lx/n0KstPjdoUeV1LAPdeO4apxA7h9\n+nA3wDvLvftYXbsu8kpEUgX82vomdh4+1e6V5jTMHo/S8t83L5MF04eHbVCRBhRYO05meqBNhh8K\nGXYfrXMzx2vGD2TBH63P0oPWRpwWFPe1V1Z6kNtnWPXrjXtrrYBvX7nniOyilh1x4MjLTHOnsWD6\ncO7887qwz53fbWoxXOlpY/BegejUYP14M71PXVTB8OI85toX7jicLH1AQZZ7EHMEA/4BLdqOH8lb\nO+6Tl+GWvJwA5Oe6CYP41SvbeGvnMXrnZTC+tJDxnjMcx/DiXtx3fXhjnXOAdOLPJy+scD+bfV4J\n7x88yctVh9xSVLQL37xXhToBeWCB1QbUJy/DrWPH6v/tbGvZ6f4B39miIo+pV7XzCs/SohzmThvC\njsMn447br1cmaQGJe6B2Po92xhitXcvvIOI9e/Xun06G3ysrnW9EdHX0tg84RvTvxTeuHeMG/F5Z\n6TG3+WgCAYnZtROsfeFTF1nbzMCCbCbYZ7vOdry7GzL8pCrp3ProSveqzfZotgN+tAbRYEDcP8o1\n4/13lMJsa+PNTAu4fWwdLcawr7beN4A5tfJZ5w5wT+ejcU7fczMjevr0tQLbzHOsLNJpoHQazCJ5\nSxj98zOZPqrYnpci3/HboyA7ndF2z4xhdinl+vMHcc7A/KglG0fkWfPHKktJC0jc9eEY3CfXno7Q\nOzeDL84cCRC154tj1ABr+hV9/EtS0Tg79LyLKnw/d8/aYpQDInlLxeNKChjh6e/uZO8fq2x72wDn\n6udoZwHOATyyjNNR3qRiTJS/SzAgnFtS4DbkR+MEtsgLm5zSYawruv2yfEf//NazFb+ySjzFcc52\nYnFuBwKt++knPzCkQ9OwuioH3O7fXSmpMnxvH+72aG6JneEHRKjom8vm+2axrrqGv66zsoBnFlzI\nbPtioEx7h8tMC7apbTa3GI6eavS9P8r55YVsvm+W+51X7pzBhQ+86DsfTg0wN+J0urhXFpu+PYvs\njCD1TS3utH592+Q2jU7Obzpe/uoM0oOBsHk4Hau+PtPNJksKs93p3TCxlHh9DryZ24qvTKe0KJtv\nzx7b7vnJy0xj7T1XgFjrf/4lQ5k7bUjcMsd3rh/L52YMj9qTJZq0YIAt35lFepQzE6fMEojT9hDN\n7+ZNaRP03v3OVb7Bzjlzitax46pxAxP+24J1QNl83ywCIsSIuTz1mWlxb8Ex0E44Iksfj9w6qc1B\nIFIgIBAy3HbhEL529Tlhn73y1RkMv/tvQHjZrb1ilaHieWL+VHc7DwQk5vYRjYiw+PMXuyWvrtTl\nAV9EZgE/BoLAw8aYLn/UoTEmboMfeDL8uugZPlili96exru+nozAyfrys9PaZPiHTzZgDPTOabsR\npgcDePfFwijd86C1oS/ySldozQK9O3YwIARpu/zebMRplEo0IEQ2bjnTa0+W6w1kWelWGaSj81Pg\nWbciEjfYg7V+2ls2ihSrgdTJ8DqQ4Lvzm5EW8M3Wo2XwzqrzO7A7Ev3bdmQ6sdobHE4X1MhtJhiQ\nuAcLp70nIy3Q5vvebe10An4iIrfzWNtHLH43q+sKXRrwRSQI/BTrmbbVwJsissgYszH2NxNjTNva\npZ9m+2KbaFfveY/8vcN6a7SutjmTy9lTU8/nZozgz55bEgBuF8re7bhJUqydqj1BrD3ys9L55nVj\nfG9f0BO8mXCsU/azhXMGFq0x2s+C6cNpCRk+HqWfdjTOuvO7AvZMdcHgIm6fPpybp8ZvEI3kbB/x\ntpN4dfRofvOpye79kJJZV2f4k4EqY8z7ACLyODAb6NKAHzKGgE+GG6k5ZDh8ooGtB/xvguZtS/V2\nd8z1NI5mpQfdU8zMKFlO73Z0K4uV4ThnK6dZKQhzm6ehsad5lzmR0+ozhXMm1pFlyctMa1OiaI/W\nDL/DX+0xwYDw5StHndZ3nXUaraHfcboB/4Mj499FNRl0daNtCeC9ZrjaHuYSkfkiskpEVh086H9J\nf0f57QPNLW3v9NccCnHlj1ZErR96A1JkcPK7NUK0Hd17djDK5/7tXpHdxqD1zoXddLuNbhNt/Z6t\ncpwMvxuWxbmPvrddJpk5yU60s6fL7V47HWkwT0VdneH7/XXCwpYxZiGwEKCysrJTQppfYKz3ubVr\nS4sJux9OpFiNb3+/4xK3DcDhjD3znGKK87P4o30VrDfgP3P7hTT5HHwA1t17RZuuncksLOB3xulL\nD3O6SHbHslwysl+P3VO9J6X5dGEG+Om/TGzXbYxTXVcH/GrAW5wsBfxvYtKJjE+O3+BzUURkwI4U\nK1Pzq7k7pZdeWelh5R3vhSZZ6W178ziiNTi1t355tvEGxg52bDijtacBszOkWrCH6ElYtEZvFa6r\nA/6bwAgRqQB2A3OAf+ni32x3ht8cMowtyWf97lpmnzeIZyJupxAZ8P/npvM51ejfowfCT2ecYJYe\nlNNuuXd8bFIZm/Ye5/YZHb8g5EzmLYGlJUHEH1Gcxyc/MIS5cZ6ToE5fsiU93a1LA74xpllEbgeW\nYHXLfNQYsyHO17pEvU+G39QSornFcMWY/owekM8zEScfkdnEdVFu4uVwRjfGuAeLzghkORlp/NeN\n4xOezpkmLMNPgv04EBDu/dC5PT0bSclJ4pKhracndXk/fGPMYmBx3BE79TfbDmto8qnh2w81yEwP\n+t7eoKMbl9OX9vzyIvchI5qRROetx7bnugmlNOAn5uw/j/bhV8P3u0ClOWRoaA6RGaX+19ErJs8r\nK2T5ly9l7rTBbvYarZFJnf4VqSp1acBPTFIG/JCxHjD+w+e3uF0afQO+/Ri3jLSA71nB6fTwquib\ni0jrlYO6gUan60Z1lJ4xJyYpA74xhk88/AYPvVjlPt3Jr0NOyEBDcwuZaQHfHjuJBCTnu1qqiE4z\nfNVRyXCBXk9KzoBPayOt0zc32j1HrJJO0PfCrEQCkhvwT3sKyU+zNdVezpm6bjOJSc6Ab1oz+ou/\nt5zlmw9EfV5ko13D98vwEwn4znc1i41OSzqqo3SbSUxSBnxMeEb/2vuHY95zJCMt4N5e4V8vrqDS\nvqtkYiUd63+N99Hp6bnqKA34iUnKgG8wYY2woZCJeVdBb4afk5HmPjGrMzJ83TyjS4bbKajupSWd\nxCRnwI/I8FuMoSXGnccy04PMmVTGsH65fHxSmaeHzenPgzbaxqfZmuooLZEmJikDfsiYsIBvTOvF\nWN/zuWI1MxhgUGE2y750KYMKszulS2VrwD/tSSQ9Dfiqo/S6lsQkZcD/8M9edR8+AtYVtc4BwO+K\n2syIJ1V1RoOrNtrGp3eyVe3lPkZQ96eEJOUut/PIqbD3VsZvvfZ7gELkg0vSNMPvFrrzqo5Khpvs\n9aSUWHveEo9fo0/kbRWc3iNJmgrFAAATUklEQVQJ9cPXRtu4dOdVHaVlwMSkxB4XCsW+cCPy9sVu\nsE5g2wpoo21cGu9VR2nAT0xK7HIhYwjZJX2/Rp/Iko5TW07kkYJpeqVtXLrzqo7SbSYxXX575DPB\nn9+q5qnV1YB/DT+ypBPvQcntEdAaflzap1q1m5186TaTmBTJ8Ftft6eG3xm9R1rLQrqBRqONtqqj\nNMNPTEKhTUQ+KiIbRCQkIpURn90lIlUiskVErkxsNjuPX8BPD/p3y0yEe2uFhKeUvHTnVR2l20xi\nEi3prAc+AvzCO1BExmA9v/ZcYBCwVERGGmN6/LHyaT7pe0YXBHzthx+frhvVXs5Jugb8xCSU4Rtj\nNhljtvh8NBt43BjTYIzZBlQBkxP5rc7SnpJOZ9B++PHpVZOqozTgJ6aravglwC7P+2p7WI/zCzKR\nJR3vw8hPl3bLjE9vnqY6ShttExO3pCMiS4EBPh/dbYx5JtrXfIb5Rk8RmQ/MBygvL483Ownzu9jH\n73YLidILr+LT2yOrjtIyYGLiBnxjzMzTmG41UOZ5XwrsiTL9hcBCgMrKygR6vreP3ylhmwy/E39H\nt8/oNMNXHaVlwMR0VUlnETBHRDJFpAIYAazsot/qkPb00nEkcvTRRtv4grrzqnZyyqtaw09Mot0y\nPywi1cA04K8isgTAGLMBeBLYCPwdWHAm9NAB/wwhciPqjLq7ZvjxaYavOkq3mcQk1C3TGPM08HSU\nz+4H7k9k+l2huzIE7Ycfn2ZrqqP0hnuJSbm1110ZgnuWoBlJVFruUh2l8T4xKbf6OlKuSeTmac53\nNaRFp13slOpeKRfw2xNjOiPxdBqZNKZFp90yVXvNmWx12Y68lbnqmJS4W6ZXd5URnJMDvfBKqcTd\nffU5/OeVo7rkqvhUogG/izglHU1ilUpcICBkBTS7T1TKHS6lA0tsEuiJ7zxSUbSKr5Q6Q6RcwG9P\nht8ZIdpt8NV4r5Q6Q6RgwO+e3ykpzAZg+qji7vnBs9jUob17ehaUSglaw+8i5X1yWHn3ZfTLy+yW\n3ztbrbnncrIztDarVHfQgO/HHieRfvgAxb2yEptACijMyejpWVAqZaRgwI8/zo0TS1m36xhfumJU\n18+QUkp1kxQM+PEjfnZGkO9/dEI3zI1SSnWflGu01euglFKpKgUDvkZ8pVRqSrmAr5RSqSrlavhe\nt0wdzNShfXp6NpRSqlsk+sSr74vIZhFZJyJPi0ih57O7RKRKRLaIyJWJz2rnu+/6sVwzfmBPz4ZS\nSnWLREs6LwBjjTHjgXeBuwBEZAwwBzgXmAX8TET06hqllOpBCQV8Y8zzxphm++3rQKn9ejbwuDGm\nwRizDagCJifyW0oppRLTmY22nwL+Zr8uAXZ5Pqu2hymllOohcRttRWQpMMDno7uNMc/Y49wNNAN/\ncL7mM77vjQpEZD4wH6C8vLwds6yUUup0xA34xpiZsT4XkVuBa4HLjHHvPlMNlHlGKwX2RJn+QmAh\nQGVlZYJ3r1FKKRVNor10ZgFfBT5kjDnl+WgRMEdEMkWkAhgBrEzkt5RSSiUm0X74PwEygRfsK1hf\nN8Z81hizQUSeBDZilXoWGGNaEvwtpZRSCUgo4Btjhsf47H7g/kSmr5RSqvPorRWUUipFaMBXSqkU\noQFfKaVSRNLcPM104HmEf/38RQk/vlAppc42SRPwQx0I4OcOKui6GVFKqTNU0pR0QpqyK6VUTEkb\n8H9126QemhOllDozJU3Aj0zw2/OwcqWUSiVJE/AjM/yAxnullAqTRAE//L1m+EopFS6JAn54xNd4\nr5RS4ZIm4JtQ+Hvx3JJ/7T1XdPPcKKXUmSdpAn7MDF+zfaWUSuKA732tAV8ppZIp4Ie/F0+U13iv\nlFJJFPAj76XjzepFU3yllEqegN8mw++Z2VBKqTNWos+0vU9E1onIGhF5XkQG2cNFRB4SkSr784md\nM7vRxbqXjgZ/pZRKPMP/vjFmvDHmPOA54B57+FVYDy4fAcwHfp7g78QVq5eOVnSUUirBgG+MqfW8\nzQWcqDsb+K2xvA4UisjARH4rnlAocoi30VYjvlJKJXw/fBG5H5gL1ADT7cElwC7PaNX2sL0+35+P\ndRZAeXn5ac9Hi2b4SikVU9wMX0SWish6n3+zAYwxdxtjyoA/ALc7X/OZlG+R3Riz0BhTaYyp7Nev\n3+kuBy2h6P3wlVJKtSPDN8bMbOe0/gj8FfgmVkZf5vmsFNjT4bnrgLY1fE9JR6O/UkolVtIRkRHG\nmK322w8Bm+3Xi4DbReRxYApQY4xpU87pTDGvtEW47/qx5GclzRMdlVKqwxKNgA+IyCggBOwAPmsP\nXwxcDVQBp4DbEvyduNqUdCJq+LdMHdzVs6CUUme0hAK+MeaGKMMNsCCRaXdUZC8dQW+toJRSXklz\npW1kLx0vvbWCUkolU8CPUdJRSimVRAE/8uZpXhr7lVIqiQJ+vEZbpZRKdckT8Nt0y/T2w9eIr5RS\nSRPw2/TS0RivlFJhkibgx7qXjlJKqSQK+G2vtNWIr5RSXskT8LVbplJKxZQ0AT+yl45SSqlwSRPw\nY908TSmlVBIF/BbtpaOUUjElTcBv+xBzjfhKKeWVtAFfM3yllAqXNAFfH3GolFKxJW/A1xRfKaXC\ndErAF5Evi4gRkb72exGRh0SkSkTWicjEzvidWLSXjlJKxZZwwBeRMuByYKdn8FXACPvffODnif5O\nPJHd8DXBV0qpcJ2R4T8IfAXwhtzZwG+N5XWgUEQGdsJvRdW2hq8RXymlvBIK+CLyIWC3MWZtxEcl\nwC7P+2p7WJdp2y1TKaWUV9yHmIvIUmCAz0d3A18DrvD7ms8w34gsIvOxyj6Ul5fHm52o9BGHSikV\nW9yAb4yZ6TdcRMYBFcBau0dMKfCWiEzGyujLPKOXAnuiTH8hsBCgsrLytNN0vZeOUkrFdtolHWPM\nO8aYYmPMEGPMEKwgP9EYsw9YBMy1e+tMBWqMMXs7Z5ajzU/4e83wlVIqXNwM/zQtBq4GqoBTwG1d\n9Duutg9A0YivlFJenRbw7SzfeW2ABZ017fbQK22VUiq2pLnSVh+AopRSsSVNwG9T0tEcXymlwiRN\nwNcrbZVSKrbkCfghQ0CDvFJKRZU0Ab/FGIKeiK+xXymlwiVNwLcyfE+Y14ivlFJhkibgt4QiM3yN\n+Eop5ZU0AT9kIOjJ8LXRVimlwiVRwDdoRUcppaJLmoDfpqSjKb5SSoVJnoCvvXSUUiqmpAn4kb10\nNMFXSqlwyRPwIzJ8pZRS4ZIm4LeECM/wtaijlFJhkibgh4wh4F0ajfdKKRUmaQJ+S8hoP3yllIoh\noYAvIveKyG4RWWP/u9rz2V0iUiUiW0TkysRnNbYWYwhoLx2llIqqM5549aAx5r+9A0RkDDAHOBcY\nBCwVkZHGmJZO+D1fxkRm+BrylVLKq6tKOrOBx40xDcaYbVjPtp3cRb8FWCWd8EZbpZRSXp0R8G8X\nkXUi8qiIFNnDSoBdnnGq7WFdpiVEeElHI75SSoWJG/BFZKmIrPf5Nxv4OTAMOA/YC/zA+ZrPpIzP\nMERkvoisEpFVBw8ePM3FcPrhe6arOb5SSoWJW8M3xsxsz4RE5JfAc/bbaqDM83EpsCfK9BcCCwEq\nKyt9Dwrtob10lFIqtkR76Qz0vP0wsN5+vQiYIyKZIlIBjABWJvJb8YQieukopZQKl2gvne+JyHlY\n5ZrtwGcAjDEbRORJYCPQDCzoyh46AI3NIdKDSXNZgVJKdbqEAr4x5pYYn90P3J/I9DuioTlEr6zW\nxdGSjlJKhUualLixOURmWtB9r422SikVLmkCfkNzC5nprYujGb5SSoVLooAfItNTw9d4r5RS4Trj\n1gpnhIbmEJnpAX4/bwqDCrP01gpKKRUhaQK+U8O/aERfwHoCllJKqVZJVNJpISNNa/hKKRVNUgR8\nY4xV0klLisVRSqkukRQRsjlkMIawgK81fKWUCpcUAb+hOQQQVtJRSikVLikiZEOTddcG74VXSiml\nwiVFwG9ssTJ8reErpVR0SREhG5rsgJ+eFIujlFJdIikipFvDD2pJRymlokmSgO/U8JNicZRSqksk\nRYRsbNaSjlJKxZMUEbK1pJMUi6OUUl0iKSKkW9JJ1xq+UkpFk3DAF5HPicgWEdkgIt/zDL9LRKrs\nz65M9HdicUs6WsNXSqmoErpbpohMB2YD440xDSJSbA8fA8wBzgUGAUtFZGRXPde2MCeDi0f0JT87\nvSsmr5RSSSHR2yP/G/CAMaYBwBhzwB4+G3jcHr5NRKqAycBrCf6er6lD+zB1aJ+umLRSSiWNRGsg\nI4GLReQNEfmniEyyh5cAuzzjVdvDlFJK9ZC4Gb6ILAUG+Hx0t/39ImAqMAl4UkSG4v+EQd8nkojI\nfGA+QHl5efvmWimlVIfFDfjGmJnRPhORfwP+YowxwEoRCQF9sTL6Ms+opcCeKNNfCCwEqKys1MdU\nKaVUF0m0pPN/wAwAERkJZACHgEXAHBHJFJEKYASwMsHfUkoplYBEG20fBR4VkfVAI3Crne1vEJEn\ngY1AM7Cgq3roKKWUap+EAr4xphH4RJTP7gfuT2T6SimlOo9eqaSUUilCA75SSqUIDfhKKZUiNOAr\npVSK0ICvlFIpQgO+UkqlCA34SimVIjTgK6VUitCAr5RSKUIDvlJKpQgN+EoplSI04CulVIrQgK+U\nUilCA75SSqUIDfhKKZUiNOArpVSKSCjgi8gTIrLG/rddRNZ4PrtLRKpEZIuIXJn4rCqllEpEok+8\n+rjzWkR+ANTYr8cAc4BzgUHAUhEZqY85VEqpntMpJR0REeBjwGP2oNnA48aYBmPMNqAKmNwZv6WU\nUur0dFYN/2JgvzFmq/2+BNjl+bzaHqaUUqqHxC3piMhSYIDPR3cbY56xX99Ea3YPID7jmyjTnw/M\nBygvL483O0oppU5T3IBvjJkZ63MRSQM+AlzgGVwNlHnelwJ7okx/IbAQoLKy0vegoJRSKnGdUdKZ\nCWw2xlR7hi0C5ohIpohUACOAlZ3wW0oppU5TQr10bHMIL+dgjNkgIk8CG4FmYIH20FFKqZ6VcMA3\nxnwyyvD7gfsTnb5SSqnO0RkZ/hnrwY9PoH9+Vk/PhlJKnRGSOuB/+PzSnp4FpZQ6Y+i9dJRSKkVo\nwFdKqRShAV8ppVKEBnyllEoRGvCVUipFaMBXSqkUoQFfKaVShAZ8pZRKEWLMmXODShE5COw4za/3\nBQ514uycDXSZU4Muc2pIZJkHG2P6xRvpjAr4iRCRVcaYyp6ej+6ky5wadJlTQ3css5Z0lFIqRWjA\nV0qpFJFMAX9hT89AD9BlTg26zKmhy5c5aWr4SimlYkumDF8ppVQMSRHwRWSWiGwRkSoRubOn56ez\niMijInJARNZ7hvUWkRdEZKv9f5E9XETkIXsdrBORiT0356dPRMpEZLmIbBKRDSLyBXt40i63iGSJ\nyEoRWWsv87fs4RUi8oa9zE+ISIY9PNN+X2V/PqQn5/90iUhQRN4Wkefs90m9vAAisl1E3hGRNSKy\nyh7Wbdv2WR/wRSQI/BS4ChgD3CQiY3p2rjrNr4FZEcPuBJYZY0YAy+z3YC3/CPvffODn3TSPna0Z\n+JIx5hxgKrDA/nsm83I3ADOMMROA84BZIjIV+C/gQXuZjwLz7PHnAUeNMcOBB+3xzkZfADZ53if7\n8jqmG2PO83TB7L5t2xhzVv8DpgFLPO/vAu7q6fnqxOUbAqz3vN8CDLRfDwS22K9/AdzkN97Z/A94\nBrg8VZYbyAHeAqZgXYSTZg93t3NgCTDNfp1mjyc9Pe8dXM5SO7jNAJ4DJJmX17Pc24G+EcO6bds+\n6zN8oATY5XlfbQ9LVv2NMXsB7P+L7eFJtx7sU/fzgTdI8uW2yxtrgAPAC8B7wDFjTLM9ine53GW2\nP68B+nTvHCfsR8BXgJD9vg/JvbwOAzwvIqtFZL49rNu27WR4pq34DEvFrkdJtR5EJA/4M3CHMaZW\nxG/xrFF9hp11y22MaQHOE5FC4GngHL/R7P/P6mUWkWuBA8aY1SJyqTPYZ9SkWN4IFxpj9ohIMfCC\niGyOMW6nL3cyZPjVQJnnfSmwp4fmpTvsF5GBAPb/B+zhSbMeRCQdK9j/wRjzF3tw0i83gDHmGPAP\nrPaLQhFxkjLvcrnLbH9eABzp3jlNyIXAh0RkO/A4VlnnRyTv8rqMMXvs/w9gHdgn043bdjIE/DeB\nEXYLfwYwB1jUw/PUlRYBt9qvb8WqcTvD59ot+1OBGuc08WwiVir/CLDJGPNDz0dJu9wi0s/O7BGR\nbGAmVmPmcuBGe7TIZXbWxY3Ai8Yu8p4NjDF3GWNKjTFDsPbXF40xN5Oky+sQkVwR6eW8Bq4A1tOd\n23ZPN2J0UkPI1cC7WHXPu3t6fjpxuR4D9gJNWEf7eVi1y2XAVvv/3va4gtVb6T3gHaCyp+f/NJf5\nIqzT1nXAGvvf1cm83MB44G17mdcD99jDhwIrgSrgKSDTHp5lv6+yPx/a08uQwLJfCjyXCstrL99a\n+98GJ1Z157atV9oqpVSKSIaSjlJKqXbQgK+UUilCA75SSqUIDfhKKZUiNOArpVSK0ICvlFIpQgO+\nUkqlCA34SimVIv4/vXOWAjZfaJcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d7b6e10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "environment = GridWorld()\n",
    "agentQ = Q_Agent(environment)\n",
    "\n",
    "# Note the learn=True argument!\n",
    "reward_per_episode = play(environment, agentQ, trials=500, learn=True)\n",
    "\n",
    "# Simple learning curve\n",
    "plt.plot(reward_per_episode)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the final Q-value table with nice formatting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0, 0)\n",
      "\tUP\n",
      "\t\t-0.30000000000000004\n",
      "\tDOWN\n",
      "\t\t-0.34409100000000004\n",
      "\tLEFT\n",
      "\t\t-0.30000000000000004\n",
      "\tRIGHT\n",
      "\t\t-0.24656472100000001\n",
      "(0, 1)\n",
      "\tUP\n",
      "\t\t-0.2\n",
      "\tDOWN\n",
      "\t\t-0.2\n",
      "\tLEFT\n",
      "\t\t-0.1\n",
      "\tRIGHT\n",
      "\t\t5.066124678571429\n",
      "(0, 2)\n",
      "\tUP\n",
      "\t\t4.215552911753549\n",
      "\tDOWN\n",
      "\t\t5.210099618907776\n",
      "\tLEFT\n",
      "\t\t1.0221592868959593\n",
      "\tRIGHT\n",
      "\t\t9.999999999999995\n",
      "(0, 3)\n",
      "\tUP\n",
      "\t\t0\n",
      "\tDOWN\n",
      "\t\t0\n",
      "\tLEFT\n",
      "\t\t0\n",
      "\tRIGHT\n",
      "\t\t0\n",
      "(0, 4)\n",
      "\tUP\n",
      "\t\t-0.1\n",
      "\tDOWN\n",
      "\t\t-0.19639\n",
      "\tLEFT\n",
      "\t\t6.5132155990000005\n",
      "\tRIGHT\n",
      "\t\t-0.19\n",
      "(1, 0)\n",
      "\tUP\n",
      "\t\t-0.542819\n",
      "\tDOWN\n",
      "\t\t-0.5677967841\n",
      "\tLEFT\n",
      "\t\t-0.5000000000000001\n",
      "\tRIGHT\n",
      "\t\t-0.47162451809000006\n",
      "(1, 1)\n",
      "\tUP\n",
      "\t\t-0.29602\n",
      "\tDOWN\n",
      "\t\t-0.33782900000000005\n",
      "\tLEFT\n",
      "\t\t-0.30000000000000004\n",
      "\tRIGHT\n",
      "\t\t6.129179200289822\n",
      "(1, 2)\n",
      "\tUP\n",
      "\t\t8.999999999999982\n",
      "\tDOWN\n",
      "\t\t3.716164488305088\n",
      "\tLEFT\n",
      "\t\t1.4945660017237155\n",
      "\tRIGHT\n",
      "\t\t-4.0951\n",
      "(1, 3)\n",
      "\tUP\n",
      "\t\t0\n",
      "\tDOWN\n",
      "\t\t0\n",
      "\tLEFT\n",
      "\t\t0\n",
      "\tRIGHT\n",
      "\t\t0\n",
      "(1, 4)\n",
      "\tUP\n",
      "\t\t1.9989243432810007\n",
      "\tDOWN\n",
      "\t\t-0.22819\n",
      "\tLEFT\n",
      "\t\t-1.0\n",
      "\tRIGHT\n",
      "\t\t-0.2\n",
      "(2, 0)\n",
      "\tUP\n",
      "\t\t-0.8326566589000002\n",
      "\tDOWN\n",
      "\t\t-0.7978664724011002\n",
      "\tLEFT\n",
      "\t\t-0.7915234100000003\n",
      "\tRIGHT\n",
      "\t\t-0.8586721375310769\n",
      "(2, 1)\n",
      "\tUP\n",
      "\t\t1.1971984067776398\n",
      "\tDOWN\n",
      "\t\t-0.7111062059647298\n",
      "\tLEFT\n",
      "\t\t-0.6059049000000001\n",
      "\tRIGHT\n",
      "\t\t-0.6007547711441001\n",
      "(2, 2)\n",
      "\tUP\n",
      "\t\t7.999999999999978\n",
      "\tDOWN\n",
      "\t\t2.83244514113278\n",
      "\tLEFT\n",
      "\t\t-0.6549507881279562\n",
      "\tRIGHT\n",
      "\t\t-0.045604428976968536\n",
      "(2, 3)\n",
      "\tUP\n",
      "\t\t-1.0\n",
      "\tDOWN\n",
      "\t\t-0.6573044338900002\n",
      "\tLEFT\n",
      "\t\t3.426992033691882\n",
      "\tRIGHT\n",
      "\t\t-0.6812706890000001\n",
      "(2, 4)\n",
      "\tUP\n",
      "\t\t-0.31456455926409993\n",
      "\tDOWN\n",
      "\t\t-0.6807482079501002\n",
      "\tLEFT\n",
      "\t\t-0.5951957000000001\n",
      "\tRIGHT\n",
      "\t\t-0.6000000000000001\n",
      "(3, 0)\n",
      "\tUP\n",
      "\t\t-1.2091569357417478\n",
      "\tDOWN\n",
      "\t\t-1.1828682248384699\n",
      "\tLEFT\n",
      "\t\t-1.2000000000000006\n",
      "\tRIGHT\n",
      "\t\t-0.33624194534918445\n",
      "(3, 1)\n",
      "\tUP\n",
      "\t\t-0.9916056404190002\n",
      "\tDOWN\n",
      "\t\t-0.9466022805243023\n",
      "\tLEFT\n",
      "\t\t-1.0111385559491004\n",
      "\tRIGHT\n",
      "\t\t4.1370895449936596\n",
      "(3, 2)\n",
      "\tUP\n",
      "\t\t6.9999999999999725\n",
      "\tDOWN\n",
      "\t\t2.962653356097409\n",
      "\tLEFT\n",
      "\t\t0.4764647521416745\n",
      "\tRIGHT\n",
      "\t\t-0.2791923307218595\n",
      "(3, 3)\n",
      "\tUP\n",
      "\t\t-0.6394159923514351\n",
      "\tDOWN\n",
      "\t\t-0.5265489617195914\n",
      "\tLEFT\n",
      "\t\t3.2836446026266795\n",
      "\tRIGHT\n",
      "\t\t-0.9701609884900001\n",
      "(3, 4)\n",
      "\tUP\n",
      "\t\t-1.1674428038245022\n",
      "\tDOWN\n",
      "\t\t-1.3072547791181952\n",
      "\tLEFT\n",
      "\t\t-1.1621297748844435\n",
      "\tRIGHT\n",
      "\t\t-1.1909739000000004\n",
      "(4, 0)\n",
      "\tUP\n",
      "\t\t-1.7689511637214377\n",
      "\tDOWN\n",
      "\t\t-1.3216120153258972\n",
      "\tLEFT\n",
      "\t\t-1.3256705826372484\n",
      "\tRIGHT\n",
      "\t\t3.993753612187077\n",
      "(4, 1)\n",
      "\tUP\n",
      "\t\t-1.513368648945791\n",
      "\tDOWN\n",
      "\t\t-1.5000000000000009\n",
      "\tLEFT\n",
      "\t\t-0.025654933305258143\n",
      "\tRIGHT\n",
      "\t\t4.9999993020040625\n",
      "(4, 2)\n",
      "\tUP\n",
      "\t\t5.999999999999967\n",
      "\tDOWN\n",
      "\t\t1.4687399741605525\n",
      "\tLEFT\n",
      "\t\t1.8187051017412166\n",
      "\tRIGHT\n",
      "\t\t0.5779038037938466\n",
      "(4, 3)\n",
      "\tUP\n",
      "\t\t-0.2877081283786905\n",
      "\tDOWN\n",
      "\t\t0.37343171309539464\n",
      "\tLEFT\n",
      "\t\t4.999999946373289\n",
      "\tRIGHT\n",
      "\t\t-0.650623071200497\n",
      "(4, 4)\n",
      "\tUP\n",
      "\t\t-1.7370563231187757\n",
      "\tDOWN\n",
      "\t\t-1.3271550826907088\n",
      "\tLEFT\n",
      "\t\t3.998205821998274\n",
      "\tRIGHT\n",
      "\t\t-1.8000000000000012\n"
     ]
    }
   ],
   "source": [
    "def pretty(d, indent=0):\n",
    "    for key, value in d.items():\n",
    "        print('\\t' * indent + str(key))\n",
    "        if isinstance(value, dict):\n",
    "            pretty(value, indent+1)\n",
    "        else:\n",
    "            print('\\t' * (indent+1) + str(value))\n",
    "\n",
    "\n",
    "pretty(agentQ.q_table)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}