{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "07a0a926",
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a0c2f6e",
   "metadata": {},
   "source": [
    "# Homework 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9992241e",
   "metadata": {},
   "source": [
    "In this homework you will implement a numerical approach to gradient descent and use it to implement the perceptron algorithm.  That will take place in stages described below.  But we'll start by describing how to perform gradient descent numerically."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5393ce8",
   "metadata": {},
   "source": [
    "### The method of finite differences\n",
    "\n",
    "Given a function $f(x)$, the analytical approach to gradient descent - i.e., to finding the value of $x$ that minimizes $f(x)$ - is to compute $f'(x)$ and iterate as follows, where $\\alpha \\in (0, 1]$ is the learning rate:\n",
    "* x = 0\n",
    "* while not converged:\n",
    "    * $x = x - \\alpha f'(x)$\n",
    "    \n",
    "If you cannot compute $f'(x)$ analytically, you can estimate it as follows, for sufficiently small $\\epsilon$: \n",
    "\n",
    "$f'(x) \\approx \\frac{f(x + \\epsilon) - f(x - \\epsilon)}{2\\epsilon}$\n",
    "\n",
    "The *method of finite differences* makes the assumption that over very small intervals the function behaves linearly and the slope of the tangent line can be estimated as the difference between function values at the ends of the interval divided by the interval width.\n",
    "\n",
    "For example, the slope of $f(x) = x^2$ at $x = 1$ can be estimated as $f'(1) = \\frac{(1 + 0.001)^2 - (1 - 0.001)^2}{2 * 0.001}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "98418213",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.9999999999998352"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "e = 0.001\n",
    "(math.pow(1 + e, 2) - math.pow(1 - e, 2))/(2*e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "302343fd",
   "metadata": {},
   "source": [
    "Note that the number above is very close to what you would get analytically by taking derivatives: $f'(x) = 2x$ so $f'(1) = 2*1 = 2$.\n",
    "\n",
    "You'll use the method of finite differences to compute the derivative of a loss function with respect to the weights of a perceptron."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e69a08ef",
   "metadata": {},
   "source": [
    "Below is a simple implementation of the method of finite differences for a univariate function.  It is overly simple, running for a fixed number of iterations, and assuming constants for $\\epsilon$ and $\\alpha$, but it works for simple cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d69740bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "def fd_demo(f, x0):\n",
    "    e = 0.001\n",
    "    a = 0.01\n",
    "    x = x0\n",
    "    for _ in range(1000):\n",
    "        g = (f(x + e) - f(x - e)) / (2*e)\n",
    "        x = x - a * g\n",
    "    return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "30421acd",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.365934714445534e-09"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# x*x is minimized at 0\n",
    "f = lambda x: x * x\n",
    "fd_demo(f, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9c2d5e19-32f3-4166-a7d1-76320a2b4284",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f(8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "47ef2ab4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.711986618234112\n",
      "4.71238898038469\n"
     ]
    }
   ],
   "source": [
    "# sin(x) is minimized at lots of points, the closest to 2 is 3pi/2\n",
    "f = lambda x: math.sin(x)\n",
    "print(fd_demo(f, 2))\n",
    "print(math.pi * 3 / 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f43f0e4",
   "metadata": {},
   "source": [
    "### Perceptron class\n",
    "\n",
    "Below is a simple Perceptron class that you can use in this assignment.  Feel free to use it as is, make changes, or throw it out and write your own.  It creates a weight vector with small random numbers when initialized, and has methods to compute the activation for an input and to produce a binary class label for an input.\n",
    "\n",
    "Note that the class below does not maintain an explicit bias term $b$.  You can add one or, better yet, make sure that all inputs, $x$, have a 1 in one of the positions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "4cf39bfb",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Perceptron:\n",
    "    \n",
    "    def __init__(self, n):\n",
    "        \"\"\"\n",
    "        n (int) - The length of x vectors that the classifier will process.\n",
    "        \"\"\"\n",
    "        \n",
    "        # Start with weights uniformly chosen from the range [-0.5, 0.5]\n",
    "        self.weights = (np.random.rand(n) - 0.5).tolist()\n",
    "        \n",
    "    def activation(self, x):\n",
    "        return np.dot(x, self.weights)\n",
    "    \n",
    "    def predict(self, x):\n",
    "        return 1 if self.activation(x) > 0 else -1\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48e92616",
   "metadata": {},
   "source": [
    "### Task 1: The hinge loss\n",
    "\n",
    "Fill in the function below to compute the hinge loss.  Recall that the hinge loss is 0 if the activation and the class label have the same sign, otherwise it is -y * activation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "69adecc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_hinge(y, activation):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afe259c2",
   "metadata": {},
   "source": [
    "### Task 2: One step of gradient descent\n",
    "\n",
    "Fill in the function below which takes the following arguments:\n",
    "* clf - An instance of the perceptron class above\n",
    "* x - A training instance\n",
    "* y - The corresponding true class label\n",
    "* learning_rate - A learning rate in the range (0, 1]\n",
    "* loss_fn - A function that takes as input the true class label and the activation and returns a real number which is the loss on that example\n",
    "* epsilon - The delta to use when using the method of finite differences\n",
    "\n",
    "The function must return a new set of weights to use in the perceptron after performing one step of gradient descent update using the training example and loss function.  To do that it will:\n",
    "* Loop over each of the weights\n",
    "* Compute the partial derivative of the loss with respect to the weight using the method of finite differences\n",
    "* Use the computed gradient (the list of partials with respect to each of the weights) to compute a new weight vector as $w = w - \\alpha g$ where $w$ is the weight vector, $\\alpha$ is the learning rate, and $g$ is the computed gradient.\n",
    "* Return the new weight vector\n",
    "\n",
    "**Note:** Be careful not to modify the weights of the preceptron in place in the routine below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20fb54a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "def gd_step(clf, x, y, learning_rate, loss_fn, epsilon = 0.001):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "182ca040",
   "metadata": {},
   "source": [
    "### Some training data\n",
    "\n",
    "The code below generates a simple 2D dataset of n positive examples followed by n negative examples.  The cell after that plots them.  The code also prepends a 1 in each example so that the bias term will simply correspond to the first weight."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1e0e5abd",
   "metadata": {},
   "outputs": [],
   "source": [
    "n = 10\n",
    "X = np.concatenate((np.random.rand(n, 2) + 1,\n",
    "                    np.random.rand(n, 2)))\n",
    "X = np.hstack((np.expand_dims(np.ones(2*n), 1), X))\n",
    "Y = [1] * n + [-1] * n\n",
    "colors = c = ['r'] * n + ['g'] * n\n",
    "\n",
    "# Randomize the order of the instances just for fun\n",
    "rng = np.random.default_rng()\n",
    "state = rng.__getstate__()\n",
    "rng.shuffle(X)\n",
    "rng.__setstate__(state)\n",
    "rng.shuffle(Y)\n",
    "rng.__setstate__(state)\n",
    "rng.shuffle(colors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "95e8d2b7",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f7da07c9490>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8WgzjOAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhSklEQVR4nO3deXwV9b3/8dcnGyQhypK4VBBQsSouoEe06nW9AmKV1hXclxbtr9pe7WJ7+9AW7bXWpfpzFy31traoaFFcEKHuVSzBBVckoFWolgjIlhAS8rl/zKCH5ISckDlnTg7v5+NxHjnn+505885k8snkO3NmzN0REZH8VRB3ABERySwVehGRPKdCLyKS51ToRUTynAq9iEieK4o7QCqVlZU+YMCAuGOIiHQZc+bM+dzdq1L15WShHzBgANXV1XHHEBHpMszsn231aehGRCTPtVvozayfmT1rZu+a2Ttm9sMU05iZ3WxmNWY218z2Teo728zmh4+zo/4GRERk09IZumkCfuTur5lZBTDHzGa4+7tJ0xwDDAofBwB3AAeYWW/gl0AC8HDeqe6+PNLvQkRE2tTuHr27f+rur4XPVwHvATu0mGw08EcPzAJ6mtn2wAhghrsvC4v7DGBkpN+BiIhsUofG6M1sADAUeLVF1w7AJ0mvF4VtbbWneu9xZlZtZtW1tbUdiSUi0jUsWAAvvgjLszuokXahN7MewMPAf7n7yqiDuPsEd0+4e6KqKuUZQiIiXdPy5XD44bDXXnDccfC1r8Hll0OWLiqZVqE3s2KCIv9nd/9rikkWA/2SXvcN29pqFxHZcpxxBrzyCtTXw4oVsHYt/O53cP/9WVl8OmfdGPB74D13/10bk00FzgrPvjkQWOHunwLTgeFm1svMegHDwzYRkS3D0qXwt7/BunUbt9fVwfXXZyVCOmfdHAycCbxlZm+Ebf8N7Ajg7ncCTwKjgBqgDjg37FtmZlcBs8P5rnT3ZZGlFxHJdV98AUVF0NDQum/p0qxEaLfQu/tLgLUzjQPfb6NvIjBxs9KJiHQls2bB3XfDypVw0klw4okwYACUlsKaNRtPW1QEI0ZkJZY+GSsiEoXf/Q6OOgruvRceegjOPz8o5O5wxx1QVgYW7jOXlEDPnnDFFVmJlpPXuhER6VJqa+G//3vj4Zk1a+DVV2HKFDj5ZOjfPxiT//BDOPJIuOQS2HbbrMRToRcR6aznngv20luOw69ZAw8/HBT6/feHBx6IJZ6GbkREOquiInV7QUEwRBMzFXqRXOUOb74Z7C2uXh13GtmUI4+E4uLW7d27w3e/m/08LajQi+SiDz+EPfaAgw+G0aODsdw774w7lbSlpASmT4c+fYK9+622Cor8b38L++0XdzqN0YvkHPfgbI0FC6C5+av2H/0I9tkHvvGN+LJJ2xIJ+PTTr/4DO+ww6N077lSACr1I7pkzJygYyUUego/P33qrCn0uKy6Go4/u+HxNTfDCC7BqFRx6KPTqFWksFXqRXLNsWXAQryV3+Oyz7OeRzHrjjeA/uPr64Dz7devguuvgoosiW4TG6EVyzbBhra+LAsGnK0ePzn4eyZympqDIL1kS7M2vXBlc8Oyyy2D27PbnT5MKvUiu6dkTfv3r4JOUG5SWBh+4Of/82GJJBjz3XLAn39LatXDXXZEtRkM3IrnoRz+CffeFW24J9vZOOAHGjYPy8riTSZRWrvzqsgjJmpsjvTmJCr1IrjriiOAh+euww1IP05WXBxdEi4iGbkRE4tKnD1x99cYXPCsvh6FDg8smRER79CIicbrkkuCU2TvvDK5df/LJcMopqT9pu5lU6EVE4nbggcEjQzR0IyKS59rdozezicA3gSXuvmeK/p8Apye93+5AVXgbwY+AVcB6oMndE1EFFxGR9KSzR38vMLKtTne/zt2HuPsQ4OfA8y3uC3tE2K8iLyISg3YLvbu/AKR7Q++xwKROJRIRkUhFNkZvZmUEe/4PJzU78LSZzTGzcVEtS6TLcIe//jW40NXBB8NttwWfehTJoijPujkO+HuLYZtD3H2xmW0DzDCz98P/EFoJ/xCMA9hxxx0jjCUSox/8AP7wh+CWchBcwOpPf4IXX9z49Lm1a4OLmW2zDRTpZDiJVpRn3YyhxbCNuy8Ovy4BpgDD2prZ3Se4e8LdE1VVVRHGEonJggVwzz1fFXmAujp45x145JHgdVNTcB51796wyy5BoY/wGiciEFGhN7OtgcOAR5Pays2sYsNzYDjwdhTLE+kSXnwRCgtbt69eDU89FTz/6U9hwoTgwlb19cH1TS69NBjuEYlIu4XezCYBrwBfN7NFZna+mV1oZhcmTfZt4Gl3T9p1YVvgJTN7E/gH8IS7PxVleJGc1qdP6uvKFxcHtwZsaAj23uvqNu6vq4Px47OTUbYI5u5xZ2glkUh4dXV13DFEOmfdOujbF2prN24vK4O5c6FHDxgwIPXB2d69YenSrMSU/GBmc9o6jV2fjBXJlJISeOaZ4DryPXoEN4zeemuYNAl23hkqK4P2VHLghtKSP3R4XyST9twTPvwwONtm7dqggJeUBH2FhXDttcEt45KHb8rK4De/iSWu5CcVepFMMwsuO5vKuecGe/bjx8PHHwd/CP7nf4KbjohEREM3InE77jiorg7uJDVtmop8Plu6FC64IDhQv802wRlWq1dnfLHaoxcRyYaGBjjggOA/t8bGoO322+Gll+DVV1PfUjAi2qMXEcmGKVPg3//+qshDUPzfew+efz6ji1ahFxHJhjlzUg/TrFsXHKzPIBV6EZFsGDQouB9sS926wU47ZXTRKvQiItkwdix0777xWHxhYfDhuFGjMrpoFXoRkWyoqICXXw5uBF5UFDyOPBL+/veMX7FUZ92IiGTLrrsGhX3NmuA6SKWlWVmsCr2ISLalGqvPIA3diIjkORV6EZE8p0IvIpLnVOhFRPKcCr2ISJ5ToRcRyXPp3DN2opktMbOUN/Y2s8PNbIWZvRE+rkjqG2lm88ysxsx+FmVwERFJTzp79PcCI9uZ5kV3HxI+rgQws0LgNuAYYA9grJnt0ZmwIiLSce0Wend/AVi2Ge89DKhx94Xuvg64Hxi9Ge8jIiKdENUY/TfM7E0zm2Zmg8O2HYBPkqZZFLalZGbjzKzazKpra2sjiiUiIlEU+teA/u6+D3AL8MjmvIm7T3D3hLsnqqqqIoglIiIQQaF395Xuvjp8/iRQbGaVwGKgX9KkfcM2ERHJok4XejPbziy4wLKZDQvfcykwGxhkZgPNrAQYA0zt7PJERKRj2r16pZlNAg4HKs1sEfBLoBjA3e8ETgK+Z2ZNQD0wxt0daDKzi4DpQCEw0d3fych3ISIibbKgJueWRCLh1dXVcccQkU1Zvhz+8hf46CM48EA4/ngoLo471RbLzOa4eyJVn65HLyId9+abcNhh0NgIdXXQowcMHBjcVKOiIu500oIugSAiHXfaabBiRVDkAVavhg8+gKuvjjeXpKRCLyId8+mnsGBB6/aGBpg0Kft5pF0q9CLSMYWFbfdl+CbXsnlU6EWkY7bZBvbaK7i5dbLSUjj33HgyySap0ItIx91/P1RVBQdeS0qCm10fcAD8+MdxJ5MU9H+WiHTczjvDxx/DY4/BJ5/A/vvDQQdB8NlJyTEq9CKyeUpK4MQT404hadDQjYhInlOhFxHJcyr0IiJ5ToVeRCTPqdCLiOQ5FXoRkTynQi8iuW/5cnj9dfjii7iTdEkq9CKSu5qa4IIL4Gtfg8MPh+23h4svhubmuJN1KSr0IpK7xo+H++6DtWth5crg68SJ8Nvfxp2sS1GhF5HcdfPNX13zfoO6OrjxxnjydFHtFnozm2hmS8zs7Tb6TzezuWb2lpm9bGb7JPV9FLa/YWa6N6CIpK+5GVatSt23fHl2s3Rx6ezR3wuM3ET/h8Bh7r4XcBUwoUX/Ee4+pK17GYqIpFRQEFwOOZX99stuli6u3ULv7i8AyzbR/7K7b/jzOgvoG1E2EdnS3XILlJV9dVXMgoLg9U03xRqrq4l6jP58YFrSaweeNrM5ZjZuUzOa2Tgzqzaz6tra2ohjiUiXdOihwQ3HTzoJdtsNTj4ZXn0VDjww7mRdSmSXKTazIwgK/SFJzYe4+2Iz2waYYWbvh/8htOLuEwiHfRKJhEeVS0S6uCFD4MEH407RpUWyR29mewP3AKPdfemGdndfHH5dAkwBhkWxPBERSV+nC72Z7Qj8FTjT3T9Iai83s4oNz4HhQMozd0REJHPaHboxs0nA4UClmS0CfgkUA7j7ncAVQB/gdgsOmDSFZ9hsC0wJ24qAv7j7Uxn4HkREZBPaLfTuPrad/u8A30nRvhDYp/UcIiKSTfpkrIhInlOhFxHJcyr0IiJ5ToVeRCTPqdCLiLT0+eeweDF4fnx2U4VeRGSDTz6BQw6Bvn1hl11g111h1qy4U3WaCr2ICMD69XDYYUFhb2gIbnJSUwNHHw2ffhp3uk5RoRcRAXjmmWDIZv36jdubmuD3v48nU0RU6EVEIBi2SXUv2rVrYcGC7OeJkAq9iAjA/vunLvTl5cHlkrswFXoREQjuZjViRHBjkw1KSmC77WDMmPhyRUCFXkRkg8mT4aqrYNAg6NcPLroIZs+G0tK4k3WKeQ6eJ5pIJLy6WvcSFxFJl5nNaeve3NqjFxHJcyr0IiJ5ToVeRCTPqdCLiOS5tAq9mU00syVmlvKerxa42cxqzGyume2b1He2mc0PH2dHFbyl+Uvnc1f1XTz4zoPUNdZlajEiIl1Ou7cSDN0L3Ar8sY3+Y4BB4eMA4A7gADPrTXCP2QTgwBwzm+ruyzsTOpm788Onfsjdr91NAQUUFhRSYAU8febTDNthWFSLERHpstLao3f3F4Blm5hkNPBHD8wCeprZ9sAIYIa7LwuL+wxgZGdDJ3ti/hNMfH0ia5vWUtdUx6p1q1jRsILjJh3H+ub17b+BiEiei2qMfgfgk6TXi8K2ttpbMbNxZlZtZtW1tbVpL3jCnAmsaVzTqr2+sZ5XFr2S9vuIiOSrnDkY6+4T3D3h7omqqqq052tY35Cy3cxYt35dVPFERLqsqAr9YqBf0uu+YVtb7ZE5fa/TKS8ub9Xu7hzU76AoFyVdzAv/fIFxj43jvEfPY+bCmeTip8BFsiGqQj8VOCs8++ZAYIW7fwpMB4abWS8z6wUMD9sic9pep3FQv4PoUdwDgJLCEkqLSrn3W/fSvah7lIuSLuQnM37CqD+P4p7X7uEPb/yBb93/LcY9Pi7uWCKxSOusGzObBBwOVJrZIoIzaYoB3P1O4ElgFFAD1AHnhn3LzOwqYHb4Vle6+6YO6nb8Gygo4qkznuKpmqd4cv6TVJZVcvY+ZzOw18AoFyNdyPufv89t/7iN+qb6L9vWNK7hL2/9he/u+12djSVbnLQKvbuPbaffge+30TcRmNjxaOkrsAJGDRrFqEGjMrkY6SKmzZ9Gs7e+rvjaxrU88cETKvSyxcmZg7EiUSkrLqOwoLBVe1FhET1KesSQSCReKvSSd07c48SU7YVWyKl7nprlNCLxU6GXvFNZVskDJz1AeXE5FSUVVJRUUFpUyj3H38OOW+8YdzyRrEv3EggiXco3d/0mn/34M6bXTKfZmxm+83C27r513LFEYqFCL3mrR0mPNodxRLYkGroREclzKvQiInlOhV5EJM9pjD4PuDuzFs1iWs00tu62NafueSp9t+obdywRyREq9F2cu3PmlDOZ8v4U6hvrKSks4fJnL+e+E+7jhN1PiDueiOQADd10cY9/8DiPvP8IdY11OE7D+gbqm+o5a8pZrFnX+jr9IrLlUaHv4u6be1/KG68UWiHPfvRsDIlEJNdo6KaLKypo40doQbGXjttwzOPxDx6nvKSc0/Y6jQE9B8QdS2SzqdB3cWftcxaPznu01V69u3PEwCNiStV1uTvnPXoeD777IPWN9RQVFPHrF37NhOMmcMbeZ8QdT2SzaOimixu+83DOHXIupUWllBSUUFZcRllxGQ+d8pBuvLIZZiycweR3J395zKOxuZH6pnrGPTaOFWtXxB1PZLNoj76LMzNuGXULFyYuZPqC6WzVbStO2P0Eepf2jjtalzTprUkpj3kUFRQxY+EMTtrjpBhSiXSOCn2eGLzNYAZvMzjuGF1eUUERhuG0vr9sm8dDRHKchm5Ekpy1z1mUFpe2am/2Zo7e6egYEol0XlqF3sxGmtk8M6sxs5+l6L/RzN4IHx+Y2RdJfeuT+qZGmF0kcv/R/z+4eNjFlBaV0r2oO+XF5ZQVlzH55MmUl5THHU9ks1hwu9dNTGBWCHwAHA0sIrjR91h3f7eN6S8Ghrr7eeHr1e7eofu3JRIJr66u7sgsIpGav3Q+02qmUV5czrd3/7aOeUjOM7M57p5I1ZfOoOMwoMbdF4Zvdj8wGkhZ6IGxwC83J6hIrhjUZxCD+gyKO4ZIJNIZutkB+CTp9aKwrRUz6w8MBJ5Jau5uZtVmNsvMvtXWQsxsXDhddW1tbRqxREQkHVEfjB0DPOTu65Pa+of/TpwG3GRmO6ea0d0nuHvC3RNVVVURxxIR2XKlU+gXA/2SXvcN21IZA0xKbnD3xeHXhcBzwNAOpxTJsGZvjjuCSMakU+hnA4PMbKCZlRAU81Znz5jZbkAv4JWktl5m1i18XgkcTNtj+yJZ9/C7D7PT/9+JwisL2fb6bbn1H7fS3gkKIl1Nuwdj3b3JzC4CpgOFwER3f8fMrgSq3X1D0R8D3O8b/5bsDtxlZs0Ef1SuaetsHZFse+KDJzhzypnUN9UDsGTNEi6beRnr1q/j0m9cGnM6kei0e3plHHR6pWTDkDuH8Oa/32zV3rN7T5b+dCkFps8TStexqdMrtSXLFmvB8gUp29esW8OqhlVZTiOSOSr0ssXatfeuKdsrulVQ0a0iy2lEMkeFXrZYVx91NaVFG1/Xpqy4jPGHj9ewjeQVbc2yxRqxywgmnzyZ3St3p6igiH5b9ePmY27momEXxR1NJFK67qps0Y7d9ViO3fXYuGOIZJT26EVE8pwKvYhInlOhFxHJcyr0IiJ5ToVeRCTPqdCLiOQ5FXoRkTynQi8ikudU6EVE8pwKvYhInlOhFxHJcyr0IiJ5Lq1Cb2YjzWyemdWY2c9S9J9jZrVm9kb4+E5S39lmNj98nB1leBERaV+7V680s0LgNuBoYBEw28ymprj36wPuflGLeXsDvwQSgANzwnmXR5JeRETalc4e/TCgxt0Xuvs64H5gdJrvPwKY4e7LwuI+Axi5eVFFRGRzpFPodwA+SXq9KGxr6UQzm2tmD5lZvw7OKxK5NevWULOshvrG+rijiMQqqoOxjwED3H1vgr32/+3oG5jZODOrNrPq2traiGLJlmh983oueeoSqq6rYuhdQ6m8rpLxz4/H3eOOJhKLdAr9YqBf0uu+YduX3H2puzeEL+8B9kt33qT3mODuCXdPVFVVpZNdJKXxz49nwmsTqG+qZ/W61dQ11nHt36/lzuo7444mEot0Cv1sYJCZDTSzEmAMMDV5AjPbPunl8cB74fPpwHAz62VmvYDhYZtIRrg7N826ibrGuo3a6xrr+M1Lv4kplUi82j3rxt2bzOwiggJdCEx093fM7Eqg2t2nAj8ws+OBJmAZcE447zIzu4rgjwXAle6+LAPfhwgAjc2NrGlck7Kvtk5DgrJlSuvm4O7+JPBki7Yrkp7/HPh5G/NOBCZ2IqNI2koKSxjQcwALly9s1TdkuyHZDySSA/TJWMk7txxzC6VFpV++Noyy4jJuGH5DjKlE4qNCL3ln1KBRPH3m0xw18Cj6btWXY3c9lhfPfZGD+h0UdzSRWKQ1dCPS1Ryy4yHMPGtm3DFEcoL26EVE8pwKvYhInlOhFxHJcyr0IiJ5ToVeRCTPqdCLiOQ5FXoRkTynQi9tml4znX3v2pceV/dgrzv2Yuq8qe3PJCI5R4VeUpo2fxonPHACr3/2Omsa1/D2krcZ+/BYHnj7gbijiUgHqdBLSj+d8VPqmlpf6veymZfFlEhENpcKfQ76bPVnTH5nMjMXzqSpuSmWDPOWzkvZ/vGKjzucaX3zem585UZ2vnlntr1+W8579Dz+tepfUcQUkTToWjc5Zvxz47nmpWsoLiwGoLyknJlnzmTwNoOzmmOHrXbgoy8+atXep6wPRQUd22zOffRcHn7v4S9vBvKnN//Ek/Of5L3vv0ev0l5RxBWRTdAefQ6ZuXAm1758LWvXr2XVulWsWreKz1Z/xjF/PoZmb85qll8d/ivKiss2aisrLuPyQy/v0Pt8uPxDJr87eaM7PjV5EysbVnL3a3dHklVENk2FPofcPvv2VrfAA1i+djmzF89OMUfmnL3P2dww/AYqyyopKiiiV/deXHXEVVw87OIOvc/rn71OSWFJq/b6pnqe/+fzUcUVkU3Q0E0OWdGwImV7gRWwet3qLKeBCxMXcsF+F7B63WrKS8opsI7vFwzoOYD1zetbtRcXFPP1Pl+PIqaItCOt31wzG2lm88ysxsx+lqL/UjN718zmmtnfzKx/Ut96M3sjfOhE7E04ZY9TWg2XADQ1N3Fg3wNjSARmRkW3is0q8gBDtxvKbpW7UVxQvFF7SWFJh/87EJHN0+5vr5kVArcBxwB7AGPNbI8Wk70OJNx9b+Ah4Nqkvnp3HxI+jo8od146Z8g5DK4aTHlxOQCFVkhZcRm3H3s75SXlMafbPGbG9DOmM2KXEZQUltCtsBu79NqFaadPY2CvgXHHE9kipDN0MwyocfeFAGZ2PzAaeHfDBO7+bNL0s4Azogy5pehW1I2XznuJye9M5pH3H2HbHtsybr9x7L3t3nFH65Q+ZX14bOxjrGpYRX1TPVVlVZhZ3LFEthjpFPodgE+SXi8CDtjE9OcD05JedzezaqAJuMbdH0k1k5mNA8YB7LjjjmnEyk8lhSWcvvfpnL736XFHiVxFtwoqulXEHUNkixPpwVgzOwNIAIclNfd398VmthPwjJm95e4LWs7r7hOACQCJRMKjzCUisiVL5wjbYqBf0uu+YdtGzOw/gV8Ax7t7w4Z2d18cfl0IPAcM7UReERHpoHQK/WxgkJkNNLMSYAyw0dkzZjYUuIugyC9Jau9lZt3C55XAwSSN7YuISOa1O3Tj7k1mdhEwHSgEJrr7O2Z2JVDt7lOB64AewOTwINvH4Rk2uwN3mVkzwR+Va9xdhV5EJIvMPfeGwxOJhFdXV8cdQ0SkyzCzOe6eSNWnSyCIiOQ5FXoRkTynQi8ikudU6EVE8pwKvYhInlOhFxHJc7oevUSm+l/VTHprEs00c+rgU2O7tLKIbEyFXiJxxbNXcMMrN7C2cS0YTJgzge8lvsf1w6+PO5rIFk9DN9Jp8z6fx3UvX0ddYx3NNNPszdQ11nFH9R3M/ffcuOOJbPFU6KXTHv/g8ZQ3L29oauDR9x+NIZGIJFOhl07rVtSNQits1V5YUEj3ou4xJBKRZCr00mkn7n5iyvYCK+CUwadkOY2ItKRCL522fcX2/H707+le1J0eJT3oUdKD7kXduX3U7fTv2b/9NxCRjNJZNxKJsXuOZfhOw3li/hO4O6MGjaKqvCruWCKCCr1EqE9ZH87a56y4Y4hICxq6ERHJcyr0IiJ5ToVeRCTPqdCLiOQ5FXoRkTyXkzcHN7Na4J8puiqBz7McpyNyOV8uZ4PczpfL2SC38+VyNsjtfB3N1t/dU57TnJOFvi1mVt3WXc5zQS7ny+VskNv5cjkb5Ha+XM4GuZ0vymwauhERyXMq9CIiea6rFfoJcQdoRy7ny+VskNv5cjkb5Ha+XM4GuZ0vsmxdaoxeREQ6rqvt0YuISAep0IuI5LmcKfRmNtLM5plZjZn9LEV/NzN7IOx/1cwGJPX9PGyfZ2YjYsh2qZm9a2ZzzexvZtY/qW+9mb0RPqZGnS3NfOeYWW1Sju8k9Z1tZvPDx9kxZLsxKdcHZvZFUl9G152ZTTSzJWb2dhv9ZmY3h9nnmtm+SX0ZXW9p5js9zPWWmb1sZvsk9X0Utr9hZtUxZDvczFYk/fyuSOrb5DaRhWw/Scr1drid9Q77MrrewmX0M7Nnw5rxjpn9MMU00W577h77AygEFgA7ASXAm8AeLab5f8Cd4fMxwAPh8z3C6bsBA8P3KcxytiOAsvD59zZkC1+vzoF1dw5wa4p5ewMLw6+9wue9spmtxfQXAxOzuO4OBfYF3m6jfxQwDTDgQODVbKy3DuQ7aMNygWM25AtffwRUxrjuDgce7+w2kYlsLaY9DngmW+stXMb2wL7h8wrggxS/s5Fue7myRz8MqHH3he6+DrgfGN1imtHA/4bPHwKOMjML2+939wZ3/xCoCd8va9nc/Vl3rwtfzgL6Rrj8TufbhBHADHdf5u7LgRnAyBizjQUmRbj8TXL3F4Blm5hkNPBHD8wCeprZ9mR+vaWVz91fDpcPWd7u0lh3benM9pqJbFnd5gDc/VN3fy18vgp4D9ihxWSRbnu5Uuh3AD5Jer2I1t/4l9O4exOwAuiT5ryZzpbsfIK/xBt0N7NqM5tlZt+KMFdH850Y/gv4kJn16+C8mc5GONw1EHgmqTnT6649beXP9HrbHC23OweeNrM5ZjYupkzfMLM3zWyamQ0O23Jm3ZlZGUGRfDipOavrzYIh6KHAqy26It32dIepCJnZGUACOCypub+7LzaznYBnzOwtd1+Q5WiPAZPcvcHMLiD4z+jILGdozxjgIXdfn9SWC+su55nZEQSF/pCk5kPCdbcNMMPM3g/3dLPlNYKf32ozGwU8AgzK4vLTcRzwd3dP3vvP2nozsx4Ef2T+y91XZmIZG+TKHv1ioF/S675hW8ppzKwI2BpYmua8mc6Gmf0n8AvgeHdv2NDu7ovDrwuB5wj+ekep3XzuvjQp0z3AfunOm+lsScbQ4l/oLKy79rSVP9PrLW1mtjfBz3S0uy/d0J607pYAU4h2OLNd7r7S3VeHz58Eis2skhxad2x6m8voejOzYoIi/2d3/2uKSaLd9jJ50KEDByeKCA4qDOSrAzSDW0zzfTY+GPtg+HwwGx+MXUi0B2PTyTaU4ADToBbtvYBu4fNKYD7RH3hKJ9/2Sc+/Dczyrw7sfBjm7BU+753NbOF0uxEcBLNsrrvwvQfQ9gHFY9n4gNg/srHeOpBvR4JjUge1aC8HKpKevwyMzHK27Tb8PAmK5cfhekxrm8hktrB/a4Jx/PIY1psBfwRu2sQ0kW57kW+YnfjmRxEcfV4A/CJsu5JgDxmgOzA53LD/AeyUNO8vwvnmAcfEkG0m8G/gjfAxNWw/CHgr3JjfAs6Pad39BngnzPEssFvSvOeF67QGODfb2cLXvwKuaTFfxtcdwd7cp0AjwVjn+cCFwIVhvwG3hdnfAhLZWm9p5rsHWJ603VWH7TuF6+3N8Of+ixiyXZS0zc0i6Y9Rqm0im9nCac4hOIkjeb6Mr7dwOYcQHAuYm/SzG5XJbU+XQBARyXO5MkYvIiIZokIvIpLnVOhFRPKcCr2ISJ5ToRcRyXMq9CIieU6FXkQkz/0foJTVS2N6QZIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X[:,1], X[:,2], c = colors)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30f70899",
   "metadata": {},
   "source": [
    "### Task 3: Full gradient descent\n",
    "\n",
    "If you've done everthing above correctly, the code below will perform gradient descent to train the classifier.\n",
    "\n",
    "Modify this code so that it runs until one epoch produces no classification errors rather than running for a fixed number of iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6671de9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = Perceptron(3)\n",
    "\n",
    "for epoch in range(100):\n",
    "    for x, y in zip(X, Y):\n",
    "        clf.weights = gd_step(clf, x, y, 0.01, loss_hinge)\n",
    "\n",
    "print(clf.weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b4f343b",
   "metadata": {},
   "source": [
    "### Task 4: Plot some hyperplanes\n",
    "\n",
    "Run full gradient descent 5 times and write a routine to convert the weights into slope/intercept form.  Then use the function below to plot the hyperplanes learned by the perceptron along with the data in one graph.  The second cell below shows how that can be done.  Write a paragraph explaining what you see in the plot, touching on how much variation there is from run to run and whether the separators seem like \"good\" ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f55bc1ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "def weights_to_slope_intercept(weights):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b9d6f313",
   "metadata": {},
   "outputs": [],
   "source": [
    "def abline(slope, intercept):\n",
    "    \"\"\"Plot a line from slope and intercept\"\"\"\n",
    "    axes = plt.gca()\n",
    "    x_vals = np.array(axes.get_xlim())\n",
    "    y_vals = intercept + slope * x_vals\n",
    "    plt.plot(x_vals, y_vals, '--')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "db3ee89d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAD8CAYAAACYebj1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA9KElEQVR4nO3dd3gU5drH8e+dXgihJNTQm9JL6E2OgIggioKoL3axC5ZjPUc9luPxeARRsaCggoCigIIizUIVpPcinVBDJz27+7x/zIIRAlnIZmd3c3+uKxeb2ZndX+KYe2fmee4RYwxKKaUUQIjdAZRSSvkPLQpKKaXO0KKglFLqDC0KSimlztCioJRS6gwtCkoppc4osCiISBUR+UVENorIehEZnM86IiLviMhWEVkjIs3zPNdDRDa7n3vG2z+AUkop7/HkSMEBPGGMuRxoAzwkIvXPWudqoI77axDwAYCIhAIj3M/XB27OZ1ullFJ+osCiYIzZb4xZ4X58CtgIVD5rtT7AGGNZDJQSkYpAK2CrMWa7MSYH+NK9rlJKKT8UdjEri0h1oBmw5KynKgN78nyf4l6W3/LW53ntQVhHGYRGx7WoVbMmcVEXFU+pouPIhsObITQSEuqCiN2JlPqL5cuXHzbGJBb2dTz+qysiJYBJwBBjzMmzn85nE3OB5ecuNGYkMBKgZJV6JrvXa1zTrDL/7FWf0rERnsZUyvtyMmBUNzhZCe6bB6Wq2p1IqXOIyC5vvI5Ho49EJByrIIwzxkzOZ5UUoEqe75OAfRdYfkF1ysXx6JV1mLp6H12HzmX+H6mexFTK+4yB6U/CwfXQ9xMtCCroeTL6SIBRwEZjzNDzrDYVuM09CqkNcMIYsx9YCtQRkRoiEgEMcK9bwHvC493qMu2RDlRPiCUxLtLjH0gpr3I5rFNHnZ+COl0vfnunE1asgLVrrQKjlJ+TgrqkikgHYD6wFnC5Fz8HVAUwxnzoLhzvAT2ADOBOY8wy9/Y9gbeBUGC0Mea1gkIlJyebZcuW4X59xH3+9oXv1lG/YklualnlzDKlipwx1lfIRU7r+flnGDAAsrLA5YLERPj2W2jSpEhiquJNRJYbY5IL/Tr+2Do7b1E4LSvXye2jf2fJjqO0q1WW//RtTNWyMTYlVEEv8xhMuR+6vwoJdS5++/37oXZtyMj46/IyZWDvXoiK8k5Opdy8VRQCZkZzVHgoE+5tw2vXN2RNygmuenseoxbswOnyv6KmApzLZRWErT9B5vFLe40xY6xTR2fLzYVp0woVT6miFDBFASAkRLi1dTVmP96JtrXK8vacLRxJz7Y7lgo2C4fBlhlw1WtQpeWlvcb+/ZCdz76ZmwsHDxYun1JFKKCKwmkV46MZdXsy0x/tSLm4KFwuw8Sle8hxuAreWKkL2TEPfn4VGt4ArQZd+utceSWUKHHuchHo3PnSX1epIhaQRQFARKhSxrqmsGDrYZ6atIZr31vA6j3H7Q2mAtvCd6BsHej9TuEmqPXsaV1Qjslz3Ss2Fq6/Hho1KnxOpYpIwFxoLsicDQf5x7frOHQqi3s61uSxrnWJjggtooQqaOVmQXoqlKpS8LoFyc6Gjz+2ri9ERMB998Gtt178KCalTktPtwYvJCSc86Gl2I0+8sTJrFz+8+Mmxi/ZTYfaCXxxT74dNZQ61+ovod7VEBVvdxKlznX8ONx9N3z/vfV9UhKMGgVXXHFmFS0KF7Bo22FCRGhTsyxZuU5ynC5KRoV7MaEKKhu+g4m3wRXPwhXa3V35oY4d4fffISfnz2WxsdbEyLp1gWI4JPVitKuVQJuaZQF456c/6D50Hj9v0hEfKh+Ht8K3D0HlZOjwuN1plDrXhg3WH/+8BQGs05PDh3v97YKyKOTVvUEF4qPDueuzZQz+ciVH0nQIq3LLybCOEELDod9nEKaNF5Uf2rEDwvM50+FwwObNXn+7oC8KTauUYtojHRjStQ7T1+6n27B5zN2iDfYUMOclOLQBbvjEOxeWlSoKTZrkP+clKgo6dfL62wV9UQCICAthSNe6fP9IR2olxlKhpLYYUECHIXDd+1D7SruTKHV+SUlwyy1/Hd4cGmrNg3ngAa+/XbEoCqfVqxDHxPvaUq9CHAD/+HYt45bswqWtMoqXE3utVhYlK0HTW+xOo1TBRo6E116DmjWt4ag33wzLl1tNFr2sWBUF4Ex31axcJ9tT03l+yjpu+WQxOw+n25xM+UTGURjdA6Y9ancSpTwXGgpDhsC2bZCaCmPHQtWiubdHsSsKp0WFhzLuntb8p28j1u89yVVvz2PkvG04nNoqI2i5XDDlPji1H1rcaXeaS7N7N0ycCPPnWz+PUl5WbIsCWEcNA1pVZfbjnelYJ5H3ft7K0YycgjdUgWnBW/DHLOjxOiS1sDvNxTEGHnwQ6tWDe+6x2mjUqWMVCaW8yON7NAezCvFRfHxbC/Yez/yzwd6yPVzfvDKRYdoqIyhs+wV+fg0a3ggt77E7zcUbN85ql5GVZX0BZGZC375QiImeSp2tWB8p5CUiJJW2ru4v2naEZyavpfe7C1i5+5jNyZRXRJSAWl2g9/DCNbqzy7vvWn1v8nI6rYlNu7xyv3alAC0K+epQJ4HRdyRzKstB3w8W8cr3G8jMyeeGKcr/nW7jUqUlDJwCkfm0sw4EJ0/mvzw0FNLSfJtFBbUCi4KIjBaRQyKy7jzP/11EVrm/1omIU0TKuJ/bKSJr3c8F1DHu3y4rz6zHOnFr66qMWrCDe8cEVHx12sznrS8/7PF1Ufr1g8jIc5dHR8Nll/k+jwpanhwpfAb0ON+Txpg3jTFNjTFNgWeBucaYo3lW6eJ+vtCNmnwtLiqcV69rxJeD2vDI32oD1lDWE5m5NidTHlk/BRaPAJcjME8Z5fXEE9YQxNMTmMLCrMeff24dLSjlJQVeaDbGzBOR6h6+3s3AhEIl8kOnm+sBDP/pDyYtT+HV6xrSvUEFG1OpCzr8B3z3MCS1gm6v2J2m8OLjYdUq62LzrFlQvbo1m7VOHbuTqSDjtWsKIhKDdUQxKc9iA8wSkeUiUoh7G/qPng0rUrZEJIPGLufh8Ss4rA32/E9OOnw1EMIig6vRXUwM3H8/TJ4MQ4dqQVBFwpsXmnsDC886ddTeGNMcuBp4SETO271JRAaJyDIRWZaa6r8N6xolxTP14fY82b0us9YfpOvQufy6+ZDdsVRee1fA8d1wwyiIr2x3GqUCijeLwgDOOnVkjNnn/vcQMAVodb6NjTEjjTHJxpjkxCLo5+FN4aEhPPy3Okwf3IHLKsRRqVS03ZFUXjU6wpC11hBUf7Z2rXUnrZQUu5ModYZXioKIxAOdge/yLIsVkbjTj4HuQL4jmAJV7XJxfDmoLXXLWw32np+ylrGLtcGebfYuh9VfWY9jy154XTsdPQpt2lhft94KtWtbs5S1bYXyAwVeaBaRCcAVQIKIpAAvAuEAxpgP3atdD8wyxuSdXVMemOJuQBcGjDfGzPBedP+Sletk15EMxi3ZzbTV+3jjhsbUSIi1O1bxkXEUJt4OCFzeCyL8+Hd/xx2wcuVf76Q1YQI0awYPPWRbLKUgSO/RbBdjDF8vS+GVHzaQ43DxWLe63NOhBmGhOkewSLlcML4/7JgLd82Eys3tTnR+J09a7Y7PvrUiWBeOt2zxfab8GGONdjp0CFq1gtKl7U6kCqD3aPZDIkL/llWY83hnOtdN5P1ftMGeT8z/H2ydDT3+498FASAjA0LO87/d+WYt+1pKCjRqZN0s/qaboFIlq5e/Kha0KBSB8iWj+GhgC34c0ulMg71xS3aR7dBWGV53+A/49XVofBMk32V3moKVLw8V8pnfEhZmdT71B717w6ZNVq+lEyesBnyvvw7Tp9udTPmAFoUiIiJUdo9KWrTtCM9PWcc17yxg+a6jBWypLkpCHRgwHnoNC4xZyyIwapQ15yDMfUkvKgrKlIFX/GCS3ZYt1pfzrA8w6enw9tu2RFK+pUXBBzrUSeDzu1qRmePkxg9/46Wp60nPdtgdK7A5cuCAezBbvav9+8Ly2f72N+tC86BB0LUrPP+81e20sh/MqTh27M9idbYjR3ybRdlC76fgI53rJjLzsU68OWMTny3ayZaDpxh/bxu7YwWu2f+EZaPh4WVQuprdaS5e3bowYoTdKc7VpEn+zQOjouC663weR/meFgUfKhEZxr/6NKRXk0pnlmXlOsnOdREfE25jsgCzbhIs+RDaPBiYBcGfRUVZ92548EHrJj7GWJ1YK1SAR/W+1sWBFgUbtKxe5szjt+f8waQVKbzSpyE9GmqDvQKlboapj0KV1tDtZbvTBKfbb4f69WH4cNi7F3r1gnvvhZIl7U6mfECLgs16Na7IvC2p3P/Fcno2qsBL1zagXFyU3bH805lGd1FWo7tQPboqMi1bwhdf2J1C2UAvNNusYeV4vnu4PX+/qh5zNh6i29B5/LJJG+zlKywKLu8NN46CkpUKXl8pddH0SMEPhIeG8FCX2lzVoAIvfLeOyqW1wd45HDlWC+wr/2l3EqWCmh4p+JHa5Uow/t42ZxrsPTt5DZ8v2qkN9lKWwbstYP8au5MoFfS0KPiprFwne49n8eLU9fT/6De2HiqmN2dPP2I1uhMgPsnuNEoFPS0KfioqPJTP72zJW/2a8MehNHoOn8+IX7aS6yxG7ZVdTph8D6Qfgv5jIKZMwdsopQpFi4IfExFuaJHE7Mc70bV+OUbO287xjFy7Y/nO3P/Ctp/h6v9CpWZ2p1GqWNALzQGgXFwU79/agv0nMkmMi7Qa7P2+m34tkogKD7U7XtFwueDAGmhyM7S4w+40ShUbeqQQQCrGW6OSftt+hH9+u46e78xn2c4gbbAXEgI3jYPewwOj0Z1SQUKLQgBqXzuBsXe3Isfhot9Hv/Hid+tIC5YGe44cmDYEju+2CkNYpN2JlCpWtCgEqI51Epk5pBO3t63OmMW7uPfzwLtTXb5mPQ/LP4X9q+1OolSxpNcUAlhsZBgvXduA3k0qnmlsmZXrJDPHSenYCHvDXYq138DvI6Htw9bMZaWUzxV4pCAio0XkkIisO8/zV4jICRFZ5f56Ic9zPURks4hsFZFnvBlc/alFtTIku5vsDZuzhW7D5jJ97X788f7b53Vok9Xormpb6PqS3WmUKrY8OX30GdCjgHXmG2Oaur9eBhCRUGAEcDVQH7hZROoXJqwqWJ8mlakYH82D41Zw/xfLOXQyy+5Invn5FYiIgRs/1UZ3StmowKJgjJkHXMoQl1bAVmPMdmNMDvAl0OcSXkddhPqVSjLlwXY8c/Vl/Lo5la5D5/LTxoN2xyrY9R/CwClQsqLdSZQq1rx1obmtiKwWkR9FpIF7WWVgT551UtzL8iUig0RkmYgsS01N9VKs4iksNIT7O9fix8EdaVKlFFXLxNgd6fy2/QI5GRAZBxUa2Z1GqWLPG0VhBVDNGNMEeBf41r08v8Hl5z3JbYwZaYxJNsYkJyYmeiGWqplYgrF3t6aOu8He09+s4dOFO3D6S4O9PUthXD/4+VW7kyil3ApdFIwxJ40xae7H04FwEUnAOjKokmfVJGBfYd9PXZqsXCcHT2Xxr2kb6PfhIrYeOmVvoPTD8PXt1n0ROv/d3ixKqTMKXRREpIKINeVURFq5X/MIsBSoIyI1RCQCGABMLez7qUsTFR7Kp3e0ZNhNTdh+OJ2ewxfw3s9/2NNgz+WESfdYhaH/GIgu7fsMSql8FThPQUQmAFcACSKSArwIhAMYYz4EbgQeEBEHkAkMMNZYSIeIPAzMBEKB0caY9UXyUyiPiAjXN0uiY51EXpy6no/n7+CmllVJjPPxrOH5Q2H7L1YLi0pNffveqvg5dgxGj4aVK6FpU7jrLiijHXfPR/xxLHtycrJZtixIZuj6sQMnsqgQH4XLZRi7eBc3tazimwZ7R7fD2knQ6Unta6SK1rZt0Lo1pKVBdjaEhUFsLCxdCnXq2J3Oq0RkuTEmubCvo20uirEK8VEALN5+hBenrufq4fNZsv1I0b1h5nEwBsrUtK4jaEFQRe2RR+DoUasgADgccOIE9Oplby4/pkVB0a52AuPvaY3TZbhp5GL+8e1aTmV5+b4NjmwYex1Mfdi7r6vUhcyaBfmdDdmyBVJSfJ8nAGhRUIBVGGYM6cg9HWowfslu7h3j5dN3M56FfSuhbkGT45XyopAL/In78Uff5Qgg2hBPnRETEcY/etXnmsYVcbk/XWXmOMnMdVKmMA321kyEZaOg3SPa6E75VoMGsGrVucvDwiBc26nkR48U1DmaVS1Ni2rW6Iy3f9pCt6Fz+X7NvktrsHdoI0wbDNXaw5UveTeoUgX56KP8r12FhcG11/o+TwDQoqAu6PpmlalcOpqHx69k0NjlHLzYBnsZR6F0DbhxNITqganysVat4M03ISLC+oqOhqgoGDdOh6Wehw5JVQVyOF2MXriDt2ZtISIshGH9m9K1fnnPX8DluvC5XaWK2t698MMPVmG49tqgLAjeGpKqH91UgcJCQxjUqRbd61fgn9+to3qCBw32lnxkDUHt9HctCMp+lSvDoEF2pwgI+n+r8lj1hFjG3t2a2uWsBnt//3o1n8zffm6Dvd1LYOZz1i01dS6CUgFFi4K6JNkOJ0fTc3j1h430/WARmw+4G+ylpcLXd0B8FbjufS0KSgUYLQrqkkSGhfLJ7ckMH9CUPUcz6PXufIbP3ojrm7sh86i70V0pu2MqpS6SXlNQl0xE6NO0Mh1qJ/Dy9xtYsWgOIguh19tQsbHd8ZRSl0CLgiq0siUiGT6gGYdOXo7k9sRZuiZjFu5gQMuqREf4oMGeUspr9PSRKrxju2DTdMqVjIKytViy/Qj/mraBHsPn8du2Imywp5TyOi0KqnBys2DibfDt/ZB5DHA32Lu3NQA3f7yYZyev5aS3G+wppYqEFgVVODOegf2r4LoP/nIHtXa1EpgxuBODOtXkq6W7GeTtBntKqSKh1xTUpVv9JSz/FNoPhsuuOefp6IhQnut5Odc0qojD9WeDvYwcB2VL+Phub0opj+iRgro0J1Jg2hCo1gH+9sIFV21SpRQtqllHEW/P2ULXoXP5btXeS2uwp5QqUgUWBREZLSKHRGTdeZ6/VUTWuL8WiUiTPM/tFJG1IrJKRPT8QTApWRmueeuiG93d0CKJamVjGfzlKu7+fBn7jmcWYUil1MXy5EjhM+BCd0bZAXQ2xjQGXgFGnvV8F2NMU280alJ+wBg4vtuaqdzsVoi7iMZ4QN3ycUx6oB3/7FWf37YdofuweczecLCIwiqlLlaBRcEYMw84eoHnFxljjrm/XQwkeSmb8keL34cRreHQpkt+idAQ4e4ONZg5pBMtq5emRkKsFwMqpQrD29cU7gby3uPOALNEZLmIXLBFoYgMEpFlIrIsNTXVy7GUV+xeDLNfgFp/g8R6hX65qmVj+PTOVtQuVwKAJyauZuS8bTicrkK/tlLq0nitKIhIF6yi8HSexe2NMc2Bq4GHRKTT+bY3xow0xiQbY5ITExO9FUt5S9ohq9FdqapF0ugu2+HkZFYu/56+iRs+WMSmAye9+vpKKc94pSiISGPgE6CPMebMFFZjzD73v4eAKUArb7yf8jGnA765y7o/Qv+xEBXv9beIDAtl5MAWvHdLM1KOZdLrnQUMnb2FbIfT6++llDq/QhcFEakKTAYGGmO25FkeKyJxpx8D3YF8RzApP2dcUO5y6DUUKjQssrcREXo1rsScxzvTu0klxv62k1NZjiJ7P6XUuQq8HaeITACuABKAg8CLQDiAMeZDEfkEuAHY5d7EYYxJFpGaWEcHYE2SG2+Mec2TUHo7Tj9ijG33REg9lU1iXCROl+HThTu4pXVVYiJ0vqVS+fHW7Tj1Hs3q/I7usPoaXfc+VGhkW4xF2w5zy8dLqFImmv/0bUz72gm2ZVE2yM2FKVNg5kzrtpp33QXVq9udyu94qyjojGaVv9ws+Pp2OL4LIkrYGqVdrQS+GtSGsJAQbv1kCU9/s4YTmdpgr1jIzIR27eDuu2H0aHjjDWjQAKZPtzvZxVu+HIYMgQcfhJ9/to7C/ZAWBZW/H5+y7rF8/UdQpobdaWhdsyw/Du7I/Z1r8c2KFO7VBnvFw0cfwfr1kJZmfZ+TAxkZMHAgOALoetPrr0OnTvDuu/DBB3DttVah88PCoCdo1blWjYcVn0OHx6De1XanOSMqPJRnrr7M3WDPmsuQmeMkLdtBYpw22AtK48dbRwtny82FVasgOQAaJezeDS+/DFlZfy5LT4eJE61TYR062JctH3qkoP7KGFg/Bap3hC7/sDtNvholxdOsqtVgb9icLXQbNpcpK1O0wV4wio7Of7nLBVFRvs1yqWbMgJB8/tRmZFjXSvyMFgX1VyIwYALc9MVFNbqzS//kJGomxPLYV6u587Ol7NUGe8HlgQcg9qw2KCJQoYJ1bSEQREXlXxRCQyEmxvd5CqBFQVmMgbn/tWYuh4ZBdCm7E3mkdrk4vr6/HS/2rs+S7UfpPnQuM9cfsDuW8pabboJbb7X+sMbGQlwcJCbC1Km2DZW+aNdeax3ZnC083PrZ/IwWBWVZ9C788hps+M7uJBctNES4s30NZj3WiTY1y57ppaSnk4KAiHWxec0aeOcd6xpDSgrUr293Ms+VKgVff20dFcTFQYkSEBkJb70Fl11md7pz6DwFBTsXwue9rbun9R8TOJ/ACvD4V6uoXb4EgzrWJCxUP/8om506ZQ2lzcmBq66CcuW8+vLemqfg/yeNVdE6dRC+uRNKV4c+I4KmIGQ7nKTnOPjvjM1MX7ufN25oTINK3u/ZpJTH4uKs02F+Tj8+FXez/wlZJ+GmsRBV0u40XhMZFspHA5P54NbmHDiRTZ/3FvK/mZvJytUGe0pdiBaF4q7Hf+DmCVA+QEZyXKSrG1VkzuOd6NO0MuN/3016dgBNeFLKBloUiqsDa8GRAzFloFYXu9MUqVIxEbzVvwmzH+tE2RJWg72P523XAqFUPrQoFEdHt8On18D0J+1O4lNlS1iznpfsOMJr0zfSfdg85m3Ru/wplZcWheImN9PqfCoCHR+3O40t2tVK4Ov72xIZHsJto3/nya9XcyJDG+wpBVoUip/pf7dOHfUdaY04KqZaVi/D9Ec78uAVtZiyci/3jtUh0MqPuFxWR9jERAgLg2bNYO5cn7y1DkktTlaOg5VjoeOTUPcqu9PYLio8lKd6XEbPRhVxuKz5Ohk5DtKyHZSLC5C+Oio4PfMMjBhh9UcCq/lfz55WYSjiJoB6pFCcVGoGLe6ALs/ZncSvNKwcT9MqpQAYNnsL3YbO45vl2mBP2SQtDd5778+CcFpmJvzrX0X+9loUigNHjvVv+frQeziEhNqbx4/d1LIqdcqV4MmvV3Pb6N/ZczSj4I2U8qaUFOuU0dmMgXVFf5v7AouCiIwWkUMikm8asbwjIltFZI2INM/zXA8R2ex+7hlvBlceMsaasTxtsN1JAkLtciWYeF9bXu7TgBW7jnHV2/OYsU4b7CkfSkrK/wZCIj7pDOvJkcJnQI8LPH81UMf9NQj4AEBEQoER7ufrAzeLSAB1sQoSi96BTd9DQj27kwSMkBDhtrbVmflYJ9rXTqBOeW2wp3yoRAmrZfjZbbWjo+HFF4v87QssCsaYecDRC6zSBxhjLIuBUiJSEWgFbDXGbDfG5ABfutdVvrJzIcz5F9S/Dto8YHeagJNUOoaPb0umVqJVFIZ8tYoRv2wl15lPG2SlvOnNN+H556FsWesIoVEj+P57aNmyyN/aG9cUKgN78nyf4l52vuX5EpFBIrJMRJalpuqEokI7dcA6bVSmBlz7btA0urNLtsNJrtPFmzM30+e9hazbe8LuSCqYhYTAc8/B4cPgdFqtw7v4pvOAN4pCfn9tzAWW58sYM9IYk2yMSU5MTPRCrGLu8B+AQP/ganRnl8iwUN6/tQUf/l8LUtOy6TNiIW/M2KQN9lTR8/EHOm8UhRSgSp7vk4B9F1iufKFGRxi82hpxpLymR8MKzHmsMzc0r8xXS/do/yQVdLxRFKYCt7lHIbUBThhj9gNLgToiUkNEIoAB7nVVUdr4PSz+wBp1FG5NwPptz2+0G9WO6NeiqTqsKh8s/UAvmhZCfEw4/72xCXMe73ymwd5Hc7eRpgVCBQFPhqROAH4D6olIiojcLSL3i8j97lWmA9uBrcDHwIMAxhgH8DAwE9gITDTGrC+Cn0GddmQbfPsArPkKnFYvnxX7V9B1bFd+S/mNLEcWe07u4cnZT/LyvJdtDhv4ysRGAPD7jqP8Z8Ymrho2j183H7I5lVKFo7fjDBa5mfBJNzixB+6bB6WrAdB7Qm9+2PID5qzLObHhsaT+PZXo8Gg70gad5buO8tQ3a9iWmk7f5pX55zX1Ke0uGkr5grdux6kzmoPFD0/CwbXQ9+MzBQFg1YFV5xQEABEh5WSKLxMGtRbVyjB9cEce+Vttpq7ax31jl9sdSalLog3xgsH+NbBqHHR6Cup2/8tT9crWy/ePv9PlpGJcRV8lLBYiw0J5ons9rm5Y8cxchowcB2lZDsqV1AZ7KjDokUIwqNgY7pkDV5zbSeTFzi8SE/7XmZEx4THc2+JeSkSU8FXCYqV+pZI0ydNg78qhc5m4dI9e3FcBQYtCgDqWeYxhv7zEEx+15r5p97E+IjrfRncdq3Vk4o0TqVm6JiESQlxEHI+1eYyh3YfakLr4ublVVS6vWJKnJq1h4ChtsKf8n15oDkCH0g/R/MOmfJh2iiuNUFsyOR4ewcQbJ3JN3WvOu12WI4vI0EhEZzf7lMtlGP/7bv7z4yacLsOwm5rQo6GeulPepReai7FX573KbWnH6WVCeJps9uEgIzeDu6fejcucvy9PVFiUFgQbhIQI/9emGrMe60SnugnULR8HaIM95Z+0KASg1PWTecWE8RW5vEvOmeWnck6x/dh2G5OpC6lUKpqPBiZT091gb/CXq3j3pz+0wZ7yK1oUAk36Ed7LyGALLu4h8y8dppwuJyUjtc9RIMh2OHEZw1uzt9D73QWsSTludySlAC0KgSemDNvrX8vAcEjLUxDCQ8JpV6Ud5WLL2ZdNeSwyLJT3bmnOyIEtOJaRw3UjFvL69I3aYE/ZLqiKwpqDa+g3sR913q3DdV9ex/J9QTaBKPM4iJB8w6e0b3E3kaGRlIwsSWx4LA3LNeSrG7+yO6G6SN0bVGDWY525qWUVvlmeQkaOFgVlr6AZffTbnt/oOrYrmbmZGAyCEB0ezfc3f0+XGr7pQ16kNkyFqY/A7dOseQnA/lP7WbF/BUklk2hSoYnNAVVhHUvPoXRshNVgb942BrapRlxUuN2xVIDQ0UdnGTJzCBm5GWdaOhgMGbkZPPLjIzYn84Ij2+C7h6BsLUj887aaFeMqck3da7QgBInTvZKW7jzK/2ZupvuwefyySRvsKd8KmqKwcv/KfJevT12P0xXAh+Q5GfDVQAgJg36fQ1ik3YlUEWtTsyyTHmhHicgw7vxsKUO+XMnR9JyCN1TKC4KmKJSJLpPv8pKRJQnNZ6ZvQDAGfngCDm2AGz6GUlUK3kYFhWZVS/P9ox0YfGUdvl+zn/vG6mRO5RtBUxSeaPtEvj1+BrcebFMiL3A5wLig89NQu6vdaZSPRYaF8li3unz/aAf+2cu6g156toODJ7NsTqaCWdB0SX2i3RMcTD/IiKUjCA8JJ9eVyx1N7uDFzi/aHe3ShYbD9R9aRwzqHE6Xk5UHrNOGzSo0C9wjwgJcVuHPuSfDZm/hq2V7eL7n5dzUsorOUFdeFzSjj047mX2SXcd3USW+CqWiSnk3mK9kHoPJg6D7q3+5sKz+tHD3QvpO7EtmbiZgHRVO6j+J9lXb25ysaO08nM7Tk9awZMdR2tYsy39uaES1srF2x1J+wFujj4KuKAQ8lwsmDIBtP8NdMyCp0P+Ng86xzGNUfbsqaTlpf1keFxHH7sd2B+6HAQ+5XIYvl+7h9ekbyXW5GNq/KT0baYO94s6nQ1JFpIeIbBaRrSJyTtN+Efm7iKxyf60TEaeIlHE/t1NE1rqfK6Z/6S/CgqHwx0y46t9aEM5j4vqJ+Tb+cxonX60L/gl8ISHCLa2rMuvxTnSpV47LKmiDPeU9BV5TEJFQYATQDUgBlorIVGPMhtPrGGPeBN50r98beMwYczTPy3Qxxhz2avJgtP1X+OU1aHgjtLrX7jR+KzUjlSzHuRdbsxxZHM4oPrtZxfhoPvi/FoBVEB6esJI65Urw4BW1iQgLmjEkysc82XNaAVuNMduNMTnAl0CfC6x/MzDBG+GKncUfQNk60Hs46AXE8+pSvQvRYdHnLI8Oi+aK6lf4PpAfyHG6CAsR3p7zB73fXcCqPcftjqQClCdFoTKwJ8/3Ke5l5xCRGKAHMCnPYgPMEpHlIjLofG8iIoNEZJmILEtNTfUgVhDqPwYGToZIvU3mhbSr0o4ra15JbPifF1hjw2O5suaVtKvSzsZk9okMC2X4gGaMuj2ZE5m59H1/Ia/9sIFM7aWkLpInRSG/j6znO3nZG1h41qmj9saY5sDVwEMi0im/DY0xI40xycaY5MTERA9iBZGV46wRR2GREJ9kdxq/JyJM7j+Z93q+R8eqHelYtSPv9XyPyf0nF/shmldeXp5Zj3diQKuqTFm5l0ztuqouUoGjj0SkLfCSMeYq9/fPAhhjXs9n3SnA18aY8ed5rZeANGPM/y70nsVq9NH6b+Hr2+GKZ+GKc67hK3XJTmTkEh8TjtNl+HDuNga2rUZJbbAXtHw5+mgpUEdEaohIBDAAmJpPoHigM/BdnmWxIhJ3+jHQHVhX2NBB4/Af8N3DkNQSOjxudxoVZOJjrAKwbOdR3pq1mW5D5zJnw0GbUyl/V2BRMMY4gIeBmcBGYKIxZr2I3C8i9+dZ9XpgljEmPc+y8sACEVkN/A78YIyZ4b34ASwn3Wp0FxYB/T6z/lWqCLSuWZYpD7andEwE94xZxiMTVnI4LdvuWMpP6eQ1u0x/Cn4faV1YrvU3u9OoYiDH4eKDX7fx3i9/0CSpFN88UDwvygcrb50+CpreRwGn4+NQubkWBOUzEWEhDO5ah6sbVSA715r8l57t4GRWLhXjzx3iq4onneHia8f3gMsJcRWgyQC706hiqG75OBolxQNWg71uQ+cxbskuXC7/O2ugfE+Lgi9lHIVPe1q31VTKD9zWtjpNqsTz/JR13PzxYnYcTi94IxXUtCj4issFU+6DU/sh+W670ygFQNWyMXxxd2v+e0NjNuw/SY+35/HDmv12x1I20msKvrLgLfhjFvT8HyS1sDuNUmeICP1bVqFzvURe/n4DDSpZ928wxhT7yYDFkRYFX9j+K/zyb2jUD1reY3capfJVvmQUI25pDrgb7I1fSa1yJXioSy0iw4LzBkbqXHr6yBci46DWldroTgWMHKeLiLAQ3vnpD3q9s4AVu4/ZHUn5iM5TKErGaBFQAe2XTYd4bspaDpzM4q72NXiie11iIvQEgz/y6U121CWa8Sz8+IzeY1kFrC6XlWPWY534v9bV+G7VvjPzG1Tw0qJQVNZNgiUfWI/1aEEFsLiocF65riE/PdGZ0rEROJwu3vnpD05k5todTRUBLQpFIXULTH0UklpBt5ftTqOUV8RHWw32lu86xvCf/qDb0LnMXH/A5lTK27QoeFt2GkwcaN0bQRvdqSDUumZZvn2wPWVLRHLf2OU8NG4Fqae0wV6w0KLgbftXw4kUuGEUxOd7gzqlAl6jpHimPtyev19Vj9kbDvLAF8vtjqS8RIcReFv19jBkLcSUsTuJUkUqPDSEh7rU5qoG5cnK02DveGYulUtpg71ApUcK3rJ3Oaz8wnqsBUEVI7XLxdGwstVgb+jsLXQfOpexv+3UBnsBSouCN2QchYm3w69vWDfPUaqYuqNddZpXK80/v1vPgJGL2Z6aZnckdZG0KBSWywWT74W0g9D/c4iItTuRUrapUiaGMXe14s0bG7PpwEl6DJ/PtNX77I6lLoJeUyiseW/C1jnQa5h10xylijkRoV9yFTrXTeSVHzbSyH1qSRvsBQaPjhREpIeIbBaRrSLyTD7PXyEiJ0RklfvrBU+3DWhHtsHc/0DjAdDiTp++9c7jO5m/az7Hs4779H2V8lS5klG8e3MzqifEYozhgS9W8ObMTWTlOu2Opi6gwCMFEQkFRgDdgBRgqYhMNcZsOGvV+caYXpe4bUA5lX2K0JBQYsrWgpu/skYc+egT0ImsE9w48UYW7llIRGgE2c5snmj7BK90eUU/hSm/les0lIgKY8Qv25ix7gBv3NCY5Oo6IMMfeXKk0ArYaozZbozJAb4E+nj4+oXZ1u+sP7Selh+3pMIbZen8emmuGnsV+ys28ul1hNu/vZ35u+eT6cjkRPYJshxZDFs8jHFrx/ksg1IXKyIshP/1a8KYu1qRleui30e/8dLU9aRnO+yOps7iSVGoDOzJ832Ke9nZ2orIahH5UUQaXOS2iMggEVkmIstSU1M9iOVbxzKP0eHTDizft5zXXSEsdEWyffsvdPi0A06Xbw6Hj2cdZ8bWGWQ7/zp7NCM3g/8t+p9PMihVGJ3qJjLrsU7c3rY6P6zdT45DG+z5G0+KQn7nJM4egLwCqGaMaQK8C3x7EdtaC40ZaYxJNsYkJyYmehDLt8auGUuOI4f+JpRHieQ9cthKLqnpqczePtsnGY5nHSc0JP+bnRzOOOyTDEoVVmxkGC9d2+AvDfbenrOFExnaYM8feFIUUoAqeb5PAv4yxswYc9IYk+Z+PB0IF5EET7YNFJsPb6ZabhafEM0CHDyN9Wnd4XKw49gOn2SoGl+VEhElzlkeKqF0q9nNJxmU8paSUVaDvRW7j/Puz1vpOmwuM9bp/aHt5klRWArUEZEaIhIBDACm5l1BRCqI+yqniLRyv+4RT7YNFO3KN2GyxJKO4SYycbiPgUIkhKYVmvokQ4iE8ME1HxATHoO4D8IiQiIoFVWKf3X5l08yKOVtrWqUYerD7SkXF8n9X6zggS+Wc+hUlt2xiq0Ci4IxxgE8DMwENgITjTHrReR+EbnfvdqNwDoRWQ28Awwwlny3LYofpKj1bXQLsyJjuC0kl31inQGLCouiecXmtElq47scl/dl7h1z6degHy0qtuDRNo+y9oG1VI2v6rMMSnlbg0rxfPtQe57qUY+fNh3iwS9W2B2p2NLbcXrCkQ1hkRzOOMzzPz3PpI2TCA8N544md/BC5xeIDtfmX0p5y7bUNDJznDSsHE9atoNj6TlUKRNjdyy/563bcWpRKMiepfD17TBgPFRqancapYqVV77fwITfd/PUVfW4rW11QkJ0Ls756D2afSH9CHx9B4SEQulqdqdRqti5s311WlYvw0vTNtDvo9/YeuiU3ZGCnhaF83E5YfI9kH4I+o+B6NJ2J1Kq2EkqHcNnd7ZkaP8mbEtNo+fwBUzVBntFShvinc/c/8K2n6H3cKjUzO40ShVbIkLf5kl0rJPIv6dvpEmS1WDP5TJ6OqkIaFHIj8sFh9ZDk1ug+e12p1FKAYlxkQy7qSlgdVx9YNxyaiaWYPCVdYgKz39Sp7p4WhTyExIC/ceCM9dnje6UUp7LdRrio8P54NdtzFx3gDdubExLbbDnFXpNIS9HNkx9BI7ttIpBWITdiZRS+YgIC+G/Nzbhi7tbk+N00e/D33jhu3WkaYO9QtOikNfM52HFGDiw1u4kSikPdKiTwMwhnbizfXVmrDuAw6kN9gpLi8Jpa76GpR9D24fh8t52p1FKeSg2MowXe1sN9krFWA32hs7azLH0HLujBSQtCgCHNsK0R6FqW+j6kt1plFKXIC5Pg733f91Gt2Fz+WHNfvxxgq4/06IA8MtrEFECbvwUQsPtTqOUKoRWNcow7ZEOVIyP5qHxK7hv7HIOntQGe57SNhcAOelwdAdUaOi791RKFSmH08WoBTsYOnsLDSvHM+mBdnZHKlLeanNRvIekbv0JqraxbqepBUGpoBIWGsJ9nWvRvUEFMnKsUUnaYK9gxff00Z7fYXx/+OkVu5MENWMMmw9vZv2h9biMf40MSctJY8LaCXy07CO2Hd1mdxxVRGokxNKgkjULeuisLXQfNo/RC3bgdPnfWRJ/UDyPFNIPW43uSlaGK562O03QWndoHdd/dT37Tu1DEEpFleKrG7+ifdX2dkdjwe4F9BzXEwCny4kLFw+1fIg3u72J6ITFoHVPxxrsOJzGy99vYNqaffz3hsbUKR9ndyy/UvyuKbic8EVf2PUb3DMbKjYpmvcp5jJzM0kalsTRzKN/WV4ivATbB28nMda++3DnOnMp/7/yHMs69pflseGxTLlpCt1q6a1Ng5kxhu9W7eNf09aTnu3kzX6N6dO0st2xCk1bZ1+qBUNh+6/Q800tCEVo6uap5DrPvRG7wzj4Ys0XNiT607xd83C6nOcsT89NZ9TKUTYkUr4kIlzXrDKzH+9MryYVaVqlFGA12FPF8fRRo/6AQPPb7E4S1A6kHSDHee7koSxHFvtO2dv6OMeZA+c5Q5TpyPRtGGWbhBKRDO3fFLCOHu77Yjk1E2IZ0rUu0RHFt8GeR0cKItJDRDaLyFYReSaf528VkTXur0Ui0iTPcztFZK2IrBIR+26nlnkMjLFultPpSW10V8Q6VO1AaMi5/2OViChB5+qdbUj0p07VOuFwndsjJzY8llsb3WpDImW3XKchoUQEH83bztXD57F4+xG7I9mmwKIgIqHACOBqoD5ws4jUP2u1HUBnY0xj4BVg5FnPdzHGNPXG+a5L4siGsdfDtw/a8vbFUYtKLeheqzsx4X8O/YsOi6ZhuYZcXftqG5NBbEQso68dTXRYNOEh1mTF2PBYulTvwg2X32BrNmWPiLAQXu/bmPH3tMZlYMDIxTw3ZS2nss49BRrsPDl91ArYaozZDiAiXwJ9gA2nVzDGLMqz/mIgyZshC23GM7BvJXR80u4kxcrX/b5m1IpRfLziYxwuBwMbD+ShVg/lewThazc1vImWlVvy+arPOZ51nF51e3FlzSsJkeJ3mU39qV1tq8HeW7M2M23NPp66qp7dkXyuwNFHInIj0MMYc4/7+4FAa2PMw+dZ/0ngsjzr7wCOAQb4yBhz9lHEObw6+mj1VzBlELR7FLrrnASllGfSsh2UiAzD4XTx9pw/uKtDDcrE+m87fV+OPsrv5Hu+lUREugB3A3kH/7c3xjTHOv30kIh0Os+2g0RkmYgsS01N9SCWBw5ugGmDoVp7uPJF77ymUqpYKBFpnUhZtec4H83bRtehc5m6el/QN9jzpCikAFXyfJ8EnDN8REQaA58AfYwxZ67SGGP2uf89BEzBOh11DmPMSGNMsjEmOTHRS2PYs09CQm13o7viN9BKKVV4ydWtBntVSkfz6ISV3DtmOQdOBG+DPU+KwlKgjojUEJEIYAAwNe8KIlIVmAwMNMZsybM8VkTiTj8GugPrvBW+QFXbwH3zIa68z95SKRV8LqtQkskPtuf5npezYGsqD41fYXekIlPgx2djjENEHgZmAqHAaGPMehG53/38h8ALQFngfXeLAIf73FZ5YIp7WRgw3hgzo0h+krwWfwgZR+CKZ8APLmoqpQJfaIhwb6eadKtfnowca/LjqaxcjqbnUK1srM3pvCf42lzsXgyfXQN1roIB43Q+glKqyLw8bQPjf9/FE93qcVeHGoSG2Pf3Rttc5Cct1Wp0F18FrntfC4JSqkgN6lSTDrUTeG36Rvq+v5DNB07ZHanQgqcouJww6W5r5nL/MRBdyu5ESqkgVyE+io9vS+bdm5uRciyTXu/O59uVe+2OVSjBMyRn7wrYtQh6vw0VG9udRilVTIgIvZtUon3tBF6fvpHmVUsDVoO9EBtPJ12q4LqmcHQ7lKnp/UDKdrtP7ObbTd9ijKHPZX2oXqq63ZGUOi9jDPeOWU6NhBge71bPJw329JrCacd2wcZp1mMtCEHpg6UfUO+9ejw952menvM0l4+4nLcXv213LKXOK9dpKFcyko/n7+Cqt+exaNthuyN5LLCLQm4WTLwNvnsIMo4WvL4KOLtP7ObxWY+T5cgiy5FFtjObLEcWz/30HFuPbrU7nlL5iggL4d/XN2LCvW0IEbjl4yU8O3lNQDTYC+yiMOMZ2L8KrvsQYsrYnUYVgSkbp+TbVsDhcjBpwyQbEinluba1yvLj4E7c16kmv2xKxeVftynPV+AWhVUTYPmn0H4IXNbT7jSqiLhM/v8XGcx5n1PKn0RHhPJsz8v56YnOxMeEk+t08caMTRxOy7Y7Wr4Csyic2AvfPwbVO8Lf/ml3GlWE+lzWB8lnvkl4SDh9L+9rQyKlLk2su8He6j3HGTV/B92GzuXblXv9rsFeYBaF+MrQezjcMEob3QW5mqVr8mqXV4kKiyI8JJywkDCiw6L5R6d/UC+h+PW6V4EvuXoZfni0A9UTYhny1Sru/nwZ+477z21gA2tIqjFwfBeUru7zTMpeW45sYdKGSRgMfS/vy2UJl9kdSalCcboMny/ayZszN3N5xTgmP9i+UK/nrSGpgVUUFr0HP78Kg36Bcpf7PphSSnnZnqMZpOc4uKxCSU5l5XI4LYcaCRffYK/4zVPY9RvMfgHqdIVE/ZSolAoOVcrEcFmFkgAMnb2FHm/P46O523A47RlIERhFIe2Q1eiudDXoM0Ib3RVTaTlpvLnwTVp93IruY7vz3abv/O4inVKFcX/nWnSqm8jrP26i7weL2Lj/pM8z+P/pI6cDxl4HKcvgnjlQoaGt2ZQ9MnMzaflxS7Yf206mw7ooFxseyyOtHuH1rq/bnE4p7zHGMH3tAV6cuo7jGbm82a8x1zdLKnC7YnT6yECFRtBrqBaEYmzc2nHsPL7zTEEASM9NZ9jiYew/td/GZEp5l4hwTeOKzH6sMzc0TyK5mjUx1+nyzQd4/y4KxkBoOPR4HZreYncaZaPvt3xPem76OcsjQiNYtGeRDYmUKlqlYyN448bGVCkTgzGGQWOW8a9p68nIcRTp+/pvUTi6Az7sCPtX251E+YFKcZUIlfw7TSbGJvo4jVK+les0VC4dzacLd9J92DwW/FF0Dfb8sygYYzW6O7EbouLtTqP8wAPJDxAZFvmXZSESQpnoMnSo2sGmVEr5RkRYCC/3acjE+9oSERrC/41awlPfrOZkETTY86goiEgPEdksIltF5Jl8nhcRecf9/BoRae7ptvk6sQcOrIHrR+pENQVAo/KNGH3taOIi4igZWZKY8BjqlqnLT7f9RIj452cbpbytVY0yTB/ckQeuqMWCPw5TFOOEChx9JCKhwBagG5ACLAVuNsZsyLNOT+ARoCfQGhhujGntybb5Sa4UapaNeQG6vnjJP5gKTtmObFYeWElcRBz1E+vn2xdJqeIgM8dJdEQouU4Xb83awrM9L/fK6CNPGge1ArYaY7YDiMiXQB8g7x/2PsAYY1WYxSJSSkQqAtU92PZckXHQ5fmL/FFUcRAZFkmbpDZ2x1DKdqfv5rYm5TijF+zw2ut6UhQqA3vyfJ+CdTRQ0DqVPdwWABEZBAxyf5stYeHrPMhmpwQgEG6npDm9S3N6l+b0Hq90iPSkKOR3fH72OafzrePJttZCY0YCIwFEZJk3DoOKUiBkBM3pbZrTuzSn94jIJdzY/lyeFIUUoEqe75OAfR6uE+HBtkoppfyEJ8M2lgJ1RKSGiEQAA4CpZ60zFbjNPQqpDXDCGLPfw22VUkr5iQKPFIwxDhF5GJgJhAKjjTHrReR+9/MfAtOxRh5tBTKAOy+0rQe5Rl7KD+NjgZARNKe3aU7v0pze45WMftkQTymllD101o9SSqkztCgopZQ6w6dFweftMoou563ufGtEZJGINMnz3E4RWSsiq7w1RKwQOa8QkRPuLKtE5AVPt/Vxzr/nybhORJwiUsb9nE9+nyIyWkQOiUi+82P8aN8sKKe/7JsF5bR93/Qgo+37pfu9qojILyKyUUTWi8jgfNbx3v5pjPHJF9aF5m1ATayhqquB+met0xP4EWt+Qxtgiafb+jhnO6C0+/HVp3O6v98JJPjJ7/MK4PtL2daXOc9avzfwsw2/z05Ac2DdeZ63fd/0MKft+6aHOf1h37xgRn/YL93vVRFo7n4ch9U6qMj+dvrySOFMuwxjTA5wuuVFXmfaZRhjFgOn22V4sq3PchpjFhljjrm/XYw1/8LXCvM78avf51luBiYUUZbzMsbMA45eYBV/2DcLzOkn+6Ynv8/z8dnv8yIz2rJfAhhj9htjVrgfnwI2YnWLyMtr+6cvi8L5WmF4so4n23rLxb7X3VgV+jQDzBKR5WK17igqnuZsKyKrReRHEWlwkdt6g8fvJSIxQA9gUp7Fvvp9FsQf9s2LZde+6Sm7902P+NN+KSLVgWbAkrOe8tr+6cmMZm/xSbsML/D4vUSkC9b/eHkb+rc3xuwTkXLAbBHZ5P5EYkfOFUA1Y0yaWJ1svwXqeLitt1zMe/UGFhpj8n5689XvsyD+sG96zOZ90xP+sG96yi/2SxEpgVWYhhhjTp79dD6bXNL+6csjhcK0y/BkW2/x6L1EpDHwCdDHGHPk9HJjzD73v4eAKViHb7bkNMacNMakuR9PB8JFJMGTbX2ZM48BnHWI7sPfZ0H8Yd/0iB/smwXyk33TU7bvlyISjlUQxhljJuezivf2T19cKHFf8AgDtgM1+POCR4Oz1rmGv14s+d3TbX2csyrW7O12Zy2PBeLyPF4E9LAxZwX+nKDYCtjt/t361e/TvV481vndWDt+n+73qM75L4zavm96mNP2fdPDnLbvmwVl9KP9UoAxwNsXWMdr+6fPTh8Ze9plFFXOF4CywPti3eTFYawOiuWBKe5lYcB4Y8wMG3PeCDwgIg4gExhgrD3F336fANcDs4wx6Xk299nvU0QmYI2ISRCRFOBFIDxPRtv3TQ9z2r5vepjT9n3Tg4xg837p1h4YCKwVkVXuZc9hfQDw+v6pbS6UUkqdoTOalVJKnaFFQSml1BlaFJRSSp2hRUEppdQZWhSUUkqdoUVBKaXUGVoUlFJKnfH/LNIfJfPmgJQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.xlim([0, 2])\n",
    "plt.ylim([0, 2])\n",
    "plt.scatter(X[:,1], X[:,2], c = colors)\n",
    "abline(-1, 2)\n",
    "abline(2, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abc138f2",
   "metadata": {},
   "source": [
    "### Task 5: Do task 4 again with a new loss\n",
    "\n",
    "Repeat The previous task, generating 5 plots of the separators correspond to hyperplanes, but this time use the loss function below.\n",
    "\n",
    "How are the results different?  Write a brief paragraph explaining why the results look different."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0286d066",
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_hinge_margin(y, activation):\n",
    "    return 0 if y * activation > 1 else -y * activation + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4bd59176",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}