{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YHI3vyhv5p85"
   },
   "source": [
    "# Exercise 1 : Conversion from Celsius to Fahrenheit (Simple Regression Analysis)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "F8YVA_634OFk"
   },
   "source": [
    "The problem we will solve is to convert from Celsius to Fahrenheit, where the approximate formula is:\n",
    "\n",
    "$$ f = c \\times 1.8 + 32 $$\n",
    "\n",
    "\n",
    "Of course, it would be simple enough to create a conventional Python function that directly performs this calculation, but that wouldn't be machine learning.\n",
    "\n",
    "\n",
    "Instead, we will give TensorFlow some sample Celsius values (0, 8, 15, 22, 38) and their corresponding Fahrenheit values (32, 46, 59, 72, 100).\n",
    "Then, we will train a model that figures out the above formula through the training process. This is a _simple regression analysis_ problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fA93WUy1zzWf"
   },
   "source": [
    "## Import dependencies\n",
    "\n",
    "First, import TensorFlow. Here, we're calling it `tf` for ease of use. We also tell it to only display errors.\n",
    "\n",
    "Next, import [NumPy](http://www.numpy.org/) as `np`. Numpy helps us to represent our data as highly performant lists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "X9uIpOS2zx7k"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.7.1\n"
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "y_WQEM5MGmg3"
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "logger = tf.get_logger()\n",
    "logger.setLevel(logging.ERROR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AC3EQFi20buB"
   },
   "source": [
    "## Set up training data\n",
    "\n",
    "As we saw before, supervised Machine Learning is all about figuring out an algorithm given a set of inputs and outputs. Since the task in this exercise is to create a model that can give the temperature in Fahrenheit when given the degrees in Celsius, we create two lists `celsius_q` and `fahrenheit_a` that we can use to train our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gg4pn6aI1vms"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-40.0 degrees Celsius = -40.0 degrees Fahrenheit\n",
      "-10.0 degrees Celsius = 14.0 degrees Fahrenheit\n",
      "0.0 degrees Celsius = 32.0 degrees Fahrenheit\n",
      "8.0 degrees Celsius = 46.0 degrees Fahrenheit\n",
      "15.0 degrees Celsius = 59.0 degrees Fahrenheit\n",
      "22.0 degrees Celsius = 72.0 degrees Fahrenheit\n",
      "38.0 degrees Celsius = 100.0 degrees Fahrenheit\n"
     ]
    }
   ],
   "source": [
    "celsius_q    = np.array([-40, -10,  0,  8, 15, 22,  38],  dtype=float)\n",
    "fahrenheit_a = np.array([-40,  14, 32, 46, 59, 72, 100],  dtype=float)\n",
    "\n",
    "for i,c in enumerate(celsius_q):\n",
    "  print(\"{} degrees Celsius = {} degrees Fahrenheit\".format(c, fahrenheit_a[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wwJGmDrQ0EoB"
   },
   "source": [
    "### Some Machine Learning terminology\n",
    "\n",
    " - **Feature** — The input(s) to our model. In this case, a single value — the degrees in Celsius.\n",
    "\n",
    " - **Labels/response variable** — The output our model predicts. In this case, a single value — the degrees in Fahrenheit. In a classification setting, we would predict labels (discrete classes), in a regression setting, we predict a continuous response variable, such as Fahrenheit.\n",
    "\n",
    " - **Example** — A pair of inputs/outputs used during training. In our case a pair of values from `celsius_q` and `fahrenheit_a` at a specific index, such as `(22,72)`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VM7_9Klvq7MO"
   },
   "source": [
    "## 1. Define the Network\n",
    "\n",
    "Next create the model. We will use the simplest possible model we can, a Dense network. Since the problem is straightforward, this network will require only a single layer, with a single neuron.\n",
    "\n",
    "### Build a layer\n",
    "\n",
    "We'll call the layer `l0` and create it by instantiating `tf.keras.layers.Dense` with the following configuration:\n",
    "\n",
    "*   `input_shape=[1]` — This specifies that the input to this layer is a single value. That is, the shape is a one-dimensional array with one member. Since this is the first (and only) layer, that input shape is the input shape of the entire model. The single value is a floating point number, representing degrees Celsius.\n",
    "\n",
    "*   `units=1` — This specifies the number of units in the layer. The number of units defines how many internal variables the layer has to try to learn how to solve the problem (more later). Since this is the final layer, it is also the size of the model's output — a single float value representing degrees Fahrenheit. (In a multi-layered network, the size and shape of the layer would need to match the `input_shape` of the next layer.)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pRllo2HLfXiu"
   },
   "outputs": [],
   "source": [
    "l0 = tf.keras.layers.Dense(units=1, input_shape=[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_F00_J9duLBD"
   },
   "source": [
    "### Assemble layers into the model\n",
    "\n",
    "Once layers are defined, they need to be assembled into a model. The Sequential model definition takes a list of layers as argument, specifying the calculation order from the input to the output.\n",
    "\n",
    "This model has just a single layer, `l0`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cSp-GpLSuMRq"
   },
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "  tf.keras.layers.Dense(units=1, input_shape=[1])\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_11\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_15 (Dense)            (None, 1)                 2         \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 2\n",
      "Trainable params: 2\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kiZG7uhm8qCF"
   },
   "source": [
    "## 2. Compile the network, with loss and optimizer functions\n",
    "\n",
    "Before training, the model has to be compiled. When compiled for training, the model is given:\n",
    "\n",
    "- **Loss function** — A way of measuring how far off predictions are from the desired outcome. (The measured difference is called the \"loss\".)\n",
    "\n",
    "- **Optimizer function** — A way of adjusting internal values in order to reduce the loss.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "m8YQN1H41L-Y"
   },
   "outputs": [],
   "source": [
    "model.compile(loss='mean_squared_error',\n",
    "              optimizer=tf.keras.optimizers.Adam(0.1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "17M3Pqv4P52R"
   },
   "source": [
    "These are used during training (`model.fit()`, below) to first calculate the loss at each point, and then improve it. In fact, the act of calculating the current loss of a model and then improving it is precisely what training is.\n",
    "\n",
    "During training, the optimizer function is used to calculate adjustments to the model's internal variables. The goal is to adjust the internal variables until the model (which is really a math function) mirrors the actual equation for converting Celsius to Fahrenheit.\n",
    "\n",
    "`TensorFlow` uses numerical analysis to perform this tuning, and all this complexity is hidden from you so we will not go into the details here. What is useful to know about these parameters are:\n",
    "\n",
    "The loss function ([mean squared error](https://en.wikipedia.org/wiki/Mean_squared_error)) and the optimizer ([Adam](https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/)) used here are standard for simple models like this one, but many others are available. It is not important to know how these specific functions work at this point.\n",
    "\n",
    "One part of the Optimizer you may need to think about when building your own models is the learning rate (`0.1` in the code above). This is the step size taken when adjusting values in the model. If the value is too small, it will take too many iterations to train the model. Too large, and accuracy goes down. Finding a good value often involves some trial and error, but the range is usually within 0.001 (default), and 0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "c-Jk4dG91dvD"
   },
   "source": [
    "## 3. Fit the model\n",
    "\n",
    "Train the model by calling the `fit` method.\n",
    "\n",
    "During training, the model takes in Celsius values, performs a calculation using the current internal variables (called \"weights\") and outputs values which are meant to be the Fahrenheit equivalent. Since the weights are initially set randomly, the output will not be close to the correct value. The difference between the actual output and the desired output is calculated using the loss function, and the optimizer function directs how the weights should be adjusted.\n",
    "\n",
    "This cycle of calculate, compare, adjust is controlled by the `fit` method. The first argument is the inputs, the second argument is the desired outputs. The `epochs` argument specifies how many times this cycle should be run, and the `verbose` argument controls how much output the method produces."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lpRrl7WK10Pq"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Finished training the model\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)\n",
    "print(\"Finished training the model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0-QsNCLD4MJZ"
   },
   "source": [
    "## 4. Evaluate the Model - Display training statistics\n",
    "\n",
    "The `fit` method returns a history object. We can use this object to plot how the loss of our model goes down after each training epoch. A high loss means that the Fahrenheit degrees the model predicts is far from the corresponding value in `fahrenheit_a`.\n",
    "\n",
    "We'll use [Matplotlib](https://matplotlib.org/) to visualize this (you could use another tool). As you can see, our model improves very quickly at first, and then has a steady, slow improvement until it is very near \"perfect\" towards the end.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IeK6BzfbdO6_"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7ff9ec12a190>]"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkLElEQVR4nO3deZhc1Xnn8e9b1VVdvS/qbkloQRKIReygYOElMWBsjB3Dk9gxxElIhmfIxHiMJ84kJpkZx06cx86TQExiOyYxMV5i7Dj2mMdDbGQ2L8EGiV2SQUIIq4XQ1vveXfXOH/dUd6nVS0nq6qru+n2ep55777m3qt4rmn77nHPPOebuiIiIzCRW7ABERKT0KVmIiMislCxERGRWShYiIjIrJQsREZlVRbEDKISWlhZfs2ZNscMQEVlQtm7detjdW6c6tyiTxZo1a9iyZUuxwxARWVDM7JXpzqkZSkREZqVkISIis1KyEBGRWSlZiIjIrJQsRERkVkoWIiIyKyULERGZlZJFjle7Brn9gRd4+XB/sUMRESkpShY5jvSNcOdDu9h1sK/YoYiIlBQlixw1lXEA+ofHihyJiEhpUbLIUVsZzX7Sp2QhInIUJYscNSFZDIwoWYiI5FKyyFGViGMGfcPpYociIlJSlCxyxGJGdSKuPgsRkUkKnizMLG5mT5nZd8PxWjP7mZntMrOvm1kylFeG413h/Jqcz7gtlL9gZm8rZLw1lRVKFiIik8xHzeJWYEfO8aeAO9z9dKATuCmU3wR0hvI7wnWY2QbgeuAc4Grgs2YWL1SwtZUV6uAWEZmkoMnCzFYC7wD+ORwbcAXwzXDJPcB1Yf/acEw4f2W4/lrgXncfdveXgV3ApYWKuaaygoER9VmIiOQqdM3i74A/BjLheAnQ5e7ZP93bgRVhfwWwFyCc7w7Xj5dP8Z45V52Mq2YhIjJJwZKFmb0TOOjuWwv1HZO+72Yz22JmWw4dOnTCn1OrPgsRkWMUsmbxBuBdZrYHuJeo+enTQKOZZdf+XgnsC/v7gFUA4XwDcCS3fIr3jHP3u9x9o7tvbG2dcr3xvKiDW0TkWAVLFu5+m7uvdPc1RB3UD7n7+4CHgXeHy24EvhP27wvHhPMPubuH8uvD01JrgfXA44WKu6aygn71WYiIHKVi9kvm3J8A95rZXwJPAV8I5V8Avmxmu4AOogSDu28zs28A24Ex4BZ3L9hv85qkxlmIiEw2L8nC3R8BHgn7u5niaSZ3HwLeM837PwF8onARTsg+DZXJOLGYzcdXioiUPI3gniQ7mWC/5ocSERmnZDHJxGSC6rcQEclSspgku6aFxlqIiExQspikJhmaoZQsRETGKVlMUqMFkEREjqFkMUm2g3tAa1qIiIxTspikOrsOt56GEhEZp2QxidbhFhE5lpLFJNk+C3Vwi4hMULKYpDqRfXRWfRYiIllKFpPEYkZNMs6AahYiIuOULKZQXVmhDm4RkRxKFlOI1uFWM5SISJaSxRRqKjVNuYhILiWLKdQktVqeiEguJYsp1KjPQkTkKEoWU4jW4VafhYhIlpLFFGor4xrBLSKSQ8liCjXJCo2zEBHJoWQxhWicRbQOt4iIKFlMqTbMPDswqn4LERFQspiSJhMUETmaksUUstOU9w4pWYiIgJLFlOpS2WQxWuRIRERKg5LFFOpSCUALIImIZClZTGGiZqFkISICShZTytYs1AwlIhJRspiCahYiIkdTsphCbTJKFj1KFiIigJLFlGIxo7ayQs1QIiKBksU06lIVaoYSEQmULKZRl6qgT8lCRARQsphWXSpB77CaoUREQMliWmqGEhGZoGQxjaiDW8lCRASULKZVl0roaSgRkUDJYhr1qQqNsxARCZQsplGXqmBkLMPwmBZAEhFRspjG+Myzql2IiBQuWZhZysweN7NnzGybmX0slK81s5+Z2S4z+7qZJUN5ZTjeFc6vyfms20L5C2b2tkLFnEvzQ4mITChkzWIYuMLdLwAuBK42s03Ap4A73P10oBO4KVx/E9AZyu8I12FmG4DrgXOAq4HPmlm8gHEDWi1PRCRXwZKFR/rCYSK8HLgC+GYovwe4LuxfG44J5680Mwvl97r7sLu/DOwCLi1U3FmaplxEZEJB+yzMLG5mTwMHgc3AS0CXu2f/XG8HVoT9FcBegHC+G1iSWz7Fe3K/62Yz22JmWw4dOnTSsWebofRElIhIgZOFu6fd/UJgJVFt4KwCftdd7r7R3Te2trae9OfVq2YhIjJuXp6Gcvcu4GHgMqDRzCrCqZXAvrC/D1gFEM43AEdyy6d4T8Gog1tEZEIhn4ZqNbPGsF8FXAXsIEoa7w6X3Qh8J+zfF44J5x9ydw/l14enpdYC64HHCxV3Vm1IFn3DShYiIhWzX3LClgP3hCeXYsA33P27ZrYduNfM/hJ4CvhCuP4LwJfNbBfQQfQEFO6+zcy+AWwHxoBb3L3gI+US8RipREzNUCIi5JEswhNJ7wPWufvHzWw1sMzdZ/zr3t2fBS6aonw3UzzN5O5DwHum+axPAJ+YLda5Fs0PpZqFiEg+zVCfJepruCEc9wKfKVhEJUTTlIuIRPJphnqdu19sZk8BuHtndtT1YleXStCjZigRkbxqFqOh38Eh6rgGMgWNqkTUq2YhIgLklyzuBL4NtJnZJ4AfA39V0KhKRLQAkmoWIiKzNkO5+1fNbCtwJWDAde6+o+CRlYC6VIUenRURYYZkYWbNOYcHga/lnnP3jkIGVgrqUgl6BpUsRERmqllsJeqnMGA10QyxBjQCvwDWFjq4YmuoSjA4mmZkLEOyQkt/iEj5mvY3oLuvdfd1wA+AX3X3FndfArwTeGC+AiymhqpofqjuQfVbiEh5y+fP5U3ufn/2wN3/A3h94UIqHY3VShYiIpDfOItXzex/AV8Jx+8DXi1cSKWjXjULEREgv5rFDUAr0eOz3wbamBjNvahlm6F6lCxEpMzl8+hsB3DrPMRSctRnISISyWciwYcJo7dzufsVBYmohGSTRdfASJEjEREprnz6LP4oZz8F/DrRVOGL3kTNoixuV0RkWvk0Q22dVPQTMyv44kOlIBGPUZOMqxlKRMpePs1QuSO5Y8AlREueloWGqoSShYiUvXyaoXJHco8BLwM3FTKoUlKvZCEikleyODusYjfOzCoLFE/JaaxO0D2oDm4RKW/5jLP4zynKHpvrQEqVmqFERGaedXYZsAKoMrOLiJqhAOqB6nmIrSQoWYiIzNwM9Tbgd4GVwO055b3AnxYwppKiZCEiMkOycPd7gHvM7Nfd/d/nMaaS0lidZGg0w9BomlQiXuxwRESKYqZmqN9y968Aa8zsDyefd/fbp3jbolOfMz+UkoWIlKuZmqFqwrZ2PgIpVbnzQ7XVp4ocjYhIcczUDPX5sP3Y/IVTeprCmhadA+q3EJHylc8I7lbgvwJrcq939/9SuLBKR1N1EoCOfo21EJHylc+gvO8APyJaXjVd2HBKT3NNlCw086yIlLN8kkW1u/9JwSMpUeM1CyULESlj+Yzg/q6ZXVPwSEpUVTJOKhGjU81QIlLG8kkWtxIljEEz6zGzXjPrKXRgpaS5OklHvzq4RaR85bOeRd18BFLKmmqS6rMQkbKWz9NQF09R3A284u5lsYRcU3VSfRYiUtby6eD+LHAx8Fw4Pg94Hmgwsz9w9wcKFVypaKpJ0t45UOwwRESKJp8+i1eBi9z9Ene/BLgQ2A1cBfx1AWMrGc3VCQ3KE5Gylk+yOMPdt2UP3H07cJa77y5cWKWlqSZJ9+AoY+lMsUMRESmKfJqhtpnZ54B7w/F7ge1htbyy+HM7O9aia3CUltqyWSRQRGRcPjWL3wV2AR8Kr92hbBS4vDBhlZamMIpbYy1EpFzl8+jsIPC34TVZ35xHVIKaQ81C/RYiUq5mrVmY2Xoz+6aZbTez3dlXHu9bZWYPh/dtM7NbQ3mzmW02s51h2xTKzczuNLNdZvZs7iO7ZnZjuH6nmd14Mjd8IhrDzLOaTFBEylU+zVD/AnwOGCNqdvoS8JU83jcGfNjdNwCbgFvMbAPwEeBBd18PPBiOAd4OrA+vm8N3YmbNwEeB1wGXAh/NJpj5kp1MUMlCRMpVPsmiyt0fBMzdX3H3PwfeMdub3H2/uz8Z9nuBHcAK4FrgnnDZPcB1Yf9a4Ese+SnQaGbLidYC3+zuHe7eCWwGrs73BufCktooWRzpG57PrxURKRn5PA01bGYxYKeZfQDYx3Gunmdma4CLgJ8BS919fzj1GrA07K8A9ua8rT2UTVc++TtuJqqRsHr16uMJb1aVFXHqUxUcVrIQkTKV70SC1cAHgUuA3wby7jcws1rg34EPuftRExC6uwOed7QzcPe73H2ju29sbW2di488SktdJYf71AwlIuUpn6ehngi7fcDvHc+Hm1mCKFF81d2/FYoPmNlyd98fmpkOhvJ9wKqct68MZfuAN08qf+R44pgLLbWVHOpVzUJEytO0ycLM7pvpje7+rpnOm5kBXwB2uPvtOafuI6qZfDJsv5NT/gEzu5eoM7s7JJTvA3+V06n9VuC2mb67EFprK9mxv6xmZhcRGTdTzeIyor6CrxH1NdhxfvYbiJqsnjOzp0PZnxIliW+Y2U3AK8BvhHP3A9cQDQAcINRi3L3DzP4CyNZwPu7uHccZy0lrqU1ySH0WIlKmZkoWy4gmC7wB+E3g/wFfy50naibu/mOmTzBXTnG9A7dM81l3A3fn872F0lJbSe/QGEOjaVKJeDFDERGZd9N2cLt72t2/5+43Eo2T2AU8Ep6IKjstddGcUEc01kJEytCMHdxhssB3ENUu1gB3At8ufFilJzuB4OHeYVY0VhU5GhGR+TVTB/eXgHOJ+hI+5u7Pz1tUJaglDMzTWAsRKUcz1Sx+C+gnGmfxwejhJiDqh3B3ry9wbCVlvGahZCEiZWjaZOHu+QzYKxutddlkoT4LESk/Sgh5SiXi1FVWaGCeiJQlJYvj0FqvUdwiUp6ULI7DsvoU+7sHix2GiMi8y2fxo5ow6yxmdoaZvSvM+VR2ltWnONCjmoWIlJ98ahY/BFJmtgJ4gGgKjy8WMqhStawhxYGeITKZOZkoV0RkwcgnWZi7DwC/BnzW3d8DnFPYsErTsoYUYxnncL9qFyJSXvJKFmZ2GfA+ovmhAMpycqRl9SkAXuseKnIkIiLzK59k8SGiKcG/7e7bzGwd8HBBoypRyxqULESkPOWz+NGjwKMAoaP7sLt/sNCBlaLxZNGjZCEi5SWfp6H+1czqzawGeB7Ybmb/s/ChlZ6WmkoqYqaahYiUnXyaoTaEtbOvA/4DWEv0RFTZicWMpfUpJQsRKTv5JItEGFdxHXCfu48CZfvs6NL6SvYrWYhImcknWXwe2APUAD80s1OBsl2MenlDlUZxi0jZmTVZuPud7r7C3a/xyCvA5fMQW0la1VzNvq5B0hqYJyJlJJ8O7gYzu93MtoTX3xLVMsrSquYqRtOuJ6JEpKzk0wx1N9AL/EZ49QD/UsigStnq5moA9nYMFDkSEZH5k0+yOM3dP+ruu8PrY8C6QgdWqrLJ4hdKFiJSRvJJFoNm9sbsgZm9ASjbHt5TGquImWoWIlJeZh3BDfw34Etm1hCOO4EbCxdSaUvEYyxvqFKyEJGyks90H88AF5hZfTjuMbMPAc8WOLaStbq5Ws1QIlJW8l4pz917wkhugD8sUDwLQpQsyrYlTkTK0Ikuq2pzGsUCs3pJNYf7hukfHit2KCIi8+JEk0VZj0hb1xINM3npUF+RIxERmR/T9lmYWS9TJwUDqgoW0QKwfmkdADsP9HH+ysbiBiMiMg+mTRbuXjefgSwka5ZUk4gbLx7sLXYoIiLz4kSbocpaRTzGupZadh1QM5SIlAclixO0fmmtahYiUjaULE7QGUvraO8cZGBET0SJyOKnZHGC1rfV4g4vHewvdigiIgWnZHGCNpxSD8Bz+7qLHImISOEpWZyg1c3VNNckeeoXncUORUSk4JQsTpCZceGqRp7e21XsUERECk7J4iRctKqRnQf76B4cLXYoIiIFVbBkYWZ3m9lBM3s+p6zZzDab2c6wbQrlZmZ3mtkuM3vWzC7Oec+N4fqdZlZSU6NftLoJgGfbu4obiIhIgRWyZvFF4OpJZR8BHnT39cCD4Rjg7cD68LoZ+BxEyQX4KPA64FLgo9kEUwrOX9WAGWzZo34LEVncCpYs3P2HQMek4muBe8L+PcB1OeVf8shPgUYzWw68Ddjs7h3u3gls5tgEVDT1qQQXrWrkkRcOFjsUEZGCmu8+i6Xuvj/svwYsDfsrgL0517WHsunKj2FmN5vZFjPbcujQobmNegZXnr2UZ9q7Odg7NG/fKSIy34rWwe3uzhxOde7ud7n7Rnff2NraOlcfO6srzmoD4JGfz1+CEhGZb/OdLA6E5iXCNtt+sw9YlXPdylA2XXnJOGtZHac0pPjetteKHYqISMHMd7K4D8g+0XQj8J2c8t8JT0VtArpDc9X3gbeaWVPo2H5rKCsZZsavXbySh184yF6tyy0ii1QhH539GvAYcKaZtZvZTcAngavMbCfwlnAMcD+wG9gF/BPwfgB37wD+AngivD4eykrKb206lZgZX3psT7FDEREpiGkXPzpZ7n7DNKeunOJaB26Z5nPuBu6ew9Dm3LKGFG8/dxn3Pr6X3/+V02iprSx2SCIic0ojuOfIh95yBoOjaf73/32eKPeJiCweShZz5PS2Wv7obWfyH8+/xof/7RmGRtPFDklEZM4UrBmqHP3+L69jZCzD7Ztf5IcvHubt5y7jzGV1rGiqYkVjFac0VlFbqX9yEVl49JtrDpkZH7xyPb+0ppl/+tFuvvVkO/0jR9cw6lMVnLeygavPWcavXnAKjdXJIkUrIpI/W4zt6xs3bvQtW7YUOwzSGedg7xCvdg2yrytsOwf5yUuH2X2on6pEnOsvXcVNb1zLyqbqYocrImXOzLa6+8apzqlmUUDxmLG8oYrlDVVccurR57a/2sM//3g3X37sFb782Cu8Z+MqPnDF6axorCpOsCIiM1DNosj2dQ3yj4+8xL1P/AKA639pNe+//DSWNyhpiMj8mqlmoWRRIvZ1DfKZh3fxjSf2EosZv3npat7/5tNoq08VOzQRKRNKFgvI3o4B/uGhXXzzyXbiZlx97jJuuHQ1m9Y1Y2bFDk9EFjEliwXolSP9/MtP9vCtJ9vpGRpjzZJq3nn+KVxz3nLOXl6nxCEic07JYgEbGk1z/3P7+ebWdn66+wgZhzVLqrnmvOW87ZxlnLeigVhMiUNETp6SxSJxpG+Y7287wP3P7eex3UdIZ5yl9ZVcefZSrtqwlNeftoTKinixwxSRBUrJYhHq7B/hoZ8f5Ac7DvDoi4cYGElTk4zzK2e2ctWGpVx+ZpsG/InIcVGyWOSGRtM89tIRHth+gB/sOMCh3mHiMePSNc28ZcNS3rphKauaNehPRGamZFFGMhnnmfYuNofE8eKBPiBa0e+qDVFz1XkrGtRBLiLHULIoY3sO9/ODHQd4YPsBtuzpIOOwtL6St4R+jk3rlpBKqJ9DRJQsJOjI9nNsP8APd0b9HKlEjNetXcKb1rfwy2e0sr6tVrUOkTKlZCHHyPZzPPriIX608xAvHeoHYFl9ijetb+FNZ7TyxtNbaK5RJ7lIudBEgnKMVCLO5We1cflZbUA03ciPdx7ihy8e5oHtB/i3re2YwbmnNPCm9S28cX0LF69uUpOVSJlSzUKOkc44z+3r5kcvHuJHOw/z5C86Gcs4yYoYF61qZNO6JVx22hIuXNWo5CGyiKgZSk5K79AoT+zp4LGXjvDT3R1se7WbjEOyIsbFqxu5bF0Lm9Y1c+HqRg0KFFnAlCxkTnUPjvLEyx38dPcRfvryEba92oM7VFbEuHBVI5ec2sTGNU1cvLpJAwNFFhAlCymo7oFRHg81j62vdLDt1R7GMtHP1elttWw8tYmLT21i46lNrG2p0dNWIiVKyULm1eBImmfau9j6Suf4q3twFIDmmiQXr27i4lMbuWBlI+etbKA+lShyxCICehpK5llVMs6mdUvYtG4JEI0q3324jy17JpLHD3YcGL9+XUsN569s4PyVjVywqoFzTmlQx7lIiVHNQoqia2CEZ9u7eba9i2fC9kDPMBCtXX7G0jouCAlkwyn1nLm0jqqkEohIIakZShaEAz1DPLO3i2fbu3mmPdpmm69iBmtbathwSgNnL69jw/J6Niyvp7WuUn0gInNEyUIWJHdnb8cg2/d3s31/Lzv297D91R72dQ2OX7OkJsmGU+o5e3k9Zy+v44yldZzWWqtmLJEToD4LWZDMjNVLqlm9pJqrz10+Xt49MMqO13rYsT96bd/fwxf/cw8jY5nwPljdXM36tjrWL61lfVst69vqOK2thuqkfuRFToT+z5EFp6E6cVQHOsBoOsPLh/vZeaCPnQd72Xmwj50Henn0xYOMpqPasxmsbKqKkkhbLWtbaljTUsPalhra1JwlMiMlC1kUEvEYZyyNmqFgohYyms7wypEBdh3sZeeBPl4MSeTHOw8zks6MX1edjHPqkhrWtdSwpqWaNUtqxpPJkpqkEomUPSULWdQS8Rint9VyelstV587UZ7OOK92DbLnSD8vH45eew73s31/D9/b9hrpzERfXl1lBae2VLOqqZqVTVWsao62K8OxmrakHOinXMpSPGasaq5mVXM1b1rfetS50XSG9s5B9hyeSCSvdAzwwoFeHvz5wfG+kawlNUlWhgSSTSgrmqo4paGKZQ0p6lMVqpnIgqdkITJJIh5jbejLuHzSuUzGOdw/zN6OQdo7B2jvnNhuf7WHzdsOHNW8BVET17L6FMsaotfyhhTLGqpYVp/dT9FcnSQWU0KR0qVkIXIcYjGjrS5FW12KS05tOuZ8JuMc7B2mvXOA13qGeK17iP3dQ+P7P9vdwYGeofG5s7KS8RitdZW01lXSUhttW2uTRx1ntzWV+t9W5p9+6kTmUCxm4zWI6aQzzpG+4aOSyP7uIQ70DHG4L0o0T+/t5Ej/CFMNg6pKxEPySNJSW8mS2iSN1Umaq5M0VidorgnHNVFZXapCtRY5aUoWIvMsHjPa6lO01ae4YIbrxtIZOgZGONw7wqG+YQ73DnOob5hDvcMcDts9R/p58hdddA2MHFNbyYoZNE1OJOG4vipBfaoibBPU5ezXV1VQlYirv0UAJQuRklURj403ec3G3ekdHqOrf5SOgRE6B0bo7B+hc2A0bLNlo+ztGODZ9i46B0aP6aw/JoaYHZNA6iqjbW1lgtrKONWVFdRUVlCTjFOdrKC2soLqyjg1yQpqwra6Mq6FsRa4BZMszOxq4NNAHPhnd/9kkUMSKRlmFv0yTyVYvaQ67/cNjabpHRqjZ2g02g6O0jM0Ss/gGL1DE/u55w/19tEzOEbf8Bj9I2NTNpVNJRG3iWSSjJJMdSJOVTJOKhEjlYhHr4o4VckYqYpwnIyTqojOV4VrqpIxKiuy+9H5ykScZDxGIm6qDRXAgkgWZhYHPgNcBbQDT5jZfe6+vbiRiSxs2V/QrXWVJ/R+d2dwNE3/cJr+kDwGRtL0DY8xMGXZGH3DaQZGxugfSTMwPEZv7yhDoxkGR9IMj6Wj/dH0UWNdjlcyHiNZEb0ScYv24zES8RiV4+Wx8fKjtpPOJeJGRTxGRcyoiBnxeIxEzIjHjIq4UREL58I1R5XHbbwsEY9F21iMeNwmPiNcF48ZMctuKbmEtyCSBXApsMvddwOY2b3AtYCShUgRmUW1hepkxQknnOmMpqOkMTSaZnh0Yn/oqP2JsuGxNCNjmeiV9rBNMzrmjKSz5ZmJa8Yy9A2P5bxnYjua3aaLN9GqGcTNiMWMeE4SiccmJxbLKYMrzmrjz96xYc7jWSjJYgWwN+e4HXhd7gVmdjNwM8Dq1avnLzIRKYhEqAkUcyXFTMYZyzjpjDOayZBOh23GGUtnz0VJJZ1xRtPh3Pj5TM51uceZic9NO2PpDGl3MhknnYGMOxmPzk8uP7osZ9+jeJc1VBXk32KhJItZuftdwF0QTVFe5HBEZBGIxYxkeOy4ivLuoI8VO4A87QNW5RyvDGUiIjIPFkqyeAJYb2ZrzSwJXA/cV+SYRETKxoJohnL3MTP7APB9okdn73b3bUUOS0SkbCyIZAHg7vcD9xc7DhGRcrRQmqFERKSIlCxERGRWShYiIjIrJQsREZmVeb6zgC0gZnYIeOUkPqIFODxH4SwUuufyoHsuDyd6z6e6e+tUJxZlsjhZZrbF3TcWO475pHsuD7rn8lCIe1YzlIiIzErJQkREZqVkMbW7ih1AEeiey4PuuTzM+T2rz0JERGalmoWIiMxKyUJERGalZJHDzK42sxfMbJeZfaTY8cwVM7vbzA6a2fM5Zc1mttnMdoZtUyg3M7sz/Bs8a2YXFy/yE2dmq8zsYTPbbmbbzOzWUL5o79vMUmb2uJk9E+75Y6F8rZn9LNzb18M0/5hZZTjeFc6vKeoNnAQzi5vZU2b23XC8qO/ZzPaY2XNm9rSZbQllBf3ZVrIIzCwOfAZ4O7ABuMHM5n4h2+L4InD1pLKPAA+6+3rgwXAM0f2vD6+bgc/NU4xzbQz4sLtvADYBt4T/nov5voeBK9z9AuBC4Goz2wR8CrjD3U8HOoGbwvU3AZ2h/I5w3UJ1K7Aj57gc7vlyd78wZzxFYX+23V2vqJP/MuD7Oce3AbcVO645vL81wPM5xy8Ay8P+cuCFsP954IaprlvIL+A7wFXlct9ANfAk0Vr1h4GKUD7+c060PsxlYb8iXGfFjv0E7nVl+OV4BfBdwMrgnvcALZPKCvqzrZrFhBXA3pzj9lC2WC119/1h/zVgadhfdP8OoanhIuBnLPL7Ds0xTwMHgc3AS0CXu4+FS3Lva/yew/luYMm8Bjw3/g74YyATjpew+O/ZgQfMbKuZ3RzKCvqzvWAWP5LCcXc3s0X5DLWZ1QL/DnzI3XvMbPzcYrxvd08DF5pZI/Bt4KziRlRYZvZO4KC7bzWzNxc5nPn0RnffZ2ZtwGYz+3nuyUL8bKtmMWEfsCrneGUoW6wOmNlygLA9GMoXzb+DmSWIEsVX3f1boXjR3zeAu3cBDxM1wTSaWfYPw9z7Gr/ncL4BODK/kZ60NwDvMrM9wL1ETVGfZnHfM+6+L2wPEv1RcCkF/tlWspjwBLA+PEWRBK4H7ityTIV0H3Bj2L+RqE0/W/474QmKTUB3TtV2wbCoCvEFYIe7355zatHet5m1hhoFZlZF1EezgyhpvDtcNvmes/8W7wYe8tCovVC4+23uvtLd1xD9P/uQu7+PRXzPZlZjZnXZfeCtwPMU+me72B01pfQCrgFeJGrn/bNixzOH9/U1YD8wStReeRNRO+2DwE7gB0BzuNaIngp7CXgO2Fjs+E/wnt9I1K77LPB0eF2zmO8bOB94Ktzz88D/CeXrgMeBXcC/AZWhPBWOd4Xz64p9Dyd5/28GvrvY7znc2zPhtS37u6rQP9ua7kNERGalZigREZmVkoWIiMxKyUJERGalZCEiIrNSshARkVkpWciiZ2bpMDtn9jVnMwqb2RrLmc13huv+3MwGwojbbFnffMYgcjI03YeUg0F3v7DYQRBNWvdh4E+KHUguM6vwiXmURKakmoWUrbAmwF+HdQEeN7PTQ/kaM3sozP3/oJmtDuVLzezbFq0X8YyZvT58VNzM/smiNSQeCKOnp3I38F4za54Ux1E1AzP7IzP787D/iJndYWZbzGyHmf2SmX0rrFnwlzkfU2FmXw3XfNPMqsP7LzGzR8OEc9/PmQ7iETP7O4vWQrj15P81ZbFTspByUDWpGeq9Oee63f084B+IZi8F+HvgHnc/H/gqcGcovxN41KP1Ii4mGj0L0ToBn3H3c4Au4NeniaOPKGEc7y/nEY/WLPhHoikcbgHOBX7XzLIzpp4JfNbdzwZ6gPeHubH+Hni3u18SvvsTOZ+bdPeN7v63xxmPlCE1Q0k5mKkZ6ms52zvC/mXAr4X9LwN/HfavAH4Hxmd37bZoNbKX3f3pcM1WorVDpnMn8LSZ/c1xxJ+do+w5YJuHeX3MbDfRBHFdwF53/0m47ivAB4HvESWVzWG23TjRtC9ZXz+OGKTMKVlIufNp9o/HcM5+GpiuGQp37zKzfyWqHWSNcXQtPzXN52cmfVeGif+HJ8fuRHMCbXP3y6YJp3+6OEUmUzOUlLv35mwfC/v/STSDKcD7gB+F/QeBP4DxRYYaTvA7bwd+n4lf9AeANjNbYmaVwDtP4DNXm1k2Kfwm8GOiFdFas+VmljCzc04wZilzShZSDib3WXwy51yTmT1L1I/wP0LZfwd+L5T/NhN9DLcCl5vZc0TNTSe0Rru7HyZag6AyHI8CHyeaBXUz8PPp3z2tF4jWGd8BNAGfc/cRomm4P2VmzxDNvPv66T9CZHqadVbKVlgwZ2P45S0iM1DNQkREZqWahYiIzEo1CxERmZWShYiIzErJQkREZqVkISIis1KyEBGRWf1/HJk7XyckzeYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.xlabel('Epoch Number')\n",
    "plt.ylabel(\"Loss Magnitude\")\n",
    "plt.plot(history.history['loss'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LtQGDMob5LOD"
   },
   "source": [
    "## 5. Use the model to predict values\n",
    "\n",
    "Now you have a model that has been trained to learn the relationship between `celsius_q` and `fahrenheit_a`. You can use the predict method to have it calculate the Fahrenheit degrees for a previously unknown Celsius degrees.\n",
    "\n",
    "So, for example, if the Celsius value is 100, what do you think the Fahrenheit result will be? Take a guess before you run this code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oxNzL4lS2Gui"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[211.31021]]\n"
     ]
    }
   ],
   "source": [
    "print(model.predict([100.0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jApk6tZ1fBg1"
   },
   "source": [
    "The correct answer is $100 \\times 1.8 + 32 = 212$, so our model is doing really well.\n",
    "\n",
    "### To review\n",
    "\n",
    "\n",
    "*   We created a model with a Dense layer\n",
    "*   We trained it with 3500 examples (7 pairs, over 500 epochs).\n",
    "\n",
    "Our model tuned the variables (weights) in the Dense layer until it was able to return the correct Fahrenheit value for any Celsius value. (Remember, 100 Celsius was not part of our training data.)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zRrOky5gm20Z"
   },
   "source": [
    "## Looking at the layer weights\n",
    "\n",
    "Finally, let's print the internal variables of the Dense layer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kmIkVdkbnZJI"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "These are the layer variables: [array([[1.8242955]], dtype=float32), array([28.880667], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "print(\"These are the layer variables: {}\".format(model.get_weights()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RSplSnMvnWC-"
   },
   "source": [
    "The first variable is close to ~1.8 and the second to ~32. These values (1.8 and 32) are the actual variables in the real conversion formula.\n",
    "\n",
    "This is really close to the values in the conversion formula. We can show how a Dense layer works, but for a single neuron with a single input and a single output, the internal math looks the same as [the equation for a line](https://en.wikipedia.org/wiki/Linear_equation#Slope%E2%80%93intercept_form), $y = mx + b$, which has the same form as the conversion equation, $f = 1.8c + 32$.\n",
    "\n",
    "Since the form is the same, the variables should converge on the standard values of 1.8 and 32, which is exactly what happened.\n",
    "\n",
    "With additional neurons, additional inputs, and additional outputs, the formula becomes much more complex, but the idea is the same.\n",
    "\n",
    "### A little experiment\n",
    "\n",
    "Just for fun, what if we created more Dense layers with different units, which therefore also has more variables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Y2zTA-rDS5Xk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Finished training the model\n",
      "[[211.74742]]\n",
      "Model predicts that 100 degrees Celsius is: [[211.74742]] degrees Fahrenheit\n",
      "These are the l0 variables: [array([[0.5968949 , 0.02139384, 0.01172368, 0.35185102]], dtype=float32), array([ 3.484818 , -3.0073252, -2.7715163,  3.010582 ], dtype=float32)]\n",
      "These are the l1 variables: [array([[ 0.20319672, -0.41102245,  0.7771168 , -0.7402758 ],\n",
      "       [-0.7281887 ,  0.9473572 , -0.15004049,  0.12950596],\n",
      "       [-0.96478873,  0.05949384, -0.2531712 ,  0.72464406],\n",
      "       [-0.10805991, -0.88057184,  1.0134443 , -0.15393376]],\n",
      "      dtype=float32), array([ 1.8714209, -3.126011 ,  3.3828623, -2.6109185], dtype=float32)]\n",
      "These are the l2 variables: [array([[ 0.23399544],\n",
      "       [-0.9353689 ],\n",
      "       [ 1.1917399 ],\n",
      "       [-0.6491724 ]], dtype=float32), array([3.2284596], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "l0 = tf.keras.layers.Dense(units=4, input_shape=[1])\n",
    "l1 = tf.keras.layers.Dense(units=4)\n",
    "l2 = tf.keras.layers.Dense(units=1)\n",
    "model = tf.keras.Sequential([l0, l1, l2])\n",
    "model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1))\n",
    "model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)\n",
    "print(\"Finished training the model\")\n",
    "print(model.predict([100.0]))\n",
    "print(\"Model predicts that 100 degrees Celsius is: {} degrees Fahrenheit\".format(model.predict([100.0])))\n",
    "print(\"These are the l0 variables: {}\".format(l0.get_weights()))\n",
    "print(\"These are the l1 variables: {}\".format(l1.get_weights()))\n",
    "print(\"These are the l2 variables: {}\".format(l2.get_weights()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xrpFFlgYhCty"
   },
   "source": [
    "As you can see, this model is also able to predict the corresponding Fahrenheit value really well. But when you look at the variables (weights) in the `l0` and `l1` layers, they are nothing even close to ~1.8 and ~32. The added complexity hides the \"simple\" form of the conversion equation.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 2 : O-Rings seen with Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook calculates a logistic regression using Keras. It's basically meant to show the principles of Keras.\n",
    "\n",
    "###  Datset\n",
    "\n",
    "We investigate the data set of the challenger flight with broken O-rings (`Y=1`\n",
    ") vs start temperature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Broken O-rings')"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZM0lEQVR4nO3deZhldX3n8ffH7kZbgqDSY7QXRR+WOC4gpag4StxY4gBmnBGUPGqMPUZJ3IaMJA4hLiORuD0uKOKuD7g8hGkV05q4LyjdoiJgG0SUblQQA26tQPOdP+4pvBSnbt3q5tS5Xbxfz3Ofe8/vbN/6ddX59FnuOakqJEma6Q59FyBJmkwGhCSplQEhSWplQEiSWhkQkqRWBoQkqVVnAZHkXUmuSvKdWcY/Pcm3k1yY5CtJHtxVLZKk+etyD+I9wGEjxv8AeExVPRB4BXB6h7VIkuZpaVcLrqovJLnPiPFfGRo8D1jVVS2SpPnrLCDm6dnAJ2cbmWQtsBZg1113PXC//fZbqLokaVHYuHHjz6pqxXzm6T0gkvwxg4B41GzTVNXpNIegpqamasOGDQtUnSQtDkl+ON95eg2IJA8CzgAOr6pr+qxFknRLvV3mmmQNcDbwZ1X1vb7qkCS162wPIsmZwCHAnkk2A38PLAOoqrcBJwF3B96aBODGqprqqh5J0vx0eRXTsXOM/wvgL7pavyRpx/hNaklSKwNCktTKgJAktTIgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSKwNCktTKgJAktTIgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSq6VdLTjJu4AnAVdV1QNaxgd4I3AE8BvgmVX1ja7q0Y4754ItnLp+E1deu5V77bGcEw7dl6MPWNnrevuqaXvr3Rm97JwLOfNrV7CtiiUJxx60mlce/cC+y1p0/TyJOgsI4D3Am4H3zTL+cGDv5nUQcFrzrgl0zgVbOPHsC9l6wzYAtly7lRPPvhCg0z/KUesFeqlplL76qSsvO+dCPnDej24e3lZ183CfIbHY+nlSdXaIqaq+APx8xCRHAe+rgfOAPZLcs6t6tGNOXb/p5j/GaVtv2Map6zf1tt6+ahplEmvaEWd+7Yp5tS+UxdbPk6rLPYi5rASGf8s2N20/njlhkrXAWoA1a9YsSHG6pSuv3Tqv9j7X23VNo/TVT13ZVjWv9oWy2Pp5Uu0UJ6mr6vSqmqqqqRUrVvRdzu3SvfZYPq/2hVhvXzWNMok17YglybzaF8pi6+dJ1WdAbAFWDw2vato0gU44dF+WL1tyi7bly5ZwwqH79rbevmoaZRJr2hHHHrR6Xu0LZbH186Tq8xDTOuD4JGcxODl9XVXd6vCSJsP0ib+FvmpknPVO0pUsffVTV6ZPRE/aVUyLrZ8nVaqjY4lJzgQOAfYEfgr8PbAMoKre1lzm+mbgMAaXuT6rqjbMtdypqanasGHOySRJQ5JsrKqp+czT2R5EVR07x/gCnt/V+iVJO2anOEktSVp4BoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWnUaEEkOS7IpyaVJXtoyfk2Szya5IMm3kxzRZT2SpPF1FhBJlgBvAQ4H7g8cm+T+MyZ7GfDhqjoAOAZ4a1f1SJLmp8s9iIcBl1bVZVV1PXAWcNSMaQq4S/N5d+DKDuuRJM1DlwGxErhiaHhz0zbsZOC4JJuBc4G/altQkrVJNiTZcPXVV3dRqyRphr5PUh8LvKeqVgFHAO9Pcquaqur0qpqqqqkVK1YseJGSdHvUZUBsAVYPDa9q2oY9G/gwQFV9FbgTsGeHNUmSxtRlQJwP7J1kryS7MDgJvW7GND8CHgeQ5I8YBITHkCRpAnQWEFV1I3A8sB64hMHVShcleXmSI5vJXgI8J8m3gDOBZ1ZVdVWTJGl8S7tceFWdy+Dk83DbSUOfLwYO7rIGSdL26fsktSRpQs0ZEElekOQuGXhnkm8keeJCFCdJ6s84exB/XlW/AJ4I3BX4M+CUTquSJPVunIBI834E8P6qumioTZK0SI0TEBuTfIpBQKxPshtwU7dlSZL6Ns5VTM8G9gcuq6rfJLk78KxOq5Ik9W6cgNi/eb9vcvORpeuSLG2+6yBJWoTGCYi3Ag8Bvs3g3MMDgIuA3ZP8ZVV9qsP6JEk9GeccxJXAAc3N8g4EDgAuA54AvKbL4iRJ/RknIPZprlwCbv72835VdVl3ZUmS+jbOIaaLkpzG4IE/AE8FLk5yR+CGziqTJPVqnD2IZwKXAi9sXpc1bTcAf9xNWZKkvs25B1FVW4HXNq+ZfnWbVyRJmghzBkSSgxk8GvTew9NX1X27K0uS1LdxzkG8E3gRsBHY1m05kqRJMU5AXFdVn+y8EknSRBknID6b5FTgbOB3041V9Y3OqpIk9W6cgDioeZ8aaivgsbd9OZKkSTHOVUxeyipJt0OzBkSS46rqA0le3Da+ql7XXVmSpL6N2oPYtXnfbSEKkSRNllkDoqrenmQJ8Iuqev0C1iRJmgAjb7VRVduAYxeoFknSBBnnKqYvJ3kz8CHg19ONXuYqSYvbfJ4o9/KhNi9zlaRFrtPLXJMcBrwRWAKcUVWntEzzPxjc66mAb1XV07Z3fZKk2844exA3S/LxqnrSmNMuAd7C4Mlzm4Hzk6xrHjg0Pc3ewInAwVX1H0n+03zqkSR1Z5znQQxbOY9pHwZcWlWXVdX1DB44dNSMaZ4DvKWq/gOgqq6aZz2SpI7MNyAumMe0K4ErhoY3c+uA2QfYJ8mXk5zXHJK6lSRrk2xIsuHqq6+eX8WSpO0yr4Coqj+/jde/FNgbOITB5bTvSLJHy3pPr6qpqppasWLFbVyCJKnNyIBIcniSLyT5WfP6fJIjxlz2FmD10PCqpm3YZmBdVd1QVT8AvscgMCRJPZs1IJI8B3gFgyuM7tu8/gE4OcnaMZZ9PrB3kr2S7AIcA6ybMc05DPYeSLIng0NOl83rJ5AkdWLUVUwvAh5VVT8favtMksOBLwGnj1pwVd2Y5HhgPYPLXN9VVRcleTmwoarWNeOemORiBk+rO6GqrtmBn0eSdBsZFRCZEQ4AVNU1ScZaeFWdC5w7o+2koc8FvLh5SZImyKhzEL9I8uCZjU3bL7srSZI0CUbtQbwEWJfk3cDGpm0KeAZwXNeFSZL6NeseRFV9icGX3e4APLN53QF4eDNOkrSIjbzVRlX9FDhp1DSSpMVpvt+kliTdThgQkqRWBoQkqdWct/tOsg9wAnDv4emrygcGSdIiNs7zID4CvA14B4NvO0uSbgfGCYgbq+q0ziuRJE2Ucc5BfCzJ85LcM8ndpl+dVyZJ6tU4exDPaN5PGGorBnd3lSQtUnMGRFXttRCFSJImy5yHmJLcOcnLkpzeDO+d5EndlyZJ6tM45yDeDVwPPLIZ3gK8srOKJEkTYZyAuF9VvQa4AaCqfgOM90AISdJOa5yAuD7JcgYnpklyP+B3nVYlSerdOFcx/T3wL8DqJB8EDmZw629J0iI2TkBsBP4UeDiDQ0svAHbrsihJUv/G+qIccENVfaKqPg6saNokSYvYOAHxfxl8m3rXJAcCH8VHjkrSojfOF+U+kWQZ8GkGh5aeXFXf67wySVKvZg2IJG+iuXKpsTvwfeD4JFTVX3ddnCSpP6P2IDbMGN7YZSGSpMkya0BU1XunPyfZBdinGdxUVTeMs/AkhwFvBJYAZ1TVKbNM998YnNt4aFXNDCZJUg/GeaLcIcB7gcsZXOa6OskzquoLc8y3BHgL8ARgM3B+knVVdfGM6XZjcOns17ajfklSR8a5ium1wBOr6jFV9WjgUOD1Y8z3MODSqrqsqq4HzgKOapnuFcA/Ar8ds2ZJ0gIYJyCWVdWm6YHmCqZlY8y3ErhiaHhz03azJA8BVlfVJ0YtKMnaJBuSbLj66qvHWLUkaUeNExAbk5yR5JDm9Q5ufQJ73pLcAXgd8JK5pq2q06tqqqqmVqxYsaOrliSNYZxbbTwXeD4wfVnrF4G3jjHfFmD10PCqpm3absADgM8lAfhDYF2SIz1RLUn9GxkQzYnmb1XVfgz+tz8f5wN7J9mLQTAcAzxtemRVXQfsObSuzwH/y3CQpMkw8hBTVW0DNiVZM98FV9WNwPHAeuAS4MNVdVGSlyc5cruqlSQtmHEOMd0VuCjJ14FfTzdW1Zwb+ao6Fzh3RttJs0x7yBi1SJIWyDgB8X86r0KSNHHGuVnf56c/J9kTuKaqasQskqRFYNZzEEkenuRzSc5OckCS7wDfAX7a3EJDkrSIjdqDeDPwtwzu4voZ4PCqOi/JfsCZDB5DKklapEZdxbS0qj5VVR8BflJV5wFU1XcXpjRJUp9GBcRNQ5+3zhjnOQhJWuRGHWJ6cJJfMLiD6/LmM83wnTqvTJLUq1HPg1iykIVIkibLODfrkyTdDhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlq1WlAJDksyaYklyZ5acv4Fye5OMm3k/xbknt3WY8kaXydBUSSJcBbgMOB+wPHJrn/jMkuAKaq6kHAR4HXdFWPJGl+utyDeBhwaVVdVlXXA2cBRw1PUFWfrarfNIPnAas6rEeSNA9dBsRK4Iqh4c1N22yeDXyyw3okSfOwtO8CAJIcB0wBj5ll/FpgLcCaNWsWsDJJuv3qcg9iC7B6aHhV03YLSR4P/B1wZFX9rm1BVXV6VU1V1dSKFSs6KVaSdEtdBsT5wN5J9kqyC3AMsG54giQHAG9nEA5XdViLJGmeOguIqroROB5YD1wCfLiqLkry8iRHNpOdCvwB8JEk30yybpbFSZIWWKfnIKrqXODcGW0nDX1+fJfrlyRtP79JLUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqtbTLhSc5DHgjsAQ4o6pOmTH+jsD7gAOBa4CnVtXlXdYk7czOuWALp67fxJXXbuVeeyznhEP35egDVgLw9Hd8lS9//+c3T3vw/e7GB5/ziLHmHTXuZedcyJlfu4JtVSxJOPag1bzy6AfucL3jjN/eZXdVc1f1TqpUVTcLTpYA3wOeAGwGzgeOraqLh6Z5HvCgqnpukmOAJ1fVU0ctd2pqqjZs2NBJzdIkO+eCLZx49oVsvWHbzW3Lly3h1X/6QD6y4Ue3CIdp0yExal5g1nEbfvhzPnDej2613OMevmbODe6odR59wMo5x2/vsruqGWbvpx2pd6FCIsnGqpqa1zwdBsQjgJOr6tBm+ESAqnr10DTrm2m+mmQp8BNgRY0oyoDQ7dXBp3yGLdduvVX7yj2Wt7ZPu/yUPxk5LzDruJ9c91u2tfw5Lkn4/quP2O56v/zSx845fnuX3VXNMHs/7Ui9c817W9megOjyENNK4Iqh4c3AQbNNU1U3JrkOuDvws+GJkqwF1gKsWbOmq3qliXblLCEwW/uOznvltVuZ7X9qbRvg+a6zq5+ny5rnO25HljsJdoqT1FV1elVNVdXUihUr+i5H6sW9mv/Fjts+7ryjxi1JWsfN1j7uOscZv73L7qrmruqdZF0GxBZg9dDwqqatdZrmENPuDE5WS5rhhEP3ZfmyJbdoW75sCSccui8H3+9urfNMt4+ad9S4Yw9aTZvZ2setd5zx27vsrmruqt5J1uUhpvOBvZPsxSAIjgGeNmOadcAzgK8CTwE+M+r8g3R7Nn0ys+1KmKMPWDnyKqZR806bbbnAdl0RNNc6x6lpe/uiq5q7qHeSdXaSGiDJEcAbGFzm+q6qelWSlwMbqmpdkjsB7wcOAH4OHFNVl41apiepJWn+Ju0kNVV1LnDujLaThj7/FvjvXdYgSdo+O8VJaknSwjMgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSq05v1teFJL8ENvVdxwx7MuMhRxNiEuuypvFY0/gmsa5JrGnfqtptPjN0erO+jmya7x0Ju5Zkw6TVBJNZlzWNx5rGN4l1TWpN853HQ0ySpFYGhCSp1c4YEKf3XUCLSawJJrMuaxqPNY1vEutaFDXtdCepJUkLY2fcg5AkLQADQpLUauIDIsmSJBck+XgzvFeSryW5NMmHkuwyIXW9J8kPknyzee2/wPVcnuTCZt0bmra7Jfl0kn9v3u86ATWdnGTLUD8dscA17ZHko0m+m+SSJI/ou59G1NVbXyXZd2i930zyiyQv7LOvRtTU9+/Ui5JclOQ7Sc5Mcqe+t1Oz1DTvbdTEn4NI8mJgCrhLVT0pyYeBs6vqrCRvA75VVadNQF3vAT5eVR9d6Fqaei4HpqrqZ0NtrwF+XlWnJHkpcNeq+t8913Qy8Kuq+qeFqmNGTe8FvlhVZzR/tHcG/pYe+2lEXS+kx74aqm0JsAU4CHg+PfdVS03Poqd+SrIS+BJw/6ra2myfzgWOoKft1IiaDmGe26iJ3oNIsgr4E+CMZjjAY4HpH/C9wNF91zXBjmLQR9BTX02SJLsDjwbeCVBV11fVtfTcTyPqmhSPA75fVT9kcn6nhmvq21JgeZKlDIL9x/S/nZpZ05Xbs5CJDgjgDcDfADc1w3cHrq2qG5vhzcDKCahr2quSfDvJ65PccYFrKuBTSTYmWdu03aOqftx8/glwjwmoCeD4pp/etcCHc/YCrgbencHhwTOS7Er//TRbXdBfXw07Bjiz+dx3X00brgl66qeq2gL8E/AjBsFwHbCRHrdTbTVV1aea0fPaRk1sQCR5EnBVVW3su5ZhI+o6EdgPeChwN2Chd7sfVVUPAQ4Hnp/k0cMja3AscaGPJ7bVdBpwP2B/Br+8r13AepYCDwFOq6oDgF8DLx2eoKd+mq2uPvsKgOZw15HAR2aO66mv2mrqrZ+aMDqKQcjfC9gVOGyh1j9uTUmOYzu2URMbEMDBwJHNceyzGOyyvRHYo9ltAljF4Dhkr3Ul+UBV/bgGfge8G3jYQhbV/K+BqroK+Odm/T9Nck+A5v2qvmuqqp9W1baqugl4BwvbT5uBzVX1tWb4oww2zL3202x19dxX0w4HvlFVP22G++6rW9XUcz89HvhBVV1dVTcAZzPYRvS5nWqr6ZHbs42a2ICoqhOralVV3YfB7uRnqurpwGeBpzSTPQP4fxNQ13FDfzRhcLzxOwtVU5Jdk+w2/Rl4YrP+dQz6CBa4r2arabqfGk9mAfupqn4CXJFk36bpccDF9NhPo+rqs6+GHMstD+X02leNW9TUcz/9CHh4kjs3f/vTv1N9bqfaarpku7ZRVTXxL35/9h3gvsDXgUsZ7GLecULq+gxwYdPpHwD+YAHruC/wreZ1EfB3TfvdgX8D/h34V+BuE1DT+5t++jaDjc09F/jfbH9gQ7P+c4C79tlPc9TVd1/tClwD7D7U1mtfzVJT3/30D8B3m7/99wN37Hs7NUtN895GTfxlrpKkfkzsISZJUr8MCElSKwNCktTKgJAktTIgJEmtls49iTQ5kkxfZgnwh8A2BrepgMEX8a7vpbAWSQ4Brq+qr3Sw7PsAlwCbqmr/pm0bg8sYpx0NrAbeDtxUVQ+4revQ4mZAaKdSVdcw+M5A73eGbWpYWr+/585MhwC/AsYOiDmWN9P3p8OhsXXGMMDlGdz++uPj1iBN8xCTdnpJDkzy+eamgOuHvjH6ueamZBsyeMbCQ5OcncGzDF7ZTHOfDJ7B8MFmmo8mufMYy31DBs+4eEGS/5rBvf8vSPKvSe7R/A//ucCLMrj3/n/J4H78Txmq+1fN+yFJvphkHYNvUC9JcmqS85sbq/3PBe1QqWFAaGcX4E3AU6rqQOBdwKuGxl9fVVPA2xjc7uD5wAOAZzaHqwD2Bd5aVX8E/AJ4XpJlcyx3l6qaqqrXMrj3/sNrcKO9s4C/qarLm3W+vqr2r6ovzvFzPAR4QVXtAzybwR04H8rgxmrPSbLXGH2xPL9/GMw/jzG9NJKHmLSzuyODDf6nB7eYYQmDO3pOW9e8XwhcVM2tqpNcxuD4/LXAFVX15Wa6DwB/DfzLHMv90NDnVcCHmj2MXYAfbMfP8fWqmp7vicCDhvY2dgf2HmO5bYeYpO1mQGhnFwYb/kfMMv53zftNQ5+nh6d//2feb6bGWO6vhz6/CXhdVa1rTkyfPMs8N9LstSe5A4MwaVtegL+qqvWzLEdaEB5i0s7ud8CKJI8ASLIsyX+e5zLWTM8PPI3BIaNN81ju7vz+ds7PGGr/JbDb0PDlwIHN5yOBZbMsbz3wl81hLpLsk98/QEhaMAaEdnY3Mbit8j8m+RbwTeCR81zGJgYPNLqEwV1UT2sulx13uScDH0myEfjZUPvHgCdPn6Rm8KyCxzTLewS33GsYdgaDW0Z/I8l3GFym6t6+Fpx3c9XtWnO10cd3tu8IzKfunfVnVP/cg5B2TtuA3ZN8c9REzZ7Lx7jlno00FvcgJEmt3IOQJLUyICRJrQwISVIrA0KS1MqAkCS1+v/YKMuQWSKjDwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as imgplot\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tempfile\n",
    "data = np.asarray(pd.read_csv('./challenger.txt', sep=','), dtype='float32')\n",
    "plt.plot(data[:,0], data[:,1], 'o')\n",
    "plt.axis([40, 85, -0.1, 1.2])\n",
    "plt.xlabel('Temperature [F]')\n",
    "plt.ylabel('Broken O-rings')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 1. 0. 0. 0. 0. 0. 0. 1. 1. 1. 0. 0. 1. 0. 0. 0. 0. 0. 0. 1. 0. 1.]\n"
     ]
    }
   ],
   "source": [
    "y_values = data[:,1]\n",
    "print(y_values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mathematical Notes\n",
    "\n",
    "We are considering the likelihood $P(y_i=1|x_i)$ for the class $y_i=1$ given the $i-$th data point $x_i$ ($x_i$ could be a vector). This is given by:\n",
    "\n",
    "$\n",
    "P(y_i=1 | x_i) = \\frac{e^{(b +  x_i w)}}{1 + e^{(b + x_i w)}} = [1 + e^{-(b + x_i w)}]^{-1}\n",
    "$\n",
    "\n",
    "If we have more than one data point, which we usually do, we have to apply the equation above to each of the N data points. In this case we can use a vectorized version with $x=(x_1,x_2,\\ldots,x_N)$ and $y=(y_1,y_2,\\ldots,y_N$)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numpy code\n",
    "This numpy code, shows the calculation for one value using `NumPy` (like a single forward pass)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.882916\n",
      "[0.999 0.998 0.998 0.998 0.999 0.996 0.996 0.998 1.    0.999 0.998 0.988\n",
      " 0.999 1.    0.999 0.993 0.998 0.978 0.992 0.985 0.993 0.992 1.   ]\n"
     ]
    }
   ],
   "source": [
    "# Data\n",
    "N = len(data)\n",
    "x = data[:,0]\n",
    "y = data[:,1]\n",
    "# Initial Value for the weights\n",
    "w = -0.20\n",
    "b = 20.0\n",
    "# predicted probabilities \n",
    "p_1 = 1 / (1 + np.exp(-x*w - b))\n",
    "# cross-entropy loss function\n",
    "cross_entropy = -np.mean(y * np.log(p_1) + (1-y) * np.log(1-p_1))\n",
    "print(cross_entropy)\n",
    "print(np.round(p_1,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Better values from intuition\n",
    "\n",
    "Now lets try to find better values for $W$ and $b$. Lets assume $W$ is given with $-1$. We want the probability\n",
    "for a dammage $P(y_i=1 | x_i)$ to be $0.5$.\n",
    "Determine an appropriate value for $b$.\n",
    "Hint: at which $x$ value should $P(y_i=1 | x_i)$ be $0.5$, look at the data. At this $x$ value the term $1 + e^{-(b + W’ x_i)}$ must be $2$.\n",
    "\n",
    "**Solution**\n",
    "\n",
    "$P(y=1 | x) = 0.5$ at $x \\approx 65$ \n",
    "\n",
    "$-(b + (-1) x_i) = 0 \\rightarrow b = 65$  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9094435\n",
      "[0.269 0.007 0.018 0.047 0.119 0.001 0.    0.007 1.    0.881 0.007 0.\n",
      " 0.119 1.    0.119 0.    0.007 0.    0.    0.    0.    0.    0.999]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAb/klEQVR4nO3de3hU9b3v8fc3kwQiIheJwYRE0CIIoQWbighbrbaVi/XS7nMqre1uty2nrezTy35s5dTH9tj21F09Z+vTWi3V1lZ3dat1W06lpT3VPm1R2URRERAFkUuQiyAIEkIu3/PHmtAhZC5JZmatrHxez8MzM2utmfXxJ3yy8ltrZszdERGR+CoJO4CIiBSWil5EJOZU9CIiMaeiFxGJORW9iEjMqehFRGIua9Gb2U/NbJeZvZRm/SfM7EUzW21mT5nZe/IfU0REeiuXI/p7gdkZ1m8CLnD3KcC3gcV5yCUiInlSmm0Dd/+zmY3NsP6plIfPAGPykEtERPIka9H30DXAb9OtNLMFwAKAIUOGvHfixIl53r2ISLw9++yzb7p7ZU+ek7eiN7P3ExT9rHTbuPtiklM7DQ0N3tjYmK/di4gMCGa2uafPyUvRm9m7gbuBOe6+Jx+vKSIi+dHnyyvNrA54FPiku7/S90giIpJPWY/ozewB4EJglJltA74JlAG4+13AjcDJwI/MDKDN3RsKFVhERHoml6tu5mdZ/1ngs3lLJCIieaV3xoqIxJyKXkQk5lT0IiIxp6IXEYk5Fb2ISMyp6EVEYk5FLyIScyp6EZGYU9GLiMScil5EJOZU9CIiMaeiFxGJORW9iEjMqehFRGJORS8iEnMqehGRmFPRi4jEnIpeRCTmVPQiIjGnohcRiTkVvYhIzKnoRURiTkUvIhJzKnoRkZhT0YuIxFxptg3M7KfApcAud6/vZr0BtwNzgUPAp939uXwHlfx5bFUTtyxbz/Z9zVQPr+C6SyZwxbSaUPcbVqbe5u2PbnhsNQ+s2Eq7Owkz5k+v5TtXTAk7VuzGOYqyFj1wL/BD4Bdp1s8Bxif/TAfuTN5KBD22qolFj66mubUdgKZ9zSx6dDVAQf9xZdovEEqmTMIap0K54bHV3P/MlqOP292PPg6z7OM2zlFl7p59I7OxwG/SHNH/GPiTuz+QfLweuNDd38j0mg1jyr3x2tG9Ci29d6Clle7+l5vB0EFlyQe5vFLKRmZ/W3bc/eB298FWWh06KMHd6MBwjJJEgjYSNLeX0EqCNkppI0GrJ6Csgr+bPBbKT4TyIcHtoBOD26GjYVgtDK+FwcN6Oxxpzbz5CZr2NR+3vGZ4Bcuvvyjv+yu0MxYtpb2b//EJMzZ+b24IiQJxG+diMLNn3b2hJ8/J5Yg+mxpga8rjbcllxxW9mS0AFgBMqjkJpn48D7uXnnh4+aa06/5x+jgg+w/+Y39SeMqyLvc7t/MO/vifWzCcEjooMT96P9HWQSntlNJOGe2U0kaZtTPEDjO47QBsexOOHIQj70Droe7zDBoGw8YEpT9iLIw7H8ZdEPxQ6KXt3ZRPpuVR113JZ1peLHEb56jKR9HnzN0XA4sBGhoanDk3F3P3AtzzQvojqH+cU7gjqB+sTb9fIP1R3ZdSMnW0B4XfcgAOvAH7tsD+rbBvK+zfFtzf9BdYcRckymHs38GZs+HMDwU/AHqgenhFt5mqk3n7m4RZ2iP6MMVtnKMqH1fdNAG1KY/HJJdJBF13yQQqyhLHLKsoS3DdJRNC22/OmUoSMPgkGFYDYxqg/iMw80sw71b4+IPwheXw9dfhU0vgnAWwbzP89jq4/T1wx3R44jvQcrDPefuj+dNre7S8WOI2zlGVjyP6JcBCM3uQ4CTs/mzz8xKezhNcxb7KIZf95iVTaTmcfkHw55Lvwp6N8MoyeHUZ/PlWeOlX8NG7oea9fc7bn3SecI3aVTdxG+eoynoy1sweAC4ERgE7gW8CZQDuflfy8sofArMJLq/8jLs3ZttxQ0ODNzZm3Uwkf15fDo8ugIM74MJFMOsrwW8JIv1Ib07G5nTVTSGo6CUUzfvg8a8GR/Z158FHfgzD68JOJZKz3hS93hkrA0vFcPjoPXDlYtixGu6cBasfCTuVSEGp6GXgMYP3fAy+8Fc4ZSL86hpY8t/p9g0GIjGgopeBa8RY+PRSmLEQnvs5rHk07EQiBaGil4EtUQofvAlOnQq/WxTM4YvEjIpepCQBH74N3tkNT3w77DQieaeiFwGongbn/DdYeQ9s09VgEi8qepFOF30Dhp4K//fL0N4WdhqRvFHRi3QaNBTm/AvsXA0r7gw7jUjeqOhFUp314eCD0J78X8GHponEgIpeJJUZzL0luL/0a7q2XmJBRS/S1fC64LNwXvktvPybsNOI9JmKXqQ7534BquqDo/qWA2GnEekTFb1IdxJlcOltwRec/ElfkCP9m4peJJ3a9wVfbrLqfl1uKf2ail4kk8lXwuF9sOWpsJOI9JqKXiSTMy6C0sHw8uNhJxHpNRW9SCblQ4Kyf/lxXWop/ZaKXiSbifNg/1Z444Wwk4j0iopeJJszZ4OVaPpG+i0VvUg2Q0ZB3QwVvfRbKnqRXEycB7vWwN7Xwk4i0mMqepFcTJwX3L68NNwcIr2gohfJxYixUDVF0zfSL6noRXI1cR5sfQYO7g47iUiP5FT0ZjbbzNab2QYzu76b9XVm9qSZrTKzF81sbv6jioRs4jzwDnjld2EnEemRrEVvZgngDmAOMAmYb2aTumx2A/CQu08DrgJ+lO+gIqEbPQWG1emji6XfyeWI/hxgg7u/5u5HgAeBy7ts48BJyfvDgO35iygSEWbBUf3GJ6HlYNhpRHKWS9HXAFtTHm9LLkv1LeBqM9sGLAX+qbsXMrMFZtZoZo27d2ueU/qhifOgvQU2/jHsJCI5y9fJ2PnAve4+BpgL3Gdmx722uy929wZ3b6isrMzTrkWKqG4GVIzQ1TfSr+RS9E1AbcrjMcllqa4BHgJw96eBwcCofAQUiZREKZw5Jzgh294adhqRnORS9CuB8WY2zszKCU62LumyzRbgYgAzO4ug6DU3I/E0cR4c3g+bl4edRCQnWYve3duAhcAyYB3B1TVrzOwmM7ssudk/A58zsxeAB4BPu+szXSWmzrgISis0fSP9RmkuG7n7UoKTrKnLbky5vxaYmd9oIhFVfgK86+Kg6Od8P7gaRyTC9M5Ykd6YOA/eboLtq8JOIpKVil6kNzo/o369PuRMok9FL9IbJ4yE0e+GrSvCTiKSlYpepLeqp8H2F/RdshJ5KnqR3qqeCi379WUkEnkqepHeOnVqcPvG82GmEMlKRS/SW6dMgkQ5bH8+7CQiGanoRXqrtByqJusSS4k8Fb1IX5w6Fd54USdkJdJU9CJ9UT1NJ2Ql8lT0In1RPTW41fSNRJiKXqQvKs+CxCBdeSORpqIX6YujJ2SfDzuJSFoqepG+qp4Kb7wAHR1hJxHplopepK+qp0HL2/DWprCTiHRLRS/SV53vkNUJWYkoFb1IX52SPCGropeIUtGL9FWiDEbXB/P0IhGkohfJh1OnBlfe6ISsRJCKXiQfqqfCkQN6h6xEkopeJB+qpwW3euOURJCKXiQfKifqhKxElopeJB8SZTB6it4hK5GkohfJF71DViJKRS+SL6dOTZ6Q3Rh2EpFj5FT0ZjbbzNab2QYzuz7NNv/VzNaa2Roz+2V+Y4r0A50nZDV9IxGTtejNLAHcAcwBJgHzzWxSl23GA4uAme4+Gfhy/qOKRFzlRCgdrCtvJHJyOaI/B9jg7q+5+xHgQeDyLtt8DrjD3d8CcPdd+Y0p0g8kSqGqXlfeSOTkUvQ1wNaUx9uSy1KdCZxpZsvN7Bkzm93dC5nZAjNrNLPG3bt39y6xSJRVTwu+Q1YnZCVC8nUythQYD1wIzAd+YmbDu27k7ovdvcHdGyorK/O0a5EIOfoOWZ2QlejIpeibgNqUx2OSy1JtA5a4e6u7bwJeISh+kYFFH1ksEZRL0a8ExpvZODMrB64ClnTZ5jGCo3nMbBTBVI4+9EMGns4TsrryRiIka9G7exuwEFgGrAMecvc1ZnaTmV2W3GwZsMfM1gJPAte5+55ChRaJrERp8h2yOqKX6CjNZSN3Xwos7bLsxpT7Dnw1+UdkYKueBs//MjghW6L3JEr49LdQJN9OnQpHDsKeDWEnEQFU9CL5Vz01uNX0jUSEil4k30ZNCE7I7ngx7CQigIpeJP8SpcHVNztfCjuJCKCiFymM0fWw4yVwDzuJiIpepCCqpsChN+HgzrCTiKjoRQpidH1wu0PTNxI+Fb1IIVRNDm53rg43hwgqepHCqBgBw2p1RC+RoKIXKZSqel15I5GgohcplNH18Oar0NocdhIZ4FT0IoVSVQ/eDrvWhZ1EBjgVvUihjJ4S3Gr6RkKmohcplBHjoGyITshK6FT0IoVSUhJcZqkjegmZil6kkPRRCBIBKnqRQqqqh5b9sH9r2ElkAFPRixRS5wlZzdNLiFT0IoV0yiTANE8voVLRixTSoBNh5DjYoc+8kfCo6EUKTR+FICFT0YsU2ugpsHcTtBwMO4kMUCp6kUKrqgccdq0NO4kMUCp6kUI7+iUkmqeXcORU9GY228zWm9kGM7s+w3YfNTM3s4b8RRTp54bVwuBhmqeX0GQtejNLAHcAc4BJwHwzm9TNdkOBLwEr8h1SpF8zC6ZvdC29hCSXI/pzgA3u/pq7HwEeBC7vZrtvA/8CHM5jPpF4qKqHnWugoyPsJDIA5VL0NUDq+7e3JZcdZWZnA7Xu/nimFzKzBWbWaGaNu3fv7nFYkX5rdD20vgNvbQo7iQxAfT4Za2YlwP8B/jnbtu6+2N0b3L2hsrKyr7sW6T+qkidkNU8vIcil6JuA2pTHY5LLOg0F6oE/mdnrwLnAEp2QFUlxyllgCV15I6HIpehXAuPNbJyZlQNXAUs6V7r7fncf5e5j3X0s8Axwmbs3FiSxSH9UVgGjxuuErIQia9G7exuwEFgGrAMecvc1ZnaTmV1W6IAisaGPQpCQlOaykbsvBZZ2WXZjmm0v7HsskRgaXQ8vPQLNb0HFiLDTyACid8aKFEtV55eFrwk3hww4KnqRYjn6UQiavpHiUtGLFMuJVXDCKNipK2+kuFT0IsVi9rcvCxcpIhW9SDFV1cOuddDeFnYSGUBU9CLFNHoKtLfAng1hJ5EBREUvUkzV04LbbSvDzSEDiopepJhGnRmckN28POwkMoCo6EWKyQzGzoTX/xp2EhlAVPQixXbaLNi/Fd7aHHYSGSBU9CLFNnZmcKvpGykSFb1IsVWeBRUjNX0jRaOiFym2khI47TwVvRSNil4kDGNnwb7NsG9r9m1F+khFLxKGsbOCW83TSxGo6EXCcMpkGDxc0zdSFCp6kTB0ztPriF6KQEUvEpbTZsLe1+Dt7WEnkZhT0YuEpXOe/nUd1UthqehFwjJ6CgwaBps1Ty+FpaIXCUtJAurO1RG9FJyKXiRMY2fBnlfhwI6wk0iMqehFwqTPvZEiUNGLhGn0e6B8qKZvpKByKnozm21m681sg5ld3836r5rZWjN70cz+aGan5T+qSAwlSqFuut44JQWVtejNLAHcAcwBJgHzzWxSl81WAQ3u/m7gEeD7+Q4qEltjZ8Gb6+Hg7rCTSEzlckR/DrDB3V9z9yPAg8DlqRu4+5Pufij58BlgTH5jisTYafrcGymsXIq+Bkj9iL1tyWXpXAP8ti+hRAaU6qlQNkTTN1Iwpfl8MTO7GmgALkizfgGwAKCuri6fuxbpvxJlwTy9juilQHI5om8CalMej0kuO4aZfQD4BnCZu7d090LuvtjdG9y9obKysjd5ReLptJmway28syfsJBJDuRT9SmC8mY0zs3LgKmBJ6gZmNg34MUHJ78p/TJGY0+fTSwFlLXp3bwMWAsuAdcBD7r7GzG4ys8uSm90CnAg8bGbPm9mSNC8nIt2pPhtKK1T0UhA5zdG7+1JgaZdlN6bc/0Cec4kMLKXlUHuO3jglBaF3xopExdhZsPMlfe6N5J2KXiQq6j8a3K68O9wcEjsqepGoOPkMmDgPVt4DRw5l314kRyp6kSiZsRCa98ILvww7icSIil4kSurODa7AefpH0NERdhqJCRW9SJSYwXkLYe9GeOV3YaeRmFDRi0TNWZfDsDp4+odhJ5GYUNGLRE2iFM79fPDmqabnwk4jMaCiF4miaZ+EQSfB03eEnURiQEUvEkWDT4KzPwVr/gP2bc2+vUgGKnqRqJr++eB2xV3h5pB+T0UvElXDa2HylfDcL+Dw22GnkX5MRS8SZTOuhZa3g7IX6SUVvUiU1ZwdfCnJirugvS3sNNJPqehFom7GQti/Fdb9Ouwk0k+p6EWi7szZcPK74C//Cq3NYaeRfkhFLxJ1JSVw8TeDz6r/909CW7dfySySlopepD+YdBl8+DbY8Ad4+DPQ3hp2IulHVPQi/cV7Pw1zboH1j8OvPquTs5KznL4zVkQiYvoCaG+B398ApYPgijuhJBF2Kok4Fb1If3PeP0HbYXjiO0HZX3p7MI8vkoaKXqQ/Ov+64KTsn2+BRDnMvTX4LHuRbqjoRfqr938jOLJ/6gdweD9c8HUYNT7sVBJBKnqR/soMPvjt4Ij+qR/A6odh/Ifg3C/A6e/XEb4cpYk9kf7MDC6+Eb6yBi5cBNtXwX1Xwo9mwLM/1xusBABz9+wbmc0GbgcSwN3ufnOX9YOAXwDvBfYAH3P31zO9ZkNDgzc2NvYytkj/9tiqJm5Ztp7t+5qpHl7BdZdM4IppNQB84idPs3zj3qPbzjxjJP/2uRk5PXdJ4yZeXHYPH2lZwqSSzbSUD2fQWXOgqp6fbTyBO9dWsMtPImHG/Om1fOeKKX3Om8v63r72DY+t5oEVW2l3z2vmQuUtBjN71t0bevScbEVvZgngFeCDwDZgJTDf3dembPNF4N3u/nkzuwq40t0/lul1VfQyUD22qolFj66mubX96LKKsgTf+8gUHm7cckzJd+os+0zPBVLWOeeWrOMzZX/g/MGvUdGy++j2u30Y6zrqWO+11NSdztz3TYITRkLFCKjovB0OibKsea+YVpN1fW/HonHzXu5/Zstxz7n63LqsZZ/7OOUvb7HKvlBFPwP4lrtfkny8CMDdv5eyzbLkNk+bWSmwA6j0DC+uopeBaubNT9C07/gplZrhFd0u7/T6zfMyPhdIu+7I/l28y7Zwlm1hom1hQslWJthWBluGd9haAkoHs7/VaPYyjngpLZTTSintGIlEKZNqRvBC0wEOt0O7l9BOCU5wbqCsrJQZp49Kniuwv93C0fMHf371TQ63th99TqfBZQlaWttxOG6dAZdMrjrmdbr60/rdx5Rxp4qy4D0H6dZdOKEy/Xhked1sz80Xu+rfelz0uZyMrQFSv8tsGzA93Tbu3mZm+4GTgTePCWi2AFgAUFdX15OcIrGxPU2Zp1ve1+du39eMM5TdPpmnmZyyxhlKM6u/1gDNb0HzXmjeB4f2wuF9wRU9bS38+i/rKaeVcmtjEEcop50SOki0dTCp/EQOth0iYR2UWjuDaMWS1WxtDocAD+qao8d9fvRmVNvbmHVzPNhG8mdCmmPFvYfSrwNGtx2A7n4GdL6ZON26PZm/4CXj62Z5bpiKetWNuy8GFkNwRF/MfYtERXWaI/d0y3N9LnR/RF89vIId+w/Tftwv2MYhGwIjxwHj0u7zx6vS/xax/FMX8bUMv2UsX3BRxv+ez2V4bveZIWHGxi/Ozfi61/TyN5/lX8ycN9PrZntu3lzb86upcrnqpgmoTXk8Jrms222SUzfDCE7KikgX110y4egUQqeKsgTXXTKBmWeM7PY5ncszPTfTuvnTa+lOuuW55s1lfW9fu1CZC5U3ynI5ol8JjDezcQSFfhXw8S7bLAH+AXga+HvgiUzz8yIDWedJu+6u3LhiWk3Gq24yPbdTutcFenUFS7Z95pKpt2NRqMyFyBtluV5eORe4jeDyyp+6+3fN7Cag0d2XmNlg4D5gGrAXuMrdX8v0mjoZKyLSc7256ianOXp3Xwos7bLsxpT7h4H/0pMdi4hIceidsSIiMaeiFxGJORW9iEjMqehFRGJORS8iEnMqehGRmFPRi4jEnIpeRCTmVPQiIjGnohcRiTkVvYhIzOX0oWYF2bHZAWB9KDtPbxRdviwlIqKYS5lyo0y5i2KuKGaa4O5De/KEon7xSBfre/oJbIVmZo1RywTRzKVMuVGm3EUxV1Qz9fQ5mroREYk5Fb2ISMyFWfSLQ9x3OlHMBNHMpUy5UabcRTFXLDKFdjJWRESKQ1M3IiIxp6IXEYm5ohW9mSXMbJWZ/Sb5eJyZrTCzDWb272ZWXqwsWXLda2abzOz55J+pRc7zupmtTu67MblspJn9wcxeTd6OiECmb5lZU8o4zS1ypuFm9oiZvWxm68xsRtjjlCFXaGNlZhNS9vu8mb1tZl8Oc6wyZAr779RXzGyNmb1kZg+Y2eCweypNph53VNHm6M3sq0ADcJK7X2pmDwGPuvuDZnYX8IK731mUMJlz3Qv8xt0fKXaWZJ7XgQZ3fzNl2feBve5+s5ldD4xw96+HnOlbwEF3v7VYObpk+jnwF3e/O/mP7wTgfxDiOGXI9WVCHKuUbAmgCZgOXEvIY9VNps8Q0jiZWQ3wV2CSuzcn+2kpMJeQeipDpgvpYUcV5YjezMYA84C7k48NuAjoDPpz4IpiZMmUK8IuJxgjCGmsosTMhgHnA/cAuPsRd99HyOOUIVdUXAxsdPfNROfvVGqmsJUCFWZWSvAD+g3C76mumbb35kWKNXVzG/A1oCP5+GRgn7u3JR9vA2qKlCVTrk7fNbMXzexfzWxQkTM58Hsze9bMFiSXVbn7G8n7O4CqCGQCWJgcp58WeZpkHLAb+JkF0253m9kQwh+ndLkgvLFKdRXwQPJ+2GPVKTUThDRO7t4E3ApsISj4/cCzhNhT3WVy998nV/eoowpe9GZ2KbDL3Z8t9L56IkOuRcBE4H3ASKDYv87OcvezgTnAtWZ2fupKD+bain1NbHeZ7gTOAKYS/CX830XMUwqcDdzp7tOAd4DrUzcIaZzS5QpzrABITiNdBjzcdV1IY9VdptDGKflD5XKCH9bVwBBgdrH2n2smM7uaXnRUMY7oZwKXJed5HyT4Veh2YHjy1xGAMQTzdMV0XC4zu9/d3/BAC/Az4Jxihkr+FMfddwH/kdz/TjM7FSB5uyvsTO6+093b3b0D+AnFHadtwDZ3X5F8/AhBwYY6TulyhTxWneYAz7n7zuTjsMfquEwhj9MHgE3uvtvdW4FHCToizJ7qLtN5vemoghe9uy9y9zHuPpbg17Qn3P0TwJPA3yc3+wfg14XOkkOuq1P+8hvBfNxLxcpkZkPMbGjnfeBDyf0vIRgjKPJYpcvUOU5JV1LEcXL3HcBWM5uQXHQxsJYQxylTrjDHKsV8jp0iCXWsko7JFPI4bQHONbMTkv/2O/9OhdlT3WVa16uOcvei/eFvZ4sBTgf+E9hA8KvboGJmyZDrCWB1cvDuB04sYo7TgReSf9YA30guPxn4I/Aq8P+AkRHIdF9ynF4kKI1Ti/z/bCrQmNz/Y8CIMMcpS66wx2oIsAcYlrIs1LFKkynscfqfwMvJf/v3AYPC7qk0mXrcUfoIBBGRmNM7Y0VEYk5FLyIScyp6EZGYU9GLiMScil5EJOZU9CIiMaeiFxGJuf8Pn37/6Rxd8c8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w_val = -1\n",
    "b_val = 65\n",
    "plt.plot(data[:,0], data[:,1], 'o')\n",
    "plt.axis([40, 85, -0.1, 1.2])\n",
    "x_pred = np.linspace(40,85)\n",
    "x_pred = np.resize(x_pred,[len(x_pred),1])\n",
    "y_pred = 1 / (1 + np.exp(-x_pred*w_val - b_val))\n",
    "plt.plot(x_pred, y_pred)\n",
    "\n",
    "# predicted probabilities\n",
    "p_1 = 1 / (1 + np.exp(-x*w_val - b_val))\n",
    "\n",
    "# cross-entropy loss function\n",
    "cross_entropy = -np.mean(y * np.log(p_1) + (1-y) * np.log(1-p_1))\n",
    "                         \n",
    "print(cross_entropy)\n",
    "print(np.round(p_1,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the value of the cross-entropy has decreased from 3.882916 to 0.9094435."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO : determine the accuracy of this logistic regression model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy:  0.8695652173913043\n"
     ]
    }
   ],
   "source": [
    "y_pred = np.round(p_1, decimals=0).astype('int')\n",
    "accuracy = np.mean(y==y_pred)\n",
    "print(\"Accuracy: \", accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO : set up a Keras model\n",
    "\n",
    "If there are two labels, we use `binary_crossentropy` as loss function. In this case, we use `sigmoid` as output layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7ff9e419c450>"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l0 = tf.keras.layers.Dense(units=1, activation = tf.nn.sigmoid, input_shape=[1])\n",
    "model = tf.keras.Sequential([l0])\n",
    "model.compile(loss='binary_crossentropy', optimizer=tf.keras.optimizers.Adam(0.01), metrics=['accuracy'])\n",
    "model.fit(x, y, epochs=10000, verbose=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7ff9ec0405d0>]"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhYUlEQVR4nO3deXxU5d3+8c83C2GLRCCiJKCACFK1gqnWpRVxBS1aVxCsW+Vpq221rU/1aW2tbX/d9/rY4lIfNxA3xBWtS1txqUFUBEQBFQiVTUHZCXx/f9yDBswyCTNzT06u9+s1r5lz5sycyyO5cnLOmXvM3RERkeQqiB1ARESyS0UvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJ12TRm9lNZrbMzF5r4PkxZvaqmc00s2fN7NOZjykiIi2Vzh79zcAJjTz/FnCku+8P/BgYn4FcIiKSIUVNLeDu/zSzvRp5/tk6k88DlRnIJSIiGdJk0TfThcAjDT1pZuOAcQCdOnU6aODAgRlevYhIsk2fPn2Fu5c35zUZK3ozO4pQ9Ec0tIy7jyd1aKeqqsqrq6sztXoRkTbBzN5p7msyUvRmdgBwAzDc3Vdm4j1FRCQzdvrySjPrDdwLnOPub+x8JBERyaQm9+jNbAIwFOhuZouBHwLFAO7+F+AHQDfgf80MoNbdq7IVWEREmiedq25GN/H8l4EvZyyRiIhklD4ZKyKScCp6EZGEU9GLiCScil5EJOFU9CIiCaeiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqnoRUQSTkUvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJp6IXEUk4Fb2ISMKp6EVEEk5FLyKScEVNLWBmNwEnAcvcfb96njfgD8AIYB1wnru/lOmgkjmTZ9Twq6lzWbJqPT3LOnD58QM4ZXBF1PXGytTSvK3R9yfPZMILi9jiTqEZow/pxU9O2T92rMRt53zUZNEDNwN/Bm5p4PnhQP/U7RDgutS95KHJM2q48t6ZrN+8BYCaVeu58t6ZAFn94WpsvUCUTI2JtZ2y5fuTZ3Lb8ws/mt7i/tF0zLJP2nbOV+buTS9kthfwYAN79H8Fnnb3CanpucBQd/9PY+9Zte9eXn3bj6CwGAqKoaDw48eFRVDYLvW4XZhfuO1xOygqqXNfAgU6ApWuw3/+JDWr1n9ifkVZB6ZdMSzKeoEomRoTaztlS78rH2ZLPT/rhWbM/9mICImCpG3nXDCz6e5e1ZzXpLNH35QKYFGd6cWpeZ8oejMbB4wDOGiPAnjgGxlYPVBQBEXtQ/EXdQj3xR3CvLr37TpBcUdo1xGKO4X7dp2gXSmUdIZ2naGkNNzadYb2u4TlzTKTMw8sqeeHqrH5Mdeb7UyNibWdsqW+km9sfq4kbTvnq0wUfdrcfTwwHqBqyKedyx6CLZth6xbYujn1uDZ1vxm2bAqPt9R9vBFqN4bpuve1G6F2Q7htXl/n8QZYtwI2rYPNqdumdVCb5j+kgiJoXwbtu0CH1H37MuiwK3TsFm6dukPHrqnp7mG6qCRbm3Gn9CzrUO8eVM/UnnWs9cbI1JhY2ylbCs0a3KOPKWnbOV9louhrgF51pitT8xpXUAxdKjOw+hbaujVV+mth0xrY+GG4bVoDG9fAxg/CbcPqcFu/6uPHqxbC+vdh3XtAA3tEHXaFzrtDaY/t77tUQJdeUNY7/GLI8Q/a5ccP2O6YKECH4kIuP35A1PXGyNSYWNspW0Yf0mu7Y/R158eUtO2crzJR9FOAS8xsIuEk7Oqmjs/nhYKCcLimpDPQo2XvsXVL+AWwbgWsWxlua1fAmmWwZmm4ffgurJwW7rdu3v71RR3CL7uyXqH8u/aFbv2g296wax8obr+z/5WfsO0EV66vckhnvfl05UWs7ZQt20645ttVN0nbzvmqyZOxZjYBGAp0B5YCPwSKAdz9L6nLK/8MnEC4vPJ8d69uasVVVVVeXd3kYsnhHv4C+GAxrF4MqxbB6m23xeGvhLXL67zAwi+BbcW/2yDosR/stm84dyAibVJWTsa6++gmnnfg4uastE0yg07dwm2PT9e/zIbV8N4CWDk/dZsH782HVyeFw0jblO0JPT4VbrvvDxVV4ZCQiEg9cnoyVprQvgv0HBxudbmHPf+ls2Hpa7B0Vri98Sj41rBMaU+orErdPgN7HBiuKhKRNk9F3xqYhZO3Zb1hwAkfz9+8IRT/4mqoqYbFL8KcKanXFIa/HPoeCX2HQq9DwiWmItLmpPWBqWxoc8foc2XtilD8i1+Ed6aF+6214YNlvQ8Jpd9nKPQ8MHxITURalZYco1fRJ93GNfDOs/DWP2DB0+EvAAjX+w8cAQO/EPb68/S6fxHZXqxPxko+K+kM+xwXbgBrlofSn/swvHYfvHRL+GTwPsfBwJOg/7Hhk8Eikhgq+ramcznsf3q41W6EBf8Ix/XnPgyv3RMO8QwYDoPPgX5H6fCOSAKo6NuyopKP9/a3boGFz8PsyTDz7nBf2hMOHA0HjgnX84tIq6Rj9PJJtRth7iPw8u0w7+/hEs7eh8HgsbDfqbp6RyQinYyVzPtgCbwyAWbcHj681akcDh4Hn/lyGMhNRHJKRS/Z4w5v/ROe+zO8+VgYp2fwWDj0YujaJ3Y6kTZDV91I9pilPnx1JCybA8/+GabfDNU3wr5fgMO+CZUHxU4pIvXQVzNJ8+22L5xyLVw6Ew7/Jsx/Gm4YBhNGw7LXY6cTkR2o6KXldtkDjrkavjULhl0Fbz8D1x0K918cRuQUkbygopedV1IKn/8OfPMV+OzXwmibfzoIHrsq9eUsIhKTil4yp2NXOP6n8PXp8KkvwrN/gj8eCNP+GL4GUkSiUNFL5pX1hi/+Bb7yTBg18/Gr4K9HwqIXYycTaZNU9JI9u+8HY+6CUXeE79i98Vh46Duw4YOmXysiGaOil+wbeCJc8m845L/gxRvg2kNgzgOxU4m0GSp6yY2SUhj+C/jyE9CxG9w5FiacDatrYicTSTwVveRW5UEw7ik45kcw/8lwOebsKbFTiSSail5yr7AYjrgUvjoNuvaFSefAg5fB5vWxk4kkkope4unWDy54DA77BlTfBNcPC8MriEhGqeglrqJ2cNyPYew9sHY5jD8Kqv8WBlETkYxQ0Ut+2PsY+Mo06P1ZePBSuOtcWL8qdiqRREir6M3sBDOba2bzzOyKep7vbWZPmdkMM3vVzEZkPqokXmkPGHtvOFH7+kPhUM6KebFTibR6TRa9mRUC1wLDgUHAaDMbtMNi3wcmuftgYBTwv5kOKm1EQUE4UXveQ7BhdRgVc/5TsVOJtGrp7NEfDMxz9wXuvgmYCJy8wzIO7JJ63AVYkrmI0ib1/ixc9CTsUgG3nRY+aCUiLZJO0VcAi+pML07Nq+tqYKyZLQYeBr5e3xuZ2Tgzqzaz6uXLl7cgrrQpu+4JFz4G/Y+Fh74ND18OW2pjpxJpdTJ1MnY0cLO7VwIjgFvN7BPv7e7j3b3K3avKy8sztGpJtJLSMFbOYV+Hf4+HO87QSVqRZkqn6GuAXnWmK1Pz6roQmATg7s8B7YHumQgoQkEhHPcTGPlneOtfcMMx8N6C2KlEWo10iv5FoL+Z9TGzdoSTrTt+Zn0hcDSAme1LKHodm5HMGnIOfOl+WLcCbhoOS2fHTiTSKjRZ9O5eC1wCTAXmEK6umWVm15jZyNRi3wYuMrNXgAnAee76xItkwV6Hw/mPhsc3j4Ca6XHziLQCFquPq6qqvLq6Osq6JQHeewtuOTl8VeHZE2GvI2InEskJM5vu7lXNeY0+GSutU9c+cMGjsEvPcPnlG4/FTiSSt1T00nrt0hPOfwTKB8DE0fDaPbETieQlFb20bp26wbkPQOVn4O4L4aVbYicSyTsqemn92ncJY+T0GwZTvh6GPBaRj6joJRnadYTRE6D/8fDgt+CVibETieQNFb0kR1EJnHkL9PkcTP4qzJocO5FIXlDRS7IUt4dRE8Ix+3suhDemxk4kEp2KXpKnpDOMuQt67Ad3ngMLno6dSCQqFb0kU/sucM594XtpJ4yGhc/HTiQSjYpekqtjVzhncrje/vYzoOal2IlEolDRS7KV9ggDobUvg9tOheVvxE4kknMqekm+LpVw7v1QUAS3nwZrlsVOJJJTKnppG7r2hbPvhLUr4I4zYdPa2IlEckZFL21HxUFw+t/gP6/AXefrawmlzVDRS9sy4AQY8Wt4cyo8/B3Q1yZIG1AUO4BIzn3mQli9CJ75HZT1gs99O3YikaxS0UvbNOwHsHoxPHENdOkFB5wZO5FI1qjopW0qKICTr4UP34XJX4POPaDvkbFTiWSFjtFL21VUAmfdBt32DkMlrHgzdiKRrFDRS9vWoQzGTILCYpgwCtavip1IJONU9CJlveGsW+H9d+DuC3TZpSSOil4EYM/D4MTfwPwn4PEfxE4jklE6GSuyzUHnwrLZ8Py10GMQDB4bO5FIRmiPXqSu434KfYfCg5fBwhdipxHJCBW9SF2FRWGYhC6VcOcYWLUodiKRnZZW0ZvZCWY218zmmdkVDSxzppnNNrNZZnZHZmOK5FDHrjB6ItRuhImjNQCatHpNFr2ZFQLXAsOBQcBoMxu0wzL9gSuBw939U8ClmY8qkkPlA+C0G+Hd18IHqjQmjrRi6ezRHwzMc/cF7r4JmAicvMMyFwHXuvv7AO6uAb+l9dvnODjmapg9GZ79U+w0Ii2WTtFXAHUPVC5OzatrH2AfM5tmZs+b2Qn1vZGZjTOzajOrXr58ecsSi+TS4d+EfUfC338Ib/0zdhqRFsnUydgioD8wFBgNXG9mZTsu5O7j3b3K3avKy8sztGqRLDKDU/4XuvUPY9ivXhw7kUizpVP0NUCvOtOVqXl1LQamuPtmd38LeINQ/CKtX0lpGBOndiNM+lK4F2lF0in6F4H+ZtbHzNoBo4ApOywzmbA3j5l1JxzKWZC5mCKRle8DX7wOaqbDI/8dO41IszRZ9O5eC1wCTAXmAJPcfZaZXWNmI1OLTQVWmtls4Cngcndfma3QIlHs+wU44jKYfjO8dEvsNCJpM4902VhVVZVXV1dHWbdIi23dAredCu88Bxc8ChVDYieSNsbMprt7VXNeo0/GijRHQSGcdhN03i2MYb92RexEIk1S0Ys0V6duYVjjtcvhngvDXr5IHlPRi7REz8Fw4q9hwdPw9M9jpxFplIpepKUGnwMHjoF//hLefDx2GpEGqehFWsoMRvwaeuwH914EqxbGTiRSLxW9yM5o1xHOvCUcp9eHqSRPqehFdla3fmGYhCUz4NErY6cR+QQVvUgm7PsFOOzrUH0jvDopdhqR7ajoRTLl6Kuh92HwwDdh2ZzYaUQ+oqIXyZTCIjjjb9Cuc/gw1cYPYycSAVT0IplVujucfhO8Nx+mfF3fTCV5QUUvkml9PgfDroJZ98ELf42dRkRFL5IVh18K+wyHx74Hi/4dO420cSp6kWwoKAjj1+9SAXedp8HPJCoVvUi2dNg1NfjZCg1+JlGp6EWyaY9Pw4hfafAziUpFL5JtQ76kwc8kKhW9SLZp8DOJTEUvkgvbDX52rgY/k5xS0YvkSrd+cMp1sOQleOS7sdNIG6KiF8mlfU+CIy6D6X+Dl26NnUbaCBW9SK4Nuwr6DoWHvg01L8VOI22Ail4k1woK4bSboPNu4ctK1q6MnUgSTkUvEkOnbuHDVGuWwT0X6MNUklVpFb2ZnWBmc81snpld0chyp5mZm1lV5iKKJFTPwXDib8KHqZ78cew0kmBNFr2ZFQLXAsOBQcBoMxtUz3KlwDeBFzIdUiSxhpwDB50Pz/wOZk+JnUYSKp09+oOBee6+wN03AROBk+tZ7sfAL4ANGcwnknzDfwEVVTD5q7D8jdhpJIHSKfoKYFGd6cWpeR8xsyFAL3d/qLE3MrNxZlZtZtXLly9vdliRRCoqCR+mKmoPd46BDR/ETiQJs9MnY82sAPgt8O2mlnX38e5e5e5V5eXlO7tqkeToUgFn3Awr58M9X9bJWcmodIq+BuhVZ7oyNW+bUmA/4Gkzexv4LDBFJ2RFmqnP58JhnDenwhPXxE4jCVKUxjIvAv3NrA+h4EcBZ2970t1XA923TZvZ08B33L06s1FF2oCDL4Jls2Ha72G3QfDps2InkgRoco/e3WuBS4CpwBxgkrvPMrNrzGxktgOKtDnDfwl7fS58ufhi7S/JzjOP9C31VVVVXl2tf8Qi9Vq7Eq4/KoxyOe4p2KVn7ESSJ8xsurs369C4Phkrko86dYPRE2HTGph4NmxeHzuRtGIqepF81WMQnHo9LHkZ7r8EIv31La2fil4knw0cAUdfBa/dDc/8NnYaaaXSuepGRGI64luwdHa45HLXvWC/02InklZGRS+S78zg5GvhgyVw31eg8+6w1+GxU0krokM3Iq1BcXsYdXvYo584GpbPjZ1IWhEVvUhr0bErjLkbCkvgttPhw3djJ5JWQkUv0prsuieMmQTrVsIdZ8LGNbETSSugohdpbXoODgOgvTsT7joPttTGTiR5TkUv0hrtcxyc+FuY9zg8dJmusZdG6aobkdaq6nxYvQj+9RvYpRKGfjd2IslTKnqR1mzYVeGyy6f/H5SUwqFfi51I8pCKXqQ1M4ORfw5j4ky9Etp1hIPOi51K8oyO0Yu0doVFcNpNsPex8MCl8MqdsRNJnlHRiyRBUTs461bY64jwJeOzp8ROJHlERS+SFMUdwtDGFQfB3RfAG4/FTiR5QkUvkiQlnWHMXWGI40nnwFv/jJ1I8oCKXiRpOpTB2Ptg1z5wxyhY+HzsRBKZil4kiTp1gy9NhtLd4dZTYf5TsRNJRCp6kaQq3R3OfySMj3PHmfD6Q7ETSSQqepEkK+0B5z0Eu+8Pd56jSy/bKBW9SNJ17Apfuh/2PAzu+y948YbYiSTHVPQibUFJabgaZ5/j4aFvw7/0/bNtiYpepK0o7gBn3Ra+c/aJH8Hfr9aol21EWkVvZieY2Vwzm2dmV9Tz/LfMbLaZvWpmT5jZnpmPKiI7rbAYTr0+jIfzzO/g/kugdlPsVJJlTRa9mRUC1wLDgUHAaDMbtMNiM4Aqdz8AuBv4ZaaDikiGFBTCSb+HI78LL98Gt5wMa1fETiVZlM4e/cHAPHdf4O6bgInAyXUXcPen3H1davJ5oDKzMUUko8zgqP+B026Emulw/TBYNid2KsmSdIq+AlhUZ3pxal5DLgQe2ZlQIpIj+58O5z8MtRvghmM1Pk5CZfRkrJmNBaqAXzXw/Dgzqzaz6uXLl2dy1SLSUpVVcNGT0HUvmHAWPHetTtImTDpFXwP0qjNdmZq3HTM7BvgeMNLdN9b3Ru4+3t2r3L2qvLy8JXlFJBu6VMIFU2HgiTD1f+CBb0BtvT/G0gqlU/QvAv3NrI+ZtQNGAdsNdm1mg4G/Ekp+WeZjikjWtesEZ9wCn/sOvHQL3HgsrJgXO5VkQJNF7+61wCXAVGAOMMndZ5nZNWY2MrXYr4DOwF1m9rKZ6VsPRFqjggI4+ioYdQesWgh//Ty8PCF2KtlJ5pGOxVVVVXl1dXWUdYtIGlbXwL0XwTvT4ICz4MTfhE/YSlRmNt3dq5rzGn0yVkTq16UCzn0Ahl4JM+8Ke/dLZsROJS2goheRhhUUwtArwgiYtRvDJZjP/gm2bomdTJpBRS8iTdvzMPjKM2FQtMe+DzceB+++FjuVpElFLyLp6dg1DIp26vXw/lsw/kh4/IewaV3Tr5WoVPQikj4zOOBMuKQaDhgF034P1x0K85+MnUwaoaIXkebr2BVOuTacrLVCuPWLcO84DY6Wp1T0ItJyfT4PX30WPv/f8Nq98Mch4UtNdDgnr6joRWTnFLeHYd+Dr06DPQ8NX2rypyEw/f9gS23sdIKKXkQypXwAnH0nnPdwGDvngW+E4/dzHtAgaZGp6EUks/Y6HC58HM66PUzfOTaMm7PgHyr8SFT0IpJ5ZrDvSfDV52Dkn8JwCreMhPFDYebdOqSTYyp6EcmewiIY8iX4xgz4wh9g01q450L442B4/jrY+GHshG2Cil5Esq+4ffhC8ov/DaMmhGP4j14Bv/sU/P3qsMcvWaPRK0UkjsXVMO0P4WStGfQ9CgaPDV9+UlQSO13easnolUXZCiMi0qjKKjjrVnj/bZhxO7x8B9x9PnTYFfY/I5T+Hp+OnTIRtEcvIvlh6xZY8DS8fDvMeRC2bIQe+8N+p8K+I6H73rET5oWW7NGr6EUk/6x/P1yd8/IdsOSlMK98IOz7BRh4UtjTN4ubMRIVvYgkz6pF8PpD8PqD4duufCt06R2O5e99NPQ+FEo6x06ZMyp6EUm2tStg7iOh9Oc/FQ7vFBRD5Weg75HQdyhUHASFxbGTZo2KXkTajk3rYNHz4RO3C56G/7wCOLTrHPbyex0cTvj2HAIdyiKHzRxddSMibUe7jtBvWLgBrHsP3n4G3vpHuJ/3dyC1I9t9QCj9bcVfPjBc299GqOhFJBk6doVBI8MNYMNqqHkJaqrDNftvPBqu6IEwhn63vaHHp6DHIOixX3jcpVciT/Kq6EUkmdp3gX5HhRuEAdXefxuWzIBls2HpLKiZDrPu/fg1xZ2gW1/o2i/8IuiWuu/aL/wiaaW/BFT0ItI2mEHXPuHGqR/P3/ABLJsDS1+DFW/Ce/Ph3VdTwytv+Xi5dqVh6IayXmHPv0sllPUOj0t3h8498vZwkIpeRNq29rtA70PCra4tm+H9d0Lxr5wHqxaGSz1XL4LFL4Zr/T/xXl2g8+5Q2iPcd94NOnWHjt2hY7fU427hr4P2ZTn7CyGtojezE4A/AIXADe7+8x2eLwFuAQ4CVgJnufvbmY0qkhyTZ9Twq6lzWbJqPT3LOnD58QM4ZXAFAGOuf45p89/7aNnD+3Xl9osOTeu1jT33/ckzmfDCIra4U2jG6EN68ZNT9t/pvOk839L3zlbmtPIWFodP43bfGzj+E++7asP7HLjLGi4eUsJhu22GNe/Ch0s/vl/0PKxZBrUb6g9oheEXw7Zbh7LU47Lwy6ddKZSUhs8IlJR+PN0CTV5eaWaFwBvAscBi4EVgtLvPrrPM14AD3P0rZjYK+KK7n9XY++rySmmrJs+o4cp7Z7J+88eHBToUF/KzU/fnruqF25X8NtvKvrHXAg0+V/3Oe9z2/MJPvO/Yz/ZusjgbW+cpgyuafL6l752tzNDwdtqZvPW+1h02rwvX/69bGa4MWrcS1qWmN6wOt/WrUo9XfTyvgV8Q9qMPMn8dvZkdClzt7senpq8M+f1ndZaZmlrmOTMrAt4Fyr2RN1fRS1t1+M+fpGbV+k/MryjrUO/8bd7++YmNvhZo8Ll3V29gSz0/joVmzP/ZiBbnnXbFsCafb+l7ZyszNLyddiZvU69tti2bw3j9m9bAxjWpxx9i/Y/NynX0FcCiOtOLgUMaWsbda81sNdANWFF3ITMbB4wD6N27d3NyiiTGkgbKvKH5O/vaJavW09AeV31F2tx1Zuu/J5uZm/vczrxvixUWh2P5Hbvu9Fvl9ItH3H28u1e5e1V5eXkuVy2SN3qm9irTnZ/uaxt7rrCBk34NzU93nek839L3zlbmbOXNZ+kUfQ3Qq850ZWpevcukDt10IZyUFZEdXH78ADoUF243r0NxIZcfP4DD+9W/97ZtfmOvbey50Yf0oj4NzU83bzrPt/S9s5U5W3nzWTqHbl4E+ptZH0KhjwLO3mGZKcC5wHPA6cCTjR2fF2nLtp20q++qj1MGVzR61U1jr92mofcFWnQFS1PrTCdTS7dFtjJnI28+S2tQMzMbAfyecHnlTe7+UzO7Bqh29ylm1h64FRgMvAeMcvcFjb2nTsaKiDRf1gY1c/eHgYd3mPeDOo83AGc0Z8UiIpIbOT0ZKyIiuaeiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqU1qFlWVmz2ITA3ysob1p0dviwlT+RjLmVKjzKlLx9z5WOmAe7erC+PTWtQsyyZ29wR2LLNzKrzLRPkZy5lSo8ypS8fc+Vrpua+RoduREQSTkUvIpJwMYt+fMR1NyQfM0F+5lKm9ChT+vIxVyIyRTsZKyIiuaFDNyIiCaeiFxFJuJwVvZkVmtkMM3swNd3HzF4ws3lmdqeZtctVliZy3Wxmb5nZy6nbgTnO87aZzUytuzo1r6uZPW5mb6bud82DTFebWU2d7TQix5nKzOxuM3vdzOaY2aGxt1MjuaJtKzMbUGe9L5vZB2Z2acxt1Uim2P+mLjOzWWb2mplNMLP2sXuqgUzN7qicHaM3s28BVcAu7n6SmU0C7nX3iWb2F+AVd78uJ2Eaz3Uz8KC7353rLKk8bwNV7r6izrxfAu+5+8/N7ApgV3f/buRMVwNr3P3XucqxQ6b/A/7l7jekfvg6Av9DxO3USK5Libit6mQrBGqAQ4CLibyt6sl0PpG2k5lVAM8Ag9x9faqfHgZGEKmnGsk0lGZ2VE726M2sEjgRuCE1bcAwYFvQ/wNOyUWWxnLlsZMJ2wgibat8YmZdgM8DNwK4+yZ3X0Xk7dRIrnxxNDDf3d8hf/5N1c0UWxHQwcyKCL+g/0P8ntox05KWvEmuDt38HvhvYGtquhuwyt1rU9OLgYocZWks1zY/NbNXzex3ZlaS40wOPGZm081sXGpeD3f/T+rxu0CPPMgEcElqO92U48MkfYDlwN8sHHa7wcw6EX87NZQL4m2rukYBE1KPY2+rbepmgkjbyd1rgF8DCwkFvxqYTsSeqi+Tuz+WerpZHZX1ojezk4Bl7j492+tqjkZyXQkMBD4DdAVy/efsEe4+BBgOXGxmn6/7pIdjbbm+Jra+TNcB/YADCf8If5PDPEXAEOA6dx8MrAWuqLtApO3UUK6Y2wqA1GGkkcBdOz4XaVvVlynadkr9UjmZ8Mu6J9AJOCFX6083k5mNpQUdlYs9+sOBkanjvBMJfwr9AShL/TkCUEk4TpdLn8hlZre5+3882Aj8DTg4l6FSv8Vx92XAfan1LzWzPQBS98tiZ3L3pe6+xd23AteT2+20GFjs7i+kpu8mFGzU7dRQrsjbapvhwEvuvjQ1HXtbfSJT5O10DPCWuy93983AvYSOiNlT9WU6rCUdlfWid/cr3b3S3fci/Jn2pLuPAZ4CTk8tdi5wf7azpJFrbJ1//EY4HvdarjKZWSczK932GDgutf4phG0EOd5WDWXatp1SvkgOt5O7vwssMrMBqVlHA7OJuJ0ayxVzW9Uxmu0PkUTdVinbZYq8nRYCnzWzjqmf/W3/pmL2VH2Z5rSoo9w9Zzc+PlsM0Bf4NzCP8KdbSS6zNJLrSWBmauPdBnTOYY6+wCup2yzge6n53YAngDeBvwNd8yDTrant9CqhNPbI8f+zA4Hq1PonA7vG3E5N5Iq9rToBK4EudeZF3VYNZIq9nX4EvJ762b8VKIndUw1kanZHaQgEEZGE0ydjRUQSTkUvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJp6IXEUm4/w8enGsxZVdAbgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(data[:,0], data[:,1], 'o')\n",
    "plt.axis([40, 85, -0.1, 1.2])\n",
    "x_pred = np.linspace(40,85)\n",
    "x_pred = np.resize(x_pred,[len(x_pred),1])\n",
    "y_pred = model.predict(x_pred)\n",
    "plt.plot(x_pred, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([[-0.23217289]], dtype=float32), array([15.043545], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "print(model.get_weights())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-entropy:  0.4416346\n",
      "Accuracy:  0.8695652173913043\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhYUlEQVR4nO3deXxU5d3+8c83C2GLRCCiJKCACFK1gqnWpRVxBS1aVxCsW+Vpq221rU/1aW2tbX/d9/rY4lIfNxA3xBWtS1txqUFUBEQBFQiVTUHZCXx/f9yDBswyCTNzT06u9+s1r5lz5sycyyO5cnLOmXvM3RERkeQqiB1ARESyS0UvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJ12TRm9lNZrbMzF5r4PkxZvaqmc00s2fN7NOZjykiIi2Vzh79zcAJjTz/FnCku+8P/BgYn4FcIiKSIUVNLeDu/zSzvRp5/tk6k88DlRnIJSIiGdJk0TfThcAjDT1pZuOAcQCdOnU6aODAgRlevYhIsk2fPn2Fu5c35zUZK3ozO4pQ9Ec0tIy7jyd1aKeqqsqrq6sztXoRkTbBzN5p7msyUvRmdgBwAzDc3Vdm4j1FRCQzdvrySjPrDdwLnOPub+x8JBERyaQm9+jNbAIwFOhuZouBHwLFAO7+F+AHQDfgf80MoNbdq7IVWEREmiedq25GN/H8l4EvZyyRiIhklD4ZKyKScCp6EZGEU9GLiCScil5EJOFU9CIiCaeiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqnoRUQSTkUvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJp6IXEUk4Fb2ISMKp6EVEEk5FLyKScEVNLWBmNwEnAcvcfb96njfgD8AIYB1wnru/lOmgkjmTZ9Twq6lzWbJqPT3LOnD58QM4ZXBF1PXGytTSvK3R9yfPZMILi9jiTqEZow/pxU9O2T92rMRt53zUZNEDNwN/Bm5p4PnhQP/U7RDgutS95KHJM2q48t6ZrN+8BYCaVeu58t6ZAFn94WpsvUCUTI2JtZ2y5fuTZ3Lb8ws/mt7i/tF0zLJP2nbOV+buTS9kthfwYAN79H8Fnnb3CanpucBQd/9PY+9Zte9eXn3bj6CwGAqKoaDw48eFRVDYLvW4XZhfuO1xOygqqXNfAgU6ApWuw3/+JDWr1n9ifkVZB6ZdMSzKeoEomRoTaztlS78rH2ZLPT/rhWbM/9mICImCpG3nXDCz6e5e1ZzXpLNH35QKYFGd6cWpeZ8oejMbB4wDOGiPAnjgGxlYPVBQBEXtQ/EXdQj3xR3CvLr37TpBcUdo1xGKO4X7dp2gXSmUdIZ2naGkNNzadYb2u4TlzTKTMw8sqeeHqrH5Mdeb7UyNibWdsqW+km9sfq4kbTvnq0wUfdrcfTwwHqBqyKedyx6CLZth6xbYujn1uDZ1vxm2bAqPt9R9vBFqN4bpuve1G6F2Q7htXl/n8QZYtwI2rYPNqdumdVCb5j+kgiJoXwbtu0CH1H37MuiwK3TsFm6dukPHrqnp7mG6qCRbm3Gn9CzrUO8eVM/UnnWs9cbI1JhY2ylbCs0a3KOPKWnbOV9louhrgF51pitT8xpXUAxdKjOw+hbaujVV+mth0xrY+GG4bVoDG9fAxg/CbcPqcFu/6uPHqxbC+vdh3XtAA3tEHXaFzrtDaY/t77tUQJdeUNY7/GLI8Q/a5ccP2O6YKECH4kIuP35A1PXGyNSYWNspW0Yf0mu7Y/R158eUtO2crzJR9FOAS8xsIuEk7Oqmjs/nhYKCcLimpDPQo2XvsXVL+AWwbgWsWxlua1fAmmWwZmm4ffgurJwW7rdu3v71RR3CL7uyXqH8u/aFbv2g296wax8obr+z/5WfsO0EV66vckhnvfl05UWs7ZQt20645ttVN0nbzvmqyZOxZjYBGAp0B5YCPwSKAdz9L6nLK/8MnEC4vPJ8d69uasVVVVVeXd3kYsnhHv4C+GAxrF4MqxbB6m23xeGvhLXL67zAwi+BbcW/2yDosR/stm84dyAibVJWTsa6++gmnnfg4uastE0yg07dwm2PT9e/zIbV8N4CWDk/dZsH782HVyeFw0jblO0JPT4VbrvvDxVV4ZCQiEg9cnoyVprQvgv0HBxudbmHPf+ls2Hpa7B0Vri98Sj41rBMaU+orErdPgN7HBiuKhKRNk9F3xqYhZO3Zb1hwAkfz9+8IRT/4mqoqYbFL8KcKanXFIa/HPoeCX2HQq9DwiWmItLmpPWBqWxoc8foc2XtilD8i1+Ed6aF+6214YNlvQ8Jpd9nKPQ8MHxITURalZYco1fRJ93GNfDOs/DWP2DB0+EvAAjX+w8cAQO/EPb68/S6fxHZXqxPxko+K+kM+xwXbgBrlofSn/swvHYfvHRL+GTwPsfBwJOg/7Hhk8Eikhgq+ramcznsf3q41W6EBf8Ix/XnPgyv3RMO8QwYDoPPgX5H6fCOSAKo6NuyopKP9/a3boGFz8PsyTDz7nBf2hMOHA0HjgnX84tIq6Rj9PJJtRth7iPw8u0w7+/hEs7eh8HgsbDfqbp6RyQinYyVzPtgCbwyAWbcHj681akcDh4Hn/lyGMhNRHJKRS/Z4w5v/ROe+zO8+VgYp2fwWDj0YujaJ3Y6kTZDV91I9pilPnx1JCybA8/+GabfDNU3wr5fgMO+CZUHxU4pIvXQVzNJ8+22L5xyLVw6Ew7/Jsx/Gm4YBhNGw7LXY6cTkR2o6KXldtkDjrkavjULhl0Fbz8D1x0K918cRuQUkbygopedV1IKn/8OfPMV+OzXwmibfzoIHrsq9eUsIhKTil4yp2NXOP6n8PXp8KkvwrN/gj8eCNP+GL4GUkSiUNFL5pX1hi/+Bb7yTBg18/Gr4K9HwqIXYycTaZNU9JI9u+8HY+6CUXeE79i98Vh46Duw4YOmXysiGaOil+wbeCJc8m845L/gxRvg2kNgzgOxU4m0GSp6yY2SUhj+C/jyE9CxG9w5FiacDatrYicTSTwVveRW5UEw7ik45kcw/8lwOebsKbFTiSSail5yr7AYjrgUvjoNuvaFSefAg5fB5vWxk4kkkope4unWDy54DA77BlTfBNcPC8MriEhGqeglrqJ2cNyPYew9sHY5jD8Kqv8WBlETkYxQ0Ut+2PsY+Mo06P1ZePBSuOtcWL8qdiqRREir6M3sBDOba2bzzOyKep7vbWZPmdkMM3vVzEZkPqokXmkPGHtvOFH7+kPhUM6KebFTibR6TRa9mRUC1wLDgUHAaDMbtMNi3wcmuftgYBTwv5kOKm1EQUE4UXveQ7BhdRgVc/5TsVOJtGrp7NEfDMxz9wXuvgmYCJy8wzIO7JJ63AVYkrmI0ib1/ixc9CTsUgG3nRY+aCUiLZJO0VcAi+pML07Nq+tqYKyZLQYeBr5e3xuZ2Tgzqzaz6uXLl7cgrrQpu+4JFz4G/Y+Fh74ND18OW2pjpxJpdTJ1MnY0cLO7VwIjgFvN7BPv7e7j3b3K3avKy8sztGpJtJLSMFbOYV+Hf4+HO87QSVqRZkqn6GuAXnWmK1Pz6roQmATg7s8B7YHumQgoQkEhHPcTGPlneOtfcMMx8N6C2KlEWo10iv5FoL+Z9TGzdoSTrTt+Zn0hcDSAme1LKHodm5HMGnIOfOl+WLcCbhoOS2fHTiTSKjRZ9O5eC1wCTAXmEK6umWVm15jZyNRi3wYuMrNXgAnAee76xItkwV6Hw/mPhsc3j4Ca6XHziLQCFquPq6qqvLq6Osq6JQHeewtuOTl8VeHZE2GvI2InEskJM5vu7lXNeY0+GSutU9c+cMGjsEvPcPnlG4/FTiSSt1T00nrt0hPOfwTKB8DE0fDaPbETieQlFb20bp26wbkPQOVn4O4L4aVbYicSyTsqemn92ncJY+T0GwZTvh6GPBaRj6joJRnadYTRE6D/8fDgt+CVibETieQNFb0kR1EJnHkL9PkcTP4qzJocO5FIXlDRS7IUt4dRE8Ix+3suhDemxk4kEp2KXpKnpDOMuQt67Ad3ngMLno6dSCQqFb0kU/sucM594XtpJ4yGhc/HTiQSjYpekqtjVzhncrje/vYzoOal2IlEolDRS7KV9ggDobUvg9tOheVvxE4kknMqekm+LpVw7v1QUAS3nwZrlsVOJJJTKnppG7r2hbPvhLUr4I4zYdPa2IlEckZFL21HxUFw+t/gP6/AXefrawmlzVDRS9sy4AQY8Wt4cyo8/B3Q1yZIG1AUO4BIzn3mQli9CJ75HZT1gs99O3YikaxS0UvbNOwHsHoxPHENdOkFB5wZO5FI1qjopW0qKICTr4UP34XJX4POPaDvkbFTiWSFjtFL21VUAmfdBt32DkMlrHgzdiKRrFDRS9vWoQzGTILCYpgwCtavip1IJONU9CJlveGsW+H9d+DuC3TZpSSOil4EYM/D4MTfwPwn4PEfxE4jklE6GSuyzUHnwrLZ8Py10GMQDB4bO5FIRmiPXqSu434KfYfCg5fBwhdipxHJCBW9SF2FRWGYhC6VcOcYWLUodiKRnZZW0ZvZCWY218zmmdkVDSxzppnNNrNZZnZHZmOK5FDHrjB6ItRuhImjNQCatHpNFr2ZFQLXAsOBQcBoMxu0wzL9gSuBw939U8ClmY8qkkPlA+C0G+Hd18IHqjQmjrRi6ezRHwzMc/cF7r4JmAicvMMyFwHXuvv7AO6uAb+l9dvnODjmapg9GZ79U+w0Ii2WTtFXAHUPVC5OzatrH2AfM5tmZs+b2Qn1vZGZjTOzajOrXr58ecsSi+TS4d+EfUfC338Ib/0zdhqRFsnUydgioD8wFBgNXG9mZTsu5O7j3b3K3avKy8sztGqRLDKDU/4XuvUPY9ivXhw7kUizpVP0NUCvOtOVqXl1LQamuPtmd38LeINQ/CKtX0lpGBOndiNM+lK4F2lF0in6F4H+ZtbHzNoBo4ApOywzmbA3j5l1JxzKWZC5mCKRle8DX7wOaqbDI/8dO41IszRZ9O5eC1wCTAXmAJPcfZaZXWNmI1OLTQVWmtls4Cngcndfma3QIlHs+wU44jKYfjO8dEvsNCJpM4902VhVVZVXV1dHWbdIi23dAredCu88Bxc8ChVDYieSNsbMprt7VXNeo0/GijRHQSGcdhN03i2MYb92RexEIk1S0Ys0V6duYVjjtcvhngvDXr5IHlPRi7REz8Fw4q9hwdPw9M9jpxFplIpepKUGnwMHjoF//hLefDx2GpEGqehFWsoMRvwaeuwH914EqxbGTiRSLxW9yM5o1xHOvCUcp9eHqSRPqehFdla3fmGYhCUz4NErY6cR+QQVvUgm7PsFOOzrUH0jvDopdhqR7ajoRTLl6Kuh92HwwDdh2ZzYaUQ+oqIXyZTCIjjjb9Cuc/gw1cYPYycSAVT0IplVujucfhO8Nx+mfF3fTCV5QUUvkml9PgfDroJZ98ELf42dRkRFL5IVh18K+wyHx74Hi/4dO420cSp6kWwoKAjj1+9SAXedp8HPJCoVvUi2dNg1NfjZCg1+JlGp6EWyaY9Pw4hfafAziUpFL5JtQ76kwc8kKhW9SLZp8DOJTEUvkgvbDX52rgY/k5xS0YvkSrd+cMp1sOQleOS7sdNIG6KiF8mlfU+CIy6D6X+Dl26NnUbaCBW9SK4Nuwr6DoWHvg01L8VOI22Ail4k1woK4bSboPNu4ctK1q6MnUgSTkUvEkOnbuHDVGuWwT0X6MNUklVpFb2ZnWBmc81snpld0chyp5mZm1lV5iKKJFTPwXDib8KHqZ78cew0kmBNFr2ZFQLXAsOBQcBoMxtUz3KlwDeBFzIdUiSxhpwDB50Pz/wOZk+JnUYSKp09+oOBee6+wN03AROBk+tZ7sfAL4ANGcwnknzDfwEVVTD5q7D8jdhpJIHSKfoKYFGd6cWpeR8xsyFAL3d/qLE3MrNxZlZtZtXLly9vdliRRCoqCR+mKmoPd46BDR/ETiQJs9MnY82sAPgt8O2mlnX38e5e5e5V5eXlO7tqkeToUgFn3Awr58M9X9bJWcmodIq+BuhVZ7oyNW+bUmA/4Gkzexv4LDBFJ2RFmqnP58JhnDenwhPXxE4jCVKUxjIvAv3NrA+h4EcBZ2970t1XA923TZvZ08B33L06s1FF2oCDL4Jls2Ha72G3QfDps2InkgRoco/e3WuBS4CpwBxgkrvPMrNrzGxktgOKtDnDfwl7fS58ufhi7S/JzjOP9C31VVVVXl2tf8Qi9Vq7Eq4/KoxyOe4p2KVn7ESSJ8xsurs369C4Phkrko86dYPRE2HTGph4NmxeHzuRtGIqepF81WMQnHo9LHkZ7r8EIv31La2fil4knw0cAUdfBa/dDc/8NnYaaaXSuepGRGI64luwdHa45HLXvWC/02InklZGRS+S78zg5GvhgyVw31eg8+6w1+GxU0krokM3Iq1BcXsYdXvYo584GpbPjZ1IWhEVvUhr0bErjLkbCkvgttPhw3djJ5JWQkUv0prsuieMmQTrVsIdZ8LGNbETSSugohdpbXoODgOgvTsT7joPttTGTiR5TkUv0hrtcxyc+FuY9zg8dJmusZdG6aobkdaq6nxYvQj+9RvYpRKGfjd2IslTKnqR1mzYVeGyy6f/H5SUwqFfi51I8pCKXqQ1M4ORfw5j4ky9Etp1hIPOi51K8oyO0Yu0doVFcNpNsPex8MCl8MqdsRNJnlHRiyRBUTs461bY64jwJeOzp8ROJHlERS+SFMUdwtDGFQfB3RfAG4/FTiR5QkUvkiQlnWHMXWGI40nnwFv/jJ1I8oCKXiRpOpTB2Ptg1z5wxyhY+HzsRBKZil4kiTp1gy9NhtLd4dZTYf5TsRNJRCp6kaQq3R3OfySMj3PHmfD6Q7ETSSQqepEkK+0B5z0Eu+8Pd56jSy/bKBW9SNJ17Apfuh/2PAzu+y948YbYiSTHVPQibUFJabgaZ5/j4aFvw7/0/bNtiYpepK0o7gBn3Ra+c/aJH8Hfr9aol21EWkVvZieY2Vwzm2dmV9Tz/LfMbLaZvWpmT5jZnpmPKiI7rbAYTr0+jIfzzO/g/kugdlPsVJJlTRa9mRUC1wLDgUHAaDMbtMNiM4Aqdz8AuBv4ZaaDikiGFBTCSb+HI78LL98Gt5wMa1fETiVZlM4e/cHAPHdf4O6bgInAyXUXcPen3H1davJ5oDKzMUUko8zgqP+B026Emulw/TBYNid2KsmSdIq+AlhUZ3pxal5DLgQe2ZlQIpIj+58O5z8MtRvghmM1Pk5CZfRkrJmNBaqAXzXw/Dgzqzaz6uXLl2dy1SLSUpVVcNGT0HUvmHAWPHetTtImTDpFXwP0qjNdmZq3HTM7BvgeMNLdN9b3Ru4+3t2r3L2qvLy8JXlFJBu6VMIFU2HgiTD1f+CBb0BtvT/G0gqlU/QvAv3NrI+ZtQNGAdsNdm1mg4G/Ekp+WeZjikjWtesEZ9wCn/sOvHQL3HgsrJgXO5VkQJNF7+61wCXAVGAOMMndZ5nZNWY2MrXYr4DOwF1m9rKZ6VsPRFqjggI4+ioYdQesWgh//Ty8PCF2KtlJ5pGOxVVVVXl1dXWUdYtIGlbXwL0XwTvT4ICz4MTfhE/YSlRmNt3dq5rzGn0yVkTq16UCzn0Ahl4JM+8Ke/dLZsROJS2goheRhhUUwtArwgiYtRvDJZjP/gm2bomdTJpBRS8iTdvzMPjKM2FQtMe+DzceB+++FjuVpElFLyLp6dg1DIp26vXw/lsw/kh4/IewaV3Tr5WoVPQikj4zOOBMuKQaDhgF034P1x0K85+MnUwaoaIXkebr2BVOuTacrLVCuPWLcO84DY6Wp1T0ItJyfT4PX30WPv/f8Nq98Mch4UtNdDgnr6joRWTnFLeHYd+Dr06DPQ8NX2rypyEw/f9gS23sdIKKXkQypXwAnH0nnPdwGDvngW+E4/dzHtAgaZGp6EUks/Y6HC58HM66PUzfOTaMm7PgHyr8SFT0IpJ5ZrDvSfDV52Dkn8JwCreMhPFDYebdOqSTYyp6EcmewiIY8iX4xgz4wh9g01q450L442B4/jrY+GHshG2Cil5Esq+4ffhC8ov/DaMmhGP4j14Bv/sU/P3qsMcvWaPRK0UkjsXVMO0P4WStGfQ9CgaPDV9+UlQSO13easnolUXZCiMi0qjKKjjrVnj/bZhxO7x8B9x9PnTYFfY/I5T+Hp+OnTIRtEcvIvlh6xZY8DS8fDvMeRC2bIQe+8N+p8K+I6H73rET5oWW7NGr6EUk/6x/P1yd8/IdsOSlMK98IOz7BRh4UtjTN4ubMRIVvYgkz6pF8PpD8PqD4duufCt06R2O5e99NPQ+FEo6x06ZMyp6EUm2tStg7iOh9Oc/FQ7vFBRD5Weg75HQdyhUHASFxbGTZo2KXkTajk3rYNHz4RO3C56G/7wCOLTrHPbyex0cTvj2HAIdyiKHzRxddSMibUe7jtBvWLgBrHsP3n4G3vpHuJ/3dyC1I9t9QCj9bcVfPjBc299GqOhFJBk6doVBI8MNYMNqqHkJaqrDNftvPBqu6IEwhn63vaHHp6DHIOixX3jcpVciT/Kq6EUkmdp3gX5HhRuEAdXefxuWzIBls2HpLKiZDrPu/fg1xZ2gW1/o2i/8IuiWuu/aL/wiaaW/BFT0ItI2mEHXPuHGqR/P3/ABLJsDS1+DFW/Ce/Ph3VdTwytv+Xi5dqVh6IayXmHPv0sllPUOj0t3h8498vZwkIpeRNq29rtA70PCra4tm+H9d0Lxr5wHqxaGSz1XL4LFL4Zr/T/xXl2g8+5Q2iPcd94NOnWHjt2hY7fU427hr4P2ZTn7CyGtojezE4A/AIXADe7+8x2eLwFuAQ4CVgJnufvbmY0qkhyTZ9Twq6lzWbJqPT3LOnD58QM4ZXAFAGOuf45p89/7aNnD+3Xl9osOTeu1jT33/ckzmfDCIra4U2jG6EN68ZNT9t/pvOk839L3zlbmtPIWFodP43bfGzj+E++7asP7HLjLGi4eUsJhu22GNe/Ch0s/vl/0PKxZBrUb6g9oheEXw7Zbh7LU47Lwy6ddKZSUhs8IlJR+PN0CTV5eaWaFwBvAscBi4EVgtLvPrrPM14AD3P0rZjYK+KK7n9XY++rySmmrJs+o4cp7Z7J+88eHBToUF/KzU/fnruqF25X8NtvKvrHXAg0+V/3Oe9z2/MJPvO/Yz/ZusjgbW+cpgyuafL6l752tzNDwdtqZvPW+1h02rwvX/69bGa4MWrcS1qWmN6wOt/WrUo9XfTyvgV8Q9qMPMn8dvZkdClzt7senpq8M+f1ndZaZmlrmOTMrAt4Fyr2RN1fRS1t1+M+fpGbV+k/MryjrUO/8bd7++YmNvhZo8Ll3V29gSz0/joVmzP/ZiBbnnXbFsCafb+l7ZyszNLyddiZvU69tti2bw3j9m9bAxjWpxx9i/Y/NynX0FcCiOtOLgUMaWsbda81sNdANWFF3ITMbB4wD6N27d3NyiiTGkgbKvKH5O/vaJavW09AeV31F2tx1Zuu/J5uZm/vczrxvixUWh2P5Hbvu9Fvl9ItH3H28u1e5e1V5eXkuVy2SN3qm9irTnZ/uaxt7rrCBk34NzU93nek839L3zlbmbOXNZ+kUfQ3Qq850ZWpevcukDt10IZyUFZEdXH78ADoUF243r0NxIZcfP4DD+9W/97ZtfmOvbey50Yf0oj4NzU83bzrPt/S9s5U5W3nzWTqHbl4E+ptZH0KhjwLO3mGZKcC5wHPA6cCTjR2fF2nLtp20q++qj1MGVzR61U1jr92mofcFWnQFS1PrTCdTS7dFtjJnI28+S2tQMzMbAfyecHnlTe7+UzO7Bqh29ylm1h64FRgMvAeMcvcFjb2nTsaKiDRf1gY1c/eHgYd3mPeDOo83AGc0Z8UiIpIbOT0ZKyIiuaeiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqU1qFlWVmz2ITA3ysob1p0dviwlT+RjLmVKjzKlLx9z5WOmAe7erC+PTWtQsyyZ29wR2LLNzKrzLRPkZy5lSo8ypS8fc+Vrpua+RoduREQSTkUvIpJwMYt+fMR1NyQfM0F+5lKm9ChT+vIxVyIyRTsZKyIiuaFDNyIiCaeiFxFJuJwVvZkVmtkMM3swNd3HzF4ws3lmdqeZtctVliZy3Wxmb5nZy6nbgTnO87aZzUytuzo1r6uZPW5mb6bud82DTFebWU2d7TQix5nKzOxuM3vdzOaY2aGxt1MjuaJtKzMbUGe9L5vZB2Z2acxt1Uim2P+mLjOzWWb2mplNMLP2sXuqgUzN7qicHaM3s28BVcAu7n6SmU0C7nX3iWb2F+AVd78uJ2Eaz3Uz8KC7353rLKk8bwNV7r6izrxfAu+5+8/N7ApgV3f/buRMVwNr3P3XucqxQ6b/A/7l7jekfvg6Av9DxO3USK5Libit6mQrBGqAQ4CLibyt6sl0PpG2k5lVAM8Ag9x9faqfHgZGEKmnGsk0lGZ2VE726M2sEjgRuCE1bcAwYFvQ/wNOyUWWxnLlsZMJ2wgibat8YmZdgM8DNwK4+yZ3X0Xk7dRIrnxxNDDf3d8hf/5N1c0UWxHQwcyKCL+g/0P8ntox05KWvEmuDt38HvhvYGtquhuwyt1rU9OLgYocZWks1zY/NbNXzex3ZlaS40wOPGZm081sXGpeD3f/T+rxu0CPPMgEcElqO92U48MkfYDlwN8sHHa7wcw6EX87NZQL4m2rukYBE1KPY2+rbepmgkjbyd1rgF8DCwkFvxqYTsSeqi+Tuz+WerpZHZX1ojezk4Bl7j492+tqjkZyXQkMBD4DdAVy/efsEe4+BBgOXGxmn6/7pIdjbbm+Jra+TNcB/YADCf8If5PDPEXAEOA6dx8MrAWuqLtApO3UUK6Y2wqA1GGkkcBdOz4XaVvVlynadkr9UjmZ8Mu6J9AJOCFX6083k5mNpQUdlYs9+sOBkanjvBMJfwr9AShL/TkCUEk4TpdLn8hlZre5+3882Aj8DTg4l6FSv8Vx92XAfan1LzWzPQBS98tiZ3L3pe6+xd23AteT2+20GFjs7i+kpu8mFGzU7dRQrsjbapvhwEvuvjQ1HXtbfSJT5O10DPCWuy93983AvYSOiNlT9WU6rCUdlfWid/cr3b3S3fci/Jn2pLuPAZ4CTk8tdi5wf7azpJFrbJ1//EY4HvdarjKZWSczK932GDgutf4phG0EOd5WDWXatp1SvkgOt5O7vwssMrMBqVlHA7OJuJ0ayxVzW9Uxmu0PkUTdVinbZYq8nRYCnzWzjqmf/W3/pmL2VH2Z5rSoo9w9Zzc+PlsM0Bf4NzCP8KdbSS6zNJLrSWBmauPdBnTOYY6+wCup2yzge6n53YAngDeBvwNd8yDTrant9CqhNPbI8f+zA4Hq1PonA7vG3E5N5Iq9rToBK4EudeZF3VYNZIq9nX4EvJ762b8VKIndUw1kanZHaQgEEZGE0ydjRUQSTkUvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJp6IXEUm4/w8enGsxZVdAbgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w_val = model.get_weights()[0]\n",
    "b_val = model.get_weights()[1]\n",
    "plt.plot(data[:,0], data[:,1], 'o')\n",
    "plt.axis([40, 85, -0.1, 1.2])\n",
    "x_pred = np.linspace(40,85)\n",
    "x_pred = np.resize(x_pred,[len(x_pred),1])\n",
    "y_pred = 1 / (1 + np.exp(-x_pred*w_val - b_val))\n",
    "plt.plot(x_pred, y_pred)\n",
    "\n",
    "# predicted probabilities\n",
    "p_1 = 1 / (1 + np.exp(-x*w_val - b_val))\n",
    "\n",
    "# cross-entropy loss function\n",
    "cross_entropy = -np.mean(y * np.log(p_1) + (1-y) * np.log(1-p_1))\n",
    "print(\"Cross-entropy: \", cross_entropy)\n",
    "y_pred = np.round(p_1, decimals=0).astype('int')\n",
    "accuracy = np.mean(y==y_pred)\n",
    "print(\"Accuracy: \", accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1/1 [==============================] - 0s 118ms/step - loss: 0.4416 - accuracy: 0.8696\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.4416346251964569, 0.8695651888847351]"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The value of the cross-entropy loss function could be decreased from 0.9094435 to 0.4416346251964569"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 3 : MNIST and Multinomial Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise we use multinomial logistic regression to predict the number of the handwritten digits of the MNIST dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO : read MNIST data and compute validation accuracy for a multinomial logistic regression model, see [Multinomial Logistic Regression](https://en.wikipedia.org/wiki/Multinomial_logistic_regression)\n",
    "\n",
    "If there are several labels, then we use `categorical_crossentropy` as loss function and the output layer should be a `softmax` layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
      "11493376/11490434 [==============================] - 0s 0us/step\n",
      "11501568/11490434 [==============================] - 0s 0us/step\n",
      "Epoch 1/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 320.4593 - accuracy: 0.8417 - val_loss: 242.9400 - val_accuracy: 0.8801\n",
      "Epoch 2/10\n",
      "1875/1875 [==============================] - 8s 4ms/step - loss: 257.2356 - accuracy: 0.8691 - val_loss: 381.1338 - val_accuracy: 0.8126\n",
      "Epoch 3/10\n",
      "1875/1875 [==============================] - 8s 5ms/step - loss: 246.9380 - accuracy: 0.8741 - val_loss: 225.4031 - val_accuracy: 0.8802\n",
      "Epoch 4/10\n",
      "1875/1875 [==============================] - 8s 4ms/step - loss: 238.4442 - accuracy: 0.8784 - val_loss: 312.3720 - val_accuracy: 0.8513\n",
      "Epoch 5/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 238.5885 - accuracy: 0.8783 - val_loss: 195.5100 - val_accuracy: 0.9018\n",
      "Epoch 6/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 238.2655 - accuracy: 0.8773 - val_loss: 253.4705 - val_accuracy: 0.8795\n",
      "Epoch 7/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 237.9472 - accuracy: 0.8798 - val_loss: 250.7999 - val_accuracy: 0.8824\n",
      "Epoch 8/10\n",
      "1875/1875 [==============================] - 8s 5ms/step - loss: 237.2793 - accuracy: 0.8802 - val_loss: 285.5836 - val_accuracy: 0.8707\n",
      "Epoch 9/10\n",
      "1875/1875 [==============================] - 8s 4ms/step - loss: 235.1660 - accuracy: 0.8808 - val_loss: 277.7219 - val_accuracy: 0.8809\n",
      "Epoch 10/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 233.3158 - accuracy: 0.8822 - val_loss: 263.0470 - val_accuracy: 0.8765\n"
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "from tensorflow.keras.datasets import mnist\n",
    "from tensorflow.keras.utils import to_categorical\n",
    "\n",
    "\n",
    "# Import TensorFlow and TensorFlow Datasets\n",
    "import tensorflow as tf\n",
    "\n",
    "# Helper libraries\n",
    "import math\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Load MNIST data\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "\n",
    "# One-hot-encoded label vector\n",
    "y_train_cat = to_categorical(y_train, 10)\n",
    "y_test_cat = to_categorical(y_test, 10)\n",
    "\n",
    "model = tf.keras.Sequential()\n",
    "model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))\n",
    "model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax, batch_input_shape=(None, 784)))\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='sgd',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "history = model.fit(X_train, \n",
    "                    y_train_cat, \n",
    "                    epochs=10, \n",
    "                    validation_data=(X_test, y_test_cat))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 [==============================] - 1s 4ms/step - loss: 263.0470 - accuracy: 0.8765\n",
      "Accuracy on test dataset: 0.8765000104904175\n"
     ]
    }
   ],
   "source": [
    "test_loss, test_accuracy = model.evaluate(X_test, y_test_cat)\n",
    "print('Accuracy on test dataset:', test_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO : use different regularization terms, see [Keras Regularizer](https://keras.io/regularizers/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 338.3148 - accuracy: 0.8376 - val_loss: 217.6844 - val_accuracy: 0.8881\n",
      "Epoch 2/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 287.3254 - accuracy: 0.8573 - val_loss: 281.7627 - val_accuracy: 0.8726\n",
      "Epoch 3/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 288.3959 - accuracy: 0.8602 - val_loss: 320.5165 - val_accuracy: 0.8475\n",
      "Epoch 4/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 296.8785 - accuracy: 0.8590 - val_loss: 222.1539 - val_accuracy: 0.8899\n",
      "Epoch 5/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 283.6608 - accuracy: 0.8618 - val_loss: 460.9875 - val_accuracy: 0.8016\n",
      "Epoch 6/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 277.9357 - accuracy: 0.8639 - val_loss: 245.5682 - val_accuracy: 0.8875\n",
      "Epoch 7/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 287.6865 - accuracy: 0.8610 - val_loss: 252.3817 - val_accuracy: 0.8678\n",
      "Epoch 8/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 284.3436 - accuracy: 0.8620 - val_loss: 228.4689 - val_accuracy: 0.8884\n",
      "Epoch 9/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 282.4845 - accuracy: 0.8623 - val_loss: 258.1628 - val_accuracy: 0.8879\n",
      "Epoch 10/10\n",
      "1875/1875 [==============================] - 9s 5ms/step - loss: 285.5985 - accuracy: 0.8620 - val_loss: 484.2548 - val_accuracy: 0.7784\n"
     ]
    }
   ],
   "source": [
    "# Import TensorFlow and TensorFlow Datasets\n",
    "import tensorflow as tf\n",
    "\n",
    "# Helper libraries\n",
    "import math\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Load MNIST data\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "\n",
    "# One-hot-encode label vector\n",
    "y_train_cat = to_categorical(y_train, 10)\n",
    "y_test_cat = to_categorical(y_test, 10)\n",
    "\n",
    "# Define Network\n",
    "model = tf.keras.Sequential()\n",
    "model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))\n",
    "model.add(tf.keras.layers.Dense(10, \n",
    "                                activation=tf.nn.softmax, \n",
    "                                batch_input_shape=(None, 784), \n",
    "                                kernel_regularizer=tf.keras.regularizers.l2(0.01)))\n",
    "\n",
    "# Compile Network\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='sgd',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Fit Network\n",
    "history = model.fit(X_train, \n",
    "                    y_train_cat, \n",
    "                    epochs=10, \n",
    "                    validation_data=(X_test, y_test_cat))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 [==============================] - 1s 3ms/step - loss: 484.2548 - accuracy: 0.7784\n",
      "Accuracy on test dataset: 0.7784000039100647\n"
     ]
    }
   ],
   "source": [
    "# Evaluate Network\n",
    "test_loss, test_accuracy = model.evaluate(X_test, y_test_cat)\n",
    "print('Accuracy on test dataset:', test_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 4 : Prediction of House Prices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, we’ll attempt to predict the median price of homes in a given Boston\n",
    "suburb in the mid-1970s, given data points about the suburb at the time, such as the\n",
    "crime rate, the local property tax rate, and so on. The dataset has relatively few data points: only\n",
    "506, split between 404 training samples and 102 test samples. And each feature in the\n",
    "input data (for example, the crime rate) has a different scale. For instance, some values\n",
    "are proportions, which take values between 0 and 1, others take values between 1\n",
    "and 12, others between 0 and 100, and so on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the Boston housing dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/boston_housing.npz\n",
      "57344/57026 [==============================] - 0s 0us/step\n",
      "65536/57026 [==================================] - 0s 0us/step\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import boston_housing\n",
    "(train_data, train_targets), (test_data, test_targets) = (boston_housing.load_data())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s look at the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(404, 13)\n",
      "(102, 13)\n"
     ]
    }
   ],
   "source": [
    "print(train_data.shape)\n",
    "print(test_data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we have 404 training samples and 102 test samples, each with 13\n",
    "numerical features, such as per capita crime rate, average number of rooms per dwelling,\n",
    "accessibility to highways, and so on.\n",
    "The targets are the median values of owner-occupied homes, in thousands of dollars:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([15.2, 42.3, 50. , 21.1, 17.7, 18.5, 11.3, 15.6, 15.6, 14.4, 12.1,\n",
       "       17.9, 23.1, 19.9, 15.7,  8.8, 50. , 22.5, 24.1, 27.5, 10.9, 30.8,\n",
       "       32.9, 24. , 18.5, 13.3, 22.9, 34.7, 16.6, 17.5, 22.3, 16.1, 14.9,\n",
       "       23.1, 34.9, 25. , 13.9, 13.1, 20.4, 20. , 15.2, 24.7, 22.2, 16.7,\n",
       "       12.7, 15.6, 18.4, 21. , 30.1, 15.1, 18.7,  9.6, 31.5, 24.8, 19.1,\n",
       "       22. , 14.5, 11. , 32. , 29.4, 20.3, 24.4, 14.6, 19.5, 14.1, 14.3,\n",
       "       15.6, 10.5,  6.3, 19.3, 19.3, 13.4, 36.4, 17.8, 13.5, 16.5,  8.3,\n",
       "       14.3, 16. , 13.4, 28.6, 43.5, 20.2, 22. , 23. , 20.7, 12.5, 48.5,\n",
       "       14.6, 13.4, 23.7, 50. , 21.7, 39.8, 38.7, 22.2, 34.9, 22.5, 31.1,\n",
       "       28.7, 46. , 41.7, 21. , 26.6, 15. , 24.4, 13.3, 21.2, 11.7, 21.7,\n",
       "       19.4, 50. , 22.8, 19.7, 24.7, 36.2, 14.2, 18.9, 18.3, 20.6, 24.6,\n",
       "       18.2,  8.7, 44. , 10.4, 13.2, 21.2, 37. , 30.7, 22.9, 20. , 19.3,\n",
       "       31.7, 32. , 23.1, 18.8, 10.9, 50. , 19.6,  5. , 14.4, 19.8, 13.8,\n",
       "       19.6, 23.9, 24.5, 25. , 19.9, 17.2, 24.6, 13.5, 26.6, 21.4, 11.9,\n",
       "       22.6, 19.6,  8.5, 23.7, 23.1, 22.4, 20.5, 23.6, 18.4, 35.2, 23.1,\n",
       "       27.9, 20.6, 23.7, 28. , 13.6, 27.1, 23.6, 20.6, 18.2, 21.7, 17.1,\n",
       "        8.4, 25.3, 13.8, 22.2, 18.4, 20.7, 31.6, 30.5, 20.3,  8.8, 19.2,\n",
       "       19.4, 23.1, 23. , 14.8, 48.8, 22.6, 33.4, 21.1, 13.6, 32.2, 13.1,\n",
       "       23.4, 18.9, 23.9, 11.8, 23.3, 22.8, 19.6, 16.7, 13.4, 22.2, 20.4,\n",
       "       21.8, 26.4, 14.9, 24.1, 23.8, 12.3, 29.1, 21. , 19.5, 23.3, 23.8,\n",
       "       17.8, 11.5, 21.7, 19.9, 25. , 33.4, 28.5, 21.4, 24.3, 27.5, 33.1,\n",
       "       16.2, 23.3, 48.3, 22.9, 22.8, 13.1, 12.7, 22.6, 15. , 15.3, 10.5,\n",
       "       24. , 18.5, 21.7, 19.5, 33.2, 23.2,  5. , 19.1, 12.7, 22.3, 10.2,\n",
       "       13.9, 16.3, 17. , 20.1, 29.9, 17.2, 37.3, 45.4, 17.8, 23.2, 29. ,\n",
       "       22. , 18. , 17.4, 34.6, 20.1, 25. , 15.6, 24.8, 28.2, 21.2, 21.4,\n",
       "       23.8, 31. , 26.2, 17.4, 37.9, 17.5, 20. ,  8.3, 23.9,  8.4, 13.8,\n",
       "        7.2, 11.7, 17.1, 21.6, 50. , 16.1, 20.4, 20.6, 21.4, 20.6, 36.5,\n",
       "        8.5, 24.8, 10.8, 21.9, 17.3, 18.9, 36.2, 14.9, 18.2, 33.3, 21.8,\n",
       "       19.7, 31.6, 24.8, 19.4, 22.8,  7.5, 44.8, 16.8, 18.7, 50. , 50. ,\n",
       "       19.5, 20.1, 50. , 17.2, 20.8, 19.3, 41.3, 20.4, 20.5, 13.8, 16.5,\n",
       "       23.9, 20.6, 31.5, 23.3, 16.8, 14. , 33.8, 36.1, 12.8, 18.3, 18.7,\n",
       "       19.1, 29. , 30.1, 50. , 50. , 22. , 11.9, 37.6, 50. , 22.7, 20.8,\n",
       "       23.5, 27.9, 50. , 19.3, 23.9, 22.6, 15.2, 21.7, 19.2, 43.8, 20.3,\n",
       "       33.2, 19.9, 22.5, 32.7, 22. , 17.1, 19. , 15. , 16.1, 25.1, 23.7,\n",
       "       28.7, 37.2, 22.6, 16.4, 25. , 29.8, 22.1, 17.4, 18.1, 30.3, 17.5,\n",
       "       24.7, 12.6, 26.5, 28.7, 13.3, 10.4, 24.4, 23. , 20. , 17.8,  7. ,\n",
       "       11.8, 24.4, 13.8, 19.4, 25.2, 19.4, 19.4, 29.1])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The prices are typically between 10000 and 50000 USD. If that sounds cheap, remember\n",
    "that this was the mid-1970s, and these prices aren’t adjusted for inflation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparing the data\n",
    "\n",
    "It would be problematic to feed into a neural network values that all take wildly different ranges. The model might be able to automatically adapt to such heterogeneous data, but it would definitely make learning more difficult. A widespread best practice for dealing with such data is to do feature-wise normalization: for each feature in the input data (a column in the input data matrix), we subtract the mean of the feature and divide by the standard deviation, so that the feature is centered around 0 and has a unit standard deviation. This is easily done in NumPy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalizing the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = train_data.mean(axis=0)\n",
    "train_data -= mean\n",
    "std = train_data.std(axis=0)\n",
    "train_data /= std\n",
    "test_data -= mean\n",
    "test_data /= std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the quantities used for normalizing the test data are computed using the\n",
    "training data. You should never use any quantity computed on the test data in your\n",
    "workflow, even for something as simple as data normalization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building your model\n",
    "\n",
    "Because so few samples are available, we’ll use a very small model with two intermediate layers, each with 64 units. In general, the less training data you have, the worse overfitting will be, and using a small model is one way to mitigate overfitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model definition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_model():\n",
    "    model = keras.Sequential([\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(64, activation=\"relu\"),\n",
    "    layers.Dense(1)\n",
    "    ])\n",
    "    model.compile(optimizer=\"rmsprop\", loss=\"mse\", metrics=[\"mae\"])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model ends with a single unit and no activation (it will be a linear layer). This is a\n",
    "typical setup for scalar regression (a regression where you’re trying to predict a single\n",
    "continuous value). Applying an activation function would constrain the range the output\n",
    "can take; for instance, if you applied a sigmoid activation function to the last layer,\n",
    "the model could only learn to predict values between 0 and 1. Here, because the last\n",
    "layer is purely linear, the model is free to learn to predict values in any range.\n",
    "Note that we compile the model with the `mse` loss function — _mean squared error_, the\n",
    "square of the difference between the predictions and the targets. This is a widely used\n",
    "loss function for regression problems.\n",
    "We’re also monitoring a new metric during training: _mean absolute error_ (`MAE`). It’s the\n",
    "absolute value of the difference between the predictions and the targets. For instance, an\n",
    "MAE of 0.5 on this problem would mean your predictions are off by 500 on average."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Validating your approach using K-fold validation\n",
    "\n",
    "To evaluate our model while we keep adjusting its parameters (such as the number of\n",
    "epochs used for training), we could split the data into a training set and a validation set, as we did in the previous examples. But because we have so few data points, the validation set would end up being very small (for instance, about 100 examples). As a consequence, the validation scores might change a lot depending on which data points we chose for validation and which we chose for training: the validation scores might have a high variance with regard to the validation split. This would prevent us from reliably evaluating our model.\n",
    "\n",
    "The best practice in such situations is to use $K$-fold cross-validation. It consists of splitting the available data into K partitions (typically $K = 4$ or $5$), instantiating $K$ identical models, and training each one on $K - 1$ partitions while evaluating on the remaining partition. The validation score for the model used is then the average of the $K$ validation scores obtained. In terms of code, this is straightforward."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### K-fold validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing fold #0\n",
      "Processing fold #1\n",
      "Processing fold #2\n",
      "Processing fold #3\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "k = 4\n",
    "num_val_samples = len(train_data) // k\n",
    "num_epochs = 100\n",
    "all_scores = []\n",
    "for i in range(k):\n",
    "    print(f\"Processing fold #{i}\")\n",
    "    # Prepares the validation data: data from partition k\n",
    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "    # Prepares the training data: data from all other partitions\n",
    "    partial_train_data = np.concatenate([train_data[:i * num_val_samples], train_data[(i + 1) * num_val_samples:]],axis=0)\n",
    "    partial_train_targets = np.concatenate([train_targets[:i * num_val_samples], train_targets[(i + 1) * num_val_samples:]],axis=0)\n",
    "    # Builds the Keras model (already compiled)\n",
    "    model = build_model()\n",
    "    # Trains the model (in silent mode, verbose=0)\n",
    "    history=model.fit(partial_train_data, partial_train_targets, epochs=num_epochs, batch_size=16, verbose=0)\n",
    "    val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
    "    all_scores.append(val_mae)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running this with `num_epochs = 100` yields the following results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1.9184445142745972,\n",
       " 2.4037296772003174,\n",
       " 2.4944815635681152,\n",
       " 2.4431681632995605]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.3149559795856476"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(all_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The different runs do indeed show rather different validation scores, from 1.9 to 2.49.\n",
    "The average (2.3) is a much more reliable metric than any single score—that’s the\n",
    "entire point of K-fold cross-validation. In this case, we’re off by 2310 USD on average, which is significant considering that the prices range from 10000 to 50000.\n",
    "Let’s try training the model a bit longer: 500 epochs. To keep a record of how well\n",
    "the model does at each epoch, we’ll modify the training loop to save the per-epoch\n",
    "validation score log for each fold."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Saving the validation logs at each fold"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing fold #0\n",
      "Processing fold #1\n",
      "Processing fold #2\n",
      "Processing fold #3\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 500\n",
    "all_mae_histories = []\n",
    "for i in range(k):\n",
    "    print(f\"Processing fold #{i}\")\n",
    "    # Prepares the validation data: data from partition #k\n",
    "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
    "    # Prepares the training data: data from all other partitions\n",
    "    partial_train_data = np.concatenate([train_data[:i * num_val_samples], train_data[(i + 1) * num_val_samples:]],axis=0)\n",
    "    partial_train_targets = np.concatenate([train_targets[:i * num_val_samples], train_targets[(i + 1) * num_val_samples:]],axis=0)\n",
    "    # Builds the Keras model (already compiled)\n",
    "    model = build_model()\n",
    "    # Trains the model (in silent mode, verbose=0)\n",
    "    history = model.fit(partial_train_data, partial_train_targets,\n",
    "    validation_data=(val_data, val_targets), epochs=num_epochs, batch_size=16, verbose=0)\n",
    "    mae_history = history.history[\"val_mae\"]\n",
    "    all_mae_histories.append(mae_history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then compute the average of the per-epoch MAE scores for all folds."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Building the history of successive mean K-fold validation scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "average_mae_history = [\n",
    "np.mean([x[i] for x in all_mae_histories]) for i in range(num_epochs)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plotting validation scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnk0lEQVR4nO3de3xcdZ3/8ddnLrlfmjZJG5peKC0gtNBC5CZiQUGoKCyyKrouKrtV1/2tPrysuBfZdS8/XRV3+YGwiCywKt4Q8cICXQSKgEALbSmX3lua9JKmSdqkSXObz++PcyadJJNpSDOZkr6fj8c85sz3nDnne9LpvOd7Lt+vuTsiIiKDRXJdAREROTopIEREJC0FhIiIpKWAEBGRtBQQIiKSlgJCRETSylpAmNkMM3vMzF4xs5fN7LNh+WQzW2ZmG8LnimHef224zAYzuzZb9RQRkfQsW/dBmFkNUOPuL5hZKbASuBL4GNDs7l83s+uBCnf/8qD3TgZWAHWAh+89091bslJZEREZImstCHff6e4vhNNtwKvAdOAK4O5wsbsJQmOwdwPL3L05DIVlwKXZqquIiAwVG4+NmNlsYBHwLDDV3XeGs3YBU9O8ZTqwPeV1fViWUWVlpc+ePfuI6ioicixZuXJlk7tXpZuX9YAwsxLgPuBz7r7fzPrnubub2REd4zKzpcBSgJkzZ7JixYojWZ2IyDHFzLYNNy+rVzGZWZwgHH7o7r8Ii3eH5yeS5yka07y1AZiR8ro2LBvC3W939zp3r6uqShuCIiIyCtm8ismA7wOvuvuNKbN+BSSvSroWeCDN2x8GLjGzivAqp0vCMhERGSfZbEG8DfgocJGZrQofS4CvAxeb2QbgXeFrzKzOzO4AcPdm4J+A58PH18IyEREZJ1m7zDUX6urqXOcgRERGzsxWuntdunm6k1pERNJSQIiISFoKCBERSUsBAdz06AaeWL8n19UQETmqKCCAWx/fxFMbm3JdDRGRo4oCAohGjN6+iXM1l4jIWFBAEAREYgJd7isiMhYUEIQtiEQi19UQETmqKCAIAqJP+SAiMoACAoia0acWhIjIAAoI1IIQEUlHAUEyIJQQIiKpFBBALGLoKlcRkYEUEEBELQgRkSEUEIQtiISaECIiqRQQQMQUECIigykggFhUASEiMpgCgqAF0auAEBEZQAFBcA5CfTGJiAykgCC4ikm9uYqIDBTL1orN7E7gcqDR3eeHZT8BTgoXmQS0uvvCNO/dCrQBfUDvcANqj5VYxOjRrdQiIgNkLSCAu4CbgXuSBe7+weS0mX0b2Jfh/Re6+7iM4hONGJ09akGIiKTKWkC4+3Izm51unpkZ8AHgomxt/42IRoyETlKLiAyQq3MQbwd2u/uGYeY78IiZrTSzpdmuTFRXMYmIDJHNQ0yZXAPcm2H++e7eYGbVwDIze83dl6dbMAyQpQAzZ84cVWWiupNaRGSIcW9BmFkMuAr4yXDLuHtD+NwI3A+clWHZ2929zt3rqqqqRlUnBYSIyFC5OMT0LuA1d69PN9PMis2sNDkNXAKszWaFohGjT/dBiIgMkLWAMLN7gWeAk8ys3syuC2d9iEGHl8zsODN7MHw5Ffi9ma0GngN+6+4PZaueoBaEiEg62byK6Zphyj+WpmwHsCSc3gycnq16paOAEBEZSndSkxyTWgEhIpJKAYF6cxURSUcBgcaDEBFJRwFBckxqBYSISCoFBMkxqRUQIiKpFBBoTGoRkXQUEKgFISKSjgICtSBERNJRQBDeB6GT1CIiAygggGgkgjsaE0JEJIUCAoiGfwW1IkREDlFAELQgAJ2HEBFJoYAgpQWhgBAR6aeA4FALQsOOiogcooAAohY86yS1iMghCgggGlULQkRkMAUEwX0QAAldxSQi0k8BQXAnNagFISKSSgFB0BcT6ByEiEgqBQRqQYiIpKOA4FALQvdBiIgckrWAMLM7zazRzNamlP2DmTWY2arwsWSY915qZuvMbKOZXZ+tOibFFBAiIkNkswVxF3BpmvLvuPvC8PHg4JlmFgVuAS4DTgGuMbNTslhPIqaAEBEZLGsB4e7LgeZRvPUsYKO7b3b3buDHwBVjWrlB1IIQERkqF+cg/tLM1oSHoCrSzJ8ObE95XR+WpWVmS81shZmt2LNnz6gqFE0GhO6DEBHpN94BcStwArAQ2Al8+0hX6O63u3udu9dVVVWNah39AZFIHGl1REQmjHENCHff7e597p4AvkdwOGmwBmBGyuvasCxrDgVENrciIvLmMq4BYWY1KS//CFibZrHngXlmdryZ5QEfAn6VzXpF+++DUEKIiCTFsrViM7sXWAxUmlk9cAOw2MwWAg5sBT4ZLnsccIe7L3H3XjP7S+BhIArc6e4vZ6uecCgglA8iIodkLSDc/Zo0xd8fZtkdwJKU1w8CQy6BzRa1IEREhtKd1Kg3VxGRdBQQpLQg+hQQIiJJCghSzkGoBSEi0k8BgXpzFRFJRwGBenMVEUlHAYH6YhIRSUcBgXpzFRFJRwEBxKIKCBGRwRQQHLoPQr25iogcMmxAmNlPU6a/MWjeI9ms1HiL6hyEiMgQmVoQ81KmLx40b3T9ah+lFBAiIkNlCohM35YT6ptUASEiMlSmzvqKzGwRQYgUhtMWPgrHo3LjRQEhIjJUpoDYCdwYTu9KmU6+njCSl7nqTmoRkUOGDQh3v3C4eWYWz051ciPWPx6EAkJEJGnEl7la4J1m9n2gPot1GndR9cUkIjLEYQPCzM4xs5uAbcADwHLg5GxXbDyZGRFTb64iIqky3Qfxr2a2AfgXYA2wCNjj7ne7e8t4VXC8RCOmFoSISIpMJ6n/DFgP3Ar82t27zGzCfoNGI6ZzECIiKTIdYqoB/hl4L7DJzP6b4HLXrI1jnUtRUwtCRCRVpquY+oCHgIfMLB+4nOD+hwYze9TdP5xpxWZ2Z/ieRnefH5Z9kyBwuoFNwMfdvTXNe7cCbUAf0OvudW98196YaMR0H4SISIoRXcXk7l3ufp+7Xw3MJQiOw7kLuHRQ2TJgvrufRnD46isZ3n+huy8cj3CA8BCTTlKLiPQbtgVhZp8/khW7+3Izmz2oLLWTvz8AVx/JNsZSNBLRISYRkRSZWhDfAv4EmAKUAKUpj5Ix2PYngP8ZZp4Dj5jZSjNbmmklZrbUzFaY2Yo9e/aMujLRiG6UExFJlemE8yLgGuA9wErgXuBR9yM/DmNmfwv0Aj8cZpHz3b3BzKqBZWb2mrsvT7egu98O3A5QV1c36rrF1IIQERlg2BaEu6929+vdfSHwfeAK4BUze9+RbNDMPkZw8vojw4WNuzeEz43A/cBZR7LNkYioBSEiMsBI7qSuImhNLCDoYqNxtBszs0uBvwbe5+4dwyxTbGalyWngEmDtaLc5UmpBiIgMlOkk9SeADwAFwM+BD4S/6EfEzO4FFgOVZlYP3EBw1VI+wWEjgD+4+6fM7DjgDndfAkwF7g/nx4AfuftIrpo6IhHTkKMiIqkynYO4g+CX+zbg3cAl4Zc2AO6e8VCTu1+Tpvj7wyy7A1gSTm8GTs9Y6yyIRSL09SkgRESSMgXEsN19T0SRiKkFISKSItOd1E+MZ0VyLaY7qUVEBhjxeBATXUQBISIygAIipBaEiMhACohQ1BQQIiKpDtt1t5mdCHwJmJW6vLtflMV6jTv15ioiMtBIxnb4GXAb8D2C7rcnpGjE6O5L5LoaIiJHjZEERK+735r1muSYhhwVERloJOcgfm1mf2FmNWY2OfnIes3GmYYcFREZaCQtiGvD5y+llDkwZ+yrkztqQYiIDHTYgHD348ejIrkWNbUgRERSjeQqpjjwaeCCsOhx4D/dvSeL9Rp30ajRm9BJahGRpJEcYroViAPfDV9/NCz7s2xVKheiZqgBISJyyEgC4q3untq76u/MbHW2KpQrsYhaECIiqUZyFVOfmZ2QfGFmc5iA90NEIobyQUTkkJG0IL4EPGZmmwEjuKP641mtVQ6oBSEiMtBIrmJ61MzmASeFRevcvSu71Rp/QW+uua6FiMjRI9OQoxe5++/M7KpBs+aaGe7+iyzXbVwFvbkqIUREkjK1IN4B/A54b5p5DkyogIioN1cRkQEyjSh3Qzj5NXffkjrPzCbczXMaD0JEZKCRXMV0X5qyn49k5WZ2p5k1mtnalLLJZrbMzDaEzxXDvPfacJkNZnZtumXGUlRjUouIDDBsQJjZyWb2fqDczK5KeXwMKBjh+u8CLh1Udj3wqLvPAx4NXw/e9mTgBuBs4CzghuGCZKxoPAgRkYEynYM4CbgcmMTA8xBtwJ+PZOXuvtzMZg8qvgJYHE7fTdB1x5cHLfNuYJm7NwOY2TKCoLl3JNsdDQWEiMhAmc5BPAA8YGbnuvszY7jNqe6+M5zeBUxNs8x0YHvK6/qwLGuikaCrDXfHzLK5KRGRN4WR3Cj3opl9BjiVlENL7v6JI924u7uZHdHPdjNbCiwFmDlz5qjXEw1DoS/hxKIKCBGRkZyk/m9gGsFhnyeAWoLDTKO128xqAMLnxjTLNAAzUl7XhmVDuPvt7l7n7nVVVVWjrlQ0DAWNCSEiEhhJQMx1978HDrj73cB7CE4ej9avODQI0bXAA2mWeRi4xMwqwpPTl4RlWZNsQSR0JZOICDCygEiO+9BqZvOBcqB6JCs3s3uBZ4CTzKzezK4Dvg5cbGYbgHeFrzGzOjO7AyA8Of1PwPPh42vJE9bZEo2oBSEikmok5yBuD3/F/z3Br/8S4KsjWbm7XzPMrHemWXYFKWNMuPudwJ0j2c5YSAaERpUTEQmMpLO+O8LJJ5hg41CniqkFISIyQKbO+j6f6Y3ufuPYVyd3ImpBiIgMkKkFURo+nwS8leDwEgQ3zT2XzUrlgloQIiIDZbpR7h8BzGw5cIa7t4Wv/wH47bjUbhxFUu6DEBGRkV3FNBXoTnndTfq7n9/UkjfHKSBERAIjuYrpHuA5M7s/fH0lQSd8E0p/C0L3QYiIACO7iulfzOx/gLeHRR939xezW63xF4sEjSm1IEREApmuYipz9/1h19tbw0dy3uRs37g23qLhwTYFhIhIIFML4kcE3X2vJBhiNMnC1xPqnoioWhAiIgNkuorp8vB5wg0vmo5aECIiA2U6xHRGpje6+wtjX53c6W9B6CS1iAiQ+RDTtzPMc+CiMa5LTkV1H4SIyACZDjFdOJ4VybVkZ30KCBGRwEjugyDs5vsUBo4od0+2KpULCggRkYEOGxBmdgOwmCAgHgQuA35PcAPdhKGAEBEZaCRdbVxNMH7DLnf/OHA6waBBE4oCQkRkoJEERKe7J4BeMysjGEN6xmHe86YTU0CIiAwwknMQK8xsEvA9gpvm2gmGEZ1Qkn0xqbtvEZFApvsgbgF+5O5/ERbdZmYPAWXuvmZcajeOkr25JnQfhIgIkLkFsR74lpnVAD8F7p2InfQlqQUhIjLQsOcg3P0/3P1c4B3AXuBOM3vNzG4wsxNHu0EzO8nMVqU89pvZ5wYts9jM9qUs89XRbm+kYhpyVERkgJF0970N+AbwDTNbBNwJfBWIjmaD7r4OWAhgZlGgAbg/zaJPJvuDGg/Jq5h6+hLjtUkRkaPaYa9iMrOYmb3XzH4I/A+wDrhqjLb/TmBTGEI5lR8L/hQ9fWpBiIhA5pPUFwPXAEuA54AfA0vd/cAYbv9DwL3DzDvXzFYDO4AvuvvLY7jdIfLCgOjq7cvmZkRE3jQyHWL6CsGYEF9w95ax3rCZ5QHvC7cz2AvALHdvN7MlwC+BecOsZymwFGDmzJmjrk8yILp7dYhJRAQyn6S+yN3vyEY4hC4DXnD33Wm2vd/d28PpB4G4mVUOU8/b3b3O3euqqqpGXZm8aLIFoYAQEYGR3UmdLdcwzOElM5tmFlx3amZnEdRzbzYrE4tGiEZMLQgRkdCIenMda2ZWDFwMfDKl7FMA7n4bQf9PnzazXqAT+JB79u9gy49FdA5CRCSUk4AIT3RPGVR2W8r0zcDN412vvFhELQgRkVAuDzEddfKiEZ2DEBEJKSBS5MfVghARSVJApFALQkTkEAVEivxYVAEhIhJSQKTI01VMIiL9FBApdBWTiMghCogU+bEI3erNVUQEUEAMkB+L0NWjgBARAQXEAPmxqFoQIiIhBUQKnaQWETlEAZEiL6qT1CIiSQqIFPlx3SgnIpKkgEihFoSIyCEKiBT58QgHe/oYh57FRUSOegqIFEV5MRKuUeVEREABMUBxXhSAA129Oa6JiEjuKSBSFOcH4yd1dOtSVxERBUSKkjAg2tWCEBFRQKQq6m9BKCBERBQQKUryg3MQ7V06xCQikrOAMLOtZvaSma0ysxVp5puZ3WRmG81sjZmdke06FeUFLQidpBYRgViOt3+huzcNM+8yYF74OBu4NXzOmuQ5CAWEiMjRfYjpCuAeD/wBmGRmNdncYLECQkSkXy4DwoFHzGylmS1NM386sD3ldX1YljVFyfsgdJmriEhODzGd7+4NZlYNLDOz19x9+RtdSRguSwFmzpx5RBXKj0WIRUwtCBERctiCcPeG8LkRuB84a9AiDcCMlNe1Ydng9dzu7nXuXldVVXVEdTIzivKiCggREXIUEGZWbGalyWngEmDtoMV+BfxpeDXTOcA+d9+Z7bpNKsqjpaMn25sRETnq5eoQ01TgfjNL1uFH7v6QmX0KwN1vAx4ElgAbgQ7g4+NRscqSPJrau8ZjUyIiR7WcBIS7bwZOT1N+W8q0A58Zz3oBVJbks3XvgfHerIjIUedovsw1JypL89nb3p3raoiI5JwCYpDKknyaO7rp7dOYECJybFNADFJVkoc7NHeoFSEixzYFxCCVJfkA7GnTiWoRObYpIAY5blIhAPUtnTmuiYhIbikgBpk1pQiA7c0dOa6JiEhuKSAGmVSUR1lBjG17FRAicmxTQKQxa0ox29SCEJFjnAIijTlVxWzc3ZbraoiI5JQCIo0F08vZse+grmQSkWOaAiKNBdPLAXipoTW3FRERySEFRBrzp5eTF4uwfP1wo6GKiEx8Cog0ivNjvPPkan6zZqe63BCRY5YCYhhXLDyOpvYunt60N9dVERHJCQXEMBafVE1pQYz7XqjPdVVERHJCATGMgniU959Ry2/X7GTXvoO5ro6IyLhTQGRw3fnHk3Dnrqe35roqIiLjTgGRwYzJRSxZUMOdv9/Cdx/fSDDInYjIsUEBcRj/fOV85k8v498eWsfDL+/OdXVERMaNAuIwJhXl8dNPnssJVcV86WerWVPfmusqiYiMCwXECMSiEe7+xFmUFcZ5381PceUtT7F8/Z5cV0tEJKvGPSDMbIaZPWZmr5jZy2b22TTLLDazfWa2Knx8dbzrOVhtRRE3f3gRsYixansrf3bPCh55eRe/XbOTf/z1y7zwegvurvMUIjJh2Hh/oZlZDVDj7i+YWSmwErjS3V9JWWYx8EV3v/yNrLuurs5XrFgxltUdYntzBy0d3Xz4e8/S3tXbXx6NGAuml9O4/yC3fOQMTqgu4ZsPreOyBdM4d84UEh4sM5zWjm5KC+IZlxGRI+PumI39/7G97V30uVNdWgDAr1fv4Ow5k6kuLWBvexeTivLS/t/e19lDeWGcHz67jdqKIk49royygjjxqNHVm6AgHk27vZYD3axp2Ed9SwdXLaolFjXi0dH93jezle5el3Zern/xmtkDwM3uviylbDFHaUAk7T/Yw42PrOeUmjLeevxkLvzW4/3zygpiRCJGa0cPAFOK8+jqTfDpxSfw+LpG+hJOS0cPX79qAfFYhBVbm/nXB1/jU+84gesvOxmARMIxY8iHebgPeHtXL109fbz4eivvfEv1gGV6+xL8es0OliyoIT+W/gOXXPfgD2VTexd72rp4S03ZqP5ObyY793Xy4Eu7+MTbZmflS2S0jvRL7cs/X0NRfpQb3nsqQPj566a8MD7gS8Xd2d7cycxwVMVM9nX2sG5XG2cdP7m/bNe+g+THIlQU57G3vYvJxXm0d/VyoKuPaeUFQ9bx+w1N5MUi9PYlOHvOFKIR476V9dzzh218+49Po6a8kIfW7qK1s4c/WjSdgz19VBTlkR+L8HpzB7Mri/vXtXz9Hm5ctp6vXXEqc6pKePH1FuZVl/KLF+u5YF4VRXlRmtq7ue6u57nrE2cxf3oZD6zawbtPnUZZQYwXXm/l2S17mVZWwNzqEr6zbD0VxXnMnlJMWUGM0oI4sagxe0oxP1mxnSsXTufxdY3UTCrkqQ1N/O+ruynOj/Gx82azbe8BfrlqB+WFceZVl7BiWwsAf3xmLe86ZSoXzKviifV7+Jv7X6L5QDcnTyvltV3B8AIF8QgVRXnMqChi7Y59LL1gDq0dPUyfVMiahn2UF8Y4bfokvvv4RramDGp25qwKfvrJc0f1A/OoDQgzmw0sB+a7+/6U8sXAfUA9sIMgLF4eZh1LgaUAM2fOPHPbtm3ZrfQwHl/XCEB+LMrfP7CWrU0H+OtLT2LdrnY6e3p58KVdI1rPnKpiNu85QEl+jNKCGGcdP5llr+ymujSfBbWTeHxdI+9ZUMNp4fTOfQd5+7xKvvv4pv51fPPq06gszef1vR288y3V3PLYJu597nU+es4s3jZ3Ck+sb6K8MM5J00ooK4iTH4syu7KIB1bt4JsPr2P1Vy+hvCgOwKX/vpzXdrWx4u/eRV4swmOvNbK1KWhFXXLKVM6bWzmg/vs6e3jstUYW1JYzp7KYLU0HSDjMrS4B4GBP35BfRWsb9jFzShGl+bHDfhFu2N1Gd1+CU48Letx1d3btP8i0sgK+/ch6aisK+dBZM9O+96G1uyjJjzG1LJ+51SXs3t/F1LJ8Nu05wMbGdm56dAOv7NzPg3/1dk45roy2g0HAl+THeGXnfp7d3ExeLMJHzp5JfUsnj7yye0iY9PQliJpx19NbeXpTEzd/+Az2tHXx2q42Fp9URTwa4ZUd+/niz1bzkXNmMntKMXmxCI++2sjnLz6ReNT41eodbNvbQWlBjNlTivnCz1Zz2fxptB3s5TMXzuXvfvkSJ08r42/f8xYK4lF27z/I2oZ9lBXGueWxjRTnxZgxuYjl6/ewfncbvYng//g7Tqzi4lOm8uPnX2dtQ/9/N75w8YkU58f42m+CRvzb5x36Ny0vjLNgejkXnzKVlxr2cevjm0i4s353OwDvO/045lWX8MT6PazY1kJpfoxzT5jCsld3E49G6O4N+jLLj0X4k3Nm8fDLu/pD5RcvNPRvp6IozoGuPrrDvs+K86Ic6O4b8m8YMaguLWDX/oMsPqmK7t4Eu/cfZNOeA/3L5McidPUO34daYTzKidNKWb29lfxYhCnFeewYdDNsxCDxBr4aT6stZ0drJ03t3YddNh41evre+PduNGL0hZXKj0V4/5m1HD+lmG8+so4P1NXyd+85ZdgWRyZHZUCYWQnwBPAv7v6LQfPKgIS7t5vZEuA/3H3e4dY5ni2Iw+ntSxBL+XX21MYmmg90U1IQY39nD1Wl+aza3kpeNMJj6xp58fVWDvb09X8o82MRygvjNLZ1BV+0ew/g4WGqqFn/f6RUVyw8jgdW7RhR/VI/bINdtWg6J00rpc+df3toXcb1zJ9extyqEuZNLWX97jbW7Wrr/zV05qwKVoa/ni4/rQaAZa/s5pMXzKG0IM663W3c90I9qR/Bq86YzpmzKvjliw2UF8bp6k0Qixibmw4wc3IRT25oIh41pk8qpLq0gNbObtbvbh+wrcnFefxxXS2v7+1gT1sX58+r5LWdbTz08qGQnj2liK17O5g+qZCG1s6M+7hgejlbmw7QFh5SnDWlaMCQtCdUFXPytDIOdPfy3JZmOlK+2E6rLWf97jYO9iT6l039Mks1qSjOlOK8YeenWz7ZSs1kbnUJGxvbR7TOpPLCOPs6D607+YUZixi9Cedtc6fQ0NJJfUsnvQmnrCDGtPICtu3toKs3wbzqEiqK8nhua/OA9RbGo3T29JEXjfR/hi8/rYanN+2l+UA3b6kp45tXn8a3HlnH4+v2UJof40/OncUP/rCNC+ZVUVYY48XXW/s/YzMmF3J8ZQmxiHHi1FJue2ITZ86qoLIkj6b2bq49bzb//r/rcQ9a9jXlhcRjEZ7f0kxxfpTmA9040NrRw1VnTGfJ/Bp27OvkopOreWpjE4tPqqa+pYPntrSQF4tw5++3UFEc5x0nVrFkQQ2PvLybZzbv5QfXnU3bwR5WbW+ltqKIiMEvVzUwa3Ix7z+zlo2N7cypKua5Lc08uaGJ3r4Ef37BHOpbOti9v4tLT53GPc9sJRoxOrr7yItF2LnvIB3dvbx9XhXxqDGvurQ//KrL8vuPBgz+rnmjjrqAMLM48BvgYXe/cQTLbwXq3D1j/9tHU0CMRl/CeXpTE/FohHnVJUQjxsbG4Mtve3MnZYUxygvj1Ld08sT6PUTMKC+M09zRzaIZk5g/vZwv/mw1q7e3MruymNqKQtbvbmPb3g4+es4sXmrYx/L1e/jw2bP44FtncO9zr1OaH+P7T22hs7uPOVUltB3sob7l0BfmiVNLmFNZ0v/l+k9XzufKhccB8L3lm1ldv4/V9a39X1R50QhXLjqOaMRYsbWFaMTYtreDg73Bl8Jwv+wWzpjEqu2tA8rKC+O0HewZ8Etu9pQiTqgqIeHO1r0dbGkKvkyjEeMDdTOob+ngyQ3pPyZvqSnj+MoitjR18OrO4Bf0tLIC3n/mdKZPKuLOp7b0f5EW5UXp6O5j4YxJxCJGnzsfO282P35uO89sPtSB43knTKEv4by8Yz/lhXHeNncK5YVxZk4ppqmti/96agvnnVDJ5qZ2mg/08JaaUhpaOvn2B06ntaOHL/5sNW0He7n2vFm0dPSwp62Lk6eV0tDayczJRUwuzuPS+dN4dWcbNeUFLN+whxOrS4nHIvzfB1/ltV1tXLVoOufPq2RL0wHee3rwt9/adICu3gRlBUGdHnxpF+eeEHypTy3P5wfPbKOiOI+3za1kY2M7JfkxWjq6eevsyZQUxCgriHPHk5tpPtDNBSdW8eSGPXT1JPjE+cezbW8HZx0/mWjEaO/qZV9ncAgEoL6lg6c2NvFHi2rJiwVfWk3tXVQU5dHTFxy+3Np0gMK8KNWl+WxuOsAJVSV09yaIRw13iISHSV6q38fc6hIK86J0hZ+fZGutpy/Bs5ubWTRzEsX5sf5/j9f3dlBbUdi/DoDu3gS9iQRFeYeWS+pL+Bs6LJP8vjyaDkEeqaMqICz4y94NNLv754ZZZhqw293dzM4Cfg7M8sNU9s0eEGNhuHMXb0RrRzddvQncYWpZPgmHDY1tnDwt/XmIju5etu3t4MSppfQmEkPOc/QlnN5EgrxohMa2Lpa9spt3nFjF1LICdrR2UpgXpaokn9ebOyjOj/H0pibOO6GSycV57O8MAqvpQBe1kwqZFR6SSTrYk/ziCPY52Sra0drJf/9hG+87/The2bmfS06ZSnlhHDOjuzdBY9tBaiuGHmvf3txBfjzCpMI84lFL+3fsSzjrdrUxb2rJiE8M9vQFLaHB6+vo7qUv4ZQWxEe0nsHr3NfZQ2VJ/ht+r0jS0RYQ5wNPAi8ByZ+TfwPMBHD328zsL4FPA71AJ/B5d3/6cOtWQIiIvDGZAmJomyvL3P33QMaft+5+M3Dz+NRIRETS0Z3UIiKSlgJCRETSUkCIiEhaCggREUlLASEiImkpIEREJC0FhIiIpJXz3lzHkpntAUbTW18lkLEbjwlI+3xs0D4fG45kn2e5e1W6GRMqIEbLzFYMdyfhRKV9PjZon48N2dpnHWISEZG0FBAiIpKWAiJwe64rkAPa52OD9vnYkJV91jkIERFJSy0IERFJ65gPCDO71MzWmdlGM7s+1/UZK2Z2p5k1mtnalLLJZrbMzDaEzxVhuZnZTeHfYI2ZnZG7mo+Omc0ws8fM7BUze9nMPhuWT9h9BjCzAjN7zsxWh/v9j2H58Wb2bLh/PzGzvLA8P3y9MZw/O6c7MEpmFjWzF83sN+HrCb2/EIysaWYvmdkqM1sRlmX1831MB4SZRYFbgMuAU4BrzOyU3NZqzNwFXDqo7Hrg0XB870fD1xDs/7zwsRS4dZzqOJZ6gS+4+ynAOcBnwn/LibzPAF3ARe5+OrAQuNTMzgG+AXzH3ecCLcB14fLXAS1h+XfC5d6MPgu8mvJ6ou9v0oXuvjDlktbsfr7d/Zh9AOcSjIudfP0V4Cu5rtcY7t9sYG3K63VATThdA6wLp/8TuCbdcm/WB/AAcPExts9FwAvA2QQ3TcXC8v7POfAwcG44HQuXs1zX/Q3uZ234ZXgRwdj2NpH3N2W/twKVg8qy+vk+plsQwHRge8rr+rBsoprq7jvD6V3A1HB6Qv0dwsMIi4BnOQb2OTzcsgpoBJYBm4BWd+8NF0ndt/79DufvA6aMa4WP3L8Df82hIYunMLH3N8mBR8xspZktDcuy+vke9yFH5ejg7m5mE+4SNjMrAe4DPufu+80OjW47UffZ3fuAhWY2CbgfODm3NcoeM7scaHT3lWa2OMfVGW/nu3uDmVUDy8zstdSZ2fh8H+stiAZgRsrr2rBsotptZjUA4XNjWD4h/g5mFicIhx+6+y/C4gm9z6ncvRV4jOAQyyQzS/4ATN23/v0O55cDe8e3pkfkbcD7zGwr8GOCw0z/wcTd337u3hA+NxL8EDiLLH++j/WAeB6YF14BkQd8CPhVjuuUTb8Crg2nryU4Tp8s/9PwyodzgH0pzdY3BQuaCt8HXnX3G1NmTdh9BjCzqrDlgJkVEpx3eZUgKK4OFxu838m/x9XA7zw8SP1m4O5fcfdad59N8P/1d+7+ESbo/iaZWbGZlSangUuAtWT7853rEy+5fgBLgPUEx23/Ntf1GcP9uhfYCfQQHH+8juDY66PABuB/gcnhskZwNdcm4CWgLtf1H8X+nk9wjHYNsCp8LJnI+xzux2nAi+F+rwW+GpbPAZ4DNgI/A/LD8oLw9cZw/pxc78MR7Pti4DfHwv6G+7c6fLyc/K7K9udbd1KLiEhax/ohJhERGYYCQkRE0lJAiIhIWgoIERFJSwEhIiJpKSBEDsPM+sIeNJOPMev118xmW0qPuyJHE3W1IXJ4ne6+MNeVEBlvakGIjFLYP/+/hX30P2dmc8Py2Wb2u7Af/kfNbGZYPtXM7g/HblhtZueFq4qa2ffC8RweCe+Ixsz+yoLxLdaY2Y9ztJtyDFNAiBxe4aBDTB9MmbfP3RcANxP0Mgrw/4C73f004IfATWH5TcATHozdcAbBHbEQ9Nl/i7ufCrQC7w/LrwcWhev5VHZ2TWR4upNa5DDMrN3dS9KUbyUYrGdz2FHgLnefYmZNBH3v94TlO9290sz2ALXu3pWyjtnAMg8GfMHMvgzE3f2fzewhoB34JfBLd2/P8q6KDKAWhMiR8WGm34iulOk+Dp0bfA9BfzpnAM+n9FYqMi4UECJH5oMpz8+E008T9DQK8BHgyXD6UeDT0D/IT/lwKzWzCDDD3R8DvkzQTfWQVoxINukXicjhFYYjtiU95O7JS10rzGwNQSvgmrDs/wD/ZWZfAvYAHw/LPwvcbmbXEbQUPk3Q4246UeAHYYgYcJMH4z2IjBudgxAZpfAcRJ27N+W6LiLZoENMIiKSlloQIiKSlloQIiKSlgJCRETSUkCIiEhaCggREUlLASEiImkpIEREJK3/D7nw54XGVMDuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Validation MAE\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It may be a little difficult to read the plot, due to a scaling issue: the validation MAE\n",
    "for the first few epochs is dramatically higher than the values that follow. Let’s omit\n",
    "the first 10 data points, which are on a different scale than the rest of the curve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plotting validation scores, excluding the first 10 data points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABBgElEQVR4nO2dd3wc1bXHf2f7qkuWe5N7xwU3jGk2xWBqAg9IIIRACEleQiDh0cOjtwQCCSWE5hcgVNMxYIxtbMBFBvfeq2xJVi+rLff9MXNnZ2ZnVytbq5U15/v56KPRzN3dO6vde+7pJIQAwzAMY18c6Z4AwzAMk15YEDAMw9gcFgQMwzA2hwUBwzCMzWFBwDAMY3Nc6Z5ASyksLBRFRUXpngbDMMwxxYoVK8qEEJ2trh1zgqCoqAjFxcXpngbDMMwxBRHtineNTUMMwzA2hwUBwzCMzWFBwDAMY3NYEDAMw9gcFgQMwzA2hwUBwzCMzWFBwDAMY3NsIwg2ldTgr19sQnltIN1TYRiGaVfYRhBsK63F37/ailIWBAzDMAZsIwg8TuVWA8FImmfCMAzTvrCNIPC6lVttCrMgYBiG0WMbQSA1gqYQCwKGYRg99hEELhYEDMMwVthOEARC4TTPhGEYpn1hG0HgdTkBAAHWCBiGYQzYSBCwaYhhGMYK2wgCzUfAUUMMwzAG7CMIOGqIYRjGEtsIAplHwD4ChmEYI7YRBKwRMAzDWGMbQeByOuAgFgQMwzBmbCMIAMVhzM5ihmEYI7YSBF6XE4EgJ5QxDMPosZUgYI2AYRgmFnsJAqeDo4YYhmFM2EoQeF0OdhYzDMOYsJUg8LhYI2AYhjFjK0HAGgHDMEwsKRMEROQjomVEtIqI1hHRPRZjbiKi9US0mojmEVHfVM0HUJ3FLAgYhmEMpFIjCACYJoQYDWAMgBlENNk05gcA44UQxwF4B8CjKZwPRw0xDMNYkDJBIBRq1T/d6o8wjZkvhKhX/1wCoFeq5gPIqCHOI2AYhtGTUh8BETmJaCWAQwDmCiGWJhh+DYA5cZ7nOiIqJqLi0tLSI56P1+Vk0xDDMIyJlAoCIURYCDEGyk5/IhGNtBpHRFcAGA/gsTjP87wQYrwQYnznzp2PeD7sI2AYhomlTaKGhBCVAOYDmGG+RkSnA7gDwPlCiEAq58GCgGEYJpZURg11JqI89dgP4AwAG01jxgL4JxQhcChVc5Gws5hhGCYWVwqfuzuAWUTkhCJw3hJCfExE9wIoFkJ8CMUUlAXgbSICgN1CiPNTNSGvy4FAkAUBwzCMnpQJAiHEagBjLc7/WXd8eqpe3wqPy4EAawQMwzAG7JVZ7FR8BEKI5gczDMPYBFsJAo9Lud1gmAUBwzCMxJaCgJPKGIZhothKEHhdTgDct5hhGEaPrQSB1Ag4hJRhGCaKvQSBUxUErBEwDMNo2EsQaD4CFgQMwzASWwkCr4s1AoZhGDO2EgSsETAMw8RiS0HAGgHDMEwUWwkCL+cRMAzDxGAzQcB5BAzDMGZsJQg4j4BhGCYWewkCziNgGIaJwV6CgKOGGIZhYrCVIPC5FR9BY5CdxQzDMBJbCYJsn9KHp6YxlOaZMAzDtB9sJQjcTgd8bgdqGoPpngrDMEy7wVaCAACyfW7WCBiGYXTYUBC4WBAwDMPosJ0gyPG5Uc2mIYZhGA3bCYJsnwvVrBEwDMNo2E4Q5Pjc7CxmGIbRYTtBwD4ChmEYIzYVBKwRMAzDSGwnCHJ8bjQGI5xdzDAMo2I7QTCwSxYAYN3+6jTPhGEYpn1gO0FwfFE+AGDFrsNpngnDMEz7wHaCoEu2Dz3z/KwRMAzDqKRMEBCRj4iWEdEqIlpHRPdYjPES0ZtEtJWIlhJRUarmoyfH70Z9E/sIGIZhgNRqBAEA04QQowGMATCDiCabxlwDoEIIMRDAEwAeSeF8NLwuB/ckYBiGUUmZIBAKteqfbvVHmIZdAGCWevwOgOlERKmak8TrcnDUEMMwjEpKfQRE5CSilQAOAZgrhFhqGtITwB4AEEKEAFQB6GTxPNcRUTERFZeWlh71vHxuJ2sEDMMwKikVBEKIsBBiDIBeACYS0cgjfJ7nhRDjhRDjO3fufNTz8rocCLBGwDAMA6CNooaEEJUA5gOYYbq0D0BvACAiF4BcAOWpno/X7eQG9gzDMCpxBQERvaU7fsR07YvmnpiIOhNRnnrsB3AGgI2mYR8CuEo9vhjAV0IIsx+h1WEfAcMwTJREGsEg3fEZpmvJ2Ge6A5hPRKsBLIfiI/iYiO4lovPVMS8C6EREWwHcBODWJOd9VHDUEMMwTBRXgmuJdubN7tqFEKsBjLU4/2fdcSOAS5p7rtaGncUMwzBREgmCDCIaC0Vr8KvHpP7422JyqYJNQwzDMFESCYIDAB5Xj0t0x/LvYxavy4lQRCAUjsDltF2VDYZhGANxBYEQ4rR414jInZrptA1et7L4N7EgYBiGST58lBSmE9GLAPamcE4px+dSbjsQZD8BwzBMs4KAiCYT0VMAdgH4AMDXAIamemKpxOt2AgAaQ+wnYBiGSZRH8CARbQHwAAAZAVQqhJglhKhoqwmmAi9rBAzDMBqJnMXXAtgM4FkAHwkhAkSU8mSvtsDrUjQCDiFlGIZJbBrqDuB+AOcB2EZE/4YSRppIeBwTaBoBm4YYhmESRg2FAXwG4DMi8gI4F0r+wD4imieE+EkbzbHV8UkfAZuGGIZhEpqGNIQQAQDvAniXiLIBXJTSWaUYGT7KSWUMwzAJBAER3dSWE2lLsn3Kbdc0htI8E4ZhmPSTSCP4C4CVAOZAaTup7xx2TDuN8zM8AICK+qY0z4RhGCb9JBIEYwFcDmAmgBUA/gNgXluUiU41eRlKYnRFHQsChmGYuFFDQohVQohb1Q5jL0LpL7xeV0L6mMXrciLL68Jh1ggYhmGSyizuDEU7GAWltMShVE+qLcjPdLNGwDAMg8TO4l8A+C8APgDvAPgvIUSHEAIAUJDhweH6YLqnwTAMk3YS+QheALAWSo2hswCcSRT1FwshjmkTUV6Gh53FDMMwSCwI4pah7ggUZHqw9VBtuqfBMAyTdhJlFi9sy4m0NTk+F2oDnEfAMAxj264sGV4X6gIhdIBoWIZhmKPCtoIgy+tCKCLQFOZ6QwzD2BvbCoIMj1J4rj7A9YYYhrE3zRadI6LBAG4G0Fc/XggxLYXzSjmZHuVWagMh5Gd60jwbhmGY9JFM9dG3ATwH4F8AOsz2OdOr3Hp9U4e5JYZhmCMiGUEQEkI8m/KZtDEZXsU0VNfEkUMMw9ibZHwEHxHRb4ioOxEVyJ+UzyzFSNNQHYeQMgxjc5LRCK5Sf9+sOycA9G/96bQdmVIjYGcxwzA2p1lBIITo1xYTaWtYI2AYhlFIJmrIDeDXAE5WTy0A8E8hxDFdsU36COrZR8AwjM1JxkfwLIDjATyj/hyvnksIEfUmovlEtJ6I1hHRDRZjconoIyJapY65uqU3cKRkqVFDdRw1xDCMzUnGRzBBCDFa9/dXRLQqiceFAPxRCPG92vB+BRHNFUKs1435LYD1Qojz1L4Hm4joNSFEysuC+t1OELFpiGEYJhmNIExEA+QfRNQfSeQTCCEOCCG+V49rAGwA0NM8DEA2KfWtswAchiJAUg4RIdPjYmcxwzC2JxmN4GYA84loO5QG9n0BtMiEQ0RFULqcLTVd+geADwHsB5AN4FIhREzxHyK6DsB1ANCnT5+WvHRCMjxO9hEwDGN7kokamkdEgwAMUU9tEkIEkn0BIsoC8C6APwghqk2XzwKwEsA0AAMAzCWiReZxQojnATwPAOPHj2+1cqGZXi5FzTAME9c0RETT1N8/AjATwED1Z6Z6rlnUiKN3AbwmhJhtMeRqALOFwlYAOwAMbdktHDmZXic+Xn0Alz3/XVu9JMMwTLsjkUZwCoCvAJxncU0AsFrYNVS7/4sANgghHo8zbDeA6QAWEVFXKFrH9uYm3VpkqLkES7YfbquXZBiGaXck6lB2t3p4rxBih/4aESWTZHYigCsBrCGileq52wH0UZ//OQD3AXiFiNZA8T/cIoQoa9EdHAWZailqQKlCKkNKGYZh7EQyK9+7AMaZzr0DJZ8gLkKIxVAW90Rj9gM4M4k5pIRM3cK/o7QOo3rlpmsqDMMwaSOuICCioQBGAMg1+QRyAPhSPbG2IEOnEWwvq2VBwDCMLUmkEQwBcC6APBj9BDUAfpnCObUZEV38UXXDMV0xg2EY5ohJ5CP4AMAHRHSCEKJDhtUEdf2KG4Pcu5hhGHuSjI/gByL6LRQzkWYSEkL8ImWzaiOMgoAzjBmGsSfJlJj4N4BuUJK/FgLoBcU8dMwTDEdtQ40hFgQMw9iTZATBQCHEXQDqhBCzoCSXTUrttNqG7rlRnzebhhiGsSvJCALpRa0kopEAcgF0Sd2U2o7bzh6Gpy4fi8IsD5uGGIaxLckIgueJKB/AXVAKxK0H8GhKZ9VG+D1OnD+6B7wuJ2sEDMPYlmSKzr2gHi7EMd6nOB4+t4N9BAzD2JZECWU3JXpggvpBxxw+txMBNg0xDGNTEmkE2ervIQAmQDELAUpy2bJUTqqt8bnZNMQwjH1JlFB2DwAQ0dcAxqldxkBE/wvgkzaZXRvhczvYWcwwjG1JxlncFYC+h3CTeq7D4HM52UfAMIxtSSaz+P8ALCOi99S/LwTwSqomlA7YNMQwjJ1pViMQQjwApZNYhfpztRDioVRPrC3x6kxD5bUBPLNgKyKRVuuIyTAM065JFDWUI4SoJqICADvVH3mtQAjRYdp66TWCG99aha83l+LkQZ0xsieXpWYYpuOTyDT0OpQy1CugtKaUkPp3h8kp8Lmi4aObSqoBAHXc1J5hGJuQKGroXPV3Mm0pj2n0CWWlNQEAQHldU6KHMAzDdBgSmYbM7SkNCCG+b/3ppIf8DA+CYYHFW8q0ZjUsCBiGsQuJTEN/TXBNAJjWynNJG5dN7I3nFm7DLe+u1s6V1wbSOCOGYZi2I5Fp6LS2nEg6yfa50Svfj1V7q7Rz5bWsETAMYw+SySOAWn56OIwdyv4vVZNKB9k+NwCgT0EGXE7CYTYNMQxjE5oVBER0N4BToQiCTwGcDWAxlESzDkOOX3krCjI9cDkI5XVsGmIYxh4kU2LiYgDTAZQIIa4GMBpKc5oORY6qERRkeuD3cKYxwzD2IRlB0CCEiAAIEVEOgEMAeqd2Wm1Pjl8RBHkZbnhdTgRCEby2dBeWbC9P88wYhmFSSzI+gmIiygPwLyjJZbUAvkvlpNJBjk95K7K8LjSFIggEw7jjvbUAgJ0Pz0zn1BiGYVJKojyCpwG8LoT4jXrqOSL6DECOEGJ1vMcdq3hdTgCAy+HQNAKGYRg7kEgj2AzgL0TUHcBbAP4jhPihbabV9oSFkknmdChF6OqauMQEwzD2IK6PQAjxpBDiBACnACgH8BIRbSSiu4locHNPTES9iWg+Ea0nonVEdEOccacS0Up1zMIjvpOjJByRgsABn8uJyvqgdq2hiXsVMAzTcUmmDPUuIcQjQoixAC6H0o9gQxLPHQLwRyHEcACTAfyWiIbrB6i+h2cAnC+EGAHgkpZNv/WYMqATAGDa0C7wuo1vy77KBkQiAs8t3IYqnYBgGIbpCDQrCIjIRUTnEdFrAOYA2ATgR809TghxQNYjUttcbgDQ0zTsJwBmCyF2q+MOtXD+rcbYPvnY/uA5mNivAF6X8W0prQng6y2leHjORtz3yfo0zZBhmGOZxmAYB6oa0j0NS+IKAiI6g4heArAXwC+h9CkeIIS4TAjxQUtehIiKAIwFsNR0aTCAfCJaQEQriOhnLZp9K+NwEAClP4GexlAYtWpZai5PzTDMkXDtrGKc8NBX6Z6GJYmcxbdB6UnwRyFExZG+ABFlAXgXwB+EENUWr388lIQ1P4DviGiJEGKz6TmuA3AdAPTp0+dIp5I0Zo2gsSmMYFiJInI7k0m9YBiGMbJ4axkAQAgBIkrzbIwkchZPE0K8cJRCwA1FCLwmhJhtMWQvgM+FEHVCiDIAX0PJXDbP5XkhxHghxPjOnTsf6XSSRoaSShqCYTQ0sSBgGCbK0/O3Yo2uUGWyBMMta4N77t8X4U9vr2rx67SElK1qpIi8FwFsEEI8HmfYBwCmqn6IDACTkJwjOqX4TM7ixmAEFfVKETq3s31JcoZh2h4hBB77fBPO+8fiFj9WWheSZe2+aryzYm+LX6clJFV99Ag5EcCVANYQ0Ur13O0A+gCAEOI5IcQGNUltNYAIgBeEEGtTOKeksNIIqhqUaKEQN7VnGNvT0l29nqZQBJne5MYK0TbrTcoEgRBiMZT+xs2NewzAY6max5EQ4yMIhlGhlqVmZzHDME0t3NXraYlG8NI3O7XjVPoW2OBtQUzUUDCMCjV/oJYFAcPYkgc+WY873lsDQNnV69lZVpd0V8PyuiatN3oiSqoacd/H0XD1mhSuPSwILNAnlGV4nGgMhlHV0DKN4JkFW7Fi1+GUzI9hmLbnX4t24LWluwEAgZCx2sCpf1mAU/+yIOYxQgj8+tUV+G5btIrx2U8uwoQHvgSQuGpBvanMzcGqxiOderOwILBAbxryuZ1oCIZR0yjzCJIrN/HoZ5vw42c7XJFWhmEQqxEA0NYIPdWNIcxZW4Lr/q845lpJVSNG3/MFindabxjNa83B6tQ1y2JBYIFHFQQ9cn3wu5UmNbIIXV1TCLe+uxrXzlqezikyDGPB4i1lWLi5tNWft6bRWFpGX504kiCARJqSzWVrAGDd/io0hSPYV2mdbWw2Q1c3pq68TSqjho5ZXA7ln3by4M5YtvMwGoJh1KrSvjYQwhvL9yR8fJgjixgmLVzxolK8oLV7iOgX61A4YtAIElUqrlajDc2RiACw+WAtACAQpxui2TRUa6FxtBasEVgwsEsWXr92Eu69YKSiETSFNTWt0qLo3NLt5Si69RPsragH0PI4YYY5Uh76dAMmPfhluqeRNFUNQcz+/shi4tfvr46xzbcV3++q1I5rAyGDRlCdYIGWgsDjil1qNx+sAQAE4qwXUiN447rJANhZnBamDCyEx+WAz+1ETWMITeEI+nfONIwJRwSenr8Vj89VKmKs3adkGR5NaBnDtIR/fr09pbbj1uaT1Qdw01ur4ppD4lFaE8A5Ty3C7bNbJ80oGI7gkc824rAaFt4cf/1ik3Zc3RAyCCSz2UhPIiEhBYGVvwGI+gh65fsBAK8t2YV1+1ueyZwMLAiawe92oqxO+aJN6FugnXcQ8OWGg3js801YukNx9mR5lb7HQe5uxjCWSDt3RZILsKQxqCyKi7a0jv1//f5qPLtgGxZubr7gsRACFfVNGNI1G4ByD/rF++XFOy0ft7eiHr9UncRW5mIpDM2CYOuhGgghtAjFXL8bXpcD28vq8MnqA83f3BHAgqAZfG4HytSY39G987TzEQEt21giP6xHk3XIMB0Zaee2MrEmQmrZrZXQebC6UX2+WFNTWW0At81eo32fQxGBiAA6ZyvpwNWNQYNp6M3iqM/w3L8v0o4/XLVfO7YSBPI90AuCBZsO4fTHv8YHK/drpqFMjwsRNcNYzqG1YUHQDD63U1Pv8jLcuOvc4Th9WBcAQL3pQ9mgCQLWCBjGCrm4mTdRzSEdqnWt1C3woLq5s4rj/8vnm/CfZbvx4ar9KKlqxCF1bGGWB4BiGopnzlm7r1oTIF2yfdr5sgTJZk3hMIQQiEQE5q4/CEDRFuoCIWR4nHA4SNtcFmaxIEgLUwYUaseZXheumdoPpw1VBIH5QykFATe+b19EIgI7y+rSPY2UkiiEsT0hBUFlQ8tMQy11ElvV6NlzuB4z/va1srhLjcAi4kdWcQiGI5j80DxM/+sCAEAndRGubgjGFQQAsPWQEg2k/58kWhOaQhFc8eJSjPzfz7FF99i6pjAyPMbATtYI0sSlE3prx1leJQTMr5agqDbtagKsEbRLnlmwFaf+ZQG2qM65jkgwcmx85qxMQ3sO1ze70DfGCbHccKAaj8/dHLPwW5lnN5bUYGNJDVbuqcAh1cFeWR/Ex6v3Gx4vw8dlGQj52l3URbiqIZhwYd9UonzOzEKmf2Gm1XAU76rAN1vLUd8U7WBWVhtAXSCkrTkSFgRpwumIFnnK9CrSWQoCc72QhmAYjcEwbn13dcrnVdMYTLq2id2Rzvz9KUzRTzft0S9VGwjF2Mbl4ihNQ3WBEE56dD7ueC9xNFA8QXH2k4vw1LwtMclXDcEwFm0pxaxvd2rnZFz+nsMNOFijfBZe+XYn/vv1H/Czl5Zh0ZZSHKhqgEstNV+809iKpSDTA5eDUF7XhMVbjU5rfTUCuauvN1kMuuf5YMUPuyu14z2HVUFQ14TyugCyfW7DWBYEaWRsnzwAgE9NCvF5lN97TSFwDU0RvP/DPqw6gmYV8fhw1X4tLFXP1Efm4/j7j5348XQiN3sduZNEqJ1poUIIjLz7c9w+e43hvCzDUFUfxP7KBkxTzS7zNyaO3tFrBPJe9aYXc3mHQDCMK19chrs/XKd7DmVh3ltRHxNyu2hLGa58cRlOeOgreNTmU2tNoZpetxN5GR48t3AbPl1TYrj2o3G9tONd5YoZsr4pBKeDcM6obgCAgmRrTwPYXV6P5TsrMLFfgeF8tjc1OcCcWZwEL/xsPD5YuR99O2UAiGoEy3YYa4Q0BMMxu4Cj5ff/+QFAbKZkS51tepbvPIxwRGBy/05HNbdjjXbWHbBVaW2NYPnOw/C7nRjZM/eIHi+/B28W78HkAQW4aKyyUNbpfASzvtupLch6zdsKvUZQ1xTG819vxpBuOdq5qoYguuZEd9x6wRGJCDQEw1i/X+mUu6eiQfMRWCE1AnNkk8fpQH6G29Lxm+GJmnB2lSuJpfVNYWS4ndoGslOmJ+E96lmjbv7OHN4VAPDEpaOxaHNZyspQsyBIgk5ZXvxiaj/tb787Nl0cUHYcrdmvIFW1RS55TimG19pp+O0VgfZnNmltQkn4CLYcrEHPfH+MA9KKZD8jew7XI8Pj1BypEv334MY3V2mCQHMW1wcNu9tmBYFuYa8LhPD0/G2G62YHbqNOcFQ3BvHrV7/Hd9uVCqA7yupQniCPwRFnsfW6HcjyWb93+kfsKq+DEAL1gTAyvErUD6CYlpKhc7YXpTUBnD+6h6YRXDS2l/YepgIWBEeA32MtCP6zbHerRgztV01PzXxHmGaImoY67hsZ0mkEmw/WYFCXLMPuMRwROOOJr3HSoEL8+5pJrfa6Jz06H16XA5vuP9tw3lwOYV9lA3rm+Q3ho1k6QaBffCMRgQc+3YBLJ/TGYDWJS68RHLDw9dz94TqtnSxgDAstr2vShACgCIJE/P2rrZbnvU5HwrLRgOIrqGsKY19lA+qDStSPFFA5cYSImTtnDkOPPD/G9clvsyb37CM4AuJpBK0dNnqgUvnAp8pBZBc0QdDB5IDeLyAj1eauP4gzn/gaH5kyUOVitGhLWbPPay52BijC5dpZxfho1X5EIgKr9lRq16w+92bN+PtdFYZs2aqGIDy6Qmx1TSGc/4/FmLfhIHYfrseLi3fgN699r13Xm3r2W5Sn2FhSY7D7S38AgGbLSPzpzMFxr+mdwB6XI67pd4K6c7/17KEgAt5cvgcNTSH43U7tvc8yOX7j0a8wExOKCprVkloTFgRHgLmDGdC82rexpBqfrkk+PfzMJxbi6leUUtf6xJSWsmR7OeZvaj6Nnjn2aNQtwLKXtrQtP/b5Rs1pCbQsDt+qdtHc9Qfx5YaDeKt4D55duA0XPP0NVuqEgRlzpcwDVQ1oCIYREYqGW1kfNIRZV9YHsXpvFa6ZVYxK1f+lXwf180+mTpH+vSmvTSwIZozsFje0s5/uvNfljGv6nT60C4rvPB1Xn9gPUwcW4rO1JagLhJHpdWr3mZ2kRpDnT96X0FqwIDgCrExDzeUOzPjbIsMORyKEsEx+kSVqASA/gZBpLlrksueX4OqXuXcCAC1Nv71Q0xg0lCFoKXozhdx1NuhCJM/7++KY68lw0MKRKqNyguEIlquNVEoShOOaTUOHqgOacOiW41ObPVn7wL5SI4j0mndzGoGZlmgEhVneuF6kPgUZ2rHH5cBd5w5Hrt+N4jtPx8e/m6pdczkdWtbv+L4F2Fpai9LaAPwel1YeQ7+BtBIKsqhlQRYLgmOCLK8Lf71kNJbePl07Z9WdCFASQ/SaQG0ghMc+34h/f7cTQgj0u+1TPPDJhoSvZyUoJJzF3DzSWRxqZ9m3t85eg9//5wdsLKk+osfrFzt5b3rTRXVjNI6/JZ8TK0FQGwhqzy8FUKkueuY2NUx0waZDaGiKDZo4VBPQ/AM91Wqah+L07f1w5T4A0Q3XnsP1+Mf8qN0+WUEgBcnhukCMWXB49xyM7qVEROX63XG/YzJSEFAEwYVje2LV3WeiMMuL4d1zLB9zXO9cCKFkGGe4ndp773E6kJehmIdyLMxE158yAJvvP9vgO2krWBAcIT8+vhe65vjw+rWT8KtT+scdd82sYoMm8P2uCjw9fxvu+mAdKtTwtBcW7wCgRDf8zzurYnZKibQN/WLAWCO/4+E2SLpav78aH69Obpe/93A0zBBQnKTXzlqOb7c1b8cHjOYSqRk2mD4Pkx+aFzM2HrO+3Yml28vx5JdbtHMyVl9udOqbwprZ5a73o0lg/1m2GxtLqvHzl5fj3o/XxyR4HappxN4KZQHvmacKgjjls3eq4ZdyIX94zkbDdfk8Eqv6O6U1AYTVf3xVQ9DgjP74d1Mx+zdT8H+/mIQ5N5wEIsIINUz2trOHGp5nYJcs7djcU8ARx4Y/rk++dpzhifoIPC6HVlI6x2/tL7DqW9AWcNTQUTJlYCGmDCzEwM5Z2F/ZiEsn9Na+fACw7VCtYfxmXZkD887mpcU78FbxXvTOzzCcDyVYwI5GIxBCtFlUQjqR715raASRiMCdH6zFlZP7YpjFjvCcp5Tqk+ce1yP5+anTqgmE8OWGQxjcNdtQ48pMYzCMCfd/aTC/yDwCc1SLzH5P5nPy1y82YVzffGwvq0OXbC8O1QRQHwwjy+uKCoJAKO7uea+aFburvA69C/yGa0u2H8aS7csARDWCgzWN8LkduOmMwXjwU2Wx757r06KC5GfTPPeNJcZSIYVZnpjY/iXbD2sLcGV9EA4C5DsjcyN8bidy1R36oz8+Dj+d1Af1umqkH/9uKoZ1z8Et7yrajjfJRTrX78aPx/XCu9/vxXmje6BTlgf3f7IBI3rkoGee31CYTnLWiK44f3Tyn5nWhjWCVuKS8b1xw+mD0C3X6Ng122Y3lVgLgkhEaLsv83oVTLCAHY0gaG+mkpQhNYJWuN+d5XV4feluS3+P4SWT8EfIEXLxlr+bs2nvrWiIscHLPAKzRiDRf06emLsZO8vqYj6bjcGIFlopF2tp4pFaan0wHNc3sLVU2fT43c6EbRV75ikbnUPVAXhdTuTqdsd6U4yMXrLyJcgKwECsmaVnnh9f6/oWVzUEm93wZHpdmDKgEBm62j4je+YaIndaslt/4KKRmHvjyThtaBcc1ysPb/3qBPjcTlx9opKPdHzffMP4v18+zjIIpa1gQZBizN3K9BqBPh56f1WDFt9oToBK5BA+mtZ9x4p/Yd3+qlaprplM0lVzyP9nc6F9yZRLlrLig5X78OtXV2iLuFWy046yOhTd+om6wEXfi5E9Fa3kyS+3IBiOxA1v1C/6T87bglP/sgDX/btYOx8KR9AUjmCPaq6SBdaigkDt2d0YintvMnPX5zFG13TLMW6OhnbPhstB2FfZAK/LYSi9UNQpGqVTVtuEX/27WKsVpWfqwKjGlOM3GjbOGtHN8L37Yv3BpJ3liezzsvREMvjcTgxScyD0TO7fCTsfnomh3YzX0mUSkrAgaGP00UD7q6IaQUlVo5bupN+5ju6Vm9g0FIxgZ1kdPltbEndM/MdGv9BCCLy6ZFfKspmPlNV7KzHzqcV4duG25gc3Q6L3MVnkTre5RUFWpj3ria9x+3vGejtr9lYZtMG3V+zFnLUlWtkQK41guboYvr9yHxqaooua3A0X76rA28V7LX1G+ysbcNnzS2LOL9hUisF3zgGg7PSBqDYqQ5brm8K4+e1VmjkmkRYp2ygGgmGDHd9sJuqdn4HzxyhmEI/LYQi97qITGlsP1eLzdUp9/mHdcwy76KmDOkffA5O9/awRXePOsTkSZV1bmYZG98rVOpe1hHF98zF1YCFeu3YSbpg+qMWPb23YR9DG6FV3mTAGACW6SA25U7/17KFYs68KGw7EjyoJhCK45d3VWLrjML648WQtEzMe4Tg10r/bXo4731+LNXur8MjFxyV/QylG2rhlyOKRIDWs1jANSUHpbmYHV90YRA/4selgDTYdrMGDF43Srp33DyWsc5Spjs8BVTgkMg3N/n6fIa9ERqEAwLfbyixNMgs3l8acM2P2LUiNoDYQwtsrYpvN/+LEfnjpmx2Gc9tKFbPSgapGbCuNbnh6F2Rgua6SZ7bPpX1OwxFhqMFjpf2+ff0JmFBUgNpACCPv/hwAkJ/hxi9P6geXRbZvjt+Nbjk+lFQ3wu92Gr5z9184MsG7AGR645tnrMxLH/z3VIuRzTOuTz5evVbJ8D5xYHx/UFvBGkEKmDLAupibuXKg3kxUUtWoKfxyV+dzOeB2UDPO4rBmY33Z9MXUI23W+ggkKQiEEJpTu6UNQ1KNW915H010lDTBxNvNVtQ14aY3V6KmMYjaQAgXPP2N4X+jp7pBagSJTUNynJlEtahkopRVefGwzufwnE470tvXN5bUxJRGBxD3XiTBcCRWEOQoguCbrdYRTKN65cTVitbtrzbE/etj8QFlZy3n3RgMG+Lme6gRRTK6BogWa9PvyLN9btwxczhumTE0RiNoCkUwqKsS7aPPHfnRuJ64YnJfyzlLMtMQutkeYEGQAl6+egLunDks5nwv9QshKxXqox92ltdhkxpPLk0EPrcTLqdD2yXd8d4aPDRng8ERGQhGtHjrz9cdjOtPkDbTJoMgUL78bxfvxV0fKOV6M5MoSNaWSGEVrzEJAHy7tSzhAivfrXAcH8GT87Zg9g/7MPv7ffhuWzlW7anEo59tshwr/zfuJE1DevZW1GOEuqMFYhPcpM+oujEUEzIcr9qs3lG6+3B9jBMZiBUE+RluQ8RTRX1TjG9hRI9c5PrdcevuZHhcePBHoyyvSbqqwsQsCIgIeeri3RAMGzZIP5nYB2/96gRcMCYaQSOFhiuO41bW8DlnVDdkeJwoKszECepmTK/1/va0gQnnCwAZFg7bn08pMlQX7YikTBAQUW8imk9E64loHRHdkGDsBCIKEdHFqZpPW+J1OXHF5L6GGGQA6Kk2pjCHh/btlIFXl+zW7KGy1Z3P7YTbSVrU0IpdFViyrdxQcrgmENQWwcN1TRh4xxz8e8kuDLz9U81mC0SdhXqnmazo+OnaaMKbVeu+QzWN+NW/i1HVwobjrYGMHImnEZRUNeInLyzFze+ssry+rbRWE5xWGsHqvZV4RW1ekul1Qa7v8bKQq02C4Cf/WoJ3LEwnVr6WLQeNocTmXfgBnc+owmQeitfsXb8bjucQ3VRifN38TI/WfxdQegE0BI3/9x55ftyqi6mfUJRv2Nxkely4+PheBpOXHCeRJiyrWlkybLMxGDGYXBwOwsR+BQZbvbzHeJE/8vrpw7pi/b0zkOt341cnD8CTl43BGWoZ5zevm4wBnbMsH6/HZSHg//f8EVh/74xmH3ssk0qNIATgj0KI4QAmA/gtEQ03DyIiJ4BHAHyRwrm0OT63E5fp2lwCQH6G8uUrKjQKAnNUxToZfeF2wOVQNIK3ivdgy6FalNUaKyne+OYqfLnBWEvorvfXIhRRnL8SKTzMpqFQOGIIabUyLby4aAc+X3cQry/bDUCxOd/z0bqYcUdDZX2TpTYjd6rxIpxk4tIPuytjIqjmrj+I6X9diO/VDlBWPoJfvBItv+F1ObTFJq4gUBf4cESgMRjGt9vKsWZv7PMv2V6OFbuMHa7M8zMnXe3T+YzMkUNVcUx28RKT9Jhj7M8e2c0QqnjLu2uwo6zeMCbD4zTU33nuiuMxVFf/X9rSpZY1oSgfv5s20JA/ISNwZDKXx+nAyz+fAMBo0rJC7/toTvuSWpG+JIXTQbhgTE88cOFI/PGMwZhQVBDv4TH892kD8WorVmg9FkiZHUAIcQDAAfW4hog2AOgJYL1p6O8AvAtgQqrmki7kwi+RX9q+uhC5X53c3+Ao1uN1O+FyEirqg/ifd5T2l6U1AVz10rKYsdleV2xsuU5zkLvFYEjvLA7j4TkbcaCqET+Z1Ad7Kxqwo8y4ewSiX2i5CMrX//O5w+Pu0rYeqkWW1xWTV2FFYzCMMffOxTVT++Guc417BantBOJoBFJTOFDViJlPLcaXN52iXVu0xegktWreoq8b1RgMa/+jeI5laaJpCIZjetrqnZJvFe/FW8VRTSESEagLGO/BbI7RRxJJh/G3W8vwyrc7sSFOGYrmFlQzd84chqtP7Ic/vLnScH63rkCdy0Hwuhzor9tBd8rywu+JCgtpS5da1vDuOfjjmUOwVLdJkU56ArDyz2fA53ZqAijP9N345PdT4dVVI9WHkTZHvio0rOz7XXJ8+F0Lo3L+dNaQFo3vCLSJj4CIigCMBbDUdL4ngIsAPNvM468jomIiKi4tbT4Cor1gLh4lF2N9J6XbzhkWoxFIfC5nzG7InJcgkc49PXpTSNDKRxCMaCGsd5wzDEO6ZqG0JhCTDCWbcZjt3onyEE5/fKEhwzoR69WoqC83HNTOHapWIk+0ZKs4gkBvgtlqyuI216238hH0yI06JRuDYU0TiKcRaIKgKazVypFzS+SnqG0KGerlA7FmOL02JjWC577eji/WH9R62ZppqSD4+ZQiOB0Enynqadfh6CKf4XGCiDTzkYzZ15uTpM1c1tuZ2E+xyetj52UTeJCy8Ou1EPO8R/TINZhS9YllzTG+qAAPXjRK8wswLSflnkEiyoKy4/+DEMK8rfkbgFuEEJFEmX9CiOcBPA8A48ePP2bSYc0fdrlo5frdeO83U7T46a7xBIHbYXCQJaJLtk8L35OELEJFDT6CUAR1gTBG98pFpteFztleNAYjqA2EDE2zpWbx2tLdGNM7TztfFwi1SjbkGrXHs34XOPFBRYhcf8oA9bWsBYE5XHLxljJc8eJSfHfbNC05SmLlI9CbVuqbwtr7Ey/3rEwtadwYDKNUbYDemIQgqG4Ixph7EiUfH1bNOZX1iaO49M1OXv/lJJTWBHDDGyvjjpc2cPP/bbf6XnXL8Wnln4kIK+48Xdtpy4geIBpUMKl/Jyy9fbr2GdbnBDxy8XF4bsE2TLQwy2Q243ztnuu3PH/akM4xtn6ng/CTSX0SPh+TmJRqBETkhiIEXhNCzLYYMh7AG0S0E8DFAJ4hogtTOae2xNzA5uyR3QEo9tSxffI1E1E884nP7UxaEBRaOOQiFhqB3kfQFA6jvimkOeakU6+sNv7O9a3iPdHz6uI8Z80BfLetHEeKzEi1yuqUZZWbwhHL4ntmO7uMbZ/17a6YmjThiMCmkhoU3foJrnppGVbtqTQ4bBuCUUEQjrNKy8qcetNQVCNQfstm5XqqG0I4bFEX3xxqnOlxgihqGtLX0j9pUCGe/ek4w3i9iWXKgEJcMKan5bzN+NzKV/+XJyklD3arhd6Gdc82fJY6ZXk1oaHXTvWlGMwbmdeunYS5N56Mnnl+3HfhSEsHrNz4DehsbQKKl7n98tUTcee5Ma5G5ihJmUZAyn/6RQAbhBCPW40RQvTTjX8FwMdCiPdTNae2ZnDXbPx0Uh+8tnQ3JvYrwIyR3bD9wXNiqhZalaQFFCeb1ZfICr87dpxh0Q/FCoKGJkUj6JGnvH7nLOULXVoTMDTk0NuyB3bJ1pKDatTSxL9W6+5Y9bf9zWsr8MxPj084d2lusYq00ZczqG8KI9dvvE9z+e9D6i59nmpmKszyas7SUERo5qeFm0vx7bYyjOyZi5MGFWLZjsMGQWA2j63ZW4XiXYe1ipkNwahpaNGWMry6ZJfmXP3ZCUXI8bnxxvKo0Kysb7IsHXHvBSMx69udOFjdiC/WH0Sm1wWv24nyuiYIIQyO3qkDC3H2qO6GxyfaWV9/ygBDzoEep2q2yfG5keNzaXN7+MfHJdW3IVFmdbIJUotvOS2hs3v+n07VBBaTWlL5Lp8I4EoA04hopfpzDhFdT0TXp/B12w1OB+GBi0Zh58Mz8davTgBgXbq2f5xdUWGWB65mEpckViYa/SJp5SNoCFprBObIIX3rQr35I565Rs+na5ovfVGnFReLNa3o4+etWiiaNYKSKmXuZbUBZHqcKNLZmsMRYVjAgmG1wbjHCb/HiYamsPb+mJ3Fv5i1HPd8tB5N4Qg8ajarvozyne+v1eaa63fHOEM3lNRopiQ9/Qszcd+FI3HSoELtfgoyPSivbcLY++Ya/DB604wkXjN1ALjxjPhOUunIdThI+787SMkojmeW0dMaVWt75WfE3QQBSnewZObCHD2pjBpaDCTfLVwI8fNUzaW90yPPj3X3nIWlO8rRI8+Pod1yEIkIOBwEtyM5We2yGKdfRKMaQXSBa2gKoTYQ1mzAUUFgXLDqA2H0yvejINNjeM54NnFzGGi8ctehsKKRSPNMeV0At7+3xpD4o29eIgXPQ3M2ID/Dg+tPGRCjRcg5VdQH0SPXZ+juFgqLmOJetYEQMj0u+N1ObD1Ui3fVnIDvd1fiD2/8gPsuHIlsnxv1unvt2ykDWw7VxkR7yczgXL87Zif7w+4Krc6+ZGi3bG1jMLl/JxRmeXHzWYPx8eoD2FfZEJM7IAXBK1dPQHVjCKcN6YwMjwv3nD/CMjzS43Tg1rOH4uE5G3Hucd0NFTn19O+chW2ldeiZ7292gf/65tOws7wu4Rjm2KN9pZHamEyvC9OGRotlyQUiWY3ASp1fr6tRpGUW63aYa/ZVaTtnAMjzu+FykKHzFKCYZDI8SrlgfeRLbSBkWf1U7mK9LgcCIaWqpT40UHLb7DV4e8VerRLjnsMNeH3pbi3RDQBKdYut1Aj+uXA7AMX0YXYW66OLcvxuFOh25i99swMzjzOaVuqbQsj0uuD3OPGtyc/x/sr9uGBMT0zoV6AVZQOUTNkth2o1B6tE5g3k+t0G/1CPXB/mbThkmNtjFx+nFV4DlGib4jtPBwBsOFCj5WzokfV/Th3SxXD+qilFMWMBZdd+/SkDNIe7ASHHAJP6FWDu+oMxGcBW9OmUgT4tiOhhjg3YANfOMfsI4jnRwhGBS47vhfGmOucSKx+BzGTOkIk/DkKvfD8+WnXAUPisTjUf5fjdhqqStYGQpTlHljSQnahk6OOt767GMwuiJQtm/6C0JDS3RvTqdtP7qxo1J7JVQTWzaUhPXoY7Jqz2k9UHDH/XNIaQ4XXGOPYl28vqsOVgDYRQasxfd3J/nDpUWYhl7X7JnLUlcDlIMzVJThvaJSb8dfqwrpbCEVBMhXqB/eo1k/CXS0ajdxILdUshkCZYLp3AkTd2hQVBO8dtWvi7WEQHAUqUy2OXjMY7v55ieV2ahKzKEOgdjo9ePBp7Kurx4mJl172/sgH1TWFkehWNQC8gKuqbYhbxg9WNuOiZbwFAM8uc/vhCbCypxhvL9+DRzzZh+c7DWLHrMNyqtlNRb23ekUjH9U9eWBrT39eqpo8k1++OG5orCUWEZhqyYuOBaryvCqyJRQW4/Zxh6GVhq9c/HxHBp1vkZZkDPfFeDwD6F0bDI28/ZyimDirExcf3SngfLUWvPw7skoV195yV1g5ZTHphQdDOMTuX9VUZ9TTXuKUprOxIZVRNvi6FP0MXtjmxXwHOGNYVbxXvxcaSakx5+Cus2FUBv9sVkxfx6GebMPOpxYZz+iQufUy5vgzxJc99hx8/+13cQnLmnXaRLoJpic580xSKoLyuCcf1ysVzV4yLKQyWjCAAoJmGrHh7xV7M+k4p1SF35PraOdL5rK+xAyjNWSQDOmfhzpnDcN+FI3GmKhQSRcPoTTTmulTJ8LtpAy3baOqRUVHSJWDXqpuMAguCdo60t583ugeeu2Iczh/TE24noWeeH9/cOg2/UFvf6aNc7rtgRMzzSE1gz+EG5PhchoQxc8ONEwcWorQmgIWbos5FqREAigZh1VEpFI7g0zVR04vePn+g0jozNvqanTCoSxamDOiEtfuqDNf0oaz6qKfyugDKa5swsEsWZozsHrOY5WV4tAqYVsh6Nhkea9OQuV+AjMzSa2X5mcpzjDSP1b0/PrcT157UH1dO7ounLh+L726bltAp20mXwZsoKigefzxzCObccFLCMRcfr9TBmjEiNueBsR8sCNo5jWpETecsL2aM7I4rJ/fFlgfOwTe3TkPPPD9G91YWIL1d/coTirTjf/1sPABgn2rb31NRj94FGQbBYTbFyOxhfcmHTK8LJw4oxMSiAjx68Wh0t0iCe2bBNjz/9Xbtb32JjTWmxd3MiQMLMfemU3DSoM4xPZv76Yr06TWOm99ejX2VDSjMUhZmc7+HxmA4bvmObQ+egzevOwE/GtsTJw0qtIxnP21IZ4tHKklWkiFqITZzbRy9hqH/3/jczmZDIvWaTap26kO6ZWPnwzMN2hZjX1gQtHO0JjVxTAnnjOqO/z5tIG4+a6jl9VMGd8ZJgwrx1FdbcfaTi7B4Sxl652cY+veaF5uh3bPhIKM5Z2DnLIzqlYu3rj8BM4/rrjmC9Zirbeo7T8ld/vShxogXbQ5qLsM1U/vFXNMvsvN0lVYXq01TZA0cuXuWDvPe+RmGRVuP00EY0i0bj186Br3yM2J6yALGdohvXDfZ8FhJV1U78Lgc6JHrw4VqJJBew7BqcZgIvbZgFm4MkwpYELRzZLRJPOei2+nAn84aEmO/lwuux+XAeaoTcMOBaoQiAn06RTWCyyf2wcXjjI5Ir8sZY1s3mz5kEppeQJkT0fTVV6VDuF+cHajcQXtcDozrk2e4phcE5pBNAOikNj+XJRAm9++ERf9zGq4+USmw9vkfTrZ8TT3m+5t748kY3kPZ7d8wfRAm9zeWghjYJQtTBxZq9YvcTge+vW06/nbZWADGBL940UHJcCSmIYZpKfwpa+ecMrgznlmwDafEMVPE47krj9f8CyN7GBe5KQM6aTWDLhnfyzLbuUee32CGGdbduGOW5ov7LhiJqoYg7v9kAzaZOmGZq68CMJQ21qPvjDapfyd8v7sSz10xDqN65RmSwqyQZh1ZZG5EjxxDqOUQ026/0EJLkIu+RFbRXPO/Z1p2bfvyplMghMA9HylV1XNMC3ZrFOMDrOsvMUxrw5+yds6k/p0sa/g0h9vp0HbIsn+rZMqAQoTVcFLzAiaRUUUXje2JkwcXGpzLAHDL2UNR3xTCWSO7YY7qIA5HBGYe112L1e9ksYDH0wj0dvEbTx+Mfp0yccbwbpoZZsGfTkV+pgej74ntXySdvjILebSuQqqZr28+zXKXneNz48ubTsHpjy80nDfftx4iws1nDUHfThk4fZgxRFSag2TpiCOlvbUOZTombBqyAW6nAxeO6YFfntQPX950CjwuB4KqjyDeQicTtc4a0RUXjY2NYe+Z58cLV01Ajs9t2P3+5tQB2uJtbswDKCGlr1w9ATeYmoUYQjJdDvzXhN4GW3xRYaZl7f3fTx+klVe4RI21t3JkS/p0yjCEteoxtxZNhkyvC1ef2C9Gq+pdkIEnLxuDZ0zVQpPl8olKcpeVtsYwrQ1vN2yCtF1L3E4HGoMRZMfRCAZ3zcaS7YfjmnL0SBu4x+nAoC7ZSlG2SBh+jxODu2ZhRI9cvKcmZWX5XDi1WxetZtH0oV3w0I9GoUsS8f5W6MMfH734ODxw0ahWKYjWGiRbEtqKBy8aiQcvGtmKs2GY+LAgsClvXncC5qw9ENcJffs5wzBjZDcM7hobTWNGOowHd8uCx+VA/86ZWLe/Gh6XA1/ceAp2l9dHBYFq85Ymj1BEHLEQAIy9bYkIHpe1EPj99EFaBFZztAc50l6EGWMPWBDYlOE9cmIcpHp8biemDEjOvi1NQ9Ip/fLVE/DdtnKtxHDvAj+mD+2CdfurNUHgVm3oydS+1+N2kqGCarKtGm86Y3BS41b++QxehBnbwYKAOWqkIBihhmB2yfYZzCJEhBd/PkErrQ0ATnWxjdckPh4epwPBcHRnby4rcbSY+wgwjB1gQcAcNUO7ZePyiX2aLVegd3zKejonD25ZWKzb5QCawrj7vOEoqWrk3TvDtAIsCJijxud24qEfjWrRY/p0ysCy26dbxvQnQobEnj6sa0rKMjOMHeHwUSZtdMnxtTg8soBNNwzT6rBGwBxTvHDVeMz+fl/cctwMw7QcFgTMMUXvggzccHr8puwMw7QcNg0xDMPYHBYEDMMwNocFAcMwjM1hQcAwDGNzWBAwDMPYHBYEDMMwNocFAcMwjM1hQcAwDGNzSLSwDHC6IaJSALuO8OGFAMpacTrHCnzf9sOu9873HZ++QgjLKo/HnCA4GoioWAgxPt3zaGv4vu2HXe+d7/vIYNMQwzCMzWFBwDAMY3PsJgieT/cE0gTft/2w673zfR8BtvIRMAzDMLHYTSNgGIZhTLAgYBiGsTm2EARENIOINhHRViK6Nd3zaW2I6CUiOkREa3XnCohoLhFtUX/nq+eJiJ5S34vVRDQufTM/OoioNxHNJ6L1RLSOiG5Qz3foeyciHxEtI6JV6n3fo57vR0RL1ft7k4g86nmv+vdW9XpRWm/gKCEiJxH9QEQfq393+Psmop1EtIaIVhJRsXqu1T7nHV4QEJETwNMAzgYwHMDlRDQ8vbNqdV4BMMN07lYA84QQgwDMU/8GlPdhkPpzHYBn22iOqSAE4I9CiOEAJgP4rfq/7ej3HgAwTQgxGsAYADOIaDKARwA8IYQYCKACwDXq+GsAVKjnn1DHHcvcAGCD7m+73PdpQogxunyB1vucCyE69A+AEwB8rvv7NgC3pXteKbjPIgBrdX9vAtBdPe4OYJN6/E8Al1uNO9Z/AHwA4Aw73TuADADfA5gEJbPUpZ7XPvcAPgdwgnrsUsdRuud+hPfbS130pgH4GADZ5L53Aig0nWu1z3mH1wgA9ASwR/f3XvVcR6erEOKAelwCoKt63CHfD1XtHwtgKWxw76p5ZCWAQwDmAtgGoFIIEVKH6O9Nu2/1ehWATm064dbjbwD+B0BE/bsT7HHfAsAXRLSCiK5Tz7Xa55yb19sAIYQgog4bJ0xEWQDeBfAHIUQ1EWnXOuq9CyHCAMYQUR6A9wAMTe+MUg8RnQvgkBBiBRGdmubptDVThRD7iKgLgLlEtFF/8Wg/53bQCPYB6K37u5d6rqNzkIi6A4D6+5B6vkO9H0TkhiIEXhNCzFZP2+LeAUAIUQlgPhSTSB4Ryc2d/t60+1av5wIob9uZtgonAjifiHYCeAOKeehJdPz7hhBin/r7EBTBPxGt+Dm3gyBYDmCQGlngAXAZgA/TPKe24EMAV6nHV0Gxn8vzP1MjCyYDqNKpl8cUpGz9XwSwQQjxuO5Sh753IuqsagIgIj8Uv8gGKALhYnWY+b7l+3ExgK+Eajw+lhBC3CaE6CWEKILyPf5KCPFTdPD7JqJMIsqWxwDOBLAWrfk5T7cTpI0cLecA2AzFjnpHuueTgvv7D4ADAIJQ7IHXQLGFzgOwBcCXAArUsQQlimobgDUAxqd7/kdx31Oh2E5XA1ip/pzT0e8dwHEAflDvey2AP6vn+wNYBmArgLcBeNXzPvXvrer1/um+h1Z4D04F8LEd7lu9v1Xqzzq5hrXm55xLTDAMw9gcO5iGGIZhmASwIGAYhrE5LAgYhmFsDgsChmEYm8OCgGEYxuawIGAYFSIKq9Ud5U+rVaoloiLSVYdlmPYEl5hgmCgNQogx6Z4Ew7Q1rBEwTDOoteAfVevBLyOiger5IiL6Sq35Po+I+qjnuxLRe2q/gFVENEV9KicR/UvtIfCFmhUMIvo9KT0VVhPRG2m6TcbGsCBgmCh+k2noUt21KiHEKAD/gFIBEwD+DmCWEOI4AK8BeEo9/xSAhULpFzAOSjYooNSHf1oIMQJAJYAfq+dvBTBWfZ7rU3NrDBMfzixmGBUiqhVCZFmc3wmlEcx2tchdiRCiExGVQanzHlTPHxBCFBJRKYBeQoiA7jmKAMwVShMRENEtANxCiPuJ6DMAtQDeB/C+EKI2xbfKMAZYI2CY5BBxjltCQHccRtRHNxNKbZhxAJbrKmkyTJvAgoBhkuNS3e/v1ONvoVTBBICfAlikHs8D8GtAayCTG+9JicgBoLcQYj6AW6CUSo7RShgmlfDOg2Gi+NWuX5LPhBAyhDSfiFZD2dVfrp77HYCXiehmAKUArlbP3wDgeSK6BsrO/9dQqsNa4QTwqiosCMBTQukxwDBtBvsIGKYZVB/BeCFEWbrnwjCpgE1DDMMwNoc1AoZhGJvDGgHDMIzNYUHAMAxjc1gQMAzD2BwWBAzDMDaHBQHDMIzN+X+PC3i7cu1EaAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "truncated_mae_history = average_mae_history[10:]\n",
    "plt.plot(range(1, len(truncated_mae_history) + 1), truncated_mae_history)\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Validation MAE\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see in Figure above, validation MAE stops improving significantly after\n",
    "120–140 epochs (this number includes the 10 epochs we omitted). Past that point,\n",
    "we start overfitting.\n",
    "Once you’re finished tuning other parameters of the model (in addition to the\n",
    "number of epochs, you could also adjust the size of the intermediate layers), you can\n",
    "train a final production model on all of the training data, with the best parameters,\n",
    "and then look at its performance on the test data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Training the final model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4/4 [==============================] - 0s 3ms/step - loss: 17.8865 - mae: 2.7644\n"
     ]
    }
   ],
   "source": [
    "# Gets a fresh, compiled model\n",
    "model = build_model()\n",
    "# Trains it on the entirety of the data\n",
    "model.fit(train_data, train_targets,\n",
    "epochs=130, batch_size=16, verbose=0)\n",
    "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here’s the final result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.7643771171569824"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_mae_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We’re still off by a bit under 2800 USD. It’s an improvement! Just like with the two previous tasks, you can try varying the number of layers in the model, or the number  of units per layer, to see if you can squeeze out a lower test error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generating predictions on new data\n",
    "When calling `predict()` on our binary classification model, we retrieved a scalar score between 0 and 1 for each input sample. With our multiclass classification model, we retrieved a probability distribution over all classes for each sample. Now, with this scalar regression model, `predict()` returns the model’s guess for the sample’s price in thousands of dollars:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([8.708372], dtype=float32)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions = model.predict(test_data)\n",
    "predictions[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first house in the test set is predicted to have a price of about 8700 USD."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Celsius to Fahrenheit.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}