Skip to content
Snippets Groups Projects
Normal view History Permalink
Solutions to Exercises - Block 2.ipynb 86.8 KiB
Newer Older
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YHI3vyhv5p85"
   },
   "source": [
    "# Exercise 1 : Conversion from Celsius to Fahrenheit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "F8YVA_634OFk"
   },
   "source": [
    "The problem we will solve is to convert from Celsius to Fahrenheit, where the approximate formula is:\n",
    "\n",
    "$$ f = c \\times 1.8 + 32 $$\n",
    "\n",
    "\n",
    "Of course, it would be simple enough to create a conventional Python function that directly performs this calculation, but that wouldn't be machine learning.\n",
    "\n",
    "\n",
    "Instead, we will give `TensorFlow` some sample Celsius values (0, 8, 15, 22, 38) and their corresponding Fahrenheit values (32, 46, 59, 72, 100).\n",
    "Then, we will train a model that figures out the above formula through the training process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fA93WUy1zzWf"
   },
   "source": [
    "## Import dependencies\n",
    "\n",
    "First, import TensorFlow. Here, we're calling it `tf` for ease of use. We also tell it to only display errors.\n",
    "\n",
    "Next, import [NumPy](http://www.numpy.org/) as `np`. Numpy helps us to represent our data as highly performant lists."
   ]
  },
  {
   "cell_type": "code",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
48 
   "execution_count": 1,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
49 50 51 52 53 54 55 56 57 58 
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "X9uIpOS2zx7k"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
59 
      "2.7.1\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
60 61 62 63 64 65 66 67 68 69 70 71 
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
72 
   "execution_count": 2,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "y_WQEM5MGmg3"
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "logger = tf.get_logger()\n",
    "logger.setLevel(logging.ERROR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AC3EQFi20buB"
   },
   "source": [
    "## Set up training data\n",
    "\n",
    "As we saw before, supervised Machine Learning is all about figuring out an algorithm given a set of inputs and outputs. Since the task in this Codelab is to create a model that can give the temperature in Fahrenheit when given the degrees in Celsius, we create two lists `celsius_q` and `fahrenheit_a` that we can use to train our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gg4pn6aI1vms"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-40.0 degrees Celsius = -40.0 degrees Fahrenheit\n",
      "-10.0 degrees Celsius = 14.0 degrees Fahrenheit\n",
      "0.0 degrees Celsius = 32.0 degrees Fahrenheit\n",
      "8.0 degrees Celsius = 46.0 degrees Fahrenheit\n",
      "15.0 degrees Celsius = 59.0 degrees Fahrenheit\n",
      "22.0 degrees Celsius = 72.0 degrees Fahrenheit\n",
      "38.0 degrees Celsius = 100.0 degrees Fahrenheit\n"
     ]
    }
   ],
   "source": [
    "celsius_q    = np.array([-40, -10,  0,  8, 15, 22,  38],  dtype=float)\n",
    "fahrenheit_a = np.array([-40,  14, 32, 46, 59, 72, 100],  dtype=float)\n",
    "\n",
    "for i,c in enumerate(celsius_q):\n",
    "  print(\"{} degrees Celsius = {} degrees Fahrenheit\".format(c, fahrenheit_a[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wwJGmDrQ0EoB"
   },
   "source": [
    "### Some Machine Learning terminology\n",
    "\n",
    " - **Feature** — The input(s) to our model. In this case, a single value — the degrees in Celsius.\n",
    "\n",
    " - **Labels** — The output our model predicts. In this case, a single value — the degrees in Fahrenheit.\n",
    "\n",
    " - **Example** — A pair of inputs/outputs used during training. In our case a pair of values from `celsius_q` and `fahrenheit_a` at a specific index, such as `(22,72)`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VM7_9Klvq7MO"
   },
   "source": [
    "## 1. Define the Network\n",
    "\n",
    "Next create the model. We will use the simplest possible model we can, a Dense network. Since the problem is straightforward, this network will require only a single layer, with a single neuron.\n",
    "\n",
    "### Build a layer\n",
    "\n",
    "We'll call the layer `l0` and create it by instantiating `tf.keras.layers.Dense` with the following configuration:\n",
    "\n",
    "*   `input_shape=[1]` — This specifies that the input to this layer is a single value. That is, the shape is a one-dimensional array with one member. Since this is the first (and only) layer, that input shape is the input shape of the entire model. The single value is a floating point number, representing degrees Celsius.\n",
    "\n",
Mirko Birbaumer's avatar
units instead of neurons  
Mirko Birbaumer committed
162 
    "*   `units=1` — This specifies the number of units in the layer. The number of units defines how many internal variables the layer has to try to learn how to solve the problem (more later). Since this is the final layer, it is also the size of the model's output — a single float value representing degrees Fahrenheit. (In a multi-layered network, the size and shape of the layer would need to match the `input_shape` of the next layer.)\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pRllo2HLfXiu"
   },
   "outputs": [],
   "source": [
    "l0 = tf.keras.layers.Dense(units=1, input_shape=[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_F00_J9duLBD"
   },
   "source": [
    "### Assemble layers into the model\n",
    "\n",
    "Once layers are defined, they need to be assembled into a model. The Sequential model definition takes a list of layers as argument, specifying the calculation order from the input to the output.\n",
    "\n",
    "This model has just a single layer, l0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cSp-GpLSuMRq"
   },
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "  tf.keras.layers.Dense(units=1, input_shape=[1])\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kiZG7uhm8qCF"
   },
   "source": [
    "## 2. Compile the network, with loss and optimizer functions\n",
    "\n",
    "Before training, the model has to be compiled. When compiled for training, the model is given:\n",
    "\n",
    "- **Loss function** — A way of measuring how far off predictions are from the desired outcome. (The measured difference is called the \"loss\".)\n",
    "\n",
    "- **Optimizer function** — A way of adjusting internal values in order to reduce the loss.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "m8YQN1H41L-Y"
   },
   "outputs": [],
   "source": [
    "model.compile(loss='mean_squared_error',\n",
    "              optimizer=tf.keras.optimizers.Adam(0.1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "17M3Pqv4P52R"
   },
   "source": [
    "These are used during training (`model.fit()`, below) to first calculate the loss at each point, and then improve it. In fact, the act of calculating the current loss of a model and then improving it is precisely what training is.\n",
    "\n",
    "During training, the optimizer function is used to calculate adjustments to the model's internal variables. The goal is to adjust the internal variables until the model (which is really a math function) mirrors the actual equation for converting Celsius to Fahrenheit.\n",
    "\n",
    "`TensorFlow` uses numerical analysis to perform this tuning, and all this complexity is hidden from you so we will not go into the details here. What is useful to know about these parameters are:\n",
    "\n",
    "The loss function ([mean squared error](https://en.wikipedia.org/wiki/Mean_squared_error)) and the optimizer ([Adam](https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/)) used here are standard for simple models like this one, but many others are available. It is not important to know how these specific functions work at this point.\n",
    "\n",
    "One part of the Optimizer you may need to think about when building your own models is the learning rate (`0.1` in the code above). This is the step size taken when adjusting values in the model. If the value is too small, it will take too many iterations to train the model. Too large, and accuracy goes down. Finding a good value often involves some trial and error, but the range is usually within 0.001 (default), and 0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "c-Jk4dG91dvD"
   },
   "source": [
    "## 3. Fit the model\n",
    "\n",
    "Train the model by calling the `fit` method.\n",
    "\n",
    "During training, the model takes in Celsius values, performs a calculation using the current internal variables (called \"weights\") and outputs values which are meant to be the Fahrenheit equivalent. Since the weights are initially set randomly, the output will not be close to the correct value. The difference between the actual output and the desired output is calculated using the loss function, and the optimizer function directs how the weights should be adjusted.\n",
    "\n",
    "This cycle of calculate, compare, adjust is controlled by the `fit` method. The first argument is the inputs, the second argument is the desired outputs. The `epochs` argument specifies how many times this cycle should be run, and the `verbose` argument controls how much output the method produces."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lpRrl7WK10Pq"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Finished training the model\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)\n",
    "print(\"Finished training the model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0-QsNCLD4MJZ"
   },
   "source": [
    "## 4. Evaluate the Model - Display training statistics\n",
    "\n",
    "The `fit` method returns a history object. We can use this object to plot how the loss of our model goes down after each training epoch. A high loss means that the Fahrenheit degrees the model predicts is far from the corresponding value in `fahrenheit_a`.\n",
    "\n",
    "We'll use [Matplotlib](https://matplotlib.org/) to visualize this (you could use another tool). As you can see, our model improves very quickly at first, and then has a steady, slow improvement until it is very near \"perfect\" towards the end.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IeK6BzfbdO6_"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
320 
       "[<matplotlib.lines.Line2D at 0x7fd953358790>]"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
321 322 323 324 325 326 327 328 
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
329 
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjTUlEQVR4nO3de5RkZXnv8e9T175fp+c+MDOCIooITBDUlRXxRogRj5qI0UgS1iEnIRFzl+SsqElMYlaEQKIoCURUIiZEDyxDIoho4g0YFOQyAsNwmR5mmJ7pnu6evlfVc/7Yb3XX9HR11cx0dVVX/T5r1aq9372r+tlDM7953733u83dERERWUys2gWIiEjtU1iIiEhJCgsRESlJYSEiIiUpLEREpKREtQuohFWrVvnmzZurXYaIyIry4IMPHnD3voW21WVYbN68me3bt1e7DBGRFcXMniu2TcNQIiJSksJCRERKUliIiEhJCgsRESlJYSEiIiUpLEREpCSFhYiIlKSwKPDCoQmuvusJnjkwVu1SRERqisKiwODYNNd9cydPvjha7VJERGqKwqJAZ3MSgOGJmSpXIiJSWxQWBTpborAYUViIiBxBYVGgLZUgZnBoXGEhIlJIYVEgFjM6mpMahhIRmUdhMU+nwkJE5CgKi3m6FBYiIkdRWMyjYSgRkaMpLObRMJSIyNEUFvMoLEREjqawmCcfFu5e7VJERGqGwmKerpYk2ZwzNp2tdikiIjVDYTGPpvwQETmawmKefFgcGp+uciUiIrVDYTFPh3oWIiJHUVjMk+9ZaDJBEZE5FQ0LM/sdM3vMzB41sy+ZWZOZbTGz+8xsp5l92cxSYd90WN8Ztm8u+J6rQvsTZvbWStbc1ZIC1LMQESlUsbAwsw3AB4Ft7v5KIA5cAnwCuMbdTwGGgMvCRy4DhkL7NWE/zOz08LlXABcCnzazeKXq1gluEZGjVXoYKgE0m1kCaAH2AhcAt4XtNwPvCMsXh3XC9jeamYX2W919yt2fAXYC51aq4NZUnHjMFBYiIgUqFhbuvgf4W+B5opAYBh4EDrl7JuzWD2wIyxuA3eGzmbB/b2H7Ap+ZZWaXm9l2M9s+MDBw3HWbGZ3NST3TQkSkQCWHobqJegVbgPVAK9EwUkW4+w3uvs3dt/X19Z3Qd2nKDxGRI1VyGOpNwDPuPuDuM8BXgNcBXWFYCmAjsCcs7wE2AYTtncDBwvYFPlMRmnlWRORIlQyL54HzzKwlnHt4I/A4cC/w7rDPpcDtYfmOsE7Y/k2PJmi6A7gkXC21BTgVuL+CddPVnNSlsyIiBRKldzk+7n6fmd0G/BDIAD8CbgD+A7jVzP4itN0YPnIj8AUz2wkMEl0Bhbs/Zmb/ShQ0GeAKd6/oxE2dzUmeOzhWyR8hIrKiVCwsANz9I8BH5jXvYoGrmdx9EviFIt/zceDjS15gEZ3NSQ6pZyEiMkt3cC+gMwxD5XKaplxEBBQWC+psTpJzODydKb2ziEgDUFgsoLMl3MWtey1ERACFxYI05YeIyJEUFgtQWIiIHElhsQCFhYjIkRQWC1BYiIgcSWGxgK4WhYWISCGFxQKak3GScU1TLiKSp7BYgKYpFxE5ksKiiA5NJigiMkthUUSXpikXEZmlsCiiqyXF0Ph0tcsQEakJCosiultSOmchIhIoLIrobkkyOKaehYgIKCyK6m5NMTGTZXKmos9ZEhFZERQWReRvzNNQlIiIwqKonpYUgE5yi4igsCiqS2EhIjJLYVFEd2s0DDU0pmEoERGFRRHd6lmIiMxSWBQxd4JbYSEiorAoIp2I05qKM6SroUREFBaL0ZQfIiIRhcUiuluTDOkubhERhcViultSGoYSEUFhsahoMkH1LEREFBaL6G5JqmchIoLCYlFdLSlGJmfIZHPVLkVEpKoUFovobknijp6YJyINT2GxiO7W/F3cCgsRaWwKi0Xkp/zQSW4RaXQKi0XMzQ+lnoWINDaFxSLy80PpLm4RaXQKi0X05M9Z6C5uEWlwCotFtKTipOIxDUOJSMOraFiYWZeZ3WZmPzGzHWZ2vpn1mNndZvZUeO8O+5qZXWdmO83sx2Z2dsH3XBr2f8rMLq1kzfPq1/xQIiKUERbhL/H3m9mfhvWTzOzcMr//WuC/3P004ExgB/Bh4B53PxW4J6wD/CxwanhdDlwffl4P8BHgNcC5wEfyAbMcelvTHBybWq4fJyJSk8rpWXwaOB94b1gfBT5V6kNm1gn8NHAjgLtPu/sh4GLg5rDbzcA7wvLFwOc98gOgy8zWAW8F7nb3QXcfAu4GLiyj7iXR25bioHoWItLgygmL17j7FcAkQPgLO1XG57YAA8A/m9mPzOyfzKwVWOPue8M++4A1YXkDsLvg8/2hrVj7EczscjPbbmbbBwYGyiivPL2tKQ4eVliISGMrJyxmzCwOOICZ9QHlTJaUAM4Grnf3s4Ax5oacAHB3z3/viXL3G9x9m7tv6+vrW4qvBKCnNc2gehYi0uDKCYvrgK8Cq83s48B3gL8s43P9QL+73xfWbyMKjxfD8BLhfX/YvgfYVPD5jaGtWPuy6G1LcXgqw+RMdrl+pIhIzSkZFu5+C/CHwF8Be4F3uPu/lfG5fcBuM3tZaHoj8DhwB5C/oulS4PawfAfwgXBC/TxgOAxXfR14i5l1hxPbbwlty6I33Guh3oWINLJEsQ3hKqS8/cCXCre5+2AZ3//bwC1mlgJ2Ab9KFFD/amaXAc8Bvxj2vRO4CNgJjId9cfdBM/tz4IGw35+V+bOXRP7GvIOHp1nf1bxcP1ZEpKYUDQvgQaLzCQacBAyF5S7geaIT2Ity94eAbQtseuMC+zpwRZHvuQm4qdTPq4TetjSALp8VkYZWdBjK3be4+1bgG8DPu/sqd+8F3gbctVwFVltvQc9CRKRRlXOC+zx3vzO/4u7/Cby2ciXVlp42nbMQEVlsGCrvBTP7v8AXw/r7gBcqV1JtaU8nSMVjHNAwlIg0sHJ6Fu8F+ogun/0qsJq5u7nrnpnR05piUMNQItLASvYswpVHVy5DLTVLU36ISKMrGRZmdi8L3GXt7hdUpKIa1NOqsBCRxlbOOYvfL1huAt4FZCpTTm1a1ZbmmQNj1S5DRKRqyhmGenBe03fN7P4K1VOTelpTuhpKRBpaOcNQhXdyx4BzgM6KVVSDelpTjE9nmZjO0pyKV7scEZFlV84wVOGd3BngGeCyShZVa1aFey0Ojk2xMdVS5WpERJZfOWHxcnefLGwws3SF6qlJPa1hyo/D02zsVliISOMp5z6L7y3Q9v2lLqSW9eoubhFpcIvNOruW6Il0zWZ2FtEwFEAH0FD/vM7PD3XgsO7iFpHGtNgw1FuBXyF62NDVBe2jwB9XsKaasyrMPHtAd3GLSIMqGhbufjNws5m9y93/fRlrqjmt6QStqTgDo+pZiEhjWmwY6v3u/kVgs5n97vzt7n71Ah+rW33tafaPTpbeUUSkDi02DNUa3tuWo5Ba19eeVs9CRBrWYsNQnw3vH1u+cmrX6vYmduwbqXYZIiJVUc4d3H3A/wY2F+7v7r9WubJqT197mv9+Sj0LEWlM5dyUdzvwP0SPV81Wtpza1deeZnQyw+RMlqakpvwQkcZSTli0uPsfVbySGtfXHl0+OzA6xaaehrrNRESkrDu4v2ZmF1W8khqXD4v9OsktIg2onLC4kigwJsxsxMxGzazhzvT2teV7Frp8VkQaTznPs2hfjkJq3eqOuWEoEZFGU87VUGcv0DwMPOfuDfPEvN7WNDFTWIhIYyrnBPengbOBR8L6GcCjQKeZ/Ya731Wp4mpJPGb0tKYZ0GSCItKAyjln8QJwlruf4+7nAK8GdgFvBv6mgrXVnNXtafaPKCxEpPGUExYvdffH8ivu/jhwmrvvqlxZtamvXT0LEWlM5QxDPWZm1wO3hvX3AI+Hp+XNVKyyGtTXnubJF0erXYaIyLIrp2fxK8BO4EPhtSu0zQBvqExZtSk/mWAu59UuRURkWZVz6ewE8Mnwmu/wkldUw1a3p8nknEMTM/SEp+eJiDSCkj0LMzvVzG4zs8fNbFf+tRzF1Zq5u7h1Y56INJZyhqH+GbgeyBANO30e+GIli6pVazqaAHRFlIg0nHLCotnd7wHM3Z9z948CP1fZsmrT2hAW+4bVsxCRxlLO1VBTZhYDnjKz3wL20KBPz8v3LPYqLESkwZQ7kWAL8EHgHOCXgUsrWVStSiVirGpLs29kotqliIgsq5Jh4e4PuPthd+93919193e6+w/K/QFmFjezH5nZ18L6FjO7z8x2mtmXzSwV2tNhfWfYvrngO64K7U+Y2VuP4ziXzLrOJvUsRKThFB2GMrM7Fvugu7+9zJ9xJbAD6AjrnwCucfdbzewzwGVEJ9AvA4bc/RQzuyTs9x4zOx24BHgFsB74hpm91N2r8tS+tZ1N7B4cr8aPFhGpmsXOWZwP7Aa+BNwH2LF+uZltJDoZ/nHgd83MgAuAXwq73Ax8lCgsLg7LALcB/xD2vxi41d2ngGfMbCdwLvD9Y61nKazrbOL+Zwar8aNFRKpmsWGotcAfA68EriWaOPCAu3/b3b9d5vf/HfCHQC6s9wKHCqY27wc2hOUNROFE2D4c9p9tX+Azs8zscjPbbmbbBwYGyizv2K3tbGJ4Yobx6YaZnV1EpHhYuHvW3f/L3S8FziOa8uNb4YqokszsbcB+d39waUpdnLvf4O7b3H1bX19fxX7Ouk5dPisijWfRS2fDZIE/B7wX2AxcB3y1zO9+HfD28PzuJqJzFtcCXWaWCL2HjUSX4hLeNwH9ZpYAOoGDBe15hZ9Zdms7moEoLLb2NeQVxCLSgIr2LMzs80TnBc4GPubuP+Xuf+7uZf1F7e5XuftGd99MdIL6m+7+PuBe4N1ht0uB28PyHcxdkvvusL+H9kvC1VJbgFOB+4/lIJdSvmehK6JEpJEs1rN4PzBGdDXTB6NzzUB0otvdvaPYB0v4I+BWM/sL4EfAjaH9RuAL4QT2IFHA4O6Pmdm/Ao8TTTlyRbWuhILonAXAvhGFhYg0jqJh4e7l3LBXFnf/FvCtsLyL6Gqm+ftMAr9Q5PMfJ7qiquqaknG6W5LsHdaNeSLSOJYsEBrJ2s5mneAWkYaisDgOuotbRBqNwuI4rO1sUs9CRBpKOQ8/ag2zzmJmLzWzt5tZsvKl1a51HU0cHJtmcqZq59lFRJZVOT2L/waazGwDcBfRrLOfq2RRtW59V3SvxQuHdJJbRBpDOWFh7j4OvBP4tLv/AtGkfg1rU08LAP1DCgsRaQxlhYWZnQ+8D/iP0BavXEm1b1NP1LPYPaTZZ0WkMZQTFh8CrgK+Gm6Q20p0F3bDWt3eRDJu7B5Uz0JEGkPJx6qGGWa/DRBOdB9w9w9WurBaFo8ZG7qa1bMQkYZRztVQ/2JmHWbWCjwKPG5mf1D50mrbpp4W+vUQJBFpEOUMQ53u7iPAO4D/BLYQXRHV0DZ2t+gEt4g0jHLCIhnuq3gHcIe7zwBe0apWgE09zRwcm2ZsSg9BEpH6V05YfBZ4FmgF/tvMTgZGKlnUSrCxW5fPikjjKBkW7n6du29w94s88hzwhmWoraZt6g6Xz+q8hYg0gHJOcHea2dX551ub2SeJehkNLX9jnq6IEpFGUM4w1E3AKPCL4TUC/HMli1oJeltTNCfjutdCRBpCyfssgJe4+7sK1j9mZg9VqJ4Vw8zY1NNMv3oWItIAyulZTJjZ6/MrZvY6QP+cJjrJvVsnuEWkAZTTs/g/wOfNrDOsDwGXVq6kleOknhZ+sOsg7k7BM8pFROpOOVdDPezuZwKvAl7l7mcBF1S8shXgJX2tjE9n2TeiByGJSH0r+0l57j4S7uQG+N0K1bOivKSvDYBdA2NVrkREpLKO97GqGnMBts6GxeEqVyIiUlnHGxYNP90HwJqONK2pOE+rZyEida7oCW4zG2XhUDCguWIVrSBmxpa+Vp5Wz0JE6lzRsHD39uUsZKV6SV8b258dqnYZIiIVdbzDUBJsXdXGC8MTTM5kq12KiEjFKCxO0Na+VtzhmQM6byEi9UthcYK29kVzKuq8hYjUM4XFCdq6SvdaiEj9U1icoOZUnA1dzepZiEhdU1gsgVNWt/HkiwoLEalfCoslcNq6dnbuH2Umm6t2KSIiFaGwWAIvX9vBTNZ13kJE6pbCYgm8fF0HADv2jpTYU0RkZVJYLIGtfa2k4jF27FNYiEh9qlhYmNkmM7vXzB43s8fM7MrQ3mNmd5vZU+G9O7SbmV1nZjvN7MdmdnbBd10a9n/KzGruwUvJeIxTVrfxk72j1S5FRKQiKtmzyAC/5+6nA+cBV5jZ6cCHgXvc/VTgnrAO8LPAqeF1OXA9ROECfAR4DXAu8JF8wNSS09a187iGoUSkTlUsLNx9r7v/MCyPAjuADcDFwM1ht5uBd4Tli4HPe+QHQJeZrQPeCtzt7oPuPgTcDVxYqbqP1xkbOhkYnWLfsJ6aJyL1Z1nOWZjZZuAs4D5gjbvvDZv2AWvC8gZgd8HH+kNbsfb5P+NyM9tuZtsHBgaW9gDK8KqNXQA83H9o2X+2iEilVTwszKwN+HfgQwWPZQXA3Z0lepCSu9/g7tvcfVtfX99SfOUxecX6DhIx4+Hdh5b9Z4uIVFpFw8LMkkRBcYu7fyU0vxiGlwjv+0P7HmBTwcc3hrZi7TWlKRnntHXt/Lh/uNqliIgsuUpeDWXAjcAOd7+6YNMdQP6KpkuB2wvaPxCuijoPGA7DVV8H3mJm3eHE9ltCW805c2MXD/cfIpfTU2dFpL5UsmfxOuCXgQvM7KHwugj4a+DNZvYU8KawDnAnsAvYCfwj8JsA7j4I/DnwQHj9WWirOWdu6mJ0MsOuA5onSkTqS9HHqp4od/8O0fO6F/LGBfZ34Ioi33UTcNPSVVcZP7W5B4D7nhnklNV6Kq2I1A/dwb2ENve20Nee5v5narLjIyJy3BQWS8jMeM2WHu7bNUjUURIRqQ8KiyX2mq297BuZ5PnB8WqXIiKyZBQWS+z8rdF5i+89fbDKlYiILB2FxRJ7SV8b6zub+NYT+0vvLCKyQigslpiZ8TOnreY7Tx1gOqMn54lIfVBYVMAbXraaseks25/VVVEiUh8UFhXw2pf0korH+MYODUWJSH1QWFRAazrBT7+0jzsf2aupP0SkLigsKuTtr17PvpFJ7tdQlIjUAYVFhbzp5atpScW5/aEXql2KiMgJU1hUSEsqwYWvWMvXHn6BsalMtcsRETkhCosKev/5JzM6leErP+yvdikiIidEYVFBZ23q4owNnXzue8/qRLeIrGgKiwoyMy57/RaeHhjjzkf3lv6AiEiNUlhU2M+fuZ6Xrmnj6rueJJPVHd0isjIpLCosHjN+/y0vY9eBMW767jPVLkdE5LgoLJbBm09fw5tevoZP3vUkTw/okasisvIoLJaBmfGX/+uVNKfi/PoXHmR4YqbaJYmIHBOFxTJZ3dHEZ95/Ds8dHOOyzz3A8LgCQ0RWDoXFMjpvay/XXnIWD/cf4p3Xf5dH9wxXuyQRkbIoLJbZRWes4+ZfO5fRyQwXf+q7/MG/Pcwj/cN6ZreI1DSrx7+ktm3b5tu3b692GYs6ND7Ntfc8xS33Pc90JsfJvS1sO7mHMzd1csrqNjZ1t7C2s4lkXHkuIsvDzB50920LblNYVNfw+Az/+ehe7n78RR7uP8SBw9Oz22IGazua2NjdwsbuZjb2RO+bwvq6ziYSChMRWSIKixXC3XlheJJnD4zRPzTOnqEJ+mdf4+wdmaTwP1c8Zmzoaubk3hY297bOvm9e1cLG7haakvHqHYyIrDiLhUViuYuR4syiv/w3dDUvuH06k2Pv8AS7B6Pw2D00zvODEzx3cIz/99AeRiczBd8F6zujIDm5t5XN+fdVLZzc00pzSkEiIuVTWKwgqUSMk3tbObm39aht7s6h8RmePTjGcwfHj3j/+mP7GBybPmL/NR3pI0LkpJ4W1nc1s7G7mb62NLGYLddhicgKoLCoE2ZGd2uK7tYUZ53UfdT24YkZnp8NkTGePTjOcwfHuPeJAQZGj5xCPRk31nY2sb4z6uVs6G5mfVf02tDVxPquZlpS+tURaST6P75BdDYnOWNjJ2ds7Dxq29hUht1D4+w9NEn/oQleKHj9YNdB9o1MMn+G9e6WZEGANLO+q4m1nc2saU+ztrOJNR1NOmciUkcUFkJrOsFpazs4bW3Hgtsz2Rwvjk7NBkj/0FyYPH9wnO8/fZDDCzwNsLM5ydqOJtZ0Nh0RIms7ovc1nWlWtWrIS2QlUFhISYl4bNET7xANc704MsmLI5PsG54My1PsC21P7BthYHTqqB5KImb0tadZ09HEqrY0fe2p8J5mVVv+lWJVe5r2dAIzBYtINSgsZEl0NifpbE7y0jXtRffJZHMcHJtm3/Ak+0Ym2T8Sve8bnmL/6CT9Q+M8tHuIwbHpo0IFIJ2IReHRnqavLXVUmPS2pulpTdHdmqS7JaUbGkWWkMJClk0iHouGnzqaOHOR/bI5Z3BsmgOHp2ZfA6NTHDg8zYHRKQYOT7Hn0CQP7R5mcOzo3kpeezoxe9K/pyUZLbekokBpSdETQiXf3tWSVMCIFKGwkJoTD0NTfe3pkvtmc87QeBQsg4enGRyfZmh8hqGxaQbHphkanw7BM82TLx5maHya8els0e9rTyfoaE7S0ZykszlBZ3OSjqao19TRnH9PzPakCrfphL7UM4WFrGjxmM0OR5VrcibLofGZI8Ik/35ofIaRyRlGJmYYmcjw7IFxRiZnGJ6YWTRkILoPJgqPBG1NSdrTCVrTcdrSSdqb5pbb0nHamhK0phK0NSVoSxe8mhI0J+M6NyM1R2EhDacpGWdtZ5y1nU3H9LmZbI6RiSg4RiYz0fvsenifyDAyMcPoVIbDkzMMjE5xeCoz+8oWGzMrELPoCrUobBK0pOI0p+K0pKIgiZbjRy6Hbfl988vzt6UTMQWRHJcVExZmdiFwLRAH/snd/7rKJUmDScZj9Lal6T2GXkwhd2dyJjcbHGNTGUYnC5bD++HJuXA5PJlhfCbL5HSW/aOTjE9Hy+MzWcans0xncsdUgxk0J+M0JaPgiF5x0sm55aZkaEvEQnt+ueAz8z6f/0wqESMVj5FMGInY3HIyHiMZD+txIx4zhdYKsyLCwsziwKeANwP9wANmdoe7P17dykTKZ2bRv/pT8bLOx5Qjk80xmckxPp1hYjrLRAiRifAan8kyMZ2J2mZCWwiZqUyWqUyOyZnofWom+p6h8Vy0nskyOZNjKr/9GINpMWZR+CZjRjJxZJDkgyWZiJEK64n43HIyHiMRiwInEYInEYsRD23ReuH73P5HbY/P225GPF74+dgR+8fMiMWI3s2IWcFyrMiyQSy2wP755RVyn9GKCAvgXGCnu+8CMLNbgYsBhYU0tEQ8Rls8Rlu68v8ruzvT2dxcwMzMhUq+bSbrzGRyzGRzTGdz0Xo2RyabYzosz20P67OvufXpzJHbxqazzGRyZHI5pjM5su5ks04m52Rzhe85cjmi9xU0oXZhcBwVKLNBU7jtyAAyM8zAgAtOW82f/NzpS17jSgmLDcDugvV+4DWFO5jZ5cDlACeddNLyVSbSIMwsDEnF6WhKVrucknI5j0IlHybZKEyyoT2TPTpo8uu53PwgypHJOjmPQjPrBcu5aDnnHtYLlwu3ET4XlnPR8vzv8bDPEd9TsF8uN/c9uZzjMLvNHdZ2Fr959kSslLAoyd1vAG6A6HkWVS5HRKosFjNiGLqieWmslDuQ9gCbCtY3hjYREVkGKyUsHgBONbMtZpYCLgHuqHJNIiINY0UMQ7l7xsx+C/g60aWzN7n7Y1UuS0SkYayIsABw9zuBO6tdh4hII1opw1AiIlJFCgsRESlJYSEiIiUpLEREpCRzr7/718xsAHjuBL5iFXBgicpZKXTMjUHH3BiO95hPdve+hTbUZVicKDPb7u7bql3HctIxNwYdc2OoxDFrGEpEREpSWIiISEkKi4XdUO0CqkDH3Bh0zI1hyY9Z5yxERKQk9SxERKQkhYWIiJSksChgZhea2RNmttPMPlztepaKmd1kZvvN7NGCth4zu9vMngrv3aHdzOy68GfwYzM7u3qVHz8z22Rm95rZ42b2mJldGdrr9rjNrMnM7jezh8Mxfyy0bzGz+8KxfTlM84+ZpcP6zrB9c1UP4ASYWdzMfmRmXwvrdX3MZvasmT1iZg+Z2fbQVtHfbYVFYGZx4FPAzwKnA+81s6V/kG11fA64cF7bh4F73P1U4J6wDtHxnxpelwPXL1ONSy0D/J67nw6cB1wR/nvW83FPARe4+5nAq4ELzew84BPANe5+CjAEXBb2vwwYCu3XhP1WqiuBHQXrjXDMb3D3VxfcT1HZ320Pz3ht9BdwPvD1gvWrgKuqXdcSHt9m4NGC9SeAdWF5HfBEWP4s8N6F9lvJL+B24M2NctxAC/BDomfVHwASoX3295zo+TDnh+VE2M+qXftxHOvG8JfjBcDXAGuAY34WWDWvraK/2+pZzNkA7C5Y7w9t9WqNu+8Ny/uANWG57v4cwlDDWcB91Plxh+GYh4D9wN3A08Ahd8+EXQqPa/aYw/ZhoHdZC14afwf8IZAL673U/zE7cJeZPWhml4e2iv5ur5iHH0nluLubWV1eQ21mbcC/Ax9y9xEzm91Wj8ft7lng1WbWBXwVOK26FVWWmb0N2O/uD5rZz1S5nOX0enffY2argbvN7CeFGyvxu62exZw9wKaC9Y2hrV69aGbrAML7/tBeN38OZpYkCopb3P0robnujxvA3Q8B9xINwXSZWf4fhoXHNXvMYXsncHB5Kz1hrwPebmbPArcSDUVdS30fM+6+J7zvJ/pHwblU+HdbYTHnAeDUcBVFCrgEuKPKNVXSHcClYflSojH9fPsHwhUU5wHDBV3bFcOiLsSNwA53v7pgU90et5n1hR4FZtZMdI5mB1FovDvsNv+Y838W7wa+6WFQe6Vw96vcfaO7byb6f/ab7v4+6viYzazVzNrzy8BbgEep9O92tU/U1NILuAh4kmic90+qXc8SHteXgL3ADNF45WVE47T3AE8B3wB6wr5GdFXY08AjwLZq13+cx/x6onHdHwMPhddF9XzcwKuAH4VjfhT409C+Fbgf2An8G5AO7U1hfWfYvrXax3CCx/8zwNfq/ZjDsT0cXo/l/66q9O+2pvsQEZGSNAwlIiIlKSxERKQkhYWIiJSksBARkZIUFiIiUpLCQuqemWXD7Jz515LNKGxmm61gNt9F9vuomY2HO27zbYeXswaRE6HpPqQRTLj7q6tdBNGkdb8H/FG1CylkZgmfm0dJZEHqWUjDCs8E+JvwXID7zeyU0L7ZzL4Z5v6/x8xOCu1rzOyrFj0v4mEze234qriZ/aNFz5C4K9w9vZCbgPeYWc+8Oo7oGZjZ75vZR8Pyt8zsGjPbbmY7zOynzOwr4ZkFf1HwNQkzuyXsc5uZtYTPn2Nm3w4Tzn29YDqIb5nZ31n0LIQrT/xPU+qdwkIaQfO8Yaj3FGwbdvczgH8gmr0U4O+Bm939VcAtwHWh/Trg2x49L+JsortnIXpOwKfc/RXAIeBdReo4TBQYx/qX87RHzyz4DNEUDlcArwR+xczyM6a+DPi0u78cGAF+M8yN9ffAu939nPCzP17wvSl33+bunzzGeqQBaRhKGsFiw1BfKni/JiyfD7wzLH8B+JuwfAHwAZid3XXYoqeRPePuD4V9HiR6dkgx1wEPmdnfHkP9+TnKHgEe8zCvj5ntIpog7hCw292/G/b7IvBB4L+IQuXuMNtunGjal7wvH0MN0uAUFtLovMjysZgqWM4CxYahcPdDZvYvRL2DvAxH9vKbinx/bt7PyjH3//D82p1oTqDH3P38IuWMFatTZD4NQ0mje0/B+/fD8veIZjAFeB/wP2H5HuA3YPYhQ53H+TOvBn6dub/oXwRWm1mvmaWBtx3Hd55kZvlQ+CXgO0RPROvLt5tZ0sxecZw1S4NTWEgjmH/O4q8LtnWb2Y+JziP8Tmj7beBXQ/svM3eO4UrgDWb2CNFw03E9o93dDxA9gyAd1meAPyOaBfVu4CfFP13UE0TPGd8BdAPXu/s00TTcnzCzh4lm3n1t8a8QKU6zzkrDCg/M2Rb+8haRRahnISIiJalnISIiJalnISIiJSksRESkJIWFiIiUpLAQEZGSFBYiIlLS/wcXyYKUCF72rgAAAABJRU5ErkJggg==\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.xlabel('Epoch Number')\n",
    "plt.ylabel(\"Loss Magnitude\")\n",
    "plt.plot(history.history['loss'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LtQGDMob5LOD"
   },
   "source": [
    "## 5. Use the model to predict values\n",
    "\n",
    "Now you have a model that has been trained to learn the relationship between `celsius_q` and `fahrenheit_a`. You can use the predict method to have it calculate the Fahrenheit degrees for a previously unknown Celsius degrees.\n",
    "\n",
    "So, for example, if the Celsius value is 100, what do you think the Fahrenheit result will be? Take a guess before you run this code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oxNzL4lS2Gui"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
374 
      "[[211.27141]]\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 
     ]
    }
   ],
   "source": [
    "print(model.predict([100.0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jApk6tZ1fBg1"
   },
   "source": [
    "The correct answer is $100 \\times 1.8 + 32 = 212$, so our model is doing really well.\n",
    "\n",
    "### To review\n",
    "\n",
    "\n",
    "*   We created a model with a Dense layer\n",
    "*   We trained it with 3500 examples (7 pairs, over 500 epochs).\n",
    "\n",
    "Our model tuned the variables (weights) in the Dense layer until it was able to return the correct Fahrenheit value for any Celsius value. (Remember, 100 Celsius was not part of our training data.)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zRrOky5gm20Z"
   },
   "source": [
    "## Looking at the layer weights\n",
    "\n",
    "Finally, let's print the internal variables of the Dense layer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kmIkVdkbnZJI"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
427 
      "These are the layer variables: [array([[1.8298433]], dtype=float32), array([28.287079], dtype=float32)]\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 
     ]
    }
   ],
   "source": [
    "print(\"These are the layer variables: {}\".format(model.get_weights()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RSplSnMvnWC-"
   },
   "source": [
    "The first variable is close to ~1.8 and the second to ~32. These values (1.8 and 32) are the actual variables in the real conversion formula.\n",
    "\n",
    "This is really close to the values in the conversion formula. We can show how a Dense layer works, but for a single neuron with a single input and a single output, the internal math looks the same as [the equation for a line](https://en.wikipedia.org/wiki/Linear_equation#Slope%E2%80%93intercept_form), $y = mx + b$, which has the same form as the conversion equation, $f = 1.8c + 32$.\n",
    "\n",
    "Since the form is the same, the variables should converge on the standard values of 1.8 and 32, which is exactly what happened.\n",
    "\n",
    "With additional neurons, additional inputs, and additional outputs, the formula becomes much more complex, but the idea is the same.\n",
    "\n",
    "### A little experiment\n",
    "\n",
    "Just for fun, what if we created more Dense layers with different units, which therefore also has more variables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Y2zTA-rDS5Xk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Finished training the model\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
469 470 471 472 473 474 475 476 477 478 479 480 481 
      "[[211.74744]]\n",
      "Model predicts that 100 degrees Celsius is: [[211.74744]] degrees Fahrenheit\n",
      "These are the l0 variables: [array([[-0.07715903, -0.20700853,  0.09303203, -0.7815757 ]],\n",
      "      dtype=float32), array([ 2.2330647, -3.0049088,  2.656292 , -3.0834572], dtype=float32)]\n",
      "These are the l1 variables: [array([[ 0.37331566,  0.670332  , -0.6500106 , -0.16718145],\n",
      "       [-0.13477354, -0.1432596 ,  0.85609394,  0.2517057 ],\n",
      "       [ 1.290541  ,  0.86174154, -0.10719326,  0.621717  ],\n",
      "       [ 0.2648061 , -0.6288689 ,  1.0755255 , -0.2527884 ]],\n",
      "      dtype=float32), array([ 2.6429222,  2.9728847, -2.9935007,  2.8113384], dtype=float32)]\n",
      "These are the l2 variables: [array([[ 0.8368167],\n",
      "       [ 0.8074423],\n",
      "       [-1.373887 ],\n",
      "       [ 0.3907403]], dtype=float32), array([2.95269], dtype=float32)]\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 
     ]
    }
   ],
   "source": [
    "l0 = tf.keras.layers.Dense(units=4, input_shape=[1])\n",
    "l1 = tf.keras.layers.Dense(units=4)\n",
    "l2 = tf.keras.layers.Dense(units=1)\n",
    "model = tf.keras.Sequential([l0, l1, l2])\n",
    "model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1))\n",
    "model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)\n",
    "print(\"Finished training the model\")\n",
    "print(model.predict([100.0]))\n",
    "print(\"Model predicts that 100 degrees Celsius is: {} degrees Fahrenheit\".format(model.predict([100.0])))\n",
    "print(\"These are the l0 variables: {}\".format(l0.get_weights()))\n",
    "print(\"These are the l1 variables: {}\".format(l1.get_weights()))\n",
    "print(\"These are the l2 variables: {}\".format(l2.get_weights()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xrpFFlgYhCty"
   },
   "source": [
    "As you can see, this model is also able to predict the corresponding Fahrenheit value really well. But when you look at the variables (weights) in the `l0` and `l1` layers, they are nothing even close to ~1.8 and ~32. The added complexity hides the \"simple\" form of the conversion equation.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 2 : O-Rings seen with Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook calculates a logistic regression using Keras. It's basically meant to show the principles of Keras.\n",
    "\n",
    "###  Datset\n",
    "\n",
    "We investigate the data set of the challenger flight with broken O-rings (Y=1) vs start temperature."
   ]
  },
  {
   "cell_type": "code",
Simon van Hemert's avatar
minor changes  
Simon van Hemert committed
531 
   "execution_count": 12,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
532 533 534 535 536 537 538 539 
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Broken O-rings')"
      ]
     },
Simon van Hemert's avatar
minor changes  
Simon van Hemert committed
540 
     "execution_count": 12,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZM0lEQVR4nO3deZhldX3n8ffH7kZbgqDSY7QXRR+WOC4gpag4StxY4gBmnBGUPGqMPUZJ3IaMJA4hLiORuD0uKOKuD7g8hGkV05q4LyjdoiJgG0SUblQQA26tQPOdP+4pvBSnbt3q5tS5Xbxfz3Ofe8/vbN/6ddX59FnuOakqJEma6Q59FyBJmkwGhCSplQEhSWplQEiSWhkQkqRWBoQkqVVnAZHkXUmuSvKdWcY/Pcm3k1yY5CtJHtxVLZKk+etyD+I9wGEjxv8AeExVPRB4BXB6h7VIkuZpaVcLrqovJLnPiPFfGRo8D1jVVS2SpPnrLCDm6dnAJ2cbmWQtsBZg1113PXC//fZbqLokaVHYuHHjz6pqxXzm6T0gkvwxg4B41GzTVNXpNIegpqamasOGDQtUnSQtDkl+ON95eg2IJA8CzgAOr6pr+qxFknRLvV3mmmQNcDbwZ1X1vb7qkCS162wPIsmZwCHAnkk2A38PLAOoqrcBJwF3B96aBODGqprqqh5J0vx0eRXTsXOM/wvgL7pavyRpx/hNaklSKwNCktTKgJAktTIgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSKwNCktTKgJAktTIgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSq6VdLTjJu4AnAVdV1QNaxgd4I3AE8BvgmVX1ja7q0Y4754ItnLp+E1deu5V77bGcEw7dl6MPWNnrevuqaXvr3Rm97JwLOfNrV7CtiiUJxx60mlce/cC+y1p0/TyJOgsI4D3Am4H3zTL+cGDv5nUQcFrzrgl0zgVbOPHsC9l6wzYAtly7lRPPvhCg0z/KUesFeqlplL76qSsvO+dCPnDej24e3lZ183CfIbHY+nlSdXaIqaq+APx8xCRHAe+rgfOAPZLcs6t6tGNOXb/p5j/GaVtv2Map6zf1tt6+ahplEmvaEWd+7Yp5tS+UxdbPk6rLPYi5rASGf8s2N20/njlhkrXAWoA1a9YsSHG6pSuv3Tqv9j7X23VNo/TVT13ZVjWv9oWy2Pp5Uu0UJ6mr6vSqmqqqqRUrVvRdzu3SvfZYPq/2hVhvXzWNMok17YglybzaF8pi6+dJ1WdAbAFWDw2vato0gU44dF+WL1tyi7bly5ZwwqH79rbevmoaZRJr2hHHHrR6Xu0LZbH186Tq8xDTOuD4JGcxODl9XVXd6vCSJsP0ib+FvmpknPVO0pUsffVTV6ZPRE/aVUyLrZ8nVaqjY4lJzgQOAfYEfgr8PbAMoKre1lzm+mbgMAaXuT6rqjbMtdypqanasGHOySRJQ5JsrKqp+czT2R5EVR07x/gCnt/V+iVJO2anOEktSVp4BoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWnUaEEkOS7IpyaVJXtoyfk2Szya5IMm3kxzRZT2SpPF1FhBJlgBvAQ4H7g8cm+T+MyZ7GfDhqjoAOAZ4a1f1SJLmp8s9iIcBl1bVZVV1PXAWcNSMaQq4S/N5d+DKDuuRJM1DlwGxErhiaHhz0zbsZOC4JJuBc4G/altQkrVJNiTZcPXVV3dRqyRphr5PUh8LvKeqVgFHAO9Pcquaqur0qpqqqqkVK1YseJGSdHvUZUBsAVYPDa9q2oY9G/gwQFV9FbgTsGeHNUmSxtRlQJwP7J1kryS7MDgJvW7GND8CHgeQ5I8YBITHkCRpAnQWEFV1I3A8sB64hMHVShcleXmSI5vJXgI8J8m3gDOBZ1ZVdVWTJGl8S7tceFWdy+Dk83DbSUOfLwYO7rIGSdL26fsktSRpQs0ZEElekOQuGXhnkm8keeJCFCdJ6s84exB/XlW/AJ4I3BX4M+CUTquSJPVunIBI834E8P6qumioTZK0SI0TEBuTfIpBQKxPshtwU7dlSZL6Ns5VTM8G9gcuq6rfJLk78KxOq5Ik9W6cgNi/eb9vcvORpeuSLG2+6yBJWoTGCYi3Ag8Bvs3g3MMDgIuA3ZP8ZVV9qsP6JEk9GeccxJXAAc3N8g4EDgAuA54AvKbL4iRJ/RknIPZprlwCbv72835VdVl3ZUmS+jbOIaaLkpzG4IE/AE8FLk5yR+CGziqTJPVqnD2IZwKXAi9sXpc1bTcAf9xNWZKkvs25B1FVW4HXNq+ZfnWbVyRJmghzBkSSgxk8GvTew9NX1X27K0uS1LdxzkG8E3gRsBHY1m05kqRJMU5AXFdVn+y8EknSRBknID6b5FTgbOB3041V9Y3OqpIk9W6cgDioeZ8aaivgsbd9OZKkSTHOVUxeyipJt0OzBkSS46rqA0le3Da+ql7XXVmSpL6N2oPYtXnfbSEKkSRNllkDoqrenmQJ8Iuqev0C1iRJmgAjb7VRVduAYxeoFknSBBnnKqYvJ3kz8CHg19ONXuYqSYvbfJ4o9/KhNi9zlaRFrtPLXJMcBrwRWAKcUVWntEzzPxjc66mAb1XV07Z3fZKk2844exA3S/LxqnrSmNMuAd7C4Mlzm4Hzk6xrHjg0Pc3ewInAwVX1H0n+03zqkSR1Z5znQQxbOY9pHwZcWlWXVdX1DB44dNSMaZ4DvKWq/gOgqq6aZz2SpI7MNyAumMe0K4ErhoY3c+uA2QfYJ8mXk5zXHJK6lSRrk2xIsuHqq6+eX8WSpO0yr4Coqj+/jde/FNgbOITB5bTvSLJHy3pPr6qpqppasWLFbVyCJKnNyIBIcniSLyT5WfP6fJIjxlz2FmD10PCqpm3YZmBdVd1QVT8AvscgMCRJPZs1IJI8B3gFgyuM7tu8/gE4OcnaMZZ9PrB3kr2S7AIcA6ybMc05DPYeSLIng0NOl83rJ5AkdWLUVUwvAh5VVT8favtMksOBLwGnj1pwVd2Y5HhgPYPLXN9VVRcleTmwoarWNeOemORiBk+rO6GqrtmBn0eSdBsZFRCZEQ4AVNU1ScZaeFWdC5w7o+2koc8FvLh5SZImyKhzEL9I8uCZjU3bL7srSZI0CUbtQbwEWJfk3cDGpm0KeAZwXNeFSZL6NeseRFV9icGX3e4APLN53QF4eDNOkrSIjbzVRlX9FDhp1DSSpMVpvt+kliTdThgQkqRWBoQkqdWct/tOsg9wAnDv4emrygcGSdIiNs7zID4CvA14B4NvO0uSbgfGCYgbq+q0ziuRJE2Ucc5BfCzJ85LcM8ndpl+dVyZJ6tU4exDPaN5PGGorBnd3lSQtUnMGRFXttRCFSJImy5yHmJLcOcnLkpzeDO+d5EndlyZJ6tM45yDeDVwPPLIZ3gK8srOKJEkTYZyAuF9VvQa4AaCqfgOM90AISdJOa5yAuD7JcgYnpklyP+B3nVYlSerdOFcx/T3wL8DqJB8EDmZw629J0iI2TkBsBP4UeDiDQ0svAHbrsihJUv/G+qIccENVfaKqPg6saNokSYvYOAHxfxl8m3rXJAcCH8VHjkrSojfOF+U+kWQZ8GkGh5aeXFXf67wySVKvZg2IJG+iuXKpsTvwfeD4JFTVX3ddnCSpP6P2IDbMGN7YZSGSpMkya0BU1XunPyfZBdinGdxUVTeMs/AkhwFvBJYAZ1TVKbNM998YnNt4aFXNDCZJUg/GeaLcIcB7gcsZXOa6OskzquoLc8y3BHgL8ARgM3B+knVVdfGM6XZjcOns17ajfklSR8a5ium1wBOr6jFV9WjgUOD1Y8z3MODSqrqsqq4HzgKOapnuFcA/Ar8ds2ZJ0gIYJyCWVdWm6YHmCqZlY8y3ErhiaHhz03azJA8BVlfVJ0YtKMnaJBuSbLj66qvHWLUkaUeNExAbk5yR5JDm9Q5ufQJ73pLcAXgd8JK5pq2q06tqqqqmVqxYsaOrliSNYZxbbTwXeD4wfVnrF4G3jjHfFmD10PCqpm3absADgM8lAfhDYF2SIz1RLUn9GxkQzYnmb1XVfgz+tz8f5wN7J9mLQTAcAzxtemRVXQfsObSuzwH/y3CQpMkw8hBTVW0DNiVZM98FV9WNwPHAeuAS4MNVdVGSlyc5cruqlSQtmHEOMd0VuCjJ14FfTzdW1Zwb+ao6Fzh3RttJs0x7yBi1SJIWyDgB8X86r0KSNHHGuVnf56c/J9kTuKaqasQskqRFYNZzEEkenuRzSc5OckCS7wDfAX7a3EJDkrSIjdqDeDPwtwzu4voZ4PCqOi/JfsCZDB5DKklapEZdxbS0qj5VVR8BflJV5wFU1XcXpjRJUp9GBcRNQ5+3zhjnOQhJWuRGHWJ6cJJfMLiD6/LmM83wnTqvTJLUq1HPg1iykIVIkibLODfrkyTdDhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlq1WlAJDksyaYklyZ5acv4Fye5OMm3k/xbknt3WY8kaXydBUSSJcBbgMOB+wPHJrn/jMkuAKaq6kHAR4HXdFWPJGl+utyDeBhwaVVdVlXXA2cBRw1PUFWfrarfNIPnAas6rEeSNA9dBsRK4Iqh4c1N22yeDXyyw3okSfOwtO8CAJIcB0wBj5ll/FpgLcCaNWsWsDJJuv3qcg9iC7B6aHhV03YLSR4P/B1wZFX9rm1BVXV6VU1V1dSKFSs6KVaSdEtdBsT5wN5J9kqyC3AMsG54giQHAG9nEA5XdViLJGmeOguIqroROB5YD1wCfLiqLkry8iRHNpOdCvwB8JEk30yybpbFSZIWWKfnIKrqXODcGW0nDX1+fJfrlyRtP79JLUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqtbTLhSc5DHgjsAQ4o6pOmTH+jsD7gAOBa4CnVtXlXdYk7czOuWALp67fxJXXbuVeeyznhEP35egDVgLw9Hd8lS9//+c3T3vw/e7GB5/ziLHmHTXuZedcyJlfu4JtVSxJOPag1bzy6AfucL3jjN/eZXdVc1f1TqpUVTcLTpYA3wOeAGwGzgeOraqLh6Z5HvCgqnpukmOAJ1fVU0ctd2pqqjZs2NBJzdIkO+eCLZx49oVsvWHbzW3Lly3h1X/6QD6y4Ue3CIdp0yExal5g1nEbfvhzPnDej2613OMevmbODe6odR59wMo5x2/vsruqGWbvpx2pd6FCIsnGqpqa1zwdBsQjgJOr6tBm+ESAqnr10DTrm2m+mmQp8BNgRY0oyoDQ7dXBp3yGLdduvVX7yj2Wt7ZPu/yUPxk5LzDruJ9c91u2tfw5Lkn4/quP2O56v/zSx845fnuX3VXNMHs/7Ui9c817W9megOjyENNK4Iqh4c3AQbNNU1U3JrkOuDvws+GJkqwF1gKsWbOmq3qliXblLCEwW/uOznvltVuZ7X9qbRvg+a6zq5+ny5rnO25HljsJdoqT1FV1elVNVdXUihUr+i5H6sW9mv/Fjts+7ryjxi1JWsfN1j7uOscZv73L7qrmruqdZF0GxBZg9dDwqqatdZrmENPuDE5WS5rhhEP3ZfmyJbdoW75sCSccui8H3+9urfNMt4+ad9S4Yw9aTZvZ2setd5zx27vsrmruqt5J1uUhpvOBvZPsxSAIjgGeNmOadcAzgK8CTwE+M+r8g3R7Nn0ys+1KmKMPWDnyKqZR806bbbnAdl0RNNc6x6lpe/uiq5q7qHeSdXaSGiDJEcAbGFzm+q6qelWSlwMbqmpdkjsB7wcOAH4OHFNVl41apiepJWn+Ju0kNVV1LnDujLaThj7/FvjvXdYgSdo+O8VJaknSwjMgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSq05v1teFJL8ENvVdxwx7MuMhRxNiEuuypvFY0/gmsa5JrGnfqtptPjN0erO+jmya7x0Ju5Zkw6TVBJNZlzWNx5rGN4l1TWpN853HQ0ySpFYGhCSp1c4YEKf3XUCLSawJJrMuaxqPNY1vEutaFDXtdCepJUkLY2fcg5AkLQADQpLUauIDIsmSJBck+XgzvFeSryW5NMmHkuwyIXW9J8kPknyzee2/wPVcnuTCZt0bmra7Jfl0kn9v3u86ATWdnGTLUD8dscA17ZHko0m+m+SSJI/ou59G1NVbXyXZd2i930zyiyQv7LOvRtTU9+/Ui5JclOQ7Sc5Mcqe+t1Oz1DTvbdTEn4NI8mJgCrhLVT0pyYeBs6vqrCRvA75VVadNQF3vAT5eVR9d6Fqaei4HpqrqZ0NtrwF+XlWnJHkpcNeq+t8913Qy8Kuq+qeFqmNGTe8FvlhVZzR/tHcG/pYe+2lEXS+kx74aqm0JsAU4CHg+PfdVS03Poqd+SrIS+BJw/6ra2myfzgWOoKft1IiaDmGe26iJ3oNIsgr4E+CMZjjAY4HpH/C9wNF91zXBjmLQR9BTX02SJLsDjwbeCVBV11fVtfTcTyPqmhSPA75fVT9kcn6nhmvq21JgeZKlDIL9x/S/nZpZ05Xbs5CJDgjgDcDfADc1w3cHrq2qG5vhzcDKCahr2quSfDvJ65PccYFrKuBTSTYmWdu03aOqftx8/glwjwmoCeD4pp/etcCHc/YCrgbencHhwTOS7Er//TRbXdBfXw07Bjiz+dx3X00brgl66qeq2gL8E/AjBsFwHbCRHrdTbTVV1aea0fPaRk1sQCR5EnBVVW3su5ZhI+o6EdgPeChwN2Chd7sfVVUPAQ4Hnp/k0cMja3AscaGPJ7bVdBpwP2B/Br+8r13AepYCDwFOq6oDgF8DLx2eoKd+mq2uPvsKgOZw15HAR2aO66mv2mrqrZ+aMDqKQcjfC9gVOGyh1j9uTUmOYzu2URMbEMDBwJHNceyzGOyyvRHYo9ltAljF4Dhkr3Ul+UBV/bgGfge8G3jYQhbV/K+BqroK+Odm/T9Nck+A5v2qvmuqqp9W1baqugl4BwvbT5uBzVX1tWb4oww2zL3202x19dxX0w4HvlFVP22G++6rW9XUcz89HvhBVV1dVTcAZzPYRvS5nWqr6ZHbs42a2ICoqhOralVV3YfB7uRnqurpwGeBpzSTPQP4fxNQ13FDfzRhcLzxOwtVU5Jdk+w2/Rl4YrP+dQz6CBa4r2arabqfGk9mAfupqn4CXJFk36bpccDF9NhPo+rqs6+GHMstD+X02leNW9TUcz/9CHh4kjs3f/vTv1N9bqfaarpku7ZRVTXxL35/9h3gvsDXgUsZ7GLecULq+gxwYdPpHwD+YAHruC/wreZ1EfB3TfvdgX8D/h34V+BuE1DT+5t++jaDjc09F/jfbH9gQ7P+c4C79tlPc9TVd1/tClwD7D7U1mtfzVJT3/30D8B3m7/99wN37Hs7NUtN895GTfxlrpKkfkzsISZJUr8MCElSKwNCktTKgJAktTIgJEmtls49iTQ5kkxfZgnwh8A2BrepgMEX8a7vpbAWSQ4Brq+qr3Sw7PsAlwCbqmr/pm0bg8sYpx0NrAbeDtxUVQ+4revQ4mZAaKdSVdcw+M5A73eGbWpYWr+/585MhwC/AsYOiDmWN9P3p8OhsXXGMMDlGdz++uPj1iBN8xCTdnpJDkzy+eamgOuHvjH6ueamZBsyeMbCQ5OcncGzDF7ZTHOfDJ7B8MFmmo8mufMYy31DBs+4eEGS/5rBvf8vSPKvSe7R/A//ucCLMrj3/n/J4H78Txmq+1fN+yFJvphkHYNvUC9JcmqS85sbq/3PBe1QqWFAaGcX4E3AU6rqQOBdwKuGxl9fVVPA2xjc7uD5wAOAZzaHqwD2Bd5aVX8E/AJ4XpJlcyx3l6qaqqrXMrj3/sNrcKO9s4C/qarLm3W+vqr2r6ovzvFzPAR4QVXtAzybwR04H8rgxmrPSbLXGH2xPL9/GMw/jzG9NJKHmLSzuyODDf6nB7eYYQmDO3pOW9e8XwhcVM2tqpNcxuD4/LXAFVX15Wa6DwB/DfzLHMv90NDnVcCHmj2MXYAfbMfP8fWqmp7vicCDhvY2dgf2HmO5bYeYpO1mQGhnFwYb/kfMMv53zftNQ5+nh6d//2feb6bGWO6vhz6/CXhdVa1rTkyfPMs8N9LstSe5A4MwaVtegL+qqvWzLEdaEB5i0s7ud8CKJI8ASLIsyX+e5zLWTM8PPI3BIaNN81ju7vz+ds7PGGr/JbDb0PDlwIHN5yOBZbMsbz3wl81hLpLsk98/QEhaMAaEdnY3Mbit8j8m+RbwTeCR81zGJgYPNLqEwV1UT2sulx13uScDH0myEfjZUPvHgCdPn6Rm8KyCxzTLewS33GsYdgaDW0Z/I8l3GFym6t6+Fpx3c9XtWnO10cd3tu8IzKfunfVnVP/cg5B2TtuA3ZN8c9REzZ7Lx7jlno00FvcgJEmt3IOQJLUyICRJrQwISVIrA0KS1MqAkCS1+v/YKMuQWSKjDwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as imgplot\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tempfile\n",
    "data = np.asarray(pd.read_csv('./challenger.txt', sep=','), dtype='float32')\n",
    "plt.plot(data[:,0], data[:,1], 'o')\n",
    "plt.axis([40, 85, -0.1, 1.2])\n",
    "plt.xlabel('Temperature [F]')\n",
    "plt.ylabel('Broken O-rings')"
   ]
  },
  {
   "cell_type": "code",
Simon van Hemert's avatar
minor changes  
Simon van Hemert committed
575 
   "execution_count": 13,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 
   "metadata": {},
   "outputs": [],
   "source": [
    "y_values = data[:,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mathematical Notes\n",
    "\n",
    "We are considering the likelihood $P(y_i=1|x_i)$ for the class $y_i=1$ given the $i-$th data point $x_i$ ($x_i$ could be a vector). This is given by:\n",
    "\n",
    "$\n",
    "P(y_i=1 | x_i) = \\frac{e^{(b +  x_i w)}}{1 + e^{(b + x_i w)}} = [1 + e^{-(b + x_i w)}]^{-1}\n",
    "$\n",
    "\n",
    "If we have more than one data point, which we usually do, we have to apply the equation above to each of the N data points. In this case we can use a vectorized version with $x=(x_1,x_2,\\ldots,x_N)$ and $y=(y_1,y_2,\\ldots,y_N$)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numpy code\n",
    "This numpy code, shows the calculation for one value using `NumPy` (like a single forward pass)"
   ]
  },
  {
   "cell_type": "code",
Simon van Hemert's avatar
minor changes  
Simon van Hemert committed
607 
   "execution_count": 14,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.882916\n",
      "[0.999 0.998 0.998 0.998 0.999 0.996 0.996 0.998 1.    0.999 0.998 0.988\n",
      " 0.999 1.    0.999 0.993 0.998 0.978 0.992 0.985 0.993 0.992 1.   ]\n"
     ]
    }
   ],
   "source": [
    "# Data\n",
    "N = len(data)\n",
    "x = data[:,0]\n",
    "y = data[:,1]\n",
    "# Initial Value for the weights\n",
    "w = -0.20\n",
    "b = 20.0\n",
    "# Log-Likelihood \n",
    "p_1 = 1 / (1 + np.exp(-x*w - b))\n",
    "like = y * np.log(p_1) + (1-y) * np.log(1-p_1)\n",
    "print(-np.mean(like))\n",
    "print(np.round(p_1,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Better values from intuition\n",
    "\n",
    "Now lets try to find better values for $W$ and $b$. Lets assume $W$ is given with $-1$. We want the probability\n",
    "for a dammage $P(y_i=1 | x_i)$ to be $0.5$.\n",
    "Determine an appropriate value for $b$.\n",
    "Hint: at which $x$ value should $P(y_i=1 | x_i)$ be $0.5$, look at the data. At this $x$ value the term $1 + e^{-(b + W’ x_i)}$ must be $2$.\n",
    "\n",
    "**Solution**\n",
    "\n",
    "$P(y=1 | x) = 0.5$ at $x \\approx 65$ \n",
    "\n",
    "$-(b + (-1) x_i) = 0 \\rightarrow b = 65$  "
   ]
  },
  {
   "cell_type": "code",
Simon van Hemert's avatar
minor changes  
Simon van Hemert committed
655 
   "execution_count": 15,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9094435\n",
      "[0.269 0.007 0.018 0.047 0.119 0.001 0.    0.007 1.    0.881 0.007 0.\n",
      " 0.119 1.    0.119 0.    0.007 0.    0.    0.    0.    0.    0.999]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAb/klEQVR4nO3de3hU9b3v8fc3kwQiIheJwYRE0CIIoQWbighbrbaVi/XS7nMqre1uty2nrezTy35s5dTH9tj21F09Z+vTWi3V1lZ3dat1W06lpT3VPm1R2URRERAFkUuQiyAIEkIu3/PHmtAhZC5JZmatrHxez8MzM2utmfXxJ3yy8ltrZszdERGR+CoJO4CIiBSWil5EJOZU9CIiMaeiFxGJORW9iEjMqehFRGIua9Gb2U/NbJeZvZRm/SfM7EUzW21mT5nZe/IfU0REeiuXI/p7gdkZ1m8CLnD3KcC3gcV5yCUiInlSmm0Dd/+zmY3NsP6plIfPAGPykEtERPIka9H30DXAb9OtNLMFwAKAIUOGvHfixIl53r2ISLw9++yzb7p7ZU+ek7eiN7P3ExT9rHTbuPtiklM7DQ0N3tjYmK/di4gMCGa2uafPyUvRm9m7gbuBOe6+Jx+vKSIi+dHnyyvNrA54FPiku7/S90giIpJPWY/ozewB4EJglJltA74JlAG4+13AjcDJwI/MDKDN3RsKFVhERHoml6tu5mdZ/1ngs3lLJCIieaV3xoqIxJyKXkQk5lT0IiIxp6IXEYk5Fb2ISMyp6EVEYk5FLyIScyp6EZGYU9GLiMScil5EJOZU9CIiMaeiFxGJORW9iEjMqehFRGJORS8iEnMqehGRmFPRi4jEnIpeRCTmVPQiIjGnohcRiTkVvYhIzKnoRURiTkUvIhJzKnoRkZhT0YuIxFxptg3M7KfApcAud6/vZr0BtwNzgUPAp939uXwHlfx5bFUTtyxbz/Z9zVQPr+C6SyZwxbSaUPcbVqbe5u2PbnhsNQ+s2Eq7Owkz5k+v5TtXTAk7VuzGOYqyFj1wL/BD4Bdp1s8Bxif/TAfuTN5KBD22qolFj66mubUdgKZ9zSx6dDVAQf9xZdovEEqmTMIap0K54bHV3P/MlqOP292PPg6z7OM2zlFl7p59I7OxwG/SHNH/GPiTuz+QfLweuNDd38j0mg1jyr3x2tG9Ci29d6Clle7+l5vB0EFlyQe5vFLKRmZ/W3bc/eB298FWWh06KMHd6MBwjJJEgjYSNLeX0EqCNkppI0GrJ6Csgr+bPBbKT4TyIcHtoBOD26GjYVgtDK+FwcN6Oxxpzbz5CZr2NR+3vGZ4Bcuvvyjv+yu0MxYtpb2b//EJMzZ+b24IiQJxG+diMLNn3b2hJ8/J5Yg+mxpga8rjbcllxxW9mS0AFgBMqjkJpn48D7uXnnh4+aa06/5x+jgg+w/+Y39SeMqyLvc7t/MO/vifWzCcEjooMT96P9HWQSntlNJOGe2U0kaZtTPEDjO47QBsexOOHIQj70Droe7zDBoGw8YEpT9iLIw7H8ZdEPxQ6KXt3ZRPpuVR113JZ1peLHEb56jKR9HnzN0XA4sBGhoanDk3F3P3AtzzQvojqH+cU7gjqB+sTb9fIP1R3ZdSMnW0B4XfcgAOvAH7tsD+rbBvK+zfFtzf9BdYcRckymHs38GZs+HMDwU/AHqgenhFt5mqk3n7m4RZ2iP6MMVtnKMqH1fdNAG1KY/HJJdJBF13yQQqyhLHLKsoS3DdJRNC22/OmUoSMPgkGFYDYxqg/iMw80sw71b4+IPwheXw9dfhU0vgnAWwbzP89jq4/T1wx3R44jvQcrDPefuj+dNre7S8WOI2zlGVjyP6JcBCM3uQ4CTs/mzz8xKezhNcxb7KIZf95iVTaTmcfkHw55Lvwp6N8MoyeHUZ/PlWeOlX8NG7oea9fc7bn3SecI3aVTdxG+eoynoy1sweAC4ERgE7gW8CZQDuflfy8sofArMJLq/8jLs3ZttxQ0ODNzZm3Uwkf15fDo8ugIM74MJFMOsrwW8JIv1Ib07G5nTVTSGo6CUUzfvg8a8GR/Z158FHfgzD68JOJZKz3hS93hkrA0vFcPjoPXDlYtixGu6cBasfCTuVSEGp6GXgMYP3fAy+8Fc4ZSL86hpY8t/p9g0GIjGgopeBa8RY+PRSmLEQnvs5rHk07EQiBaGil4EtUQofvAlOnQq/WxTM4YvEjIpepCQBH74N3tkNT3w77DQieaeiFwGongbn/DdYeQ9s09VgEi8qepFOF30Dhp4K//fL0N4WdhqRvFHRi3QaNBTm/AvsXA0r7gw7jUjeqOhFUp314eCD0J78X8GHponEgIpeJJUZzL0luL/0a7q2XmJBRS/S1fC64LNwXvktvPybsNOI9JmKXqQ7534BquqDo/qWA2GnEekTFb1IdxJlcOltwRec/ElfkCP9m4peJJ3a9wVfbrLqfl1uKf2ail4kk8lXwuF9sOWpsJOI9JqKXiSTMy6C0sHw8uNhJxHpNRW9SCblQ4Kyf/lxXWop/ZaKXiSbifNg/1Z444Wwk4j0iopeJJszZ4OVaPpG+i0VvUg2Q0ZB3QwVvfRbKnqRXEycB7vWwN7Xwk4i0mMqepFcTJwX3L68NNwcIr2gohfJxYixUDVF0zfSL6noRXI1cR5sfQYO7g47iUiP5FT0ZjbbzNab2QYzu76b9XVm9qSZrTKzF81sbv6jioRs4jzwDnjld2EnEemRrEVvZgngDmAOMAmYb2aTumx2A/CQu08DrgJ+lO+gIqEbPQWG1emji6XfyeWI/hxgg7u/5u5HgAeBy7ts48BJyfvDgO35iygSEWbBUf3GJ6HlYNhpRHKWS9HXAFtTHm9LLkv1LeBqM9sGLAX+qbsXMrMFZtZoZo27d2ueU/qhifOgvQU2/jHsJCI5y9fJ2PnAve4+BpgL3Gdmx722uy929wZ3b6isrMzTrkWKqG4GVIzQ1TfSr+RS9E1AbcrjMcllqa4BHgJw96eBwcCofAQUiZREKZw5Jzgh294adhqRnORS9CuB8WY2zszKCU62LumyzRbgYgAzO4ug6DU3I/E0cR4c3g+bl4edRCQnWYve3duAhcAyYB3B1TVrzOwmM7ssudk/A58zsxeAB4BPu+szXSWmzrgISis0fSP9RmkuG7n7UoKTrKnLbky5vxaYmd9oIhFVfgK86+Kg6Od8P7gaRyTC9M5Ykd6YOA/eboLtq8JOIpKVil6kNzo/o369PuRMok9FL9IbJ4yE0e+GrSvCTiKSlYpepLeqp8H2F/RdshJ5KnqR3qqeCi379WUkEnkqepHeOnVqcPvG82GmEMlKRS/SW6dMgkQ5bH8+7CQiGanoRXqrtByqJusSS4k8Fb1IX5w6Fd54USdkJdJU9CJ9UT1NJ2Ql8lT0In1RPTW41fSNRJiKXqQvKs+CxCBdeSORpqIX6YujJ2SfDzuJSFoqepG+qp4Kb7wAHR1hJxHplopepK+qp0HL2/DWprCTiHRLRS/SV53vkNUJWYkoFb1IX52SPCGropeIUtGL9FWiDEbXB/P0IhGkohfJh1OnBlfe6ISsRJCKXiQfqqfCkQN6h6xEkopeJB+qpwW3euOURJCKXiQfKifqhKxElopeJB8SZTB6it4hK5GkohfJF71DViJKRS+SL6dOTZ6Q3Rh2EpFj5FT0ZjbbzNab2QYzuz7NNv/VzNaa2Roz+2V+Y4r0A50nZDV9IxGTtejNLAHcAcwBJgHzzWxSl23GA4uAme4+Gfhy/qOKRFzlRCgdrCtvJHJyOaI/B9jg7q+5+xHgQeDyLtt8DrjD3d8CcPdd+Y0p0g8kSqGqXlfeSOTkUvQ1wNaUx9uSy1KdCZxpZsvN7Bkzm93dC5nZAjNrNLPG3bt39y6xSJRVTwu+Q1YnZCVC8nUythQYD1wIzAd+YmbDu27k7ovdvcHdGyorK/O0a5EIOfoOWZ2QlejIpeibgNqUx2OSy1JtA5a4e6u7bwJeISh+kYFFH1ksEZRL0a8ExpvZODMrB64ClnTZ5jGCo3nMbBTBVI4+9EMGns4TsrryRiIka9G7exuwEFgGrAMecvc1ZnaTmV2W3GwZsMfM1gJPAte5+55ChRaJrERp8h2yOqKX6CjNZSN3Xwos7bLsxpT7Dnw1+UdkYKueBs//MjghW6L3JEr49LdQJN9OnQpHDsKeDWEnEQFU9CL5Vz01uNX0jUSEil4k30ZNCE7I7ngx7CQigIpeJP8SpcHVNztfCjuJCKCiFymM0fWw4yVwDzuJiIpepCCqpsChN+HgzrCTiKjoRQpidH1wu0PTNxI+Fb1IIVRNDm53rg43hwgqepHCqBgBw2p1RC+RoKIXKZSqel15I5GgohcplNH18Oar0NocdhIZ4FT0IoVSVQ/eDrvWhZ1EBjgVvUihjJ4S3Gr6RkKmohcplBHjoGyITshK6FT0IoVSUhJcZqkjegmZil6kkPRRCBIBKnqRQqqqh5b9sH9r2ElkAFPRixRS5wlZzdNLiFT0IoV0yiTANE8voVLRixTSoBNh5DjYoc+8kfCo6EUKTR+FICFT0YsU2ugpsHcTtBwMO4kMUCp6kUKrqgccdq0NO4kMUCp6kUI7+iUkmqeXcORU9GY228zWm9kGM7s+w3YfNTM3s4b8RRTp54bVwuBhmqeX0GQtejNLAHcAc4BJwHwzm9TNdkOBLwEr8h1SpF8zC6ZvdC29hCSXI/pzgA3u/pq7HwEeBC7vZrtvA/8CHM5jPpF4qKqHnWugoyPsJDIA5VL0NUDq+7e3JZcdZWZnA7Xu/nimFzKzBWbWaGaNu3fv7nFYkX5rdD20vgNvbQo7iQxAfT4Za2YlwP8B/jnbtu6+2N0b3L2hsrKyr7sW6T+qkidkNU8vIcil6JuA2pTHY5LLOg0F6oE/mdnrwLnAEp2QFUlxyllgCV15I6HIpehXAuPNbJyZlQNXAUs6V7r7fncf5e5j3X0s8Axwmbs3FiSxSH9UVgGjxuuErIQia9G7exuwEFgGrAMecvc1ZnaTmV1W6IAisaGPQpCQlOaykbsvBZZ2WXZjmm0v7HsskRgaXQ8vPQLNb0HFiLDTyACid8aKFEtV55eFrwk3hww4KnqRYjn6UQiavpHiUtGLFMuJVXDCKNipK2+kuFT0IsVi9rcvCxcpIhW9SDFV1cOuddDeFnYSGUBU9CLFNHoKtLfAng1hJ5EBREUvUkzV04LbbSvDzSEDiopepJhGnRmckN28POwkMoCo6EWKyQzGzoTX/xp2EhlAVPQixXbaLNi/Fd7aHHYSGSBU9CLFNnZmcKvpGykSFb1IsVWeBRUjNX0jRaOiFym2khI47TwVvRSNil4kDGNnwb7NsG9r9m1F+khFLxKGsbOCW83TSxGo6EXCcMpkGDxc0zdSFCp6kTB0ztPriF6KQEUvEpbTZsLe1+Dt7WEnkZhT0YuEpXOe/nUd1UthqehFwjJ6CgwaBps1Ty+FpaIXCUtJAurO1RG9FJyKXiRMY2fBnlfhwI6wk0iMqehFwqTPvZEiUNGLhGn0e6B8qKZvpKByKnozm21m681sg5ld3836r5rZWjN70cz+aGan5T+qSAwlSqFuut44JQWVtejNLAHcAcwBJgHzzWxSl81WAQ3u/m7gEeD7+Q4qEltjZ8Gb6+Hg7rCTSEzlckR/DrDB3V9z9yPAg8DlqRu4+5Pufij58BlgTH5jisTYafrcGymsXIq+Bkj9iL1tyWXpXAP8ti+hRAaU6qlQNkTTN1Iwpfl8MTO7GmgALkizfgGwAKCuri6fuxbpvxJlwTy9juilQHI5om8CalMej0kuO4aZfQD4BnCZu7d090LuvtjdG9y9obKysjd5ReLptJmway28syfsJBJDuRT9SmC8mY0zs3LgKmBJ6gZmNg34MUHJ78p/TJGY0+fTSwFlLXp3bwMWAsuAdcBD7r7GzG4ys8uSm90CnAg8bGbPm9mSNC8nIt2pPhtKK1T0UhA5zdG7+1JgaZdlN6bc/0Cec4kMLKXlUHuO3jglBaF3xopExdhZsPMlfe6N5J2KXiQq6j8a3K68O9wcEjsqepGoOPkMmDgPVt4DRw5l314kRyp6kSiZsRCa98ILvww7icSIil4kSurODa7AefpH0NERdhqJCRW9SJSYwXkLYe9GeOV3YaeRmFDRi0TNWZfDsDp4+odhJ5GYUNGLRE2iFM79fPDmqabnwk4jMaCiF4miaZ+EQSfB03eEnURiQEUvEkWDT4KzPwVr/gP2bc2+vUgGKnqRqJr++eB2xV3h5pB+T0UvElXDa2HylfDcL+Dw22GnkX5MRS8SZTOuhZa3g7IX6SUVvUiU1ZwdfCnJirugvS3sNNJPqehFom7GQti/Fdb9Ouwk0k+p6EWi7szZcPK74C//Cq3NYaeRfkhFLxJ1JSVw8TeDz6r/909CW7dfySySlopepD+YdBl8+DbY8Ad4+DPQ3hp2IulHVPQi/cV7Pw1zboH1j8OvPquTs5KznL4zVkQiYvoCaG+B398ApYPgijuhJBF2Kok4Fb1If3PeP0HbYXjiO0HZX3p7MI8vkoaKXqQ/Ov+64KTsn2+BRDnMvTX4LHuRbqjoRfqr938jOLJ/6gdweD9c8HUYNT7sVBJBKnqR/soMPvjt4Ij+qR/A6odh/Ifg3C/A6e/XEb4cpYk9kf7MDC6+Eb6yBi5cBNtXwX1Xwo9mwLM/1xusBABz9+wbmc0GbgcSwN3ufnOX9YOAXwDvBfYAH3P31zO9ZkNDgzc2NvYytkj/9tiqJm5Ztp7t+5qpHl7BdZdM4IppNQB84idPs3zj3qPbzjxjJP/2uRk5PXdJ4yZeXHYPH2lZwqSSzbSUD2fQWXOgqp6fbTyBO9dWsMtPImHG/Om1fOeKKX3Om8v63r72DY+t5oEVW2l3z2vmQuUtBjN71t0bevScbEVvZgngFeCDwDZgJTDf3dembPNF4N3u/nkzuwq40t0/lul1VfQyUD22qolFj66mubX96LKKsgTf+8gUHm7cckzJd+os+0zPBVLWOeeWrOMzZX/g/MGvUdGy++j2u30Y6zrqWO+11NSdztz3TYITRkLFCKjovB0OibKsea+YVpN1fW/HonHzXu5/Zstxz7n63LqsZZ/7OOUvb7HKvlBFPwP4lrtfkny8CMDdv5eyzbLkNk+bWSmwA6j0DC+uopeBaubNT9C07/gplZrhFd0u7/T6zfMyPhdIu+7I/l28y7Zwlm1hom1hQslWJthWBluGd9haAkoHs7/VaPYyjngpLZTTSintGIlEKZNqRvBC0wEOt0O7l9BOCU5wbqCsrJQZp49Kniuwv93C0fMHf371TQ63th99TqfBZQlaWttxOG6dAZdMrjrmdbr60/rdx5Rxp4qy4D0H6dZdOKEy/Xhked1sz80Xu+rfelz0uZyMrQFSv8tsGzA93Tbu3mZm+4GTgTePCWi2AFgAUFdX15OcIrGxPU2Zp1ve1+du39eMM5TdPpmnmZyyxhlKM6u/1gDNb0HzXmjeB4f2wuF9wRU9bS38+i/rKaeVcmtjEEcop50SOki0dTCp/EQOth0iYR2UWjuDaMWS1WxtDocAD+qao8d9fvRmVNvbmHVzPNhG8mdCmmPFvYfSrwNGtx2A7n4GdL6ZON26PZm/4CXj62Z5bpiKetWNuy8GFkNwRF/MfYtERXWaI/d0y3N9LnR/RF89vIId+w/Tftwv2MYhGwIjxwHj0u7zx6vS/xax/FMX8bUMv2UsX3BRxv+ez2V4bveZIWHGxi/Ozfi61/TyN5/lX8ycN9PrZntu3lzb86upcrnqpgmoTXk8Jrms222SUzfDCE7KikgX110y4egUQqeKsgTXXTKBmWeM7PY5ncszPTfTuvnTa+lOuuW55s1lfW9fu1CZC5U3ynI5ol8JjDezcQSFfhXw8S7bLAH+AXga+HvgiUzz8yIDWedJu+6u3LhiWk3Gq24yPbdTutcFenUFS7Z95pKpt2NRqMyFyBtluV5eORe4jeDyyp+6+3fN7Cag0d2XmNlg4D5gGrAXuMrdX8v0mjoZKyLSc7256ianOXp3Xwos7bLsxpT7h4H/0pMdi4hIceidsSIiMaeiFxGJORW9iEjMqehFRGJORS8iEnMqehGRmFPRi4jEnIpeRCTmVPQiIjGnohcRiTkVvYhIzOX0oWYF2bHZAWB9KDtPbxRdviwlIqKYS5lyo0y5i2KuKGaa4O5De/KEon7xSBfre/oJbIVmZo1RywTRzKVMuVGm3EUxV1Qz9fQ5mroREYk5Fb2ISMyFWfSLQ9x3OlHMBNHMpUy5UabcRTFXLDKFdjJWRESKQ1M3IiIxp6IXEYm5ohW9mSXMbJWZ/Sb5eJyZrTCzDWb272ZWXqwsWXLda2abzOz55J+pRc7zupmtTu67MblspJn9wcxeTd6OiECmb5lZU8o4zS1ypuFm9oiZvWxm68xsRtjjlCFXaGNlZhNS9vu8mb1tZl8Oc6wyZAr779RXzGyNmb1kZg+Y2eCweypNph53VNHm6M3sq0ADcJK7X2pmDwGPuvuDZnYX8IK731mUMJlz3Qv8xt0fKXaWZJ7XgQZ3fzNl2feBve5+s5ldD4xw96+HnOlbwEF3v7VYObpk+jnwF3e/O/mP7wTgfxDiOGXI9WVCHKuUbAmgCZgOXEvIY9VNps8Q0jiZWQ3wV2CSuzcn+2kpMJeQeipDpgvpYUcV5YjezMYA84C7k48NuAjoDPpz4IpiZMmUK8IuJxgjCGmsosTMhgHnA/cAuPsRd99HyOOUIVdUXAxsdPfNROfvVGqmsJUCFWZWSvAD+g3C76mumbb35kWKNXVzG/A1oCP5+GRgn7u3JR9vA2qKlCVTrk7fNbMXzexfzWxQkTM58Hsze9bMFiSXVbn7G8n7O4CqCGQCWJgcp58WeZpkHLAb+JkF0253m9kQwh+ndLkgvLFKdRXwQPJ+2GPVKTUThDRO7t4E3ApsISj4/cCzhNhT3WVy998nV/eoowpe9GZ2KbDL3Z8t9L56IkOuRcBE4H3ASKDYv87OcvezgTnAtWZ2fupKD+bain1NbHeZ7gTOAKYS/CX830XMUwqcDdzp7tOAd4DrUzcIaZzS5QpzrABITiNdBjzcdV1IY9VdptDGKflD5XKCH9bVwBBgdrH2n2smM7uaXnRUMY7oZwKXJed5HyT4Veh2YHjy1xGAMQTzdMV0XC4zu9/d3/BAC/Az4Jxihkr+FMfddwH/kdz/TjM7FSB5uyvsTO6+093b3b0D+AnFHadtwDZ3X5F8/AhBwYY6TulyhTxWneYAz7n7zuTjsMfquEwhj9MHgE3uvtvdW4FHCToizJ7qLtN5vemoghe9uy9y9zHuPpbg17Qn3P0TwJPA3yc3+wfg14XOkkOuq1P+8hvBfNxLxcpkZkPMbGjnfeBDyf0vIRgjKPJYpcvUOU5JV1LEcXL3HcBWM5uQXHQxsJYQxylTrjDHKsV8jp0iCXWsko7JFPI4bQHONbMTkv/2O/9OhdlT3WVa16uOcvei/eFvZ4sBTgf+E9hA8KvboGJmyZDrCWB1cvDuB04sYo7TgReSf9YA30guPxn4I/Aq8P+AkRHIdF9ynF4kKI1Ti/z/bCrQmNz/Y8CIMMcpS66wx2oIsAcYlrIs1LFKkynscfqfwMvJf/v3AYPC7qk0mXrcUfoIBBGRmNM7Y0VEYk5FLyIScyp6EZGYU9GLiMScil5EJOZU9CIiMaeiFxGJuf8Pn37/6Rxd8c8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w_val = -1\n",
    "b_val = 65\n",
    "plt.plot(data[:,0], data[:,1], 'o')\n",
    "plt.axis([40, 85, -0.1, 1.2])\n",
    "x_pred = np.linspace(40,85)\n",
    "x_pred = np.resize(x_pred,[len(x_pred),1])\n",
    "y_pred = 1 / (1 + np.exp(-x_pred*w_val - b_val))\n",
    "plt.plot(x_pred, y_pred)\n",
    "\n",
    "p_1 = 1 / (1 + np.exp(-x*w_val - b_val))\n",
    "like = y * np.log(p_1) + (1-y) * np.log(1-p_1)\n",
    "print(-np.mean(like))\n",
    "print(np.round(p_1,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO : set up a Keras model"
   ]
  },
  {
   "cell_type": "code",
Simon van Hemert's avatar
minor changes  
Simon van Hemert committed
705 
   "execution_count": 16,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
706 707 708 709 710 
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
711 
       "<keras.callbacks.History at 0x7fd95118a410>"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
712 713 
      ]
     },
Simon van Hemert's avatar
minor changes  
Simon van Hemert committed
714 
     "execution_count": 16,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tensorflow.keras.utils import to_categorical\n",
    "y_binary = to_categorical(y)\n",
    "\n",
    "l0 = tf.keras.layers.Dense(units=1, activation = tf.nn.sigmoid, input_shape=[1])\n",
    "model = tf.keras.Sequential([l0])\n",
    "model.compile(loss='binary_crossentropy', optimizer=tf.keras.optimizers.Adam(0.01))\n",
    "model.fit(x, y, epochs=10000, verbose=False)"
   ]
  },
  {
   "cell_type": "code",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
731 
   "execution_count": 21,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
732 733 734 735 736 
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
737 
       "[<matplotlib.lines.Line2D at 0x7fd950f0bc90>]"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
738 739 
      ]
     },
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
740 
     "execution_count": 21,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
741 742 743 744 745 
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
746 
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhcUlEQVR4nO3dd5xU5d3+8c93C8tSFxYEYUGKiGILsrFGxZIoaBCNBewmkZjE9MdEok+KSX7xiXmeNDURjTGxgGgQiTFiYkmxhSWoiAQFLCxID03qLt/fH/egC26ZXWbmnj17vV+vec3MmTNzLo/stbOn3MfcHRERSa6C2AFERCS7VPQiIgmnohcRSTgVvYhIwqnoRUQSTkUvIpJwTRa9md1pZivN7JUGXr/IzF42s7lm9qyZHZ75mCIi0lLpfKO/Czi9kdffAE5090OB7wGTMpBLREQypKipGdz9b2Y2oJHXn63z9HmgIgO5REQkQ5os+mb6FPCnhl40swnABICOHTuOOPDAAzO8eBGRZJs9e/Zqd+/ZnPdkrOjN7CRC0X+koXncfRKpTTuVlZVeVVWVqcWLiLQJZvZWc9+TkaI3s8OAO4BR7r4mE58pIiKZsdeHV5pZf2AacIm7v7b3kUREJJOa/EZvZpOBkUAPM6sGvg0UA7j7r4BvAeXArWYGUOPuldkKLCIizZPOUTfjm3j908CnM5ZIREQySmfGiogknIpeRCThVPQiIgmnohcRSTgVvYhIwqnoRUQSTkUvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJp6IXEUk4Fb2ISMKp6EVEEk5FLyKScCp6EZGEU9GLiCScil5EJOFU9CIiCaeiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gkXFFTM5jZncCZwEp3P6Se1w34GTAa2Axc7u7/ynRQyZzpc5Zy08wFLFu3hT5lpVxz2lDGDu8bdbmxMrU0b2t0/fS5TH5hCbXuFJox/qh+fH/sobFjJW4956Mmix64C7gZ+F0Dr48ChqRuRwG/TN1LHpo+ZykTp81ly45aAJau28LEaXMBsvrD1dhygSiZGhNrPWXL9dPncs/zb7/3vNb9vecxyz5p6zlfmbs3PZPZAOCRBr7R3wY87e6TU88XACPd/Z3GPrPyoAFedc93obAYCoqhoPD9x4VFUNgudds1LfW4qAQKS6CoXeq+BMxa9B/fFh1345MsXbflA9P7lpXyzLUnR1kuECVTY2Ktp2wZPPFRauv5WS80Y9EPR0dIFCRtPeeCmc1298rmvCedb/RN6QssqfO8OjXtA0VvZhOACQAj9i2AP3wxA4sn/CIoah9Kv7g03BeVQnH7ML24NHXrCO06QHEHaNfx/fuSztCuU7gv6QQlXcLz9l3C+xP0i2RZPT9UjU2PudxsZ2pMrPWULfWVfGPTcyVp6zlfZaLo0+buk4BJAJVHHO585Y9QuwN21sLOHanHNan7HVC7PTyu3f7+45pt4XHNNqjdBjXbU/fboGYr7NgKNVtS96nbppWwYzNs3xzud2wO09NR2A7al0H7rlCaum9fBqXdoEN5uHUsf/9xhx7QsUf46yMP9SkrrfcbVJ/UN+tYy42RqTGx1lO2FJo1+I0+pqSt53yViaJfCvSr87wiNa1xBcXQtSIDi2+hnbWp8n8Xtm2C7Rth28bU402wbQNs3QBb18PWdeF+yzrYvBbWLoYt/wm3hnQoh069oXOv3e+79oWu/aCsf/hlkeMftGtOG7rbNlGA0uJCrjltaNTlxsjUmFjrKVvGH9Vvt230dafHlLT1nK8yUfQzgKvNbAphJ+z6prbP54WCwtSmms7QuYWfUVsTyn7zati8JtzeXR3+gti0Itw2LodVC8LjnTW7v7+4Y/hlV9YvlH/3QVC+P5QPhm4DwiaoDNu1gyvXRzmks9x8OvIi1nrKll07XPPtqJukred81eTOWDObDIwEegArgG8DxQDu/qvU4ZU3A6cTDq+8wt2rmlpwZWWlV1U1OVty7NwZfhFsqIZ1S2B9NaxfAuveDo/XvbX7XwhWEMq/fP9w6zUMeh0C+xwU9iuISJuUlZ2x7j6+idcd+HxzFtomFRRAp57h1md4/fNs+Q+sWQxrFsLaReF+zSJY8kLYnASAQfeBsE+q+HsfChWV0Ll3zv5TRKR1yenOWGlCaTeoGBFude3cCevfhhXzUrdXwv2//wik/iLr2i8UfsWHw633YeGoIxFp81T0rUFBQdhm320AHHjG+9O3b4blc2FpFVSnbvMeSr2nOPzlMOhEGDQylH8WtvmLSP5L64SpbGhz2+hzZePyVOnPgreegaX/Aq8N5xXsd0wo/YEnhm/8BRrqSKS1iXXClOSTzr3hoDPDDcJhoW8+A2/8FRY/DX/+VpjeqTccOBoO+jgMOD5vj/sXkb2nok+69l1DoR+YOs19wzuh9Bc8Ci9Ngao7wzwHjAq/HAafEs4eFpHEUNG3NV32hcPHhduOLbDoKZj/h1D8L08Jm3gOOhOGXwwDTtDmHZEEUNG3ZcWl73/br90RtunPmw6vTIO5D0DX/jD8Ijh8PHTbL3ZaEWkh7YyVD9qxJRy6OeduWPxXwMMO3OGXwLAxOnpHJKKW7IxV0Uvj1r0NL06GF+8JjzvvC0d9BkZcEQZ5E5GcUtFL9uzcCYuehOd+EY7eadcJjrgMjv5sGKtHRHKiJUWvPW2SnoICGHIqXPowfOZvMHQ0vPAr+Nnh8Psr4Z2XYycUkQao6KX59j0cPnE7fOml8I1+waNw2/Ew9bIwNo+I5BUVvbRcWT847QfwlXlw4jfg9T/DLUfCI1+FjStipxORFBW97L3SMjjpm/ClF8NO2n/9Fn7+IXjy++HiLSISlYpeMqfTPnDGj+Hz/4QDToe/3RQK/4XbwkVaRCQKFb1kXvlgOO83MOHpMGb+n74Od5wMy16MnUykTVLRS/b0GR6O0jn3N2GMndtPgpnXhevyikjOqOglu8zgkHPg6lnhuPvnboZbj4HXHo+dTKTNUNFLbpSWwcd/Clc8FsbYue88eOByHZ0jkgMqesmt/Y6Bq/4OJ10XxtO59WhY8FjsVCKJpqKX3CsqgRO/Dlf9A7r0hckXwJ+uhZptsZOJJJKKXuLpORQ+/Rc48jPwwi/hjlNh9cLYqUQSR0UvcRW3h9E/gnGTYf0SuO2EMFqmiGSMil7yw4Gj4apnwiGZ06+CaRNg28bYqUQSIa2iN7PTzWyBmS00s2vreb2/mT1lZnPM7GUzG535qJJ4XfvCZTNg5DfDFa7uOBXWvhE7lUir12TRm1khcAswChgGjDezYXvMdj0w1d2HA+OAWzMdVNqIgkIY+Q24ZDpsXA63nwxvPhM7lUirls43+iOBhe6+2N23A1OAs/aYx4EuqcddgWWZiyht0qAT4conoUM5/O4s+NfdsROJtFrpFH1fYEmd59WpaXV9B7jYzKqBR4Ev1PdBZjbBzKrMrGrVqlUtiCttSvngcFTOgI/AjKvh8ethZ23sVCKtTqZ2xo4H7nL3CmA0cLeZfeCz3X2Su1e6e2XPnj0ztGhJtNIyuOhBOHICPPsLmHKhdtKKNFM6Rb8UqHtR0IrUtLo+BUwFcPfngPZAj0wEFKGwCEbfBKN/HC5u8uuPwbolTb9PRID0in4WMMTMBppZO8LO1hl7zPM2cAqAmR1EKHptm5HMOvJKuPj3sH4p3Hk6rH49diKRVqHJonf3GuBqYCYwn3B0zTwzu8HMxqRm+xpwpZm9BEwGLnd3z1ZoacMGnwRX/BFqt4Wy10XJRZpksfq4srLSq6qqoixbEmD1wnA0zraNcNED0P+o2IlEcsLMZrt7ZXPeozNjpXXqsT988jHoWA53j4VFT8VOJJK3VPTSepX1C+PbdxsI950fhj0WkQ9Q0Uvr1rkXXP4I9D4M7r8EXro/diKRvKOil9avQ3e4dDrsdyw8NAHm3Bs7kUheUdFLMpR0DidWDTopnEX7yu9jJxLJGyp6SY7i9jDuXuh3VBjmeMGfYicSyQsqekmWdh3hwqlhm/3US2HRk7ETiUSnopfkad8lnEHb4wCYfCG89WzsRCJRqeglmTp0D2Pad62Ae8+H6tmxE4lEo6KX5OrUM1yxqkN3uOccWP5K7EQiUajoJdm69All365jGDJhzaLYiURyTkUvyddtAFz6MPhOuPdceHd17EQiOaWil7ahxxAYPwU2LIPJ42D75tiJRHJGRS9tR/+j4JzboboKpl2pyxJKm6Gil7Zl2Bg4/Yfw70fgsYmgyyZIG1AUO4BIzh392XApwudvgbL+cOzVsROJZJWKXtqmj30fNlTD49dB175w8NmxE4lkjYpe2qaCAjh7EmxcAdM+A516w37HxE4lkhXaRi9tV3F7GD85bL6ZMh7WLo6dSCQrVPTStnXoHq45C3DfONi6IW4ekSxQ0Yt0Hwjn/w7WLITff1qHXUriqOhFAAaeAKP+B16fCU/cEDuNSEZpZ6zILkdeCStfhWd+CvsMg8MviJ1IJCP0jV6krlE/gv0+AjO+oKGNJTFU9CJ1FRaH7fWde8OUC8PYOCKtXFpFb2anm9kCM1toZtc2MM/5Zvaqmc0zs/syG1MkhzqWhwHQtm8KZb9jS+xEInulyaI3s0LgFmAUMAwYb2bD9phnCDAROM7dDwa+nPmoIjnUaxicMwmWvQgzvqgxcaRVS+cb/ZHAQndf7O7bgSnAWXvMcyVwi7v/B8DdV2Y2pkgEB54BJ10Hc6fCC7fFTiPSYukUfV9gSZ3n1alpdR0AHGBmz5jZ82Z2en0fZGYTzKzKzKpWrVrVssQiuXT812DoGWFMHF1kXFqpTO2MLQKGACOB8cDtZla250zuPsndK929smfPnhlatEgWFRTA2b8MV6maehlseCd2IpFmS6folwL96jyvSE2rqxqY4e473P0N4DVC8Yu0fu27wgX3wPZ3YeqlULM9diKRZkmn6GcBQ8xsoJm1A8YBM/aYZzrh2zxm1oOwKUcjREly7HMQjL0Fqv8JM78ZO41IszRZ9O5eA1wNzATmA1PdfZ6Z3WBmY1KzzQTWmNmrwFPANe6+JluhRaI4+Gw49gsw63Z4cXLsNCJpM4902FhlZaVXVVVFWbZIi9XWwN1joXoWfOpx2Pfw2ImkjTGz2e5e2Zz36MxYkeYoLIJzfwMdyuH+i2Hz2tiJRJqkohdprk494fy7YeNymHYl7NwZO5FIo1T0Ii1RMSIMa7zwL/C3m2KnEWmUil6kpUZcAYeNg6d/CAufiJ1GpEEqepGWMoMzfxIOvfz9p2F9dexEIvVS0YvsjXYdwvb62h3hzFmdTCV5SEUvsrd67A9n3QxLq+Dx62OnEfkAFb1IJhw8Fo7+PPzzNpj7YOw0IrtR0Ytkyke/C/2OCuPXr1oQO43Ie1T0IplSWAzn3QXFpXD/JbBtU+xEIoCKXiSzuvSBc38Na16HP3xJV6aSvKCiF8m0QSPhpG/CKw/CrDtipxFR0YtkxUe+BkNOg8cmQrUG75O4VPQi2VBQAGf/CrrsG46vf1ejdks8KnqRbOnQHc77Lby7MjX4WW3sRNJGqehFsqnvEWHws0VPaPAziUZFL5Jt7w1+dmMY7VIkx1T0Itm22+BnV8K6JbETSRujohfJhbqDnz1wOdRsi51I2hAVvUiu9Ngfxt4SBj97bGLsNNKGqOhFcmnYWXDsF6Hq1zDn3thppI1Q0Yvk2infhgHHwyNfgWUvxk4jbYCKXiTXCovC4Gcde4bBzzavjZ1IEk5FLxJDxx5wwe9g03J48JM6mUqyKq2iN7PTzWyBmS00s2sbme8TZuZmVpm5iCIJ1XcEjP4xLH4KnvpB7DSSYE0WvZkVArcAo4BhwHgzG1bPfJ2BLwEvZDqkSGKNuAyOuAz+/r8w/5HYaSSh0vlGfySw0N0Xu/t2YApwVj3zfQ/4H2BrBvOJJN/om6DPEfDQVbD69dhpJIHSKfq+QN1T+apT095jZkcA/dz9j419kJlNMLMqM6tatWpVs8OKJFJRCVxwd7ifchFs3RA7kSTMXu+MNbMC4P+ArzU1r7tPcvdKd6/s2bPn3i5aJDm6VsB5v4E1CzXSpWRcOkW/FOhX53lFatounYFDgKfN7E3gaGCGdsiKNNPAE8JIl689Bk9+L3YaSZCiNOaZBQwxs4GEgh8HXLjrRXdfD/TY9dzMngb+y911WR2R5vrwp2HFPPjHT2CfYXDY+bETSQI0+Y3e3WuAq4GZwHxgqrvPM7MbzGxMtgOKtClmMOpHsN9x8PDVsHR27ESSAOaRrlJfWVnpVVX60i9Sr3fXwO0joWY7THg6XJJQBDCz2e7erE3jOjNWJB91LIfxU2D7JphyIezYEjuRtGIqepF81etgOGcSLJsDM74Akf76ltZPRS+Szw48A06+HuY+EHbQirRAOkfdiEhMx38NVr4KT3wXug2AQ86JnUhaGRW9SL4zg7NuhQ3L4KHPQOfesN+xsVNJK6JNNyKtQXF7GHcflO0Hk8fDqgWxE0kroqIXaS06dIeLH4TCdnDPubBxRexE0kqo6EVak24D4ML7YfNquO882LYpdiJpBVT0Iq1N3yPCpQiXz4UHLofamtiJJM+p6EVaowNOgzP+Dxb+Gf74FR1jL43SUTcirVXlFbB+Sbg6VZcKGPmN2IkkT6noRVqzk/87HHb59P+Dks5wzOdiJ5I8pKIXac3MYMzNYUycmROhXQcYcXnsVJJntI1epLUrLIJP3An7fxT+8GV46f7YiSTPqOhFkqCoXbju7ICPwPSr4NWHYyeSPKKiF0mK4tIwtHHFh+HBT8Frj8dOJHlCRS+SJCWd4KIHoNcwuP9iWPzX2IkkD6joRZKmfVe4+CHoPiiMi/PWc7ETSWQqepEk6lgOlz4cLkF4zzmw6KnYiSQiFb1IUnXuBZc/Ct0Gwn3nw/xHYieSSFT0IknWuRdc/gj0PgymXgovTYmdSCJQ0YskXYfucOl0GHBcuHDJP2+PnUhyTEUv0haUdIYLH4ADRsGj/xXGx5E2Q0Uv0lYUtw8nVR16HjxxA/z52xr1so1Iq+jN7HQzW2BmC83s2npe/6qZvWpmL5vZE2a2X+ajisheKyyGsydB5SfhmZ/C9M9BzbbYqSTLmix6MysEbgFGAcOA8WY2bI/Z5gCV7n4Y8CDwo0wHFZEMKSgIY9mfeC28dB/8dgy8uzp2KsmidL7RHwksdPfF7r4dmAKcVXcGd3/K3Tennj4PVGQ2pohklBmcNBHOvRPeeRFuPwlWvBo7lWRJOkXfF1hS53l1alpDPgX8aW9CiUiOHPIJuOJRqNkOv/4oLHgsdiLJgozujDWzi4FK4KYGXp9gZlVmVrVq1apMLlpEWqrvCJjwFJQPhsnj4NlfaCdtwqRT9EuBfnWeV6Sm7cbMTgWuA8a4e717d9x9krtXuntlz549W5JXRLKhSx+44k8wbAw8fj3MuFo7aRMknaKfBQwxs4Fm1g4YB8yoO4OZDQduI5T8yszHFJGsa9cRzr0LTvg6zLkH7jgFVr8eO5VkQJNF7+41wNXATGA+MNXd55nZDWY2JjXbTUAn4AEze9HMZjTwcSKSzwoK4OTrwrj265fCbSfCnHu1KaeVM4/0P7CystKrqqqiLFtE0rBhGUybAG/+HQ45F878CbTvEjtVm2dms929sjnv0ZmxIlK/Ln3CUMcnXQ/zHoLbjofq2bFTSQuo6EWkYQWFcOI14RDMnbVw58fgHz8Nj6XVUNGLSNP6Hw1X/R2GjoK/fBvuOBXeeTl2KkmTil5E0lPaDc6/Gz7xa1i/BCaNhMf/G7a/GzuZNEFFLyLpM4NDz4XP/xOGXwTP/hxuPRpe/0vsZNIIFb2INF+H7jDmF+FShYUlcO8n4MFPwiadRpOPVPQi0nIDjoPPPgMjJ8L8P8AvRsDffqzNOXlGRS8ie6eoBEZeC599FgYcD09+D34+HKruhNodsdMJKnoRyZQeQ2D8ffDJmdBtIDzylbD9/tWHdWZtZCp6Ecms/kfDJx8LwygUFMHUS8O4OYueVOFHoqIXkcwzC8fcf/ZZOOtW2LgC7j47nF378lRt0skxFb2IZE9BYTgM84v/gjE3hwucTLsSfvYhePZm2LohdsI2QUUvItlXVAJHXAKfex4unArdBsDj18FPDg4nXa17O3bCRNPolSISx9LZ4Vv9q9PDtvtBJ8LwS+DAM6C4NHa6vNWS0StV9CIS17q34cX7wrj369+Gkq7h7NvhF0Of4WF7v7xHRS8irdfOnfDm30Lhz58BNVthn2FwyDlw0BjoOTR2wrygoheRZNiyDuZNC9/0q2eFaeVD4KCPw0FnQp8j2uw3fRW9iCTPhmXw7z+GIRbe/Ad4LXTpG7blDz4lDMNQ0jl2ypxR0YtIsm1eC689BvMfgUVPhM07BUXQdwQMGgkDT4SKD0NRu9hJs0ZFLyJtx46tsOQFWPw0vPFXWDYHfCcUd4D+x0C/I6GiMvwSKO0WO23GtKToi7IVRkQkq4rbh0MyB50Ynm9ZFzbtvPHXcP/0jUDqi2z5kFD6FZVh+/4+B7WpQzhV9CKSDKVlYUftQWeG51s3hG/5S6ugugoW/gVemhxeswLoPhh6Hbz7rWt/KEjeeaQqehFJpvZddv/G7x6O2V82B1a+CivmwTsvhRO2dikqhfLB0H0QlO+fejw4PO7Yo9Ue6aOiF5G2wQy67RduB499f/q2TbByPqx4BVa/DmsXhV8ECx6FnTXvz1fcEbpWQFk/6Nov9bh/eNy5d7jl6eYgFb2ItG0lnaDfh8OtrtoaWPcWrF0MaxaGvwbWvR0ujL5sDmxeU89ndYXOvaBT6ta5N3QoD38NdChP3XqESzG2L8vZZqK0it7MTgd+BhQCd7j7jXu8XgL8DhgBrAEucPc3MxtVJDmmz1nKTTMXsGzdFvqUlXLNaUMZO7wvABfd/hzPLFr73rzHDe7OvVcek9Z7G3vt+ulzmfzCEmrdKTRj/FH9+P7YQ/c6bzqvt/Szs5U5rbyFRWHTTflgGPLRD3zuf7b+h8O7vMsXRpRw7D47YONy2LTi/fvqWeEaujVb6g9oBdC+a51bWbgvLYOSLuHcgJLO0K7T+49beL5Ak4dXmlkh8BrwUaAamAWMd/dX68zzOeAwd7/KzMYBZ7v7BY19rg6vlLZq+pylTJw2ly07at+bVlpcyA/POZQHqt7ereR32VX2jb0XaPC1qrfWcs/zHxwh8uKj+zdZnI0tc+zwvk2+3tLPzlZmaHg97U3eBt+7fTNsXh3+Ati8JpwL8G7q+db1qdu6cL9l3fvTGvgFYd/dkPnj6M3sGOA77n5a6vlEAHf/YZ15Zqbmec7MioDlQE9v5MNV9NJWHXfjkyxd98Ef4r5lpfVO3+XNG89o9L1Ag68tX7+V2np+HAvNWPTD0S3O+8y1Jzf5eks/O1uZoeH1tDd5m3pvs9XWwPaNYR/C9k2wbSNs24gNOTUrx9H3BZbUeV4NHNXQPO5eY2brgXJgdd2ZzGwCMAGgf//+zckpkhjLGijzhqbv7XuXrdtCQ9+46ivS5i4zW/892czc3Nf25nNbrLAonOiVgZO9cnrAqLtPcvdKd6/s2bNnLhctkjf6lNV/ZEZD09N9b2OvFTZwWGBD09NdZjqvt/Szs5U5W3nzWTpFvxToV+d5RWpavfOkNt10JeyUFZE9XHPaUEqLC3ebVlpcyDWnDeW4wd3rfc+u6Y29t7HXxh/Vj/o0ND3dvOm83tLPzlbmbOXNZ+lsupkFDDGzgYRCHwdcuMc8M4DLgOeAc4EnG9s+L9KW7dppV99RH2OH9230qJvG3rtLQ58LtOgIlqaWmU6mlq6LbGXORt58ltagZmY2Gvgp4fDKO939B2Z2A1Dl7jPMrD1wNzAcWAuMc/fFjX2mdsaKiDRf1gY1c/dHgUf3mPatOo+3Auc1Z8EiIpIbyRu9R0REdqOiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqU1qFlWFmy2EVgQZeEN68EeF0vJE/mYS5nSo0zpy8dc+ZhpqLs36+KxaQ1qliULmjsCW7aZWVW+ZYL8zKVM6VGm9OVjrnzN1Nz3aNONiEjCqehFRBIuZtFPirjshuRjJsjPXMqUHmVKXz7mSkSmaDtjRUQkN7TpRkQk4VT0IiIJl7OiN7NCM5tjZo+kng80sxfMbKGZ3W9m7XKVpYlcd5nZG2b2Yur2oRznedPM5qaWXZWa1t3M/mxmr6fuu+VBpu+Y2dI662l0jjOVmdmDZvZvM5tvZsfEXk+N5Iq2rsxsaJ3lvmhmG8zsyzHXVSOZYv+b+oqZzTOzV8xsspm1j91TDWRqdkflbBu9mX0VqAS6uPuZZjYVmObuU8zsV8BL7v7LnIRpPNddwCPu/mCus6TyvAlUuvvqOtN+BKx19xvN7Fqgm7t/I3Km7wCb3P3HucqxR6bfAn939ztSP3wdgG8ScT01kuvLRFxXdbIVAkuBo4DPE3ld1ZPpCiKtJzPrC/wDGObuW1L99Cgwmkg91UimkTSzo3Lyjd7MKoAzgDtSzw04GdgV9LfA2FxkaSxXHjuLsI4g0rrKJ2bWFTgB+DWAu29393VEXk+N5MoXpwCL3P0t8uffVN1MsRUBpWZWRPgF/Q7xe2rPTMta8iG52nTzU+DrwM7U83JgnbvXpJ5XA31zlKWxXLv8wMxeNrOfmFlJjjM58LiZzTazCalpvdz9ndTj5UCvPMgEcHVqPd2Z480kA4FVwG8sbHa7w8w6En89NZQL4q2rusYBk1OPY6+rXepmgkjryd2XAj8G3iYU/HpgNhF7qr5M7v546uVmdVTWi97MzgRWuvvsbC+rORrJNRE4EPgw0B3I9Z+zH3H3I4BRwOfN7IS6L3rY1pbrY2Lry/RLYDDwIcI/wv/NYZ4i4Ajgl+4+HHgXuLbuDJHWU0O5Yq4rAFKbkcYAD+z5WqR1VV+maOsp9UvlLMIv6z5AR+D0XC0/3UxmdjEt6KhcfKM/DhiT2s47hfCn0M+AstSfIwAVhO10ufSBXGZ2j7u/48E24DfAkbkMlfotjruvBB5KLX+Fme0LkLpfGTuTu69w91p33wncTm7XUzVQ7e4vpJ4/SCjYqOupoVyR19Uuo4B/ufuK1PPY6+oDmSKvp1OBN9x9lbvvAKYROiJmT9WX6diWdFTWi97dJ7p7hbsPIPyZ9qS7XwQ8BZybmu0y4OFsZ0kj18V1/vEbYXvcK7nKZGYdzazzrsfAx1LLn0FYR5DjddVQpl3rKeVscrie3H05sMTMhqYmnQK8SsT11FiumOuqjvHsvokk6rpK2S1T5PX0NnC0mXVI/ezv+jcVs6fqyzS/RR3l7jm78f7eYoBBwD+BhYQ/3UpymaWRXE8Cc1Mr7x6gUw5zDAJeSt3mAdelppcDTwCvA38BuudBprtT6+llQmnsm+P/Zx8CqlLLnw50i7memsgVe111BNYAXetMi7quGsgUez19F/h36mf/bqAkdk81kKnZHaUhEEREEk5nxoqIJJyKXkQk4VT0IiIJp6IXEUk4Fb2ISMKp6EVEEk5FLyKScP8fcHJtND9r1YoAAAAASUVORK5CYII=\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(data[:,0], data[:,1], 'o')\n",
    "plt.axis([40, 85, -0.1, 1.2])\n",
    "x_pred = np.linspace(40,85)\n",
    "x_pred = np.resize(x_pred,[len(x_pred),1])\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
762 
    "y_pred = model.predict(x_pred)\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
763 764 765 766 767 
    "plt.plot(x_pred, y_pred)"
   ]
  },
  {
   "cell_type": "code",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
768 
   "execution_count": 22,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
769 770 771 772 773 774 
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
775 
      "[array([[-0.23204683]], dtype=float32), array([15.034954], dtype=float32)]\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
776 777 778 779 780 781 782 783 784 
     ]
    }
   ],
   "source": [
    "print(model.get_weights())"
   ]
  },
  {
   "cell_type": "code",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
785 
   "execution_count": 23,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
786 787 788 789 790 791 
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
Mirko Birbaumer's avatar
minor editing  
Mirko Birbaumer committed
792 
      "0.4416347\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
793 
      "[[0.43  0.23  0.274 0.322 0.375 0.158 0.13  0.23  0.859 0.603 0.23  0.045\n",
Simon van Hemert's avatar
minor changes  
Simon van Hemert committed
794 
      "  0.375 0.939 0.375 0.086 0.23  0.023 0.069 0.036 0.086 0.069 0.829]]\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
795 796 797 798 
     ]
    },
    {
     "data": {
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
799 
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhcUlEQVR4nO3dd5xU5d3+8c93C8tSFxYEYUGKiGILsrFGxZIoaBCNBewmkZjE9MdEok+KSX7xiXmeNDURjTGxgGgQiTFiYkmxhSWoiAQFLCxID03qLt/fH/egC26ZXWbmnj17vV+vec3MmTNzLo/stbOn3MfcHRERSa6C2AFERCS7VPQiIgmnohcRSTgVvYhIwqnoRUQSTkUvIpJwTRa9md1pZivN7JUGXr/IzF42s7lm9qyZHZ75mCIi0lLpfKO/Czi9kdffAE5090OB7wGTMpBLREQypKipGdz9b2Y2oJHXn63z9HmgIgO5REQkQ5os+mb6FPCnhl40swnABICOHTuOOPDAAzO8eBGRZJs9e/Zqd+/ZnPdkrOjN7CRC0X+koXncfRKpTTuVlZVeVVWVqcWLiLQJZvZWc9+TkaI3s8OAO4BR7r4mE58pIiKZsdeHV5pZf2AacIm7v7b3kUREJJOa/EZvZpOBkUAPM6sGvg0UA7j7r4BvAeXArWYGUOPuldkKLCIizZPOUTfjm3j908CnM5ZIREQySmfGiogknIpeRCThVPQiIgmnohcRSTgVvYhIwqnoRUQSTkUvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJp6IXEUk4Fb2ISMKp6EVEEk5FLyKScCp6EZGEU9GLiCScil5EJOFU9CIiCaeiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gkXFFTM5jZncCZwEp3P6Se1w34GTAa2Axc7u7/ynRQyZzpc5Zy08wFLFu3hT5lpVxz2lDGDu8bdbmxMrU0b2t0/fS5TH5hCbXuFJox/qh+fH/sobFjJW4956Mmix64C7gZ+F0Dr48ChqRuRwG/TN1LHpo+ZykTp81ly45aAJau28LEaXMBsvrD1dhygSiZGhNrPWXL9dPncs/zb7/3vNb9vecxyz5p6zlfmbs3PZPZAOCRBr7R3wY87e6TU88XACPd/Z3GPrPyoAFedc93obAYCoqhoPD9x4VFUNgudds1LfW4qAQKS6CoXeq+BMxa9B/fFh1345MsXbflA9P7lpXyzLUnR1kuECVTY2Ktp2wZPPFRauv5WS80Y9EPR0dIFCRtPeeCmc1298rmvCedb/RN6QssqfO8OjXtA0VvZhOACQAj9i2AP3wxA4sn/CIoah9Kv7g03BeVQnH7ML24NHXrCO06QHEHaNfx/fuSztCuU7gv6QQlXcLz9l3C+xP0i2RZPT9UjU2PudxsZ2pMrPWULfWVfGPTcyVp6zlfZaLo0+buk4BJAJVHHO585Y9QuwN21sLOHanHNan7HVC7PTyu3f7+45pt4XHNNqjdBjXbU/fboGYr7NgKNVtS96nbppWwYzNs3xzud2wO09NR2A7al0H7rlCaum9fBqXdoEN5uHUsf/9xhx7QsUf46yMP9SkrrfcbVJ/UN+tYy42RqTGx1lO2FJo1+I0+pqSt53yViaJfCvSr87wiNa1xBcXQtSIDi2+hnbWp8n8Xtm2C7Rth28bU402wbQNs3QBb18PWdeF+yzrYvBbWLoYt/wm3hnQoh069oXOv3e+79oWu/aCsf/hlkeMftGtOG7rbNlGA0uJCrjltaNTlxsjUmFjrKVvGH9Vvt230dafHlLT1nK8yUfQzgKvNbAphJ+z6prbP54WCwtSmms7QuYWfUVsTyn7zati8JtzeXR3+gti0Itw2LodVC8LjnTW7v7+4Y/hlV9YvlH/3QVC+P5QPhm4DwiaoDNu1gyvXRzmks9x8OvIi1nrKll07XPPtqJukred81eTOWDObDIwEegArgG8DxQDu/qvU4ZU3A6cTDq+8wt2rmlpwZWWlV1U1OVty7NwZfhFsqIZ1S2B9NaxfAuveDo/XvbX7XwhWEMq/fP9w6zUMeh0C+xwU9iuISJuUlZ2x7j6+idcd+HxzFtomFRRAp57h1md4/fNs+Q+sWQxrFsLaReF+zSJY8kLYnASAQfeBsE+q+HsfChWV0Ll3zv5TRKR1yenOWGlCaTeoGBFude3cCevfhhXzUrdXwv2//wik/iLr2i8UfsWHw633YeGoIxFp81T0rUFBQdhm320AHHjG+9O3b4blc2FpFVSnbvMeSr2nOPzlMOhEGDQylH8WtvmLSP5L64SpbGhz2+hzZePyVOnPgreegaX/Aq8N5xXsd0wo/YEnhm/8BRrqSKS1iXXClOSTzr3hoDPDDcJhoW8+A2/8FRY/DX/+VpjeqTccOBoO+jgMOD5vj/sXkb2nok+69l1DoR+YOs19wzuh9Bc8Ci9Ngao7wzwHjAq/HAafEs4eFpHEUNG3NV32hcPHhduOLbDoKZj/h1D8L08Jm3gOOhOGXwwDTtDmHZEEUNG3ZcWl73/br90RtunPmw6vTIO5D0DX/jD8Ijh8PHTbL3ZaEWkh7YyVD9qxJRy6OeduWPxXwMMO3OGXwLAxOnpHJKKW7IxV0Uvj1r0NL06GF+8JjzvvC0d9BkZcEQZ5E5GcUtFL9uzcCYuehOd+EY7eadcJjrgMjv5sGKtHRHKiJUWvPW2SnoICGHIqXPowfOZvMHQ0vPAr+Nnh8Psr4Z2XYycUkQao6KX59j0cPnE7fOml8I1+waNw2/Ew9bIwNo+I5BUVvbRcWT847QfwlXlw4jfg9T/DLUfCI1+FjStipxORFBW97L3SMjjpm/ClF8NO2n/9Fn7+IXjy++HiLSISlYpeMqfTPnDGj+Hz/4QDToe/3RQK/4XbwkVaRCQKFb1kXvlgOO83MOHpMGb+n74Od5wMy16MnUykTVLRS/b0GR6O0jn3N2GMndtPgpnXhevyikjOqOglu8zgkHPg6lnhuPvnboZbj4HXHo+dTKTNUNFLbpSWwcd/Clc8FsbYue88eOByHZ0jkgMqesmt/Y6Bq/4OJ10XxtO59WhY8FjsVCKJpqKX3CsqgRO/Dlf9A7r0hckXwJ+uhZptsZOJJJKKXuLpORQ+/Rc48jPwwi/hjlNh9cLYqUQSR0UvcRW3h9E/gnGTYf0SuO2EMFqmiGSMil7yw4Gj4apnwiGZ06+CaRNg28bYqUQSIa2iN7PTzWyBmS00s2vreb2/mT1lZnPM7GUzG535qJJ4XfvCZTNg5DfDFa7uOBXWvhE7lUir12TRm1khcAswChgGjDezYXvMdj0w1d2HA+OAWzMdVNqIgkIY+Q24ZDpsXA63nwxvPhM7lUirls43+iOBhe6+2N23A1OAs/aYx4EuqcddgWWZiyht0qAT4conoUM5/O4s+NfdsROJtFrpFH1fYEmd59WpaXV9B7jYzKqBR4Ev1PdBZjbBzKrMrGrVqlUtiCttSvngcFTOgI/AjKvh8ethZ23sVCKtTqZ2xo4H7nL3CmA0cLeZfeCz3X2Su1e6e2XPnj0ztGhJtNIyuOhBOHICPPsLmHKhdtKKNFM6Rb8UqHtR0IrUtLo+BUwFcPfngPZAj0wEFKGwCEbfBKN/HC5u8uuPwbolTb9PRID0in4WMMTMBppZO8LO1hl7zPM2cAqAmR1EKHptm5HMOvJKuPj3sH4p3Hk6rH49diKRVqHJonf3GuBqYCYwn3B0zTwzu8HMxqRm+xpwpZm9BEwGLnd3z1ZoacMGnwRX/BFqt4Wy10XJRZpksfq4srLSq6qqoixbEmD1wnA0zraNcNED0P+o2IlEcsLMZrt7ZXPeozNjpXXqsT988jHoWA53j4VFT8VOJJK3VPTSepX1C+PbdxsI950fhj0WkQ9Q0Uvr1rkXXP4I9D4M7r8EXro/diKRvKOil9avQ3e4dDrsdyw8NAHm3Bs7kUheUdFLMpR0DidWDTopnEX7yu9jJxLJGyp6SY7i9jDuXuh3VBjmeMGfYicSyQsqekmWdh3hwqlhm/3US2HRk7ETiUSnopfkad8lnEHb4wCYfCG89WzsRCJRqeglmTp0D2Pad62Ae8+H6tmxE4lEo6KX5OrUM1yxqkN3uOccWP5K7EQiUajoJdm69All365jGDJhzaLYiURyTkUvyddtAFz6MPhOuPdceHd17EQiOaWil7ahxxAYPwU2LIPJ42D75tiJRHJGRS9tR/+j4JzboboKpl2pyxJKm6Gil7Zl2Bg4/Yfw70fgsYmgyyZIG1AUO4BIzh392XApwudvgbL+cOzVsROJZJWKXtqmj30fNlTD49dB175w8NmxE4lkjYpe2qaCAjh7EmxcAdM+A516w37HxE4lkhXaRi9tV3F7GD85bL6ZMh7WLo6dSCQrVPTStnXoHq45C3DfONi6IW4ekSxQ0Yt0Hwjn/w7WLITff1qHXUriqOhFAAaeAKP+B16fCU/cEDuNSEZpZ6zILkdeCStfhWd+CvsMg8MviJ1IJCP0jV6krlE/gv0+AjO+oKGNJTFU9CJ1FRaH7fWde8OUC8PYOCKtXFpFb2anm9kCM1toZtc2MM/5Zvaqmc0zs/syG1MkhzqWhwHQtm8KZb9jS+xEInulyaI3s0LgFmAUMAwYb2bD9phnCDAROM7dDwa+nPmoIjnUaxicMwmWvQgzvqgxcaRVS+cb/ZHAQndf7O7bgSnAWXvMcyVwi7v/B8DdV2Y2pkgEB54BJ10Hc6fCC7fFTiPSYukUfV9gSZ3n1alpdR0AHGBmz5jZ82Z2en0fZGYTzKzKzKpWrVrVssQiuXT812DoGWFMHF1kXFqpTO2MLQKGACOB8cDtZla250zuPsndK929smfPnhlatEgWFRTA2b8MV6maehlseCd2IpFmS6folwL96jyvSE2rqxqY4e473P0N4DVC8Yu0fu27wgX3wPZ3YeqlULM9diKRZkmn6GcBQ8xsoJm1A8YBM/aYZzrh2zxm1oOwKUcjREly7HMQjL0Fqv8JM78ZO41IszRZ9O5eA1wNzATmA1PdfZ6Z3WBmY1KzzQTWmNmrwFPANe6+JluhRaI4+Gw49gsw63Z4cXLsNCJpM4902FhlZaVXVVVFWbZIi9XWwN1joXoWfOpx2Pfw2ImkjTGz2e5e2Zz36MxYkeYoLIJzfwMdyuH+i2Hz2tiJRJqkohdprk494fy7YeNymHYl7NwZO5FIo1T0Ii1RMSIMa7zwL/C3m2KnEWmUil6kpUZcAYeNg6d/CAufiJ1GpEEqepGWMoMzfxIOvfz9p2F9dexEIvVS0YvsjXYdwvb62h3hzFmdTCV5SEUvsrd67A9n3QxLq+Dx62OnEfkAFb1IJhw8Fo7+PPzzNpj7YOw0IrtR0Ytkyke/C/2OCuPXr1oQO43Ie1T0IplSWAzn3QXFpXD/JbBtU+xEIoCKXiSzuvSBc38Na16HP3xJV6aSvKCiF8m0QSPhpG/CKw/CrDtipxFR0YtkxUe+BkNOg8cmQrUG75O4VPQi2VBQAGf/CrrsG46vf1ejdks8KnqRbOnQHc77Lby7MjX4WW3sRNJGqehFsqnvEWHws0VPaPAziUZFL5Jt7w1+dmMY7VIkx1T0Itm22+BnV8K6JbETSRujohfJhbqDnz1wOdRsi51I2hAVvUiu9Ngfxt4SBj97bGLsNNKGqOhFcmnYWXDsF6Hq1zDn3thppI1Q0Yvk2infhgHHwyNfgWUvxk4jbYCKXiTXCovC4Gcde4bBzzavjZ1IEk5FLxJDxx5wwe9g03J48JM6mUqyKq2iN7PTzWyBmS00s2sbme8TZuZmVpm5iCIJ1XcEjP4xLH4KnvpB7DSSYE0WvZkVArcAo4BhwHgzG1bPfJ2BLwEvZDqkSGKNuAyOuAz+/r8w/5HYaSSh0vlGfySw0N0Xu/t2YApwVj3zfQ/4H2BrBvOJJN/om6DPEfDQVbD69dhpJIHSKfq+QN1T+apT095jZkcA/dz9j419kJlNMLMqM6tatWpVs8OKJFJRCVxwd7ifchFs3RA7kSTMXu+MNbMC4P+ArzU1r7tPcvdKd6/s2bPn3i5aJDm6VsB5v4E1CzXSpWRcOkW/FOhX53lFatounYFDgKfN7E3gaGCGdsiKNNPAE8JIl689Bk9+L3YaSZCiNOaZBQwxs4GEgh8HXLjrRXdfD/TY9dzMngb+y911WR2R5vrwp2HFPPjHT2CfYXDY+bETSQI0+Y3e3WuAq4GZwHxgqrvPM7MbzGxMtgOKtClmMOpHsN9x8PDVsHR27ESSAOaRrlJfWVnpVVX60i9Sr3fXwO0joWY7THg6XJJQBDCz2e7erE3jOjNWJB91LIfxU2D7JphyIezYEjuRtGIqepF81etgOGcSLJsDM74Akf76ltZPRS+Szw48A06+HuY+EHbQirRAOkfdiEhMx38NVr4KT3wXug2AQ86JnUhaGRW9SL4zg7NuhQ3L4KHPQOfesN+xsVNJK6JNNyKtQXF7GHcflO0Hk8fDqgWxE0kroqIXaS06dIeLH4TCdnDPubBxRexE0kqo6EVak24D4ML7YfNquO882LYpdiJpBVT0Iq1N3yPCpQiXz4UHLofamtiJJM+p6EVaowNOgzP+Dxb+Gf74FR1jL43SUTcirVXlFbB+Sbg6VZcKGPmN2IkkT6noRVqzk/87HHb59P+Dks5wzOdiJ5I8pKIXac3MYMzNYUycmROhXQcYcXnsVJJntI1epLUrLIJP3An7fxT+8GV46f7YiSTPqOhFkqCoXbju7ICPwPSr4NWHYyeSPKKiF0mK4tIwtHHFh+HBT8Frj8dOJHlCRS+SJCWd4KIHoNcwuP9iWPzX2IkkD6joRZKmfVe4+CHoPiiMi/PWc7ETSWQqepEk6lgOlz4cLkF4zzmw6KnYiSQiFb1IUnXuBZc/Ct0Gwn3nw/xHYieSSFT0IknWuRdc/gj0PgymXgovTYmdSCJQ0YskXYfucOl0GHBcuHDJP2+PnUhyTEUv0haUdIYLH4ADRsGj/xXGx5E2Q0Uv0lYUtw8nVR16HjxxA/z52xr1so1Iq+jN7HQzW2BmC83s2npe/6qZvWpmL5vZE2a2X+ajisheKyyGsydB5SfhmZ/C9M9BzbbYqSTLmix6MysEbgFGAcOA8WY2bI/Z5gCV7n4Y8CDwo0wHFZEMKSgIY9mfeC28dB/8dgy8uzp2KsmidL7RHwksdPfF7r4dmAKcVXcGd3/K3Tennj4PVGQ2pohklBmcNBHOvRPeeRFuPwlWvBo7lWRJOkXfF1hS53l1alpDPgX8aW9CiUiOHPIJuOJRqNkOv/4oLHgsdiLJgozujDWzi4FK4KYGXp9gZlVmVrVq1apMLlpEWqrvCJjwFJQPhsnj4NlfaCdtwqRT9EuBfnWeV6Sm7cbMTgWuA8a4e717d9x9krtXuntlz549W5JXRLKhSx+44k8wbAw8fj3MuFo7aRMknaKfBQwxs4Fm1g4YB8yoO4OZDQduI5T8yszHFJGsa9cRzr0LTvg6zLkH7jgFVr8eO5VkQJNF7+41wNXATGA+MNXd55nZDWY2JjXbTUAn4AEze9HMZjTwcSKSzwoK4OTrwrj265fCbSfCnHu1KaeVM4/0P7CystKrqqqiLFtE0rBhGUybAG/+HQ45F878CbTvEjtVm2dms929sjnv0ZmxIlK/Ln3CUMcnXQ/zHoLbjofq2bFTSQuo6EWkYQWFcOI14RDMnbVw58fgHz8Nj6XVUNGLSNP6Hw1X/R2GjoK/fBvuOBXeeTl2KkmTil5E0lPaDc6/Gz7xa1i/BCaNhMf/G7a/GzuZNEFFLyLpM4NDz4XP/xOGXwTP/hxuPRpe/0vsZNIIFb2INF+H7jDmF+FShYUlcO8n4MFPwiadRpOPVPQi0nIDjoPPPgMjJ8L8P8AvRsDffqzNOXlGRS8ie6eoBEZeC599FgYcD09+D34+HKruhNodsdMJKnoRyZQeQ2D8ffDJmdBtIDzylbD9/tWHdWZtZCp6Ecms/kfDJx8LwygUFMHUS8O4OYueVOFHoqIXkcwzC8fcf/ZZOOtW2LgC7j47nF378lRt0skxFb2IZE9BYTgM84v/gjE3hwucTLsSfvYhePZm2LohdsI2QUUvItlXVAJHXAKfex4unArdBsDj18FPDg4nXa17O3bCRNPolSISx9LZ4Vv9q9PDtvtBJ8LwS+DAM6C4NHa6vNWS0StV9CIS17q34cX7wrj369+Gkq7h7NvhF0Of4WF7v7xHRS8irdfOnfDm30Lhz58BNVthn2FwyDlw0BjoOTR2wrygoheRZNiyDuZNC9/0q2eFaeVD4KCPw0FnQp8j2uw3fRW9iCTPhmXw7z+GIRbe/Ad4LXTpG7blDz4lDMNQ0jl2ypxR0YtIsm1eC689BvMfgUVPhM07BUXQdwQMGgkDT4SKD0NRu9hJs0ZFLyJtx46tsOQFWPw0vPFXWDYHfCcUd4D+x0C/I6GiMvwSKO0WO23GtKToi7IVRkQkq4rbh0MyB50Ynm9ZFzbtvPHXcP/0jUDqi2z5kFD6FZVh+/4+B7WpQzhV9CKSDKVlYUftQWeG51s3hG/5S6ugugoW/gVemhxeswLoPhh6Hbz7rWt/KEjeeaQqehFJpvZddv/G7x6O2V82B1a+CivmwTsvhRO2dikqhfLB0H0QlO+fejw4PO7Yo9Ue6aOiF5G2wQy67RduB499f/q2TbByPqx4BVa/DmsXhV8ECx6FnTXvz1fcEbpWQFk/6Nov9bh/eNy5d7jl6eYgFb2ItG0lnaDfh8OtrtoaWPcWrF0MaxaGvwbWvR0ujL5sDmxeU89ndYXOvaBT6ta5N3QoD38NdChP3XqESzG2L8vZZqK0it7MTgd+BhQCd7j7jXu8XgL8DhgBrAEucPc3MxtVJDmmz1nKTTMXsGzdFvqUlXLNaUMZO7wvABfd/hzPLFr73rzHDe7OvVcek9Z7G3vt+ulzmfzCEmrdKTRj/FH9+P7YQ/c6bzqvt/Szs5U5rbyFRWHTTflgGPLRD3zuf7b+h8O7vMsXRpRw7D47YONy2LTi/fvqWeEaujVb6g9oBdC+a51bWbgvLYOSLuHcgJLO0K7T+49beL5Ak4dXmlkh8BrwUaAamAWMd/dX68zzOeAwd7/KzMYBZ7v7BY19rg6vlLZq+pylTJw2ly07at+bVlpcyA/POZQHqt7ereR32VX2jb0XaPC1qrfWcs/zHxwh8uKj+zdZnI0tc+zwvk2+3tLPzlZmaHg97U3eBt+7fTNsXh3+Ati8JpwL8G7q+db1qdu6cL9l3fvTGvgFYd/dkPnj6M3sGOA77n5a6vlEAHf/YZ15Zqbmec7MioDlQE9v5MNV9NJWHXfjkyxd98Ef4r5lpfVO3+XNG89o9L1Ag68tX7+V2np+HAvNWPTD0S3O+8y1Jzf5eks/O1uZoeH1tDd5m3pvs9XWwPaNYR/C9k2wbSNs24gNOTUrx9H3BZbUeV4NHNXQPO5eY2brgXJgdd2ZzGwCMAGgf//+zckpkhjLGijzhqbv7XuXrdtCQ9+46ivS5i4zW/892czc3Nf25nNbrLAonOiVgZO9cnrAqLtPcvdKd6/s2bNnLhctkjf6lNV/ZEZD09N9b2OvFTZwWGBD09NdZjqvt/Szs5U5W3nzWTpFvxToV+d5RWpavfOkNt10JeyUFZE9XHPaUEqLC3ebVlpcyDWnDeW4wd3rfc+u6Y29t7HXxh/Vj/o0ND3dvOm83tLPzlbmbOXNZ+lsupkFDDGzgYRCHwdcuMc8M4DLgOeAc4EnG9s+L9KW7dppV99RH2OH9230qJvG3rtLQ58LtOgIlqaWmU6mlq6LbGXORt58ltagZmY2Gvgp4fDKO939B2Z2A1Dl7jPMrD1wNzAcWAuMc/fFjX2mdsaKiDRf1gY1c/dHgUf3mPatOo+3Auc1Z8EiIpIbyRu9R0REdqOiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqU1qFlWFmy2EVgQZeEN68EeF0vJE/mYS5nSo0zpy8dc+ZhpqLs36+KxaQ1qliULmjsCW7aZWVW+ZYL8zKVM6VGm9OVjrnzN1Nz3aNONiEjCqehFRBIuZtFPirjshuRjJsjPXMqUHmVKXz7mSkSmaDtjRUQkN7TpRkQk4VT0IiIJl7OiN7NCM5tjZo+kng80sxfMbKGZ3W9m7XKVpYlcd5nZG2b2Yur2oRznedPM5qaWXZWa1t3M/mxmr6fuu+VBpu+Y2dI662l0jjOVmdmDZvZvM5tvZsfEXk+N5Iq2rsxsaJ3lvmhmG8zsyzHXVSOZYv+b+oqZzTOzV8xsspm1j91TDWRqdkflbBu9mX0VqAS6uPuZZjYVmObuU8zsV8BL7v7LnIRpPNddwCPu/mCus6TyvAlUuvvqOtN+BKx19xvN7Fqgm7t/I3Km7wCb3P3HucqxR6bfAn939ztSP3wdgG8ScT01kuvLRFxXdbIVAkuBo4DPE3ld1ZPpCiKtJzPrC/wDGObuW1L99Cgwmkg91UimkTSzo3Lyjd7MKoAzgDtSzw04GdgV9LfA2FxkaSxXHjuLsI4g0rrKJ2bWFTgB+DWAu29393VEXk+N5MoXpwCL3P0t8uffVN1MsRUBpWZWRPgF/Q7xe2rPTMta8iG52nTzU+DrwM7U83JgnbvXpJ5XA31zlKWxXLv8wMxeNrOfmFlJjjM58LiZzTazCalpvdz9ndTj5UCvPMgEcHVqPd2Z480kA4FVwG8sbHa7w8w6En89NZQL4q2rusYBk1OPY6+rXepmgkjryd2XAj8G3iYU/HpgNhF7qr5M7v546uVmdVTWi97MzgRWuvvsbC+rORrJNRE4EPgw0B3I9Z+zH3H3I4BRwOfN7IS6L3rY1pbrY2Lry/RLYDDwIcI/wv/NYZ4i4Ajgl+4+HHgXuLbuDJHWU0O5Yq4rAFKbkcYAD+z5WqR1VV+maOsp9UvlLMIv6z5AR+D0XC0/3UxmdjEt6KhcfKM/DhiT2s47hfCn0M+AstSfIwAVhO10ufSBXGZ2j7u/48E24DfAkbkMlfotjruvBB5KLX+Fme0LkLpfGTuTu69w91p33wncTm7XUzVQ7e4vpJ4/SCjYqOupoVyR19Uuo4B/ufuK1PPY6+oDmSKvp1OBN9x9lbvvAKYROiJmT9WX6diWdFTWi97dJ7p7hbsPIPyZ9qS7XwQ8BZybmu0y4OFsZ0kj18V1/vEbYXvcK7nKZGYdzazzrsfAx1LLn0FYR5DjddVQpl3rKeVscrie3H05sMTMhqYmnQK8SsT11FiumOuqjvHsvokk6rpK2S1T5PX0NnC0mXVI/ezv+jcVs6fqyzS/RR3l7jm78f7eYoBBwD+BhYQ/3UpymaWRXE8Cc1Mr7x6gUw5zDAJeSt3mAdelppcDTwCvA38BuudBprtT6+llQmnsm+P/Zx8CqlLLnw50i7memsgVe111BNYAXetMi7quGsgUez19F/h36mf/bqAkdk81kKnZHaUhEEREEk5nxoqIJJyKXkQk4VT0IiIJp6IXEUk4Fb2ISMKp6EVEEk5FLyKScP8fcHJtND9r1YoAAAAASUVORK5CYII=\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w_val = model.get_weights()[0]\n",
    "b_val = model.get_weights()[1]\n",
    "plt.plot(data[:,0], data[:,1], 'o')\n",
    "plt.axis([40, 85, -0.1, 1.2])\n",
    "x_pred = np.linspace(40,85)\n",
    "x_pred = np.resize(x_pred,[len(x_pred),1])\n",
    "y_pred = 1 / (1 + np.exp(-x_pred*w_val - b_val))\n",
    "plt.plot(x_pred, y_pred)\n",
    "\n",
    "p_1 = 1 / (1 + np.exp(-x*w_val - b_val))\n",
    "like = y * np.log(p_1) + (1-y) * np.log(1-p_1)\n",
    "print(-np.mean(like))\n",
    "print(np.round(p_1,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 3 : MNIST and Multinomial Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise we use multinomial logistic regression to predict the number of the handwritten digits of the MNIST dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO : read MNIST data and compute validation accuracy for the multinomial logistic regression model"
   ]
  },
  {
   "cell_type": "code",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
849 
   "execution_count": 24,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
850 851 852 853 854 855 856 
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
857 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 321.2496 - accuracy: 0.8419 - val_loss: 243.0990 - val_accuracy: 0.8799\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
858 
      "Epoch 2/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
859 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 263.8232 - accuracy: 0.8672 - val_loss: 203.2465 - val_accuracy: 0.8986\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
860 
      "Epoch 3/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
861 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 253.0255 - accuracy: 0.8741 - val_loss: 208.3061 - val_accuracy: 0.8858\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
862 
      "Epoch 4/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
863 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 247.8131 - accuracy: 0.8757 - val_loss: 522.1936 - val_accuracy: 0.7770\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
864 
      "Epoch 5/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
865 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 244.3168 - accuracy: 0.8773 - val_loss: 264.0963 - val_accuracy: 0.8677\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
866 
      "Epoch 6/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
867 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 239.8172 - accuracy: 0.8792 - val_loss: 251.0652 - val_accuracy: 0.8803\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
868 
      "Epoch 7/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
869 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 243.0666 - accuracy: 0.8798 - val_loss: 245.4965 - val_accuracy: 0.8903\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
870 
      "Epoch 8/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
871 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 236.9614 - accuracy: 0.8811 - val_loss: 270.9126 - val_accuracy: 0.8773\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
872 
      "Epoch 9/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
873 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 235.9632 - accuracy: 0.8825 - val_loss: 208.9754 - val_accuracy: 0.8894\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
874 
      "Epoch 10/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
875 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 229.1799 - accuracy: 0.8818 - val_loss: 288.4063 - val_accuracy: 0.8719\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "from tensorflow.keras.datasets import mnist\n",
    "from tensorflow.keras.utils import to_categorical\n",
    "\n",
    "\n",
    "# Import TensorFlow and TensorFlow Datasets\n",
    "import tensorflow as tf\n",
    "\n",
    "# Helper libraries\n",
    "import math\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Load MNIST data\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "\n",
    "# One-hot-encoded label vector\n",
    "y_train_cat = to_categorical(y_train, 10)\n",
    "y_test_cat = to_categorical(y_test, 10)\n",
    "\n",
    "model = tf.keras.Sequential()\n",
    "model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))\n",
    "model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax, batch_input_shape=(None, 784)))\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='sgd',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "history = model.fit(X_train, \n",
    "                    y_train_cat, \n",
    "                    epochs=10, \n",
    "                    validation_data=(X_test, y_test_cat))"
   ]
  },
  {
   "cell_type": "code",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
915 
   "execution_count": 25,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
916 917 918 919 920 921 
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
922 923 
      "313/313 [==============================] - 1s 4ms/step - loss: 288.4063 - accuracy: 0.8719\n",
      "Accuracy on test dataset: 0.8719000220298767\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 
     ]
    }
   ],
   "source": [
    "test_loss, test_accuracy = model.evaluate(X_test, y_test_cat)\n",
    "print('Accuracy on test dataset:', test_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO : use different regularization terms, see [Keras Regularizer](https://keras.io/regularizers/)"
   ]
  },
  {
   "cell_type": "code",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
941 
   "execution_count": 26,
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
942 943 944 945 946 947 948 
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
949 
      "1875/1875 [==============================] - 10s 5ms/step - loss: 333.7498 - accuracy: 0.8395 - val_loss: 261.9417 - val_accuracy: 0.8583\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
950 
      "Epoch 2/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
951 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 291.4153 - accuracy: 0.8573 - val_loss: 244.7811 - val_accuracy: 0.8729\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
952 
      "Epoch 3/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
953 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 282.5951 - accuracy: 0.8601 - val_loss: 412.3524 - val_accuracy: 0.8189\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
954 
      "Epoch 4/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
955 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 289.8486 - accuracy: 0.8601 - val_loss: 258.7878 - val_accuracy: 0.8644\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
956 
      "Epoch 5/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
957 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 279.0330 - accuracy: 0.8620 - val_loss: 324.4070 - val_accuracy: 0.8390\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
958 
      "Epoch 6/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
959 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 285.5763 - accuracy: 0.8607 - val_loss: 223.8089 - val_accuracy: 0.8819\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
960 
      "Epoch 7/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
961 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 278.4053 - accuracy: 0.8630 - val_loss: 241.8423 - val_accuracy: 0.8798\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
962 
      "Epoch 8/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
963 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 287.0358 - accuracy: 0.8609 - val_loss: 278.3787 - val_accuracy: 0.8620\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
964 
      "Epoch 9/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
965 
      "1875/1875 [==============================] - 8s 4ms/step - loss: 284.3555 - accuracy: 0.8619 - val_loss: 321.4152 - val_accuracy: 0.8505\n",
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
966 
      "Epoch 10/10\n",
Mirko Birbaumer's avatar
model.predict_classes needs to be replaced by model.predict  
Mirko Birbaumer committed
967 
      "1875/1875 [==============================] - 9s 5ms/step - loss: 282.8943 - accuracy: 0.8621 - val_loss: 218.7821 - val_accuracy: 0.8939\n"
Simon van Hemert's avatar
Added Notebooks and Requirements
Simon van Hemert committed
968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 
     ]
    }
   ],
   "source": [
    "# Import TensorFlow and TensorFlow Datasets\n",
    "import tensorflow as tf\n",
    "\n",
    "# Helper libraries\n",
    "import math\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Load MNIST data\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "\n",
    "# One-hot-encode label vector\n",
    "y_train_cat = to_categorical(y_train, 10)\n",
    "y_test_cat = to_categorical(y_test, 10)\n",
    "\n",
    "# Define Network\n",
    "model = tf.keras.Sequential()\n",
    "model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))\n",
    "model.add(tf.keras.layers.Dense(10, \n",
    "                                activation=tf.nn.softmax, \n",
    "                                batch_input_shape=(None, 784), \n",
    "                                kernel_regularizer=tf.keras.regularizers.l2(0.01)))\n",
    "\n",
    "# Compile Network\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='sgd',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Fit Network\n",
Show full blame

This GitLab is a part of renkulab.io