Newer
Older
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "YHI3vyhv5p85"
},
"source": [
"# Exercise 1 : Conversion from Celsius to Fahrenheit (Simple Regression Analysis)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "F8YVA_634OFk"
},
"source": [
"The problem we will solve is to convert from Celsius to Fahrenheit, where the approximate formula is:\n",
"\n",
"$$ f = c \\times 1.8 + 32 $$\n",
"\n",
"\n",
"Of course, it would be simple enough to create a conventional Python function that directly performs this calculation, but that wouldn't be machine learning.\n",
"\n",
"\n",
"Instead, we will give TensorFlow some sample Celsius values (0, 8, 15, 22, 38) and their corresponding Fahrenheit values (32, 46, 59, 72, 100).\n",
"Then, we will train a model that figures out the above formula through the training process. This is a _simple regression analysis_ problem."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "fA93WUy1zzWf"
},
"source": [
"## Import dependencies\n",
"\n",
"First, import TensorFlow. Here, we're calling it `tf` for ease of use. We also tell it to only display errors.\n",
"\n",
"Next, import [NumPy](http://www.numpy.org/) as `np`. Numpy helps us to represent our data as highly performant lists."
]
},
{
"cell_type": "code",
"metadata": {
"colab": {},
"colab_type": "code",
"id": "X9uIpOS2zx7k"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
]
}
],
"source": [
"from __future__ import absolute_import, division, print_function, unicode_literals\n",
"import tensorflow as tf\n",
"print(tf.__version__)\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {},
"colab_type": "code",
"id": "y_WQEM5MGmg3"
},
"outputs": [],
"source": [
"import logging\n",
"logger = tf.get_logger()\n",
"logger.setLevel(logging.ERROR)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "AC3EQFi20buB"
},
"source": [
"## Set up training data\n",
"\n",
"As we saw before, supervised Machine Learning is all about figuring out an algorithm given a set of inputs and outputs. Since the task in this exercise is to create a model that can give the temperature in Fahrenheit when given the degrees in Celsius, we create two lists `celsius_q` and `fahrenheit_a` that we can use to train our model."
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
"metadata": {
"colab": {},
"colab_type": "code",
"id": "gg4pn6aI1vms"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"-40.0 degrees Celsius = -40.0 degrees Fahrenheit\n",
"-10.0 degrees Celsius = 14.0 degrees Fahrenheit\n",
"0.0 degrees Celsius = 32.0 degrees Fahrenheit\n",
"8.0 degrees Celsius = 46.0 degrees Fahrenheit\n",
"15.0 degrees Celsius = 59.0 degrees Fahrenheit\n",
"22.0 degrees Celsius = 72.0 degrees Fahrenheit\n",
"38.0 degrees Celsius = 100.0 degrees Fahrenheit\n"
]
}
],
"source": [
"celsius_q = np.array([-40, -10, 0, 8, 15, 22, 38], dtype=float)\n",
"fahrenheit_a = np.array([-40, 14, 32, 46, 59, 72, 100], dtype=float)\n",
"\n",
"for i,c in enumerate(celsius_q):\n",
" print(\"{} degrees Celsius = {} degrees Fahrenheit\".format(c, fahrenheit_a[i]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "wwJGmDrQ0EoB"
},
"source": [
"### Some Machine Learning terminology\n",
"\n",
" - **Feature** — The input(s) to our model. In this case, a single value — the degrees in Celsius.\n",
"\n",
" - **Labels/response variable** — The output our model predicts. In this case, a single value — the degrees in Fahrenheit. In a classification setting, we would predict labels (discrete classes), in a regression setting, we predict a continuous response variable, such as Fahrenheit.\n",
"\n",
" - **Example** — A pair of inputs/outputs used during training. In our case a pair of values from `celsius_q` and `fahrenheit_a` at a specific index, such as `(22,72)`.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "VM7_9Klvq7MO"
},
"source": [
"## 1. Define the Network\n",
"\n",
"Next create the model. We will use the simplest possible model we can, a Dense network. Since the problem is straightforward, this network will require only a single layer, with a single neuron.\n",
"\n",
"### Build a layer\n",
"\n",
"We'll call the layer `l0` and create it by instantiating `tf.keras.layers.Dense` with the following configuration:\n",
"\n",
"* `input_shape=[1]` — This specifies that the input to this layer is a single value. That is, the shape is a one-dimensional array with one member. Since this is the first (and only) layer, that input shape is the input shape of the entire model. The single value is a floating point number, representing degrees Celsius.\n",
"\n",
"* `units=1` — This specifies the number of units in the layer. The number of units defines how many internal variables the layer has to try to learn how to solve the problem (more later). Since this is the final layer, it is also the size of the model's output — a single float value representing degrees Fahrenheit. (In a multi-layered network, the size and shape of the layer would need to match the `input_shape` of the next layer.)\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "pRllo2HLfXiu"
},
"outputs": [],
"source": [
"l0 = tf.keras.layers.Dense(units=1, input_shape=[1])"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "_F00_J9duLBD"
},
"source": [
"### Assemble layers into the model\n",
"\n",
"Once layers are defined, they need to be assembled into a model. The Sequential model definition takes a list of layers as argument, specifying the calculation order from the input to the output.\n",
"\n",
"This model has just a single layer, `l0`."
]
},
{
"cell_type": "code",
"execution_count": 6,
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
"metadata": {
"colab": {},
"colab_type": "code",
"id": "cSp-GpLSuMRq"
},
"outputs": [],
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Dense(units=1, input_shape=[1])\n",
"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "kiZG7uhm8qCF"
},
"source": [
"## 2. Compile the network, with loss and optimizer functions\n",
"\n",
"Before training, the model has to be compiled. When compiled for training, the model is given:\n",
"\n",
"- **Loss function** — A way of measuring how far off predictions are from the desired outcome. (The measured difference is called the \"loss\".)\n",
"\n",
"- **Optimizer function** — A way of adjusting internal values in order to reduce the loss.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
"metadata": {
"colab": {},
"colab_type": "code",
"id": "m8YQN1H41L-Y"
},
"outputs": [],
"source": [
"model.compile(loss='mean_squared_error',\n",
" optimizer=tf.keras.optimizers.Adam(0.1))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "17M3Pqv4P52R"
},
"source": [
"These are used during training (`model.fit()`, below) to first calculate the loss at each point, and then improve it. In fact, the act of calculating the current loss of a model and then improving it is precisely what training is.\n",
"\n",
"During training, the optimizer function is used to calculate adjustments to the model's internal variables. The goal is to adjust the internal variables until the model (which is really a math function) mirrors the actual equation for converting Celsius to Fahrenheit.\n",
"\n",
"`TensorFlow` uses numerical analysis to perform this tuning, and all this complexity is hidden from you so we will not go into the details here. What is useful to know about these parameters are:\n",
"\n",
"The loss function ([mean squared error](https://en.wikipedia.org/wiki/Mean_squared_error)) and the optimizer ([Adam](https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/)) used here are standard for simple models like this one, but many others are available. It is not important to know how these specific functions work at this point.\n",
"\n",
"One part of the Optimizer you may need to think about when building your own models is the learning rate (`0.1` in the code above). This is the step size taken when adjusting values in the model. If the value is too small, it will take too many iterations to train the model. Too large, and accuracy goes down. Finding a good value often involves some trial and error, but the range is usually within 0.001 (default), and 0.1"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "c-Jk4dG91dvD"
},
"source": [
"## 3. Fit the model\n",
"\n",
"Train the model by calling the `fit` method.\n",
"\n",
"During training, the model takes in Celsius values, performs a calculation using the current internal variables (called \"weights\") and outputs values which are meant to be the Fahrenheit equivalent. Since the weights are initially set randomly, the output will not be close to the correct value. The difference between the actual output and the desired output is calculated using the loss function, and the optimizer function directs how the weights should be adjusted.\n",
"\n",
"This cycle of calculate, compare, adjust is controlled by the `fit` method. The first argument is the inputs, the second argument is the desired outputs. The `epochs` argument specifies how many times this cycle should be run, and the `verbose` argument controls how much output the method produces."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "lpRrl7WK10Pq"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Finished training the model\n"
]
}
],
"source": [
"history = model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)\n",
"print(\"Finished training the model\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "0-QsNCLD4MJZ"
},
"source": [
"## 4. Evaluate the Model - Display training statistics\n",
"\n",
"The `fit` method returns a history object. We can use this object to plot how the loss of our model goes down after each training epoch. A high loss means that the Fahrenheit degrees the model predicts is far from the corresponding value in `fahrenheit_a`.\n",
"\n",
"We'll use [Matplotlib](https://matplotlib.org/) to visualize this (you could use another tool). As you can see, our model improves very quickly at first, and then has a steady, slow improvement until it is very near \"perfect\" towards the end.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "IeK6BzfbdO6_"
},
"outputs": [
{
"data": {
"text/plain": [
"[<matplotlib.lines.Line2D at 0x7fd953358790>]"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjTUlEQVR4nO3de5RkZXnv8e9T175fp+c+MDOCIooITBDUlRXxRogRj5qI0UgS1iEnIRFzl+SsqElMYlaEQKIoCURUIiZEDyxDIoho4g0YFOQyAsNwmR5mmJ7pnu6evlfVc/7Yb3XX9HR11cx0dVVX/T5r1aq9372r+tlDM7953733u83dERERWUys2gWIiEjtU1iIiEhJCgsRESlJYSEiIiUpLEREpKREtQuohFWrVvnmzZurXYaIyIry4IMPHnD3voW21WVYbN68me3bt1e7DBGRFcXMniu2TcNQIiJSksJCRERKUliIiEhJCgsRESlJYSEiIiUpLEREpCSFhYiIlKSwKPDCoQmuvusJnjkwVu1SRERqisKiwODYNNd9cydPvjha7VJERGqKwqJAZ3MSgOGJmSpXIiJSWxQWBTpborAYUViIiBxBYVGgLZUgZnBoXGEhIlJIYVEgFjM6mpMahhIRmUdhMU+nwkJE5CgKi3m6FBYiIkdRWMyjYSgRkaMpLObRMJSIyNEUFvMoLEREjqawmCcfFu5e7VJERGqGwmKerpYk2ZwzNp2tdikiIjVDYTGPpvwQETmawmKefFgcGp+uciUiIrVDYTFPh3oWIiJHUVjMk+9ZaDJBEZE5FQ0LM/sdM3vMzB41sy+ZWZOZbTGz+8xsp5l92cxSYd90WN8Ztm8u+J6rQvsTZvbWStbc1ZIC1LMQESlUsbAwsw3AB4Ft7v5KIA5cAnwCuMbdTwGGgMvCRy4DhkL7NWE/zOz08LlXABcCnzazeKXq1gluEZGjVXoYKgE0m1kCaAH2AhcAt4XtNwPvCMsXh3XC9jeamYX2W919yt2fAXYC51aq4NZUnHjMFBYiIgUqFhbuvgf4W+B5opAYBh4EDrl7JuzWD2wIyxuA3eGzmbB/b2H7Ap+ZZWaXm9l2M9s+MDBw3HWbGZ3NST3TQkSkQCWHobqJegVbgPVAK9EwUkW4+w3uvs3dt/X19Z3Qd2nKDxGRI1VyGOpNwDPuPuDuM8BXgNcBXWFYCmAjsCcs7wE2AYTtncDBwvYFPlMRmnlWRORIlQyL54HzzKwlnHt4I/A4cC/w7rDPpcDtYfmOsE7Y/k2PJmi6A7gkXC21BTgVuL+CddPVnNSlsyIiBRKldzk+7n6fmd0G/BDIAD8CbgD+A7jVzP4itN0YPnIj8AUz2wkMEl0Bhbs/Zmb/ShQ0GeAKd6/oxE2dzUmeOzhWyR8hIrKiVCwsANz9I8BH5jXvYoGrmdx9EviFIt/zceDjS15gEZ3NSQ6pZyEiMkt3cC+gMwxD5XKaplxEBBQWC+psTpJzODydKb2ziEgDUFgsoLMl3MWtey1ERACFxYI05YeIyJEUFgtQWIiIHElhsQCFhYjIkRQWC1BYiIgcSWGxgK4WhYWISCGFxQKak3GScU1TLiKSp7BYgKYpFxE5ksKiiA5NJigiMkthUUSXpikXEZmlsCiiqyXF0Ph0tcsQEakJCosiultSOmchIhIoLIrobkkyOKaehYgIKCyK6m5NMTGTZXKmos9ZEhFZERQWReRvzNNQlIiIwqKonpYUgE5yi4igsCiqS2EhIjJLYVFEd2s0DDU0pmEoERGFRRHd6lmIiMxSWBQxd4JbYSEiorAoIp2I05qKM6SroUREFBaL0ZQfIiIRhcUiuluTDOkubhERhcViultSGoYSEUFhsahoMkH1LEREFBaL6G5JqmchIoLCYlFdLSlGJmfIZHPVLkVEpKoUFovobknijp6YJyINT2GxiO7W/F3cCgsRaWwKi0Xkp/zQSW4RaXQKi0XMzQ+lnoWINDaFxSLy80PpLm4RaXQKi0X05M9Z6C5uEWlwCotFtKTipOIxDUOJSMOraFiYWZeZ3WZmPzGzHWZ2vpn1mNndZvZUeO8O+5qZXWdmO83sx2Z2dsH3XBr2f8rMLq1kzfPq1/xQIiKUERbhL/H3m9mfhvWTzOzcMr//WuC/3P004ExgB/Bh4B53PxW4J6wD/CxwanhdDlwffl4P8BHgNcC5wEfyAbMcelvTHBybWq4fJyJSk8rpWXwaOB94b1gfBT5V6kNm1gn8NHAjgLtPu/sh4GLg5rDbzcA7wvLFwOc98gOgy8zWAW8F7nb3QXcfAu4GLiyj7iXR25bioHoWItLgygmL17j7FcAkQPgLO1XG57YAA8A/m9mPzOyfzKwVWOPue8M++4A1YXkDsLvg8/2hrVj7EczscjPbbmbbBwYGyiivPL2tKQ4eVliISGMrJyxmzCwOOICZ9QHlTJaUAM4Grnf3s4Ax5oacAHB3z3/viXL3G9x9m7tv6+vrW4qvBKCnNc2gehYi0uDKCYvrgK8Cq83s48B3gL8s43P9QL+73xfWbyMKjxfD8BLhfX/YvgfYVPD5jaGtWPuy6G1LcXgqw+RMdrl+pIhIzSkZFu5+C/CHwF8Be4F3uPu/lfG5fcBuM3tZaHoj8DhwB5C/oulS4PawfAfwgXBC/TxgOAxXfR14i5l1hxPbbwlty6I33Guh3oWINLJEsQ3hKqS8/cCXCre5+2AZ3//bwC1mlgJ2Ab9KFFD/amaXAc8Bvxj2vRO4CNgJjId9cfdBM/tz4IGw35+V+bOXRP7GvIOHp1nf1bxcP1ZEpKYUDQvgQaLzCQacBAyF5S7geaIT2Ity94eAbQtseuMC+zpwRZHvuQm4qdTPq4TetjSALp8VkYZWdBjK3be4+1bgG8DPu/sqd+8F3gbctVwFVltvQc9CRKRRlXOC+zx3vzO/4u7/Cby2ciXVlp42nbMQEVlsGCrvBTP7v8AXw/r7gBcqV1JtaU8nSMVjHNAwlIg0sHJ6Fu8F+ogun/0qsJq5u7nrnpnR05piUMNQItLASvYswpVHVy5DLTVLU36ISKMrGRZmdi8L3GXt7hdUpKIa1NOqsBCRxlbOOYvfL1huAt4FZCpTTm1a1ZbmmQNj1S5DRKRqyhmGenBe03fN7P4K1VOTelpTuhpKRBpaOcNQhXdyx4BzgM6KVVSDelpTjE9nmZjO0pyKV7scEZFlV84wVOGd3BngGeCyShZVa1aFey0Ojk2xMdVS5WpERJZfOWHxcnefLGwws3SF6qlJPa1hyo/D02zsVliISOMp5z6L7y3Q9v2lLqSW9eoubhFpcIvNOruW6Il0zWZ2FtEwFEAH0FD/vM7PD3XgsO7iFpHGtNgw1FuBXyF62NDVBe2jwB9XsKaasyrMPHtAd3GLSIMqGhbufjNws5m9y93/fRlrqjmt6QStqTgDo+pZiEhjWmwY6v3u/kVgs5n97vzt7n71Ah+rW33tafaPTpbeUUSkDi02DNUa3tuWo5Ba19eeVs9CRBrWYsNQnw3vH1u+cmrX6vYmduwbqXYZIiJVUc4d3H3A/wY2F+7v7r9WubJqT197mv9+Sj0LEWlM5dyUdzvwP0SPV81Wtpza1deeZnQyw+RMlqakpvwQkcZSTli0uPsfVbySGtfXHl0+OzA6xaaehrrNRESkrDu4v2ZmF1W8khqXD4v9OsktIg2onLC4kigwJsxsxMxGzazhzvT2teV7Frp8VkQaTznPs2hfjkJq3eqOuWEoEZFGU87VUGcv0DwMPOfuDfPEvN7WNDFTWIhIYyrnBPengbOBR8L6GcCjQKeZ/Ya731Wp4mpJPGb0tKYZ0GSCItKAyjln8QJwlruf4+7nAK8GdgFvBv6mgrXVnNXtafaPKCxEpPGUExYvdffH8ivu/jhwmrvvqlxZtamvXT0LEWlM5QxDPWZm1wO3hvX3AI+Hp+XNVKyyGtTXnubJF0erXYaIyLIrp2fxK8BO4EPhtSu0zQBvqExZtSk/mWAu59UuRURkWZVz6ewE8Mnwmu/wkldUw1a3p8nknEMTM/SEp+eJiDSCkj0LMzvVzG4zs8fNbFf+tRzF1Zq5u7h1Y56INJZyhqH+GbgeyBANO30e+GIli6pVazqaAHRFlIg0nHLCotnd7wHM3Z9z948CP1fZsmrT2hAW+4bVsxCRxlLO1VBTZhYDnjKz3wL20KBPz8v3LPYqLESkwZQ7kWAL8EHgHOCXgUsrWVStSiVirGpLs29kotqliIgsq5Jh4e4PuPthd+93919193e6+w/K/QFmFjezH5nZ18L6FjO7z8x2mtmXzSwV2tNhfWfYvrngO64K7U+Y2VuP4ziXzLrOJvUsRKThFB2GMrM7Fvugu7+9zJ9xJbAD6AjrnwCucfdbzewzwGVEJ9AvA4bc/RQzuyTs9x4zOx24BHgFsB74hpm91N2r8tS+tZ1N7B4cr8aPFhGpmsXOWZwP7Aa+BNwH2LF+uZltJDoZ/nHgd83MgAuAXwq73Ax8lCgsLg7LALcB/xD2vxi41d2ngGfMbCdwLvD9Y61nKazrbOL+Zwar8aNFRKpmsWGotcAfA68EriWaOPCAu3/b3b9d5vf/HfCHQC6s9wKHCqY27wc2hOUNROFE2D4c9p9tX+Azs8zscjPbbmbbBwYGyizv2K3tbGJ4Yobx6YaZnV1EpHhYuHvW3f/L3S8FziOa8uNb4YqokszsbcB+d39waUpdnLvf4O7b3H1bX19fxX7Ouk5dPisijWfRS2fDZIE/B7wX2AxcB3y1zO9+HfD28PzuJqJzFtcCXWaWCL2HjUSX4hLeNwH9ZpYAOoGDBe15hZ9Zdms7moEoLLb2NeQVxCLSgIr2LMzs80TnBc4GPubuP+Xuf+7uZf1F7e5XuftGd99MdIL6m+7+PuBe4N1ht0uB28PyHcxdkvvusL+H9kvC1VJbgFOB+4/lIJdSvmehK6JEpJEs1rN4PzBGdDXTB6NzzUB0otvdvaPYB0v4I+BWM/sL4EfAjaH9RuAL4QT2IFHA4O6Pmdm/Ao8TTTlyRbWuhILonAXAvhGFhYg0jqJh4e7l3LBXFnf/FvCtsLyL6Gqm+ftMAr9Q5PMfJ7qiquqaknG6W5LsHdaNeSLSOJYsEBrJ2s5mneAWkYaisDgOuotbRBqNwuI4rO1sUs9CRBpKOQ8/ag2zzmJmLzWzt5tZsvKl1a51HU0cHJtmcqZq59lFRJZVOT2L/waazGwDcBfRrLOfq2RRtW59V3SvxQuHdJJbRBpDOWFh7j4OvBP4tLv/AtGkfg1rU08LAP1DCgsRaQxlhYWZnQ+8D/iP0BavXEm1b1NP1LPYPaTZZ0WkMZQTFh8CrgK+Gm6Q20p0F3bDWt3eRDJu7B5Uz0JEGkPJx6qGGWa/DRBOdB9w9w9WurBaFo8ZG7qa1bMQkYZRztVQ/2JmHWbWCjwKPG5mf1D50mrbpp4W+vUQJBFpEOUMQ53u7iPAO4D/BLYQXRHV0DZ2t+gEt4g0jHLCIhnuq3gHcIe7zwBe0apWgE09zRwcm2ZsSg9BEpH6V05YfBZ4FmgF/tvMTgZGKlnUSrCxW5fPikjjKBkW7n6du29w94s88hzwhmWoraZt6g6Xz+q8hYg0gHJOcHea2dX551ub2SeJehkNLX9jnq6IEpFGUM4w1E3AKPCL4TUC/HMli1oJeltTNCfjutdCRBpCyfssgJe4+7sK1j9mZg9VqJ4Vw8zY1NNMv3oWItIAyulZTJjZ6/MrZvY6QP+cJjrJvVsnuEWkAZTTs/g/wOfNrDOsDwGXVq6kleOknhZ+sOsg7k7BM8pFROpOOVdDPezuZwKvAl7l7mcBF1S8shXgJX2tjE9n2TeiByGJSH0r+0l57j4S7uQG+N0K1bOivKSvDYBdA2NVrkREpLKO97GqGnMBts6GxeEqVyIiUlnHGxYNP90HwJqONK2pOE+rZyEida7oCW4zG2XhUDCguWIVrSBmxpa+Vp5Wz0JE6lzRsHD39uUsZKV6SV8b258dqnYZIiIVdbzDUBJsXdXGC8MTTM5kq12KiEjFKCxO0Na+VtzhmQM6byEi9UthcYK29kVzKuq8hYjUM4XFCdq6SvdaiEj9U1icoOZUnA1dzepZiEhdU1gsgVNWt/HkiwoLEalfCoslcNq6dnbuH2Umm6t2KSIiFaGwWAIvX9vBTNZ13kJE6pbCYgm8fF0HADv2jpTYU0RkZVJYLIGtfa2k4jF27FNYiEh9qlhYmNkmM7vXzB43s8fM7MrQ3mNmd5vZU+G9O7SbmV1nZjvN7MdmdnbBd10a9n/KzGruwUvJeIxTVrfxk72j1S5FRKQiKtmzyAC/5+6nA+cBV5jZ6cCHgXvc/VTgnrAO8LPAqeF1OXA9ROECfAR4DXAu8JF8wNSS09a187iGoUSkTlUsLNx9r7v/MCyPAjuADcDFwM1ht5uBd4Tli4HPe+QHQJeZrQPeCtzt7oPuPgTcDVxYqbqP1xkbOhkYnWLfsJ6aJyL1Z1nOWZjZZuAs4D5gjbvvDZv2AWvC8gZgd8HH+kNbsfb5P+NyM9tuZtsHBgaW9gDK8KqNXQA83H9o2X+2iEilVTwszKwN+HfgQwWPZQXA3Z0lepCSu9/g7tvcfVtfX99SfOUxecX6DhIx4+Hdh5b9Z4uIVFpFw8LMkkRBcYu7fyU0vxiGlwjv+0P7HmBTwcc3hrZi7TWlKRnntHXt/Lh/uNqliIgsuUpeDWXAjcAOd7+6YNMdQP6KpkuB2wvaPxCuijoPGA7DVV8H3mJm3eHE9ltCW805c2MXD/cfIpfTU2dFpL5UsmfxOuCXgQvM7KHwugj4a+DNZvYU8KawDnAnsAvYCfwj8JsA7j4I/DnwQHj9WWirOWdu6mJ0MsOuA5onSkTqS9HHqp4od/8O0fO6F/LGBfZ34Ioi33UTcNPSVVcZP7W5B4D7nhnklNV6Kq2I1A/dwb2ENve20Nee5v5narLjIyJy3BQWS8jMeM2WHu7bNUjUURIRqQ8KiyX2mq297BuZ5PnB8WqXIiKyZBQWS+z8rdF5i+89fbDKlYiILB2FxRJ7SV8b6zub+NYT+0vvLCKyQigslpiZ8TOnreY7Tx1gOqMn54lIfVBYVMAbXraaseks25/VVVEiUh8UFhXw2pf0korH+MYODUWJSH1QWFRAazrBT7+0jzsf2aupP0SkLigsKuTtr17PvpFJ7tdQlIjUAYVFhbzp5atpScW5/aEXql2KiMgJU1hUSEsqwYWvWMvXHn6BsalMtcsRETkhCosKev/5JzM6leErP+yvdikiIidEYVFBZ23q4owNnXzue8/qRLeIrGgKiwoyMy57/RaeHhjjzkf3lv6AiEiNUlhU2M+fuZ6Xrmnj6rueJJPVHd0isjIpLCosHjN+/y0vY9eBMW767jPVLkdE5LgoLJbBm09fw5tevoZP3vUkTw/okasisvIoLJaBmfGX/+uVNKfi/PoXHmR4YqbaJYmIHBOFxTJZ3dHEZ95/Ds8dHOOyzz3A8LgCQ0RWDoXFMjpvay/XXnIWD/cf4p3Xf5dH9wxXuyQRkbIoLJbZRWes4+ZfO5fRyQwXf+q7/MG/Pcwj/cN6ZreI1DSrx7+ktm3b5tu3b692GYs6ND7Ntfc8xS33Pc90JsfJvS1sO7mHMzd1csrqNjZ1t7C2s4lkXHkuIsvDzB50920LblNYVNfw+Az/+ehe7n78RR7uP8SBw9Oz22IGazua2NjdwsbuZjb2RO+bwvq6ziYSChMRWSIKixXC3XlheJJnD4zRPzTOnqEJ+mdf4+wdmaTwP1c8Zmzoaubk3hY297bOvm9e1cLG7haakvHqHYyIrDiLhUViuYuR4syiv/w3dDUvuH06k2Pv8AS7B6Pw2D00zvODEzx3cIz/99AeRiczBd8F6zujIDm5t5XN+fdVLZzc00pzSkEiIuVTWKwgqUSMk3tbObm39aht7s6h8RmePTjGcwfHj3j/+mP7GBybPmL/NR3pI0LkpJ4W1nc1s7G7mb62NLGYLddhicgKoLCoE2ZGd2uK7tYUZ53UfdT24YkZnp8NkTGePTjOcwfHuPeJAQZGj5xCPRk31nY2sb4z6uVs6G5mfVf02tDVxPquZlpS+tURaST6P75BdDYnOWNjJ2ds7Dxq29hUht1D4+w9NEn/oQleKHj9YNdB9o1MMn+G9e6WZEGANLO+q4m1nc2saU+ztrOJNR1NOmciUkcUFkJrOsFpazs4bW3Hgtsz2Rwvjk7NBkj/0FyYPH9wnO8/fZDDCzwNsLM5ydqOJtZ0Nh0RIms7ovc1nWlWtWrIS2QlUFhISYl4bNET7xANc704MsmLI5PsG54My1PsC21P7BthYHTqqB5KImb0tadZ09HEqrY0fe2p8J5mVVv+lWJVe5r2dAIzBYtINSgsZEl0NifpbE7y0jXtRffJZHMcHJtm3/Ak+0Ym2T8Sve8bnmL/6CT9Q+M8tHuIwbHpo0IFIJ2IReHRnqavLXVUmPS2pulpTdHdmqS7JaUbGkWWkMJClk0iHouGnzqaOHOR/bI5Z3BsmgOHp2ZfA6NTHDg8zYHRKQYOT7Hn0CQP7R5mcOzo3kpeezoxe9K/pyUZLbekokBpSdETQiXf3tWSVMCIFKGwkJoTD0NTfe3pkvtmc87QeBQsg4enGRyfZmh8hqGxaQbHphkanw7BM82TLx5maHya8els0e9rTyfoaE7S0ZykszlBZ3OSjqao19TRnH9PzPakCrfphL7UM4WFrGjxmM0OR5VrcibLofGZI8Ik/35ofIaRyRlGJmYYmcjw7IFxRiZnGJ6YWTRkILoPJgqPBG1NSdrTCVrTcdrSSdqb5pbb0nHamhK0phK0NSVoSxe8mhI0J+M6NyM1R2EhDacpGWdtZ5y1nU3H9LmZbI6RiSg4RiYz0fvsenifyDAyMcPoVIbDkzMMjE5xeCoz+8oWGzMrELPoCrUobBK0pOI0p+K0pKIgiZbjRy6Hbfl988vzt6UTMQWRHJcVExZmdiFwLRAH/snd/7rKJUmDScZj9Lal6T2GXkwhd2dyJjcbHGNTGUYnC5bD++HJuXA5PJlhfCbL5HSW/aOTjE9Hy+MzWcans0xncsdUgxk0J+M0JaPgiF5x0sm55aZkaEvEQnt+ueAz8z6f/0wqESMVj5FMGInY3HIyHiMZD+txIx4zhdYKsyLCwsziwKeANwP9wANmdoe7P17dykTKZ2bRv/pT8bLOx5Qjk80xmckxPp1hYjrLRAiRifAan8kyMZ2J2mZCWwiZqUyWqUyOyZnofWom+p6h8Vy0nskyOZNjKr/9GINpMWZR+CZjRjJxZJDkgyWZiJEK64n43HIyHiMRiwInEYInEYsRD23ReuH73P5HbY/P225GPF74+dgR+8fMiMWI3s2IWcFyrMiyQSy2wP755RVyn9GKCAvgXGCnu+8CMLNbgYsBhYU0tEQ8Rls8Rlu68v8ruzvT2dxcwMzMhUq+bSbrzGRyzGRzTGdz0Xo2RyabYzosz20P67OvufXpzJHbxqazzGRyZHI5pjM5su5ks04m52Rzhe85cjmi9xU0oXZhcBwVKLNBU7jtyAAyM8zAgAtOW82f/NzpS17jSgmLDcDugvV+4DWFO5jZ5cDlACeddNLyVSbSIMwsDEnF6WhKVrucknI5j0IlHybZKEyyoT2TPTpo8uu53PwgypHJOjmPQjPrBcu5aDnnHtYLlwu3ET4XlnPR8vzv8bDPEd9TsF8uN/c9uZzjMLvNHdZ2Fr959kSslLAoyd1vAG6A6HkWVS5HRKosFjNiGLqieWmslDuQ9gCbCtY3hjYREVkGKyUsHgBONbMtZpYCLgHuqHJNIiINY0UMQ7l7xsx+C/g60aWzN7n7Y1UuS0SkYayIsABw9zuBO6tdh4hII1opw1AiIlJFCgsRESlJYSEiIiUpLEREpCRzr7/718xsAHjuBL5iFXBgicpZKXTMjUHH3BiO95hPdve+hTbUZVicKDPb7u7bql3HctIxNwYdc2OoxDFrGEpEREpSWIiISEkKi4XdUO0CqkDH3Bh0zI1hyY9Z5yxERKQk9SxERKQkhYWIiJSksChgZhea2RNmttPMPlztepaKmd1kZvvN7NGCth4zu9vMngrv3aHdzOy68GfwYzM7u3qVHz8z22Rm95rZ42b2mJldGdrr9rjNrMnM7jezh8Mxfyy0bzGz+8KxfTlM84+ZpcP6zrB9c1UP4ASYWdzMfmRmXwvrdX3MZvasmT1iZg+Z2fbQVtHfbYVFYGZx4FPAzwKnA+81s6V/kG11fA64cF7bh4F73P1U4J6wDtHxnxpelwPXL1ONSy0D/J67nw6cB1wR/nvW83FPARe4+5nAq4ELzew84BPANe5+CjAEXBb2vwwYCu3XhP1WqiuBHQXrjXDMb3D3VxfcT1HZ320Pz3ht9BdwPvD1gvWrgKuqXdcSHt9m4NGC9SeAdWF5HfBEWP4s8N6F9lvJL+B24M2NctxAC/BDomfVHwASoX3295zo+TDnh+VE2M+qXftxHOvG8JfjBcDXAGuAY34WWDWvraK/2+pZzNkA7C5Y7w9t9WqNu+8Ny/uANWG57v4cwlDDWcB91Plxh+GYh4D9wN3A08Ahd8+EXQqPa/aYw/ZhoHdZC14afwf8IZAL673U/zE7cJeZPWhml4e2iv5ur5iHH0nluLubWV1eQ21mbcC/Ax9y9xEzm91Wj8ft7lng1WbWBXwVOK26FVWWmb0N2O/uD5rZz1S5nOX0enffY2argbvN7CeFGyvxu62exZw9wKaC9Y2hrV69aGbrAML7/tBeN38OZpYkCopb3P0robnujxvA3Q8B9xINwXSZWf4fhoXHNXvMYXsncHB5Kz1hrwPebmbPArcSDUVdS30fM+6+J7zvJ/pHwblU+HdbYTHnAeDUcBVFCrgEuKPKNVXSHcClYflSojH9fPsHwhUU5wHDBV3bFcOiLsSNwA53v7pgU90et5n1hR4FZtZMdI5mB1FovDvsNv+Y838W7wa+6WFQe6Vw96vcfaO7byb6f/ab7v4+6viYzazVzNrzy8BbgEep9O92tU/U1NILuAh4kmic90+qXc8SHteXgL3ADNF45WVE47T3AE8B3wB6wr5GdFXY08AjwLZq13+cx/x6onHdHwMPhddF9XzcwKuAH4VjfhT409C+Fbgf2An8G5AO7U1hfWfYvrXax3CCx/8zwNfq/ZjDsT0cXo/l/66q9O+2pvsQEZGSNAwlIiIlKSxERKQkhYWIiJSksBARkZIUFiIiUpLCQuqemWXD7Jz515LNKGxmm61gNt9F9vuomY2HO27zbYeXswaRE6HpPqQRTLj7q6tdBNGkdb8H/FG1CylkZgmfm0dJZEHqWUjDCs8E+JvwXID7zeyU0L7ZzL4Z5v6/x8xOCu1rzOyrFj0v4mEze234qriZ/aNFz5C4K9w9vZCbgPeYWc+8Oo7oGZjZ75vZR8Pyt8zsGjPbbmY7zOynzOwr4ZkFf1HwNQkzuyXsc5uZtYTPn2Nm3w4Tzn29YDqIb5nZ31n0LIQrT/xPU+qdwkIaQfO8Yaj3FGwbdvczgH8gmr0U4O+Bm939VcAtwHWh/Trg2x49L+JsortnIXpOwKfc/RXAIeBdReo4TBQYx/qX87RHzyz4DNEUDlcArwR+xczyM6a+DPi0u78cGAF+M8yN9ffAu939nPCzP17wvSl33+bunzzGeqQBaRhKGsFiw1BfKni/JiyfD7wzLH8B+JuwfAHwAZid3XXYoqeRPePuD4V9HiR6dkgx1wEPmdnfHkP9+TnKHgEe8zCvj5ntIpog7hCw292/G/b7IvBB4L+IQuXuMNtunGjal7wvH0MN0uAUFtLovMjysZgqWM4CxYahcPdDZvYvRL2DvAxH9vKbinx/bt7PyjH3//D82p1oTqDH3P38IuWMFatTZD4NQ0mje0/B+/fD8veIZjAFeB/wP2H5HuA3YPYhQ53H+TOvBn6dub/oXwRWm1mvmaWBtx3Hd55kZvlQ+CXgO0RPROvLt5tZ0sxecZw1S4NTWEgjmH/O4q8LtnWb2Y+JziP8Tmj7beBXQ/svM3eO4UrgDWb2CNFw03E9o93dDxA9gyAd1meAPyOaBfVu4CfFP13UE0TPGd8BdAPXu/s00TTcnzCzh4lm3n1t8a8QKU6zzkrDCg/M2Rb+8haRRahnISIiJalnISIiJalnISIiJSksRESkJIWFiIiUpLAQEZGSFBYiIlLS/wcXyYKUCF72rgAAAABJRU5ErkJggg==\n",
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"plt.xlabel('Epoch Number')\n",
"plt.ylabel(\"Loss Magnitude\")\n",
"plt.plot(history.history['loss'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "LtQGDMob5LOD"
},
"source": [
"## 5. Use the model to predict values\n",
"\n",
"Now you have a model that has been trained to learn the relationship between `celsius_q` and `fahrenheit_a`. You can use the predict method to have it calculate the Fahrenheit degrees for a previously unknown Celsius degrees.\n",
"\n",
"So, for example, if the Celsius value is 100, what do you think the Fahrenheit result will be? Take a guess before you run this code."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "oxNzL4lS2Gui"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[211.27141]]\n"
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
]
}
],
"source": [
"print(model.predict([100.0]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "jApk6tZ1fBg1"
},
"source": [
"The correct answer is $100 \\times 1.8 + 32 = 212$, so our model is doing really well.\n",
"\n",
"### To review\n",
"\n",
"\n",
"* We created a model with a Dense layer\n",
"* We trained it with 3500 examples (7 pairs, over 500 epochs).\n",
"\n",
"Our model tuned the variables (weights) in the Dense layer until it was able to return the correct Fahrenheit value for any Celsius value. (Remember, 100 Celsius was not part of our training data.)\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "zRrOky5gm20Z"
},
"source": [
"## Looking at the layer weights\n",
"\n",
"Finally, let's print the internal variables of the Dense layer. "
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "kmIkVdkbnZJI"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"These are the layer variables: [array([[1.8298433]], dtype=float32), array([28.287079], dtype=float32)]\n"
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
]
}
],
"source": [
"print(\"These are the layer variables: {}\".format(model.get_weights()))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "RSplSnMvnWC-"
},
"source": [
"The first variable is close to ~1.8 and the second to ~32. These values (1.8 and 32) are the actual variables in the real conversion formula.\n",
"\n",
"This is really close to the values in the conversion formula. We can show how a Dense layer works, but for a single neuron with a single input and a single output, the internal math looks the same as [the equation for a line](https://en.wikipedia.org/wiki/Linear_equation#Slope%E2%80%93intercept_form), $y = mx + b$, which has the same form as the conversion equation, $f = 1.8c + 32$.\n",
"\n",
"Since the form is the same, the variables should converge on the standard values of 1.8 and 32, which is exactly what happened.\n",
"\n",
"With additional neurons, additional inputs, and additional outputs, the formula becomes much more complex, but the idea is the same.\n",
"\n",
"### A little experiment\n",
"\n",
"Just for fun, what if we created more Dense layers with different units, which therefore also has more variables?"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {},
"colab_type": "code",
"id": "Y2zTA-rDS5Xk"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Finished training the model\n",
"[[211.74745]]\n",
"Model predicts that 100 degrees Celsius is: [[211.74745]] degrees Fahrenheit\n",
"These are the l0 variables: [array([[-0.47996262, 0.25047022, -0.1492717 , 0.11854655]],\n",
" dtype=float32), array([-3.2115366, 3.1785958, -2.868187 , 2.7906501], dtype=float32)]\n",
"These are the l1 variables: [array([[ 1.1949005 , 0.80858207, 0.48969162, 0.57684636],\n",
" [-0.31273484, -1.1010896 , -0.36613086, 0.34995782],\n",
" [-0.07984556, 0.8192047 , -0.63080776, -0.01728256],\n",
" [-0.8315819 , 0.24608922, 0.9040395 , 0.02908712]],\n",
" dtype=float32), array([-3.1562395 , -3.0874147 , 2.3403091 , -0.54884386], dtype=float32)]\n",
"These are the l2 variables: [array([[-1.1158108 ],\n",
" [-1.3494587 ],\n",
" [ 0.6974325 ],\n",
" [-0.21758062]], dtype=float32), array([3.0711133], dtype=float32)]\n"
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
]
}
],
"source": [
"l0 = tf.keras.layers.Dense(units=4, input_shape=[1])\n",
"l1 = tf.keras.layers.Dense(units=4)\n",
"l2 = tf.keras.layers.Dense(units=1)\n",
"model = tf.keras.Sequential([l0, l1, l2])\n",
"model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1))\n",
"model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)\n",
"print(\"Finished training the model\")\n",
"print(model.predict([100.0]))\n",
"print(\"Model predicts that 100 degrees Celsius is: {} degrees Fahrenheit\".format(model.predict([100.0])))\n",
"print(\"These are the l0 variables: {}\".format(l0.get_weights()))\n",
"print(\"These are the l1 variables: {}\".format(l1.get_weights()))\n",
"print(\"These are the l2 variables: {}\".format(l2.get_weights()))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "xrpFFlgYhCty"
},
"source": [
"As you can see, this model is also able to predict the corresponding Fahrenheit value really well. But when you look at the variables (weights) in the `l0` and `l1` layers, they are nothing even close to ~1.8 and ~32. The added complexity hides the \"simple\" form of the conversion equation.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise 2 : O-Rings seen with Logistic Regression"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook calculates a logistic regression using Keras. It's basically meant to show the principles of Keras.\n",
"\n",
"### Datset\n",
"\n",
"We investigate the data set of the challenger flight with broken O-rings (`Y=1`\n",
") vs start temperature."
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Text(0, 0.5, 'Broken O-rings')"
]
},
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZM0lEQVR4nO3deZhldX3n8ffH7kZbgqDSY7QXRR+WOC4gpag4StxY4gBmnBGUPGqMPUZJ3IaMJA4hLiORuD0uKOKuD7g8hGkV05q4LyjdoiJgG0SUblQQA26tQPOdP+4pvBSnbt3q5tS5Xbxfz3Ofe8/vbN/6ddX59FnuOakqJEma6Q59FyBJmkwGhCSplQEhSWplQEiSWhkQkqRWBoQkqVVnAZHkXUmuSvKdWcY/Pcm3k1yY5CtJHtxVLZKk+etyD+I9wGEjxv8AeExVPRB4BXB6h7VIkuZpaVcLrqovJLnPiPFfGRo8D1jVVS2SpPnrLCDm6dnAJ2cbmWQtsBZg1113PXC//fZbqLokaVHYuHHjz6pqxXzm6T0gkvwxg4B41GzTVNXpNIegpqamasOGDQtUnSQtDkl+ON95eg2IJA8CzgAOr6pr+qxFknRLvV3mmmQNcDbwZ1X1vb7qkCS162wPIsmZwCHAnkk2A38PLAOoqrcBJwF3B96aBODGqprqqh5J0vx0eRXTsXOM/wvgL7pavyRpx/hNaklSKwNCktTKgJAktTIgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSKwNCktTKgJAktTIgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSq6VdLTjJu4AnAVdV1QNaxgd4I3AE8BvgmVX1ja7q0Y4754ItnLp+E1deu5V77bGcEw7dl6MPWNnrevuqaXvr3Rm97JwLOfNrV7CtiiUJxx60mlce/cC+y1p0/TyJOgsI4D3Am4H3zTL+cGDv5nUQcFrzrgl0zgVbOPHsC9l6wzYAtly7lRPPvhCg0z/KUesFeqlplL76qSsvO+dCPnDej24e3lZ183CfIbHY+nlSdXaIqaq+APx8xCRHAe+rgfOAPZLcs6t6tGNOXb/p5j/GaVtv2Map6zf1tt6+ahplEmvaEWd+7Yp5tS+UxdbPk6rLPYi5rASGf8s2N20/njlhkrXAWoA1a9YsSHG6pSuv3Tqv9j7X23VNo/TVT13ZVjWv9oWy2Pp5Uu0UJ6mr6vSqmqqqqRUrVvRdzu3SvfZYPq/2hVhvXzWNMok17YglybzaF8pi6+dJ1WdAbAFWDw2vato0gU44dF+WL1tyi7bly5ZwwqH79rbevmoaZRJr2hHHHrR6Xu0LZbH186Tq8xDTOuD4JGcxODl9XVXd6vCSJsP0ib+FvmpknPVO0pUsffVTV6ZPRE/aVUyLrZ8nVaqjY4lJzgQOAfYEfgr8PbAMoKre1lzm+mbgMAaXuT6rqjbMtdypqanasGHOySRJQ5JsrKqp+czT2R5EVR07x/gCnt/V+iVJO2anOEktSVp4BoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWnUaEEkOS7IpyaVJXtoyfk2Szya5IMm3kxzRZT2SpPF1FhBJlgBvAQ4H7g8cm+T+MyZ7GfDhqjoAOAZ4a1f1SJLmp8s9iIcBl1bVZVV1PXAWcNSMaQq4S/N5d+DKDuuRJM1DlwGxErhiaHhz0zbsZOC4JJuBc4G/altQkrVJNiTZcPXVV3dRqyRphr5PUh8LvKeqVgFHAO9Pcquaqur0qpqqqqkVK1YseJGSdHvUZUBsAVYPDa9q2oY9G/gwQFV9FbgTsGeHNUmSxtRlQJwP7J1kryS7MDgJvW7GND8CHgeQ5I8YBITHkCRpAnQWEFV1I3A8sB64hMHVShcleXmSI5vJXgI8J8m3gDOBZ1ZVdVWTJGl8S7tceFWdy+Dk83DbSUOfLwYO7rIGSdL26fsktSRpQs0ZEElekOQuGXhnkm8keeJCFCdJ6s84exB/XlW/AJ4I3BX4M+CUTquSJPVunIBI834E8P6qumioTZK0SI0TEBuTfIpBQKxPshtwU7dlSZL6Ns5VTM8G9gcuq6rfJLk78KxOq5Ik9W6cgNi/eb9vcvORpeuSLG2+6yBJWoTGCYi3Ag8Bvs3g3MMDgIuA3ZP8ZVV9qsP6JEk9GeccxJXAAc3N8g4EDgAuA54AvKbL4iRJ/RknIPZprlwCbv72835VdVl3ZUmS+jbOIaaLkpzG4IE/AE8FLk5yR+CGziqTJPVqnD2IZwKXAi9sXpc1bTcAf9xNWZKkvs25B1FVW4HXNq+ZfnWbVyRJmghzBkSSgxk8GvTew9NX1X27K0uS1LdxzkG8E3gRsBHY1m05kqRJMU5AXFdVn+y8EknSRBknID6b5FTgbOB3041V9Y3OqpIk9W6cgDioeZ8aaivgsbd9OZKkSTHOVUxeyipJt0OzBkSS46rqA0le3Da+ql7XXVmSpL6N2oPYtXnfbSEKkSRNllkDoqrenmQJ8Iuqev0C1iRJmgAjb7VRVduAYxeoFknSBBnnKqYvJ3kz8CHg19ONXuYqSYvbfJ4o9/KhNi9zlaRFrtPLXJMcBrwRWAKcUVWntEzzPxjc66mAb1XV07Z3fZKk2844exA3S/LxqnrSmNMuAd7C4Mlzm4Hzk6xrHjg0Pc3ewInAwVX1H0n+03zqkSR1Z5znQQxbOY9pHwZcWlWXVdX1DB44dNSMaZ4DvKWq/gOgqq6aZz2SpI7MNyAumMe0K4ErhoY3c+uA2QfYJ8mXk5zXHJK6lSRrk2xIsuHqq6+eX8WSpO0yr4Coqj+/jde/FNgbOITB5bTvSLJHy3pPr6qpqppasWLFbVyCJKnNyIBIcniSLyT5WfP6fJIjxlz2FmD10PCqpm3YZmBdVd1QVT8AvscgMCRJPZs1IJI8B3gFgyuM7tu8/gE4OcnaMZZ9PrB3kr2S7AIcA6ybMc05DPYeSLIng0NOl83rJ5AkdWLUVUwvAh5VVT8favtMksOBLwGnj1pwVd2Y5HhgPYPLXN9VVRcleTmwoarWNeOemORiBk+rO6GqrtmBn0eSdBsZFRCZEQ4AVNU1ScZaeFWdC5w7o+2koc8FvLh5SZImyKhzEL9I8uCZjU3bL7srSZI0CUbtQbwEWJfk3cDGpm0KeAZwXNeFSZL6NeseRFV9icGX3e4APLN53QF4eDNOkrSIjbzVRlX9FDhp1DSSpMVpvt+kliTdThgQkqRWBoQkqdWct/tOsg9wAnDv4emrygcGSdIiNs7zID4CvA14B4NvO0uSbgfGCYgbq+q0ziuRJE2Ucc5BfCzJ85LcM8ndpl+dVyZJ6tU4exDPaN5PGGorBnd3lSQtUnMGRFXttRCFSJImy5yHmJLcOcnLkpzeDO+d5EndlyZJ6tM45yDeDVwPPLIZ3gK8srOKJEkTYZyAuF9VvQa4AaCqfgOM90AISdJOa5yAuD7JcgYnpklyP+B3nVYlSerdOFcx/T3wL8DqJB8EDmZw629J0iI2TkBsBP4UeDiDQ0svAHbrsihJUv/G+qIccENVfaKqPg6saNokSYvYOAHxfxl8m3rXJAcCH8VHjkrSojfOF+U+kWQZ8GkGh5aeXFXf67wySVKvZg2IJG+iuXKpsTvwfeD4JFTVX3ddnCSpP6P2IDbMGN7YZSGSpMkya0BU1XunPyfZBdinGdxUVTeMs/AkhwFvBJYAZ1TVKbNM998YnNt4aFXNDCZJUg/GeaLcIcB7gcsZXOa6OskzquoLc8y3BHgL8ARgM3B+knVVdfGM6XZjcOns17ajfklSR8a5ium1wBOr6jFV9WjgUOD1Y8z3MODSqrqsqq4HzgKOapnuFcA/Ar8ds2ZJ0gIYJyCWVdWm6YHmCqZlY8y3ErhiaHhz03azJA8BVlfVJ0YtKMnaJBuSbLj66qvHWLUkaUeNExAbk5yR5JDm9Q5ufQJ73pLcAXgd8JK5pq2q06tqqqqmVqxYsaOrliSNYZxbbTwXeD4wfVnrF4G3jjHfFmD10PCqpm3absADgM8lAfhDYF2SIz1RLUn9GxkQzYnmb1XVfgz+tz8f5wN7J9mLQTAcAzxtemRVXQfsObSuzwH/y3CQpMkw8hBTVW0DNiVZM98FV9WNwPHAeuAS4MNVdVGSlyc5cruqlSQtmHEOMd0VuCjJ14FfTzdW1Zwb+ao6Fzh3RttJs0x7yBi1SJIWyDgB8X86r0KSNHHGuVnf56c/J9kTuKaqasQskqRFYNZzEEkenuRzSc5OckCS7wDfAX7a3EJDkrSIjdqDeDPwtwzu4voZ4PCqOi/JfsCZDB5DKklapEZdxbS0qj5VVR8BflJV5wFU1XcXpjRJUp9GBcRNQ5+3zhjnOQhJWuRGHWJ6cJJfMLiD6/LmM83wnTqvTJLUq1HPg1iykIVIkibLODfrkyTdDhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlq1WlAJDksyaYklyZ5acv4Fye5OMm3k/xbknt3WY8kaXydBUSSJcBbgMOB+wPHJrn/jMkuAKaq6kHAR4HXdFWPJGl+utyDeBhwaVVdVlXXA2cBRw1PUFWfrarfNIPnAas6rEeSNA9dBsRK4Iqh4c1N22yeDXyyw3okSfOwtO8CAJIcB0wBj5ll/FpgLcCaNWsWsDJJuv3qcg9iC7B6aHhV03YLSR4P/B1wZFX9rm1BVXV6VU1V1dSKFSs6KVaSdEtdBsT5wN5J9kqyC3AMsG54giQHAG9nEA5XdViLJGmeOguIqroROB5YD1wCfLiqLkry8iRHNpOdCvwB8JEk30yybpbFSZIWWKfnIKrqXODcGW0nDX1+fJfrlyRtP79JLUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqZUBIkloZEJKkVgaEJKmVASFJamVASJJaGRCSpFYGhCSplQEhSWplQEiSWhkQkqRWBoQkqZUBIUlqtbTLhSc5DHgjsAQ4o6pOmTH+jsD7gAOBa4CnVtXlXdYk7czOuWALp67fxJXXbuVeeyznhEP35egDVgLw9Hd8lS9//+c3T3vw/e7GB5/ziLHmHTXuZedcyJlfu4JtVSxJOPag1bzy6AfucL3jjN/eZXdVc1f1TqpUVTcLTpYA3wOeAGwGzgeOraqLh6Z5HvCgqnpukmOAJ1fVU0ctd2pqqjZs2NBJzdIkO+eCLZx49oVsvWHbzW3Lly3h1X/6QD6y4Ue3CIdp0yExal5g1nEbfvhzPnDej2613OMevmbODe6odR59wMo5x2/vsruqGWbvpx2pd6FCIsnGqpqa1zwdBsQjgJOr6tBm+ESAqnr10DTrm2m+mmQp8BNgRY0oyoDQ7dXBp3yGLdduvVX7yj2Wt7ZPu/yUPxk5LzDruJ9c91u2tfw5Lkn4/quP2O56v/zSx845fnuX3VXNMHs/7Ui9c817W9megOjyENNK4Iqh4c3AQbNNU1U3JrkOuDvws+GJkqwF1gKsWbOmq3qliXblLCEwW/uOznvltVuZ7X9qbRvg+a6zq5+ny5rnO25HljsJdoqT1FV1elVNVdXUihUr+i5H6sW9mv/Fjts+7ryjxi1JWsfN1j7uOscZv73L7qrmruqdZF0GxBZg9dDwqqatdZrmENPuDE5WS5rhhEP3ZfmyJbdoW75sCSccui8H3+9urfNMt4+ad9S4Yw9aTZvZ2setd5zx27vsrmruqt5J1uUhpvOBvZPsxSAIjgGeNmOadcAzgK8CTwE+M+r8g3R7Nn0ys+1KmKMPWDnyKqZR806bbbnAdl0RNNc6x6lpe/uiq5q7qHeSdXaSGiDJEcAbGFzm+q6qelWSlwMbqmpdkjsB7wcOAH4OHFNVl41apiepJWn+Ju0kNVV1LnDujLaThj7/FvjvXdYgSdo+O8VJaknSwjMgJEmtDAhJUisDQpLUyoCQJLUyICRJrQwISVIrA0KS1MqAkCS1MiAkSa0MCElSq05v1teFJL8ENvVdxwx7MuMhRxNiEuuypvFY0/gmsa5JrGnfqtptPjN0erO+jmya7x0Ju5Zkw6TVBJNZlzWNx5rGN4l1TWpN853HQ0ySpFYGhCSp1c4YEKf3XUCLSawJJrMuaxqPNY1vEutaFDXtdCepJUkLY2fcg5AkLQADQpLUauIDIsmSJBck+XgzvFeSryW5NMmHkuwyIXW9J8kPknyzee2/wPVcnuTCZt0bmra7Jfl0kn9v3u86ATWdnGTLUD8dscA17ZHko0m+m+SSJI/ou59G1NVbXyXZd2i930zyiyQv7LOvRtTU9+/Ui5JclOQ7Sc5Mcqe+t1Oz1DTvbdTEn4NI8mJgCrhLVT0pyYeBs6vqrCRvA75VVadNQF3vAT5eVR9d6Fqaei4HpqrqZ0NtrwF+XlWnJHkpcNeq+t8913Qy8Kuq+qeFqmNGTe8FvlhVZzR/tHcG/pYe+2lEXS+kx74aqm0JsAU4CHg+PfdVS03Poqd+SrIS+BJw/6ra2myfzgWOoKft1IiaDmGe26iJ3oNIsgr4E+CMZjjAY4HpH/C9wNF91zXBjmLQR9BTX02SJLsDjwbeCVBV11fVtfTcTyPqmhSPA75fVT9kcn6nhmvq21JgeZKlDIL9x/S/nZpZ05Xbs5CJDgjgDcDfADc1w3cHrq2qG5vhzcDKCahr2quSfDvJ65PccYFrKuBTSTYmWdu03aOqftx8/glwjwmoCeD4pp/etcCHc/YCrgbencHhwTOS7Er//TRbXdBfXw07Bjiz+dx3X00brgl66qeq2gL8E/AjBsFwHbCRHrdTbTVV1aea0fPaRk1sQCR5EnBVVW3su5ZhI+o6EdgPeChwN2Chd7sfVVUPAQ4Hnp/k0cMja3AscaGPJ7bVdBpwP2B/Br+8r13AepYCDwFOq6oDgF8DLx2eoKd+mq2uPvsKgOZw15HAR2aO66mv2mrqrZ+aMDqKQcjfC9gVOGyh1j9uTUmOYzu2URMbEMDBwJHNceyzGOyyvRHYo9ltAljF4Dhkr3Ul+UBV/bgGfge8G3jYQhbV/K+BqroK+Odm/T9Nck+A5v2qvmuqqp9W1baqugl4BwvbT5uBzVX1tWb4oww2zL3202x19dxX0w4HvlFVP22G++6rW9XUcz89HvhBVV1dVTcAZzPYRvS5nWqr6ZHbs42a2ICoqhOralVV3YfB7uRnqurpwGeBpzSTPQP4fxNQ13FDfzRhcLzxOwtVU5Jdk+w2/Rl4YrP+dQz6CBa4r2arabqfGk9mAfupqn4CXJFk36bpccDF9NhPo+rqs6+GHMstD+X02leNW9TUcz/9CHh4kjs3f/vTv1N9bqfaarpku7ZRVTXxL35/9h3gvsDXgUsZ7GLecULq+gxwYdPpHwD+YAHruC/wreZ1EfB3TfvdgX8D/h34V+BuE1DT+5t++jaDjc09F/jfbH9gQ7P+c4C79tlPc9TVd1/tClwD7D7U1mtfzVJT3/30D8B3m7/99wN37Hs7NUtN895GTfxlrpKkfkzsISZJUr8MCElSKwNCktTKgJAktTIgJEmtls49iTQ5kkxfZgnwh8A2BrepgMEX8a7vpbAWSQ4Brq+qr3Sw7PsAlwCbqmr/pm0bg8sYpx0NrAbeDtxUVQ+4revQ4mZAaKdSVdcw+M5A73eGbWpYWr+/585MhwC/AsYOiDmWN9P3p8OhsXXGMMDlGdz++uPj1iBN8xCTdnpJDkzy+eamgOuHvjH6ueamZBsyeMbCQ5OcncGzDF7ZTHOfDJ7B8MFmmo8mufMYy31DBs+4eEGS/5rBvf8vSPKvSe7R/A//ucCLMrj3/n/J4H78Txmq+1fN+yFJvphkHYNvUC9JcmqS85sbq/3PBe1QqWFAaGcX4E3AU6rqQOBdwKuGxl9fVVPA2xjc7uD5wAOAZzaHqwD2Bd5aVX8E/AJ4XpJlcyx3l6qaqqrXMrj3/sNrcKO9s4C/qarLm3W+vqr2r6ovzvFzPAR4QVXtAzybwR04H8rgxmrPSbLXGH2xPL9/GMw/jzG9NJKHmLSzuyODDf6nB7eYYQmDO3pOW9e8XwhcVM2tqpNcxuD4/LXAFVX15Wa6DwB/DfzLHMv90NDnVcCHmj2MXYAfbMfP8fWqmp7vicCDhvY2dgf2HmO5bYeYpO1mQGhnFwYb/kfMMv53zftNQ5+nh6d//2feb6bGWO6vhz6/CXhdVa1rTkyfPMs8N9LstSe5A4MwaVtegL+qqvWzLEdaEB5i0s7ud8CKJI8ASLIsyX+e5zLWTM8PPI3BIaNN81ju7vz+ds7PGGr/JbDb0PDlwIHN5yOBZbMsbz3wl81hLpLsk98/QEhaMAaEdnY3Mbit8j8m+RbwTeCR81zGJgYPNLqEwV1UT2sulx13uScDH0myEfjZUPvHgCdPn6Rm8KyCxzTLewS33GsYdgaDW0Z/I8l3GFym6t6+Fpx3c9XtWnO10cd3tu8IzKfunfVnVP/cg5B2TtuA3ZN8c9REzZ7Lx7jlno00FvcgJEmt3IOQJLUyICRJrQwISVIrA0KS1MqAkCS1+v/YKMuQWSKjDwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import tensorflow as tf\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.image as imgplot\n",
"import numpy as np\n",
"import pandas as pd\n",
"import tempfile\n",
"data = np.asarray(pd.read_csv('./challenger.txt', sep=','), dtype='float32')\n",
"plt.plot(data[:,0], data[:,1], 'o')\n",
"plt.axis([40, 85, -0.1, 1.2])\n",
"plt.xlabel('Temperature [F]')\n",
"plt.ylabel('Broken O-rings')"
]
},
{
"cell_type": "code",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[0. 1. 0. 0. 0. 0. 0. 0. 1. 1. 1. 0. 0. 1. 0. 0. 0. 0. 0. 0. 1. 0. 1.]\n"
]
}
],
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Mathematical Notes\n",
"\n",
"We are considering the likelihood $P(y_i=1|x_i)$ for the class $y_i=1$ given the $i-$th data point $x_i$ ($x_i$ could be a vector). This is given by:\n",
"\n",
"$\n",
"P(y_i=1 | x_i) = \\frac{e^{(b + x_i w)}}{1 + e^{(b + x_i w)}} = [1 + e^{-(b + x_i w)}]^{-1}\n",
"$\n",
"\n",
"If we have more than one data point, which we usually do, we have to apply the equation above to each of the N data points. In this case we can use a vectorized version with $x=(x_1,x_2,\\ldots,x_N)$ and $y=(y_1,y_2,\\ldots,y_N$)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Numpy code\n",
"This numpy code, shows the calculation for one value using `NumPy` (like a single forward pass)"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3.882916\n",
"[0.999 0.998 0.998 0.998 0.999 0.996 0.996 0.998 1. 0.999 0.998 0.988\n",
" 0.999 1. 0.999 0.993 0.998 0.978 0.992 0.985 0.993 0.992 1. ]\n"
]
}
],
"source": [
"# Data\n",
"N = len(data)\n",
"x = data[:,0]\n",
"y = data[:,1]\n",
"# Initial Value for the weights\n",
"w = -0.20\n",
"b = 20.0\n",
"p_1 = 1 / (1 + np.exp(-x*w - b))\n",
"# cross-entropy loss function\n",
"cross_entropy = -np.mean(y * np.log(p_1) + (1-y) * np.log(1-p_1))\n",
"print(cross_entropy)\n",
"print(np.round(p_1,3))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Better values from intuition\n",
"\n",
"Now lets try to find better values for $W$ and $b$. Lets assume $W$ is given with $-1$. We want the probability\n",
"for a dammage $P(y_i=1 | x_i)$ to be $0.5$.\n",
"Determine an appropriate value for $b$.\n",
"Hint: at which $x$ value should $P(y_i=1 | x_i)$ be $0.5$, look at the data. At this $x$ value the term $1 + e^{-(b + W’ x_i)}$ must be $2$.\n",
"\n",
"**Solution**\n",
"\n",
"$P(y=1 | x) = 0.5$ at $x \\approx 65$ \n",
"\n",
"$-(b + (-1) x_i) = 0 \\rightarrow b = 65$ "
]
},
{
"cell_type": "code",
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.9094435\n",
"[0.269 0.007 0.018 0.047 0.119 0.001 0. 0.007 1. 0.881 0.007 0.\n",
" 0.119 1. 0.119 0. 0.007 0. 0. 0. 0. 0. 0.999]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAb/klEQVR4nO3de3hU9b3v8fc3kwQiIheJwYRE0CIIoQWbighbrbaVi/XS7nMqre1uty2nrezTy35s5dTH9tj21F09Z+vTWi3V1lZ3dat1W06lpT3VPm1R2URRERAFkUuQiyAIEkIu3/PHmtAhZC5JZmatrHxez8MzM2utmfXxJ3yy8ltrZszdERGR+CoJO4CIiBSWil5EJOZU9CIiMaeiFxGJORW9iEjMqehFRGIua9Gb2U/NbJeZvZRm/SfM7EUzW21mT5nZe/IfU0REeiuXI/p7gdkZ1m8CLnD3KcC3gcV5yCUiInlSmm0Dd/+zmY3NsP6plIfPAGPykEtERPIka9H30DXAb9OtNLMFwAKAIUOGvHfixIl53r2ISLw9++yzb7p7ZU+ek7eiN7P3ExT9rHTbuPtiklM7DQ0N3tjYmK/di4gMCGa2uafPyUvRm9m7gbuBOe6+Jx+vKSIi+dHnyyvNrA54FPiku7/S90giIpJPWY/ozewB4EJglJltA74JlAG4+13AjcDJwI/MDKDN3RsKFVhERHoml6tu5mdZ/1ngs3lLJCIieaV3xoqIxJyKXkQk5lT0IiIxp6IXEYk5Fb2ISMyp6EVEYk5FLyIScyp6EZGYU9GLiMScil5EJOZU9CIiMaeiFxGJORW9iEjMqehFRGJORS8iEnMqehGRmFPRi4jEnIpeRCTmVPQiIjGnohcRiTkVvYhIzKnoRURiTkUvIhJzKnoRkZhT0YuIxFxptg3M7KfApcAud6/vZr0BtwNzgUPAp939uXwHlfx5bFUTtyxbz/Z9zVQPr+C6SyZwxbSaUPcbVqbe5u2PbnhsNQ+s2Eq7Owkz5k+v5TtXTAk7VuzGOYqyFj1wL/BD4Bdp1s8Bxif/TAfuTN5KBD22qolFj66mubUdgKZ9zSx6dDVAQf9xZdovEEqmTMIap0K54bHV3P/MlqOP292PPg6z7OM2zlFl7p59I7OxwG/SHNH/GPiTuz+QfLweuNDd38j0mg1jyr3x2tG9Ci29d6Clle7+l5vB0EFlyQe5vFLKRmZ/W3bc/eB298FWWh06KMHd6MBwjJJEgjYSNLeX0EqCNkppI0GrJ6Csgr+bPBbKT4TyIcHtoBOD26GjYVgtDK+FwcN6Oxxpzbz5CZr2NR+3vGZ4Bcuvvyjv+yu0MxYtpb2b//EJMzZ+b24IiQJxG+diMLNn3b2hJ8/J5Yg+mxpga8rjbcllxxW9mS0AFgBMqjkJpn48D7uXnnh4+aa06/5x+jgg+w/+Y39SeMqyLvc7t/MO/vifWzCcEjooMT96P9HWQSntlNJOGe2U0kaZtTPEDjO47QBsexOOHIQj70Droe7zDBoGw8YEpT9iLIw7H8ZdEPxQ6KXt3ZRPpuVR113JZ1peLHEb56jKR9HnzN0XA4sBGhoanDk3F3P3AtzzQvojqH+cU7gjqB+sTb9fIP1R3ZdSMnW0B4XfcgAOvAH7tsD+rbBvK+zfFtzf9BdYcRckymHs38GZs+HMDwU/AHqgenhFt5mqk3n7m4RZ2iP6MMVtnKMqH1fdNAG1KY/HJJdJBF13yQQqyhLHLKsoS3DdJRNC22/OmUoSMPgkGFYDYxqg/iMw80sw71b4+IPwheXw9dfhU0vgnAWwbzP89jq4/T1wx3R44jvQcrDPefuj+dNre7S8WOI2zlGVjyP6JcBCM3uQ4CTs/mzz8xKezhNcxb7KIZf95iVTaTmcfkHw55Lvwp6N8MoyeHUZ/PlWeOlX8NG7oea9fc7bn3SecI3aVTdxG+eoynoy1sweAC4ERgE7gW8CZQDuflfy8sofArMJLq/8jLs3ZttxQ0ODNzZm3Uwkf15fDo8ugIM74MJFMOsrwW8JIv1Ib07G5nTVTSGo6CUUzfvg8a8GR/Z158FHfgzD68JOJZKz3hS93hkrA0vFcPjoPXDlYtixGu6cBasfCTuVSEGp6GXgMYP3fAy+8Fc4ZSL86hpY8t/p9g0GIjGgopeBa8RY+PRSmLEQnvs5rHk07EQiBaGil4EtUQofvAlOnQq/WxTM4YvEjIpepCQBH74N3tkNT3w77DQieaeiFwGongbn/DdYeQ9s09VgEi8qepFOF30Dhp4K//fL0N4WdhqRvFHRi3QaNBTm/AvsXA0r7gw7jUjeqOhFUp314eCD0J78X8GHponEgIpeJJUZzL0luL/0a7q2XmJBRS/S1fC64LNwXvktvPybsNOI9JmKXqQ7534BquqDo/qWA2GnEekTFb1IdxJlcOltwRec/ElfkCP9m4peJJ3a9wVfbrLqfl1uKf2ail4kk8lXwuF9sOWpsJOI9JqKXiSTMy6C0sHw8uNhJxHpNRW9SCblQ4Kyf/lxXWop/ZaKXiSbifNg/1Z444Wwk4j0iopeJJszZ4OVaPpG+i0VvUg2Q0ZB3QwVvfRbKnqRXEycB7vWwN7Xwk4i0mMqepFcTJwX3L68NNwcIr2gohfJxYixUDVF0zfSL6noRXI1cR5sfQYO7g47iUiP5FT0ZjbbzNab2QYzu76b9XVm9qSZrTKzF81sbv6jioRs4jzwDnjld2EnEemRrEVvZgngDmAOMAmYb2aTumx2A/CQu08DrgJ+lO+gIqEbPQWG1emji6XfyeWI/hxgg7u/5u5HgAeBy7ts48BJyfvDgO35iygSEWbBUf3GJ6HlYNhpRHKWS9HXAFtTHm9LLkv1LeBqM9sGLAX+qbsXMrMFZtZoZo27d2ueU/qhifOgvQU2/jHsJCI5y9fJ2PnAve4+BpgL3Gdmx722uy929wZ3b6isrMzTrkWKqG4GVIzQ1TfSr+RS9E1AbcrjMcllqa4BHgJw96eBwcCofAQUiZREKZw5Jzgh294adhqRnORS9CuB8WY2zszKCU62LumyzRbgYgAzO4ug6DU3I/E0cR4c3g+bl4edRCQnWYve3duAhcAyYB3B1TVrzOwmM7ssudk/A58zsxeAB4BPu+szXSWmzrgISis0fSP9RmkuG7n7UoKTrKnLbky5vxaYmd9oIhFVfgK86+Kg6Od8P7gaRyTC9M5Ykd6YOA/eboLtq8JOIpKVil6kNzo/o369PuRMok9FL9IbJ4yE0e+GrSvCTiKSlYpepLeqp8H2F/RdshJ5KnqR3qqeCi379WUkEnkqepHeOnVqcPvG82GmEMlKRS/SW6dMgkQ5bH8+7CQiGanoRXqrtByqJusSS4k8Fb1IX5w6Fd54USdkJdJU9CJ9UT1NJ2Ql8lT0In1RPTW41fSNRJiKXqQvKs+CxCBdeSORpqIX6YujJ2SfDzuJSFoqepG+qp4Kb7wAHR1hJxHplopepK+qp0HL2/DWprCTiHRLRS/SV53vkNUJWYkoFb1IX52SPCGropeIUtGL9FWiDEbXB/P0IhGkohfJh1OnBlfe6ISsRJCKXiQfqqfCkQN6h6xEkopeJB+qpwW3euOURJCKXiQfKifqhKxElopeJB8SZTB6it4hK5GkohfJF71DViJKRS+SL6dOTZ6Q3Rh2EpFj5FT0ZjbbzNab2QYzuz7NNv/VzNaa2Roz+2V+Y4r0A50nZDV9IxGTtejNLAHcAcwBJgHzzWxSl23GA4uAme4+Gfhy/qOKRFzlRCgdrCtvJHJyOaI/B9jg7q+5+xHgQeDyLtt8DrjD3d8CcPdd+Y0p0g8kSqGqXlfeSOTkUvQ1wNaUx9uSy1KdCZxpZsvN7Bkzm93dC5nZAjNrNLPG3bt39y6xSJRVTwu+Q1YnZCVC8nUythQYD1wIzAd+YmbDu27k7ovdvcHdGyorK/O0a5EIOfoOWZ2QlejIpeibgNqUx2OSy1JtA5a4e6u7bwJeISh+kYFFH1ksEZRL0a8ExpvZODMrB64ClnTZ5jGCo3nMbBTBVI4+9EMGns4TsrryRiIka9G7exuwEFgGrAMecvc1ZnaTmV2W3GwZsMfM1gJPAte5+55ChRaJrERp8h2yOqKX6CjNZSN3Xwos7bLsxpT7Dnw1+UdkYKueBs//MjghW6L3JEr49LdQJN9OnQpHDsKeDWEnEQFU9CL5Vz01uNX0jUSEil4k30ZNCE7I7ngx7CQigIpeJP8SpcHVNztfCjuJCKCiFymM0fWw4yVwDzuJiIpepCCqpsChN+HgzrCTiKjoRQpidH1wu0PTNxI+Fb1IIVRNDm53rg43hwgqepHCqBgBw2p1RC+RoKIXKZSqel15I5GgohcplNH18Oar0NocdhIZ4FT0IoVSVQ/eDrvWhZ1EBjgVvUihjJ4S3Gr6RkKmohcplBHjoGyITshK6FT0IoVSUhJcZqkjegmZil6kkPRRCBIBKnqRQqqqh5b9sH9r2ElkAFPRixRS5wlZzdNLiFT0IoV0yiTANE8voVLRixTSoBNh5DjYoc+8kfCo6EUKTR+FICFT0YsU2ugpsHcTtBwMO4kMUCp6kUKrqgccdq0NO4kMUCp6kUI7+iUkmqeXcORU9GY228zWm9kGM7s+w3YfNTM3s4b8RRTp54bVwuBhmqeX0GQtejNLAHcAc4BJwHwzm9TNdkOBLwEr8h1SpF8zC6ZvdC29hCSXI/pzgA3u/pq7HwEeBC7vZrtvA/8CHM5jPpF4qKqHnWugoyPsJDIA5VL0NUDq+7e3JZcdZWZnA7Xu/nimFzKzBWbWaGaNu3fv7nFYkX5rdD20vgNvbQo7iQxAfT4Za2YlwP8B/jnbtu6+2N0b3L2hsrKyr7sW6T+qkidkNU8vIcil6JuA2pTHY5LLOg0F6oE/mdnrwLnAEp2QFUlxyllgCV15I6HIpehXAuPNbJyZlQNXAUs6V7r7fncf5e5j3X0s8Axwmbs3FiSxSH9UVgGjxuuErIQia9G7exuwEFgGrAMecvc1ZnaTmV1W6IAisaGPQpCQlOaykbsvBZZ2WXZjmm0v7HsskRgaXQ8vPQLNb0HFiLDTyACid8aKFEtV55eFrwk3hww4KnqRYjn6UQiavpHiUtGLFMuJVXDCKNipK2+kuFT0IsVi9rcvCxcpIhW9SDFV1cOuddDeFnYSGUBU9CLFNHoKtLfAng1hJ5EBREUvUkzV04LbbSvDzSEDiopepJhGnRmckN28POwkMoCo6EWKyQzGzoTX/xp2EhlAVPQixXbaLNi/Fd7aHHYSGSBU9CLFNnZmcKvpGykSFb1IsVWeBRUjNX0jRaOiFym2khI47TwVvRSNil4kDGNnwb7NsG9r9m1F+khFLxKGsbOCW83TSxGo6EXCcMpkGDxc0zdSFCp6kTB0ztPriF6KQEUvEpbTZsLe1+Dt7WEnkZhT0YuEpXOe/nUd1UthqehFwjJ6CgwaBps1Ty+FpaIXCUtJAurO1RG9FJyKXiRMY2fBnlfhwI6wk0iMqehFwqTPvZEiUNGLhGn0e6B8qKZvpKByKnozm21m681sg5ld3836r5rZWjN70cz+aGan5T+qSAwlSqFuut44JQWVtejNLAHcAcwBJgHzzWxSl81WAQ3u/m7gEeD7+Q4qEltjZ8Gb6+Hg7rCTSEzlckR/DrDB3V9z9yPAg8DlqRu4+5Pufij58BlgTH5jisTYafrcGymsXIq+Bkj9iL1tyWXpXAP8ti+hRAaU6qlQNkTTN1Iwpfl8MTO7GmgALkizfgGwAKCuri6fuxbpvxJlwTy9juilQHI5om8CalMej0kuO4aZfQD4BnCZu7d090LuvtjdG9y9obKysjd5ReLptJmway28syfsJBJDuRT9SmC8mY0zs3LgKmBJ6gZmNg34MUHJ78p/TJGY0+fTSwFlLXp3bwMWAsuAdcBD7r7GzG4ys8uSm90CnAg8bGbPm9mSNC8nIt2pPhtKK1T0UhA5zdG7+1JgaZdlN6bc/0Cec4kMLKXlUHuO3jglBaF3xopExdhZsPMlfe6N5J2KXiQq6j8a3K68O9wcEjsqepGoOPkMmDgPVt4DRw5l314kRyp6kSiZsRCa98ILvww7icSIil4kSurODa7AefpH0NERdhqJCRW9SJSYwXkLYe9GeOV3YaeRmFDRi0TNWZfDsDp4+odhJ5GYUNGLRE2iFM79fPDmqabnwk4jMaCiF4miaZ+EQSfB03eEnURiQEUvEkWDT4KzPwVr/gP2bc2+vUgGKnqRqJr++eB2xV3h5pB+T0UvElXDa2HylfDcL+Dw22GnkX5MRS8SZTOuhZa3g7IX6SUVvUiU1ZwdfCnJirugvS3sNNJPqehFom7GQti/Fdb9Ouwk0k+p6EWi7szZcPK74C//Cq3NYaeRfkhFLxJ1JSVw8TeDz6r/909CW7dfySySlopepD+YdBl8+DbY8Ad4+DPQ3hp2IulHVPQi/cV7Pw1zboH1j8OvPquTs5KznL4zVkQiYvoCaG+B398ApYPgijuhJBF2Kok4Fb1If3PeP0HbYXjiO0HZX3p7MI8vkoaKXqQ/Ov+64KTsn2+BRDnMvTX4LHuRbqjoRfqr938jOLJ/6gdweD9c8HUYNT7sVBJBKnqR/soMPvjt4Ij+qR/A6odh/Ifg3C/A6e/XEb4cpYk9kf7MDC6+Eb6yBi5cBNtXwX1Xwo9mwLM/1xusBABz9+wbmc0GbgcSwN3ufnOX9YOAXwDvBfYAH3P31zO9ZkNDgzc2NvYytkj/9tiqJm5Ztp7t+5qpHl7BdZdM4IppNQB84idPs3zj3qPbzjxjJP/2uRk5PXdJ4yZeXHYPH2lZwqSSzbSUD2fQWXOgqp6fbTyBO9dWsMtPImHG/Om1fOeKKX3Om8v63r72DY+t5oEVW2l3z2vmQuUtBjN71t0bevScbEVvZgngFeCDwDZgJTDf3dembPNF4N3u/nkzuwq40t0/lul1VfQyUD22qolFj66mubX96LKKsgTf+8gUHm7cckzJd+os+0zPBVLWOeeWrOMzZX/g/MGvUdGy++j2u30Y6zrqWO+11NSdztz3TYITRkLFCKjovB0OibKsea+YVpN1fW/HonHzXu5/Zstxz7n63LqsZZ/7OOUvb7HKvlBFPwP4lrtfkny8CMDdv5eyzbLkNk+bWSmwA6j0DC+uopeBaubNT9C07/gplZrhFd0u7/T6zfMyPhdIu+7I/l28y7Zwlm1hom1hQslWJthWBluGd9haAkoHs7/VaPYyjngpLZTTSintGIlEKZNqRvBC0wEOt0O7l9BOCU5wbqCsrJQZp49Kniuwv93C0fMHf371TQ63th99TqfBZQlaWttxOG6dAZdMrjrmdbr60/rdx5Rxp4qy4D0H6dZdOKEy/Xhked1sz80Xu+rfelz0uZyMrQFSv8tsGzA93Tbu3mZm+4GTgTePCWi2AFgAUFdX15OcIrGxPU2Zp1ve1+du39eMM5TdPpmnmZyyxhlKM6u/1gDNb0HzXmjeB4f2wuF9wRU9bS38+i/rKaeVcmtjEEcop50SOki0dTCp/EQOth0iYR2UWjuDaMWS1WxtDocAD+qao8d9fvRmVNvbmHVzPNhG8mdCmmPFvYfSrwNGtx2A7n4GdL6ZON26PZm/4CXj62Z5bpiKetWNuy8GFkNwRF/MfYtERXWaI/d0y3N9LnR/RF89vIId+w/Tftwv2MYhGwIjxwHj0u7zx6vS/xax/FMX8bUMv2UsX3BRxv+ez2V4bveZIWHGxi/Ozfi61/TyN5/lX8ycN9PrZntu3lzb86upcrnqpgmoTXk8Jrms222SUzfDCE7KikgX110y4egUQqeKsgTXXTKBmWeM7PY5ncszPTfTuvnTa+lOuuW55s1lfW9fu1CZC5U3ynI5ol8JjDezcQSFfhXw8S7bLAH+AXga+HvgiUzz8yIDWedJu+6u3LhiWk3Gq24yPbdTutcFenUFS7Z95pKpt2NRqMyFyBtluV5eORe4jeDyyp+6+3fN7Cag0d2XmNlg4D5gGrAXuMrdX8v0mjoZKyLSc7256ianOXp3Xwos7bLsxpT7h4H/0pMdi4hIceidsSIiMaeiFxGJORW9iEjMqehFRGJORS8iEnMqehGRmFPRi4jEnIpeRCTmVPQiIjGnohcRiTkVvYhIzOX0oWYF2bHZAWB9KDtPbxRdviwlIqKYS5lyo0y5i2KuKGaa4O5De/KEon7xSBfre/oJbIVmZo1RywTRzKVMuVGm3EUxV1Qz9fQ5mroREYk5Fb2ISMyFWfSLQ9x3OlHMBNHMpUy5UabcRTFXLDKFdjJWRESKQ1M3IiIxp6IXEYm5ohW9mSXMbJWZ/Sb5eJyZrTCzDWb272ZWXqwsWXLda2abzOz55J+pRc7zupmtTu67MblspJn9wcxeTd6OiECmb5lZU8o4zS1ypuFm9oiZvWxm68xsRtjjlCFXaGNlZhNS9vu8mb1tZl8Oc6wyZAr779RXzGyNmb1kZg+Y2eCweypNph53VNHm6M3sq0ADcJK7X2pmDwGPuvuDZnYX8IK731mUMJlz3Qv8xt0fKXaWZJ7XgQZ3fzNl2feBve5+s5ldD4xw96+HnOlbwEF3v7VYObpk+jnwF3e/O/mP7wTgfxDiOGXI9WVCHKuUbAmgCZgOXEvIY9VNps8Q0jiZWQ3wV2CSuzcn+2kpMJeQeipDpgvpYUcV5YjezMYA84C7k48NuAjoDPpz4IpiZMmUK8IuJxgjCGmsosTMhgHnA/cAuPsRd99HyOOUIVdUXAxsdPfNROfvVGqmsJUCFWZWSvAD+g3C76mumbb35kWKNXVzG/A1oCP5+GRgn7u3JR9vA2qKlCVTrk7fNbMXzexfzWxQkTM58Hsze9bMFiSXVbn7G8n7O4CqCGQCWJgcp58WeZpkHLAb+JkF0253m9kQwh+ndLkgvLFKdRXwQPJ+2GPVKTUThDRO7t4E3ApsISj4/cCzhNhT3WVy998nV/eoowpe9GZ2KbDL3Z8t9L56IkOuRcBE4H3ASKDYv87OcvezgTnAtWZ2fupKD+bain1NbHeZ7gTOAKYS/CX830XMUwqcDdzp7tOAd4DrUzcIaZzS5QpzrABITiNdBjzcdV1IY9VdptDGKflD5XKCH9bVwBBgdrH2n2smM7uaXnRUMY7oZwKXJed5HyT4Veh2YHjy1xGAMQTzdMV0XC4zu9/d3/BAC/Az4Jxihkr+FMfddwH/kdz/TjM7FSB5uyvsTO6+093b3b0D+AnFHadtwDZ3X5F8/AhBwYY6TulyhTxWneYAz7n7zuTjsMfquEwhj9MHgE3uvtvdW4FHCToizJ7qLtN5vemoghe9uy9y9zHuPpbg17Qn3P0TwJPA3yc3+wfg14XOkkOuq1P+8hvBfNxLxcpkZkPMbGjnfeBDyf0vIRgjKPJYpcvUOU5JV1LEcXL3HcBWM5uQXHQxsJYQxylTrjDHKsV8jp0iCXWsko7JFPI4bQHONbMTkv/2O/9OhdlT3WVa16uOcvei/eFvZ4sBTgf+E9hA8KvboGJmyZDrCWB1cvDuB04sYo7TgReSf9YA30guPxn4I/Aq8P+AkRHIdF9ynF4kKI1Ti/z/bCrQmNz/Y8CIMMcpS66wx2oIsAcYlrIs1LFKkynscfqfwMvJf/v3AYPC7qk0mXrcUfoIBBGRmNM7Y0VEYk5FLyIScyp6EZGYU9GLiMScil5EJOZU9CIiMaeiFxGJuf8Pn37/6Rxd8c8AAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"w_val = -1\n",
"b_val = 65\n",
"plt.plot(data[:,0], data[:,1], 'o')\n",
"plt.axis([40, 85, -0.1, 1.2])\n",
"x_pred = np.linspace(40,85)\n",
"x_pred = np.resize(x_pred,[len(x_pred),1])\n",
"y_pred = 1 / (1 + np.exp(-x_pred*w_val - b_val))\n",
"plt.plot(x_pred, y_pred)\n",
"\n",
"p_1 = 1 / (1 + np.exp(-x*w_val - b_val))\n",
"\n",
"# cross-entropy loss function\n",
"cross_entropy = -np.mean(y * np.log(p_1) + (1-y) * np.log(1-p_1))\n",
" \n",
"print(cross_entropy)\n",
"print(np.round(p_1,3))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
"We can see that the value of the cross-entropy has decreased from 3.882916 to 0.9094435."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## TODO : determine the accuracy of this logistic regression model"
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Accuracy: 0.8695652173913043\n"
]
}
],
"source": [
"y_pred = np.round(p_1, decimals=0).astype('int')\n",
"accuracy = np.mean(y==y_pred)\n",
"print(\"Accuracy: \", accuracy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## TODO : set up a Keras model\n",
"\n",
"If there are two labels, we use `binary_crossentropy` as loss function. In this case, we use `softmax` as output layer."
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"l0 = tf.keras.layers.Dense(units=1, activation = tf.nn.sigmoid, input_shape=[1])\n",
"model = tf.keras.Sequential([l0])\n",
"model.compile(loss='binary_crossentropy', optimizer=tf.keras.optimizers.Adam(0.01), metrics=['accuracy'])\n",
"model.fit(x, y, epochs=10000, verbose=False)"
]
},
{
"cell_type": "code",
{
"name": "stdout",
"output_type": "stream",
"text": [
"WARNING:tensorflow:5 out of the last 9 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7ff9ec4d09e0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n"
]
},
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhYUlEQVR4nO3deZwU9Z3G8c+XmWEYYGA4JqjDIXigxBMmGiUq8QiCBo3xACUeUdm4ssaYNauJm5hsssnG3Im6QaJGUUAJInhrPDaeYRAVkaAcCgyRS0G5r+/+8WtkhDl6hu7+9dQ879erXt1VXd31WDLP1FRVV5m7IyIiydUqdgAREckuFb2ISMKp6EVEEk5FLyKScCp6EZGEU9GLiCRcg0VvZreb2XIze7OO1y8wszfMbJaZvWhmh2c+poiINFU6W/R3AqfW8/pC4AR3PxT4L2BMBnKJiEiGFDY0g7v/n5ntW8/rL9YYfRnonoFcIiKSIQ0WfSNdCjxa14tmNgoYBdCuXbsBBx10UIYXLyKSbDNmzFjp7uWNeU/Git7Mvkgo+i/UNY+7jyG1a6eystKrqqoytXgRkRbBzN5r7HsyUvRmdhgwFhji7qsy8ZkiIpIZe3x6pZn1BCYDX3P3t/c8koiIZFKDW/RmNh4YBHQ1syXAD4AiAHf/X+D7QBfgFjMD2OruldkKLCIijZPOWTcjGnj9MuCyjCUSEZGM0jdjRUQSTkUvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJp6IXEUk4Fb2ISMKp6EVEEk5FLyKScCp6EZGEU9GLiCScil5EJOFU9CIiCaeiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqnoRUQSrrChGczsduB0YLm7H1LL6wb8FhgKrAcudvdXMx1UMmfKzGpuenwuS1dvYJ+yEq4d3Jczj6yIutxYmZqatzm6Ycosxr+ymG3uFJgx4uge/PjMQ2PHStx6zkcNFj1wJ/AH4K46Xh8CHJAajgZuTT1KHpoys5rrJ89iw5ZtAFSv3sD1k2cBZPWHq77lAlEy1SfWesqWG6bMYtzLiz4Z3+b+yXjMsk/aes5X5u4Nz2S2L/BQHVv0fwSedffxqfG5wCB3/2d9n1l58L5eNe6HUFAErYqgVcHO5wWFUNA69bx1mF6w43lrKCyu8VgMrbQHKl0Df/Y01as37Da9oqyEF647McpygSiZ6hNrPWXLftc/wrZaftYLzJj/06EREgVJW8+5YGYz3L2yMe9JZ4u+IRXA4hrjS1LTdit6MxsFjAIYsHcrmHZVBhYPtCqEwjah+AtLoKhNjcc2UFSSGtpB67ZQ1BZatwvTWreD1qVQ3B6KS6F1eyjukBrvEOYxy0zOPLC0lh+q+qbHXG62M9Un1nrKltpKvr7puZK09ZyvMlH0aXP3McAYgMr+hzvfehi2bYHt22D7ltTzranHLbBtc3i+rebzTbB1Uxiv+bh1E2zdGIYtG2o83wjrVsDm9bAlNWxeD1vT/IfUqghKyqBNR2iTeiwpg5JO0LbL7kO7rtC2KxS2ztZq3CP7lJXUugW1T2rLOtZyY2SqT6z1lC0FZnVu0ceUtPWcrzJR9NVAjxrj3VPT6teqCDp2z8Dim2j79lTpr4PNa2HTx2HYvBY2rYVNH4Vh4xrYsDo8blwdhg/fhQ0fhoE6tohKOkPpXtC+W+rxM9B+L+hYAR17QFnP8Ishxz9o1w7u+6l9ogAlRQVcO7hv1OXGyFSfWOspW0Yc3eNT++hrTo8paes5X2Wi6KcCo81sAuEg7JqG9s/nhVatUrtn2gPdmvYZ27eFsl+/auewbiWsXQ5rl4Xh4/dh1bzwuH3Lp99fWBJ+2ZX1COXfuQ902R+67AedeoddTxm24wBXrs9ySGe5+XTmRaz1lC07Drjm21k3SVvP+arBg7FmNh4YBHQFlgE/AIoA3P1/U6dX/gE4lXB65SXuXtXQgisrK72qqsHZksMd1n8AHy2B1YthzRJYsxhWLwrPV78XflF8wkL5d+kDXQ6Abv2g2yHwmYPDsQQRaZGycjDW3Uc08LoDVzZmoS2SGbTrEoa9D699ng2r4YP5sGpB6nEerJoPr0+AzR/vnK+sVyj9bv1gr0OhojLsEhIRqUVOD8ZKA0rKoGJAGGpyD1v/y2bDsjdh2Vvh+duPgm8P85TuA90rU8PnYO8jwhlGItLiqeibA7Nw8LasJ/QdsnP6lo2h+JdUQXUVLJkOc6am3lMQ/nLocwL0GQQ9jg6niopIi5PWF6ayocXto8+VtSt2lv67L4Tn27eGL5b1PDqUfu9BsM8R4UtqItKsNGUfvYo+6TZ9DO+9CAueg4XPhb8AIJzrf9BQOHgY9D4+fNlMRPJerG/GSj4rLoUDB4cBwhb/wudg7iPw5gPw6l3hm8EHDoaDT4f9T0mdcioiSaGib2nal8OhZ4dh66awpT9naqr4J4VdPAcNhSNHQp8vaveOSAKo6FuywmI48Eth2L4NFr0Mb02BWZNg9gPQoQIOHwFHnB++xCUizZL20cvutm6CuY/CzHEw/6/hFM5eA8NW/mfPyso3dkUkPToYK5n30VJ4fXwo/Q8WQLtyOOpf4HOXQtvOsdOJtDgqeske93AQ98U/wLwnw6WejxwJn/9X6Nw7djqRFkNn3Uj2mIVz8PsMCt/MfekPUHUHTB8bTtEceNXu3+gVkbygWzNJ43XrB2feAlfPgmOvgvnPwG0nwoQLYMXc2OlEZBcqemm6DnvDKT+Ea2bDiTfAwv+DWz4PD46GNQ3fkkBEckNFL3uuuBSOvxaueg2OvgLemAi/7w9Pfj91cxYRiUlFL5nTrguc+t8wugr6nQkv/A5+ezi8+PtwG0gRiUJFL5nXqRec9Uf4xvPQ/Sh44gYYMyhcZVNEck5FL9mz1yEwchKcd0+4u9bYk+GRa2HjR7GTibQoKnrJvoNPhytfgaMuh7/fBjcfDXMeip1KpMVQ0UtutOkAQ2+Cy56Ckk4w8YJwOuZHS2MnE0k8Fb3kVvdK+Jfn4OQbYd5TcMsxMGda7FQiiaail9wrKIIvfAuueBE67QsTR8JD18CWDbGTiSSSil7i6bIfXPokHDMaqv4Et50Ey/8RO5VI4qjoJa7C1jD4J3DBJFi7LJyGOePOcBE1EckIFb3khwNOgSteCDcwn/ZNuP9i2LA6diqRREir6M3sVDOba2bzzOy6Wl7vaWbPmNlMM3vDzIZmPqokXuleMPIBOOkH4QDt2JNg5bzYqUSavQaL3swKgJuBIUA/YISZ9dtlthuA+9z9SGA4cEumg0oL0aoVHHcNXPxQuE7O2BNhwbOxU4k0a+ls0R8FzHP3Be6+GZgAnLHLPA50SD3vCOjkaNkzvY6Fy5+G0n3g7rNg+p9iJxJpttIp+gpgcY3xJalpNd0IjDSzJcAjwL/V9kFmNsrMqsysasWKFU2IKy1Kp33h0idg/5Pg4Wvgke/Atq2xU4k0O5k6GDsCuNPduwNDgbvNbLfPdvcx7l7p7pXl5eUZWrQkWpsOMGJCOAXz73+Ee8/RQVqRRkqn6KuBHjXGu6em1XQpcB+Au78EtAG6ZiKgCK0KwimYX/5duLnJ2JPhg4WxU4k0G+kU/XTgADPrbWatCQdbp+4yzyLgJAAzO5hQ9No3I5k14CL42hRYvxJuPxWWz4mdSKRZaLDo3X0rMBp4HJhDOLtmtpn9yMyGpWb7NnC5mb0OjAcudtc3XiQLeh8HFz8CONwxFKpfjZ1IJO9ZrD6urKz0qirdiEKa6IMFcNcZsP5DOH8i7DswdiKRnDCzGe5e2Zj36Jux0jx17gOXPBZuUD7uLHjnydiJRPKWil6ar44VcMmj0PVAGD8CZj8QO5FIXlLRS/PWrmv4Fm33Spj0dXj17tiJRPKOil6avzYdYeRk6DMIpo4OV78UkU+o6CUZWrcNX6w64Esw7Wp4fWLsRCJ5Q0UvyVFYDOfeFU7BnHIFvLXr1z1EWiYVvSRLUQkMHw8VA8I++7efiJ1IJDoVvSRPcXu44H7o1g/u+1q4bIJIC6ail2QqKQs3MenUG+4dDoteiZ1IJBoVvSRXuy5w4RQo7Qb3nA1LX4udSCQKFb0kW+lecOHUcArm3V+Ble/ETiSScyp6Sb6yHnDhg2CtYNxXYe3y2IlEckpFLy1Dl/3g/PtCyd97LmxeFzuRSM6o6KXl6D4AzrkD/vl6OPVStyWUFkJFLy1L3yEw9CZ4+zF49FrQbROkBSiMHUAk5z53GaxeDC/8Bjr2gOOuiZ1IJKtU9NIynfQDWLME/vrDUPaHnRM7kUjWqOilZWrVCs68BdYuC9fFKe0GvY+PnUokK7SPXlquwmI4b1w4I2fCSFg5L3YikaxQ0UvLVlIWTrssKITxw2HD6tiJRDJORS/SqVfYsv9wIfzlUti+LXYikYxS0YsA9DoWTvslzHsKnvx+7DQiGaWDsSI7DLgYlr0FL/0Bun0Wjjg/diKRjNAWvUhNg/8bep8A076pSxtLYqjoRWoqKIRz7oSO3WHiyHCuvUgzl1bRm9mpZjbXzOaZ2XV1zHOumb1lZrPN7N7MxhTJobadw43Gt2yACefD5vWxE4nskQaL3swKgJuBIUA/YISZ9dtlngOA64GB7v5Z4OrMRxXJofK+cPbt8M834MErdU0cadbS2aI/Cpjn7gvcfTMwAThjl3kuB2529w8B3F0X/Jbm78Avwck/gNmT4aWbY6cRabJ0ir4CWFxjfElqWk0HAgea2Qtm9rKZnVrbB5nZKDOrMrOqFStWNC2xSC4NvBoOHhZOuVz4t9hpRJokUwdjC4EDgEHACOA2MyvbdSZ3H+Pule5eWV5enqFFi2SRWbgmTpf94P6LYU117EQijZZO0VcDPWqMd09Nq2kJMNXdt7j7QuBtQvGLNH/FpXDePbB1E9z3tfAo0oykU/TTgQPMrLeZtQaGA1N3mWcKYWseM+tK2JWzIHMxRSIrPzBs2VfPgEf/I3YakUZpsOjdfSswGngcmAPc5+6zzexHZjYsNdvjwCozewt4BrjW3VdlK7RIFP2GwRe+BTPugFfvjp1GJG3mkU4bq6ys9KqqqijLFmmy7dtg3Fnw3kvw9cegon/sRNLCmNkMd69szHv0zViRxmhVAF+9Hdp/Bu67ENbpD1fJfyp6kcZq1wXOvQvWLtdljaVZUNGLNEVFfxh6Eyx4Bp77n9hpROqlohdpqv4XwhEXwHM/h3eeip1GpE4qepGmMoOhvwjXrp98GaxeFDuRSK1U9CJ7onXbsL9++za47yJ9mUrykopeZE912Q/OuBmWvgqPfzd2GpHdqOhFMqHfMDhmNEwfC2/cHzuNyKeo6EUy5eQboecxMO0qWD4ndhqRT6joRTKloAjOvgNat4eJX4NNH8dOJAKo6EUyq8Pe4c5UH8yHqf+mO1NJXlDRi2Ra7+PgxP+E2Q/A38fETiOiohfJioFXw4FD4PHvweLpsdNIC6eiF8mGVq3gK7dCh33g/otg3crYiaQFU9GLZEtJp/BlqnUr4S+X6eJnEo2KXiSb9jkChv5cFz+TqFT0ItnW/yI4/Hxd/EyiUdGLZJsZnPZLXfxMolHRi+SCLn4mEanoRXKl5sXPHrsudhppQVT0IrnUbxgM/CZU3Q4zx8VOIy2Eil4k1078PvQ+AR66BpbOjJ1GWgAVvUiuFRSG6+G0K4eJF8K6VbETScKp6EViaNcVzrsb1i6Dv3xdX6aSrEqr6M3sVDOba2bzzKzOo0hm9lUzczOrzFxEkYSq6A+n/QIWPAtP/zh2GkmwBovezAqAm4EhQD9ghJn1q2W+UuCbwCuZDimSWP0vhAEXw/O/gjnTYqeRhEpni/4oYJ67L3D3zcAE4Ixa5vsv4H+AjRnMJ5J8Q34OFQPggStgxdux00gCpVP0FcDiGuNLUtM+YWb9gR7u/nB9H2Rmo8ysysyqVqxY0eiwIolUWBy+TFVYDBMvgI0fxU4kCbPHB2PNrBXwK+DbDc3r7mPcvdLdK8vLy/d00SLJ0bE7nHMnrJoPky/XwVnJqHSKvhroUWO8e2raDqXAIcCzZvYu8Hlgqg7IijRS7+NgyP/A24/BX38UO40kSGEa80wHDjCz3oSCHw6cv+NFd18DdN0xbmbPAv/u7lWZjSrSAnzuMlg2G174TbgI2mHnxk4kCdDgFr27bwVGA48Dc4D73H22mf3IzIZlO6BIi2IGQ2+CXl+AB0fDkhmxE0kCmEe6S31lZaVXVWmjX6RW61bBbYNg62YY9Sx02Dt2IskTZjbD3Ru1a1zfjBXJR+26wIgJsHktTDgftmyInUiaMRW9SL7q9lk4a0y48NnUf4NIf31L86eiF8lnB50GJ94As+6H538dO400U+mcdSMiMR33bVj+Fvz1h9BpXzjkrNiJpJlR0YvkOzM44xb4aCk88C9Quhf0OjZ2KmlGtOtGpDkoagPD74WyXjB+BKyYGzuRNCMqepHmom1nGDkJClrDuLPh42WxE0kzoaIXaU467QvnT4T1K+Hec2DT2tiJpBlQ0Ys0NxX9wwXQ3p8Fky6BbVtjJ5I8p6IXaY4OHAyn/QreeQIevkbn2Eu9dNaNSHNVeQmsWQx/+2W4zPEJ34mdSPKUil6kOTvxP2FNNTzzEyguhc9fETuR5CEVvUhzZgZn3ByuifPYdVDUFgZcFDuV5Bntoxdp7goK4ezbYf9TYNo34Y37YieSPKOiF0mCwmI4727Y9wvwwDdgzrTYiSSPqOhFkqKoJFzauGIA3H8JvPNk7ESSJ1T0IklS3B4uuB+69YOJI2Hh32InkjygohdJmpIyGPkAdOoN954Hi16JnUgiU9GLJFG7LnDhlHCly7u/AguejZ1IIlLRiyRV6V5wyaPQqRfccw784+HYiSQSFb1IkpV2g4sfhr0OhYlf06mXLZSKXiTp2naGCx8MNyuZPAqmj42dSHJMRS/SEhSXhrNxDhwMD38b/var2Ikkh1T0Ii1FUQmcNw4O+Wq4/+xTN+qqly1EWkVvZqea2Vwzm2dm19Xy+jVm9paZvWFmfzWzXpmPKiJ7rKAIzroNBlwMz/8aHhwNWzfHTiVZ1mDRm1kBcDMwBOgHjDCzfrvMNhOodPfDgEnAzzMdVEQypFUBnP4bOOE/4LVxcNcZsG5l7FSSRels0R8FzHP3Be6+GZgAnFFzBnd/xt3Xp0ZfBrpnNqaIZJQZfPG78NU/QfUMuO1EWD4ndirJknSKvgJYXGN8SWpaXS4FHt2TUCKSI4eeDZc8Als3wthTdH2chMrowVgzGwlUAjfV8fooM6sys6oVK1ZkctEi0lTdK+Hyp6HzvnDvufDSzTpImzDpFH010KPGePfUtE8xs5OB7wHD3H1TbR/k7mPcvdLdK8vLy5uSV0SyoWN3+PrjcNBp8Ph3YdpVsLXWH2NphtIp+unAAWbW28xaA8OBqTVnMLMjgT8SSn555mOKSNa1bgfn3AXH/Tu8ehf86RRYNT92KsmABove3bcCo4HHgTnAfe4+28x+ZGbDUrPdBLQH7jez18xsah0fJyL5rFUrOOk/Yfi9sHoR/PF4eH1C7FSyh8wj7YurrKz0qqqqKMsWkTSsqYbJl8N7L8Bh58FpvwzfsJWozGyGu1c25j36ZqyI1K5jBVw0DQZdD7PuD1v3S2fGTiVNoKIXkbq1KoBB14UrYG7dFE7BfPH3sH1b7GTSCCp6EWlYr2PhG8+Hi6I9cQP86Uvw/puxU0maVPQikp62ncNF0c66DT5cCGNOgCd/AJvXN/xeiUpFLyLpM4PDzoXRVXDYcHjhN3DrMTD/6djJpB4qehFpvLad4cybw8FaKwj3pZ08ShdHy1MqehFput7HwxUvwvHfgTcnw+/6h5uaaHdOXlHRi8ieKWoDJ34PrngBeh0Tbmry+/4w48+wbWvsdIKKXkQypbwvnD8RLn4kXDtn2lVh//2cabpIWmQqehHJrH0HwqVPwnn3hPGJI8N1cxY8p8KPREUvIplnBgefDle8BMN+Hy6ncNcwGDMIZk3SLp0cU9GLSPYUFEL/C+GqmfDl38LmdfCXS+F3R8LLt8Kmj2MnbBFU9CKSfUVtwg3Jr/w7jJgQ9uE/dh38+rPw1I1hi1+yRlevFJE4llTBi78LB2sB+nwRjhwZbn5SWBw3Wx5rytUrC7MVRkSkXt0r4dy74MN34bV7YeY9MOkSKOkEh54TSn/vw2OnTARt0YtIfti+DRY+BzPHwZyHYNsm6HYoHHIWHDwMuu4fO2FeaMoWvYpeRPLPhg/D2Tmv3QtLXw3Tyg+Cg78MB50etvTN4maMREUvIsmzejH842H4x0Phble+HTr2DPvy9z8Jeh4Dxe1jp8wZFb2IJNu6VfD2o2HXzvynw+6dVkXQ/XPQ5wToMwgqBkBBUeykWaOiF5GWY8sGWPRy2K+/4FlY+hrg0Lp92MrvcVQ44LtPfygpi5s1g3TWjYi0HEUlsN8XwwCw/gN49/lQ/O8+D/OeAlIbsl37htLfUfzlB4Vz+1sIFb2IJEPbztBvWBgANq6B6lehuiqcs//2Y/Ba6vo7VgBd9odun4Vu/aDbIeF5xx6JPMiroheRZGrT8dNb/O7hnP2lM2H5W7BsNlTPgNmTd76nqB106QOd9wu/CLqkHjvvF36RNNNfAip6EWkZzKBz7zBw1s7pGz+C5XNg2Zuw8h34YD68/0bq8srbds7XujRcuqGsR3js2CMMZT2gdC9ov1fe7g5S0YtIy9amA/Q8Ogw1bdsCH74Xin/VPFi9KJzquWYxLJkezvXf7bM6hsIv7QbtU0O7rtC2K7TtknreJfx10KYsZ38hpFX0ZnYq8FugABjr7j/b5fVi4C5gALAKOM/d381sVJHkmDKzmpsen8vS1RvYp6yEawf35cwjKwC44LaXeGH+B5/MO3C/ztxz+TFpvbe+126YMovxryxmmzsFZow4ugc/PvPQPc6bzutN/exsZU4rb0FR+DZu1/2Bwbt97uqNH3JEh7Vc2b+YYz+zBda+Dx8v2/m4+BVYuxy2bqw9oBWEXww7hpKy1POy8MundSkUl4bvCBSX7hxvggZPrzSzAuBt4BRgCTAdGOHub9WY51+Bw9z9G2Y2HPiKu59X3+fq9EppqabMrOb6ybPYsGXnboGSogJ+etah3F+16FMlv8OOsq/vvUCdr1W99wHjXl602+eO/HzPBouzvmWeeWRFg6839bOzlRnqXk97krfW97rDlvXhpunrV4Uzg9avgvWp8Y1rwrBhder56p3T6vgFYT/8KPPn0ZvZMcCN7j44NX59yO8/rTHP46l5XjKzQuB9oNzr+XAVvbRUA3/2NNWrN+w2vaKspNbpO7z7s9PqfS9Q52vvr9nItlp+HAvMmP/ToU3O+8J1Jzb4elM/O1uZoe71tCd5G3pvo23bEq7Xv3ktbFqbev4xdsApWTmPvgJYXGN8CXB0XfO4+1YzWwN0AVbWnMnMRgGjAHr27NmYnCKJsbSOMq9r+p6+d+nqDdS1xVVbkTZ2mdn678lm5sa+tief22QFRWFfftvOe/xROb3xiLuPcfdKd68sLy/P5aJF8sY+qa3KdKen+976Xiuo46BfXdPTXWY6rzf1s7OVOVt581k6RV8N9Kgx3j01rdZ5UrtuOhIOyorILq4d3JeSooJPTSspKuDawX0ZuF/tW287ptf33vpeG3F0D2pT1/R086bzelM/O1uZs5U3n6Wz62Y6cICZ9SYU+nDg/F3mmQpcBLwEnA08Xd/+eZGWbMdBu9rO+jjzyIp6z7qp77071PW5QJPOYGlomelkauq6yFbmbOTNZ2ld1MzMhgK/IZxeebu7/8TMfgRUuftUM2sD3A0cCXwADHf3BfV9pg7Giog0XtYuaubujwCP7DLt+zWebwTOacyCRUQkN3J6MFZERHJPRS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqnoRUQSTkUvIpJwKnoRkYRL66JmWVmw2cfA3CgLr1tXdrlZSp7Ix1zKlB5lSl8+5srHTH3dvVE3j03romZZMrexV2DLNjOryrdMkJ+5lCk9ypS+fMyVr5ka+x7tuhERSTgVvYhIwsUs+jERl12XfMwE+ZlLmdKjTOnLx1yJyBTtYKyIiOSGdt2IiCScil5EJOFyVvRmVmBmM83sodR4bzN7xczmmdlEM2udqywN5LrTzBaa2Wup4Ygc53nXzGalll2VmtbZzJ40s3dSj53yINONZlZdYz0NzXGmMjObZGb/MLM5ZnZM7PVUT65o68rM+tZY7mtm9pGZXR1zXdWTKfa/qW+Z2Wwze9PMxptZm9g9VUemRndUzvbRm9k1QCXQwd1PN7P7gMnuPsHM/hd43d1vzUmY+nPdCTzk7pNynSWV512g0t1X1pj2c+ADd/+ZmV0HdHL3/4ic6UZgrbv/Ilc5dsn0Z+Bv7j429cPXFvguEddTPbmuJuK6qpGtAKgGjgauJPK6qiXTJURaT2ZWATwP9HP3Dal+egQYSqSeqifTIBrZUTnZojez7sBpwNjUuAEnAjuC/hk4MxdZ6suVx84grCOItK7yiZl1BI4H/gTg7pvdfTWR11M9ufLFScB8d3+P/Pk3VTNTbIVAiZkVEn5B/5P4PbVrpqVN+ZBc7br5DfAdYHtqvAuw2t23psaXABU5ylJfrh1+YmZvmNmvzaw4x5kceMLMZpjZqNS0bu7+z9Tz94FueZAJYHRqPd2e490kvYEVwB0WdruNNbN2xF9PdeWCeOuqpuHA+NTz2Otqh5qZINJ6cvdq4BfAIkLBrwFmELGnasvk7k+kXm5UR2W96M3sdGC5u8/I9rIao55c1wMHAZ8DOgO5/nP2C+7eHxgCXGlmx9d80cO+tlyfE1tbpluB/YAjCP8If5nDPIVAf+BWdz8SWAdcV3OGSOuprlwx1xUAqd1Iw4D7d30t0rqqLVO09ZT6pXIG4Zf1PkA74NRcLT/dTGY2kiZ0VC626AcCw1L7eScQ/hT6LVCW+nMEoDthP10u7ZbLzMa5+z892ATcARyVy1Cp3+K4+3LggdTyl5nZ3gCpx+WxM7n7Mnff5u7bgdvI7XpaAixx91dS45MIBRt1PdWVK/K62mEI8Kq7L0uNx15Xu2WKvJ5OBha6+wp33wJMJnREzJ6qLdOxTemorBe9u1/v7t3dfV/Cn2lPu/sFwDPA2anZLgIezHaWNHKNrPGP3wj7497MVSYza2dmpTueA19KLX8qYR1BjtdVXZl2rKeUr5DD9eTu7wOLzaxvatJJwFtEXE/15Yq5rmoYwad3kURdVymfyhR5PS0CPm9mbVM/+zv+TcXsqdoyzWlSR7l7zgZ2Hi0G6AP8HZhH+NOtOJdZ6sn1NDArtfLGAe1zmKMP8HpqmA18LzW9C/BX4B3gKaBzHmS6O7We3iCUxt45/n92BFCVWv4UoFPM9dRArtjrqh2wCuhYY1rUdVVHptjr6YfAP1I/+3cDxbF7qo5Mje4oXQJBRCTh9M1YEZGEU9GLiCScil5EJOFU9CIiCaeiFxFJOBW9iEjCqehFRBLu/wHKEWw4E3VywwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.plot(data[:,0], data[:,1], 'o')\n",
"plt.axis([40, 85, -0.1, 1.2])\n",
"x_pred = np.linspace(40,85)\n",
"x_pred = np.resize(x_pred,[len(x_pred),1])\n",
"y_pred = model.predict(x_pred)\n",
"plt.plot(x_pred, y_pred)"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[array([[-0.23215847]], dtype=float32), array([15.042496], dtype=float32)]\n"
]
}
],
"source": [
"print(model.get_weights())"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cross-entropy: 0.4416346\n",
"Accuracy: 0.8695652173913043\n"
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhYUlEQVR4nO3deZwU9Z3G8c+XmWEYYGA4JqjDIXigxBMmGiUq8QiCBo3xACUeUdm4ssaYNauJm5hsssnG3Im6QaJGUUAJInhrPDaeYRAVkaAcCgyRS0G5r+/+8WtkhDl6hu7+9dQ879erXt1VXd31WDLP1FRVV5m7IyIiydUqdgAREckuFb2ISMKp6EVEEk5FLyKScCp6EZGEU9GLiCRcg0VvZreb2XIze7OO1y8wszfMbJaZvWhmh2c+poiINFU6W/R3AqfW8/pC4AR3PxT4L2BMBnKJiEiGFDY0g7v/n5ntW8/rL9YYfRnonoFcIiKSIQ0WfSNdCjxa14tmNgoYBdCuXbsBBx10UIYXLyKSbDNmzFjp7uWNeU/Git7Mvkgo+i/UNY+7jyG1a6eystKrqqoytXgRkRbBzN5r7HsyUvRmdhgwFhji7qsy8ZkiIpIZe3x6pZn1BCYDX3P3t/c8koiIZFKDW/RmNh4YBHQ1syXAD4AiAHf/X+D7QBfgFjMD2OruldkKLCIijZPOWTcjGnj9MuCyjCUSEZGM0jdjRUQSTkUvIpJwKnoRkYRT0YuIJJyKXkQk4VT0IiIJp6IXEUk4Fb2ISMKp6EVEEk5FLyKScCp6EZGEU9GLiCScil5EJOFU9CIiCaeiFxFJOBW9iEjCqehFRBJORS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqnoRUQSrrChGczsduB0YLm7H1LL6wb8FhgKrAcudvdXMx1UMmfKzGpuenwuS1dvYJ+yEq4d3Jczj6yIutxYmZqatzm6Ycosxr+ymG3uFJgx4uge/PjMQ2PHStx6zkcNFj1wJ/AH4K46Xh8CHJAajgZuTT1KHpoys5rrJ89iw5ZtAFSv3sD1k2cBZPWHq77lAlEy1SfWesqWG6bMYtzLiz4Z3+b+yXjMsk/aes5X5u4Nz2S2L/BQHVv0fwSedffxqfG5wCB3/2d9n1l58L5eNe6HUFAErYqgVcHO5wWFUNA69bx1mF6w43lrKCyu8VgMrbQHKl0Df/Y01as37Da9oqyEF647McpygSiZ6hNrPWXLftc/wrZaftYLzJj/06EREgVJW8+5YGYz3L2yMe9JZ4u+IRXA4hrjS1LTdit6MxsFjAIYsHcrmHZVBhYPtCqEwjah+AtLoKhNjcc2UFSSGtpB67ZQ1BZatwvTWreD1qVQ3B6KS6F1eyjukBrvEOYxy0zOPLC0lh+q+qbHXG62M9Un1nrKltpKvr7puZK09ZyvMlH0aXP3McAYgMr+hzvfehi2bYHt22D7ltTzranHLbBtc3i+rebzTbB1Uxiv+bh1E2zdGIYtG2o83wjrVsDm9bAlNWxeD1vT/IfUqghKyqBNR2iTeiwpg5JO0LbL7kO7rtC2KxS2ztZq3CP7lJXUugW1T2rLOtZyY2SqT6z1lC0FZnVu0ceUtPWcrzJR9NVAjxrj3VPT6teqCDp2z8Dim2j79lTpr4PNa2HTx2HYvBY2rYVNH4Vh4xrYsDo8blwdhg/fhQ0fhoE6tohKOkPpXtC+W+rxM9B+L+hYAR17QFnP8Ishxz9o1w7u+6l9ogAlRQVcO7hv1OXGyFSfWOspW0Yc3eNT++hrTo8paes5X2Wi6KcCo81sAuEg7JqG9s/nhVatUrtn2gPdmvYZ27eFsl+/auewbiWsXQ5rl4Xh4/dh1bzwuH3Lp99fWBJ+2ZX1COXfuQ902R+67AedeoddTxm24wBXrs9ySGe5+XTmRaz1lC07Drjm21k3SVvP+arBg7FmNh4YBHQFlgE/AIoA3P1/U6dX/gE4lXB65SXuXtXQgisrK72qqsHZksMd1n8AHy2B1YthzRJYsxhWLwrPV78XflF8wkL5d+kDXQ6Abv2g2yHwmYPDsQQRaZGycjDW3Uc08LoDVzZmoS2SGbTrEoa9D699ng2r4YP5sGpB6nEerJoPr0+AzR/vnK+sVyj9bv1gr0OhojLsEhIRqUVOD8ZKA0rKoGJAGGpyD1v/y2bDsjdh2Vvh+duPgm8P85TuA90rU8PnYO8jwhlGItLiqeibA7Nw8LasJ/QdsnP6lo2h+JdUQXUVLJkOc6am3lMQ/nLocwL0GQQ9jg6niopIi5PWF6ayocXto8+VtSt2lv67L4Tn27eGL5b1PDqUfu9BsM8R4UtqItKsNGUfvYo+6TZ9DO+9CAueg4XPhb8AIJzrf9BQOHgY9D4+fNlMRPJerG/GSj4rLoUDB4cBwhb/wudg7iPw5gPw6l3hm8EHDoaDT4f9T0mdcioiSaGib2nal8OhZ4dh66awpT9naqr4J4VdPAcNhSNHQp8vaveOSAKo6FuywmI48Eth2L4NFr0Mb02BWZNg9gPQoQIOHwFHnB++xCUizZL20cvutm6CuY/CzHEw/6/hFM5eA8NW/mfPyso3dkUkPToYK5n30VJ4fXwo/Q8WQLtyOOpf4HOXQtvOsdOJtDgqeske93AQ98U/wLwnw6WejxwJn/9X6Nw7djqRFkNn3Uj2mIVz8PsMCt/MfekPUHUHTB8bTtEceNXu3+gVkbygWzNJ43XrB2feAlfPgmOvgvnPwG0nwoQLYMXc2OlEZBcqemm6DnvDKT+Ea2bDiTfAwv+DWz4PD46GNQ3fkkBEckNFL3uuuBSOvxaueg2OvgLemAi/7w9Pfj91cxYRiUlFL5nTrguc+t8wugr6nQkv/A5+ezi8+PtwG0gRiUJFL5nXqRec9Uf4xvPQ/Sh44gYYMyhcZVNEck5FL9mz1yEwchKcd0+4u9bYk+GRa2HjR7GTibQoKnrJvoNPhytfgaMuh7/fBjcfDXMeip1KpMVQ0UtutOkAQ2+Cy56Ckk4w8YJwOuZHS2MnE0k8Fb3kVvdK+Jfn4OQbYd5TcMsxMGda7FQiiaail9wrKIIvfAuueBE67QsTR8JD18CWDbGTiSSSil7i6bIfXPokHDMaqv4Et50Ey/8RO5VI4qjoJa7C1jD4J3DBJFi7LJyGOePOcBE1EckIFb3khwNOgSteCDcwn/ZNuP9i2LA6diqRREir6M3sVDOba2bzzOy6Wl7vaWbPmNlMM3vDzIZmPqokXuleMPIBOOkH4QDt2JNg5bzYqUSavQaL3swKgJuBIUA/YISZ9dtlthuA+9z9SGA4cEumg0oL0aoVHHcNXPxQuE7O2BNhwbOxU4k0a+ls0R8FzHP3Be6+GZgAnLHLPA50SD3vCOjkaNkzvY6Fy5+G0n3g7rNg+p9iJxJpttIp+gpgcY3xJalpNd0IjDSzJcAjwL/V9kFmNsrMqsysasWKFU2IKy1Kp33h0idg/5Pg4Wvgke/Atq2xU4k0O5k6GDsCuNPduwNDgbvNbLfPdvcx7l7p7pXl5eUZWrQkWpsOMGJCOAXz73+Ee8/RQVqRRkqn6KuBHjXGu6em1XQpcB+Au78EtAG6ZiKgCK0KwimYX/5duLnJ2JPhg4WxU4k0G+kU/XTgADPrbWatCQdbp+4yzyLgJAAzO5hQ9No3I5k14CL42hRYvxJuPxWWz4mdSKRZaLDo3X0rMBp4HJhDOLtmtpn9yMyGpWb7NnC5mb0OjAcudtc3XiQLeh8HFz8CONwxFKpfjZ1IJO9ZrD6urKz0qirdiEKa6IMFcNcZsP5DOH8i7DswdiKRnDCzGe5e2Zj36Jux0jx17gOXPBZuUD7uLHjnydiJRPKWil6ar44VcMmj0PVAGD8CZj8QO5FIXlLRS/PWrmv4Fm33Spj0dXj17tiJRPKOil6avzYdYeRk6DMIpo4OV78UkU+o6CUZWrcNX6w64Esw7Wp4fWLsRCJ5Q0UvyVFYDOfeFU7BnHIFvLXr1z1EWiYVvSRLUQkMHw8VA8I++7efiJ1IJDoVvSRPcXu44H7o1g/u+1q4bIJIC6ail2QqKQs3MenUG+4dDoteiZ1IJBoVvSRXuy5w4RQo7Qb3nA1LX4udSCQKFb0kW+lecOHUcArm3V+Ble/ETiSScyp6Sb6yHnDhg2CtYNxXYe3y2IlEckpFLy1Dl/3g/PtCyd97LmxeFzuRSM6o6KXl6D4AzrkD/vl6OPVStyWUFkJFLy1L3yEw9CZ4+zF49FrQbROkBSiMHUAk5z53GaxeDC/8Bjr2gOOuiZ1IJKtU9NIynfQDWLME/vrDUPaHnRM7kUjWqOilZWrVCs68BdYuC9fFKe0GvY+PnUokK7SPXlquwmI4b1w4I2fCSFg5L3YikaxQ0UvLVlIWTrssKITxw2HD6tiJRDJORS/SqVfYsv9wIfzlUti+LXYikYxS0YsA9DoWTvslzHsKnvx+7DQiGaWDsSI7DLgYlr0FL/0Bun0Wjjg/diKRjNAWvUhNg/8bep8A076pSxtLYqjoRWoqKIRz7oSO3WHiyHCuvUgzl1bRm9mpZjbXzOaZ2XV1zHOumb1lZrPN7N7MxhTJobadw43Gt2yACefD5vWxE4nskQaL3swKgJuBIUA/YISZ9dtlngOA64GB7v5Z4OrMRxXJofK+cPbt8M834MErdU0cadbS2aI/Cpjn7gvcfTMwAThjl3kuB2529w8B3F0X/Jbm78Avwck/gNmT4aWbY6cRabJ0ir4CWFxjfElqWk0HAgea2Qtm9rKZnVrbB5nZKDOrMrOqFStWNC2xSC4NvBoOHhZOuVz4t9hpRJokUwdjC4EDgEHACOA2MyvbdSZ3H+Pule5eWV5enqFFi2SRWbgmTpf94P6LYU117EQijZZO0VcDPWqMd09Nq2kJMNXdt7j7QuBtQvGLNH/FpXDePbB1E9z3tfAo0oykU/TTgQPMrLeZtQaGA1N3mWcKYWseM+tK2JWzIHMxRSIrPzBs2VfPgEf/I3YakUZpsOjdfSswGngcmAPc5+6zzexHZjYsNdvjwCozewt4BrjW3VdlK7RIFP2GwRe+BTPugFfvjp1GJG3mkU4bq6ys9KqqqijLFmmy7dtg3Fnw3kvw9cegon/sRNLCmNkMd69szHv0zViRxmhVAF+9Hdp/Bu67ENbpD1fJfyp6kcZq1wXOvQvWLtdljaVZUNGLNEVFfxh6Eyx4Bp77n9hpROqlohdpqv4XwhEXwHM/h3eeip1GpE4qepGmMoOhvwjXrp98GaxeFDuRSK1U9CJ7onXbsL9++za47yJ9mUrykopeZE912Q/OuBmWvgqPfzd2GpHdqOhFMqHfMDhmNEwfC2/cHzuNyKeo6EUy5eQboecxMO0qWD4ndhqRT6joRTKloAjOvgNat4eJX4NNH8dOJAKo6EUyq8Pe4c5UH8yHqf+mO1NJXlDRi2Ra7+PgxP+E2Q/A38fETiOiohfJioFXw4FD4PHvweLpsdNIC6eiF8mGVq3gK7dCh33g/otg3crYiaQFU9GLZEtJp/BlqnUr4S+X6eJnEo2KXiSb9jkChv5cFz+TqFT0ItnW/yI4/Hxd/EyiUdGLZJsZnPZLXfxMolHRi+SCLn4mEanoRXKl5sXPHrsudhppQVT0IrnUbxgM/CZU3Q4zx8VOIy2Eil4k1078PvQ+AR66BpbOjJ1GWgAVvUiuFRSG6+G0K4eJF8K6VbETScKp6EViaNcVzrsb1i6Dv3xdX6aSrEqr6M3sVDOba2bzzKzOo0hm9lUzczOrzFxEkYSq6A+n/QIWPAtP/zh2GkmwBovezAqAm4EhQD9ghJn1q2W+UuCbwCuZDimSWP0vhAEXw/O/gjnTYqeRhEpni/4oYJ67L3D3zcAE4Ixa5vsv4H+AjRnMJ5J8Q34OFQPggStgxdux00gCpVP0FcDiGuNLUtM+YWb9gR7u/nB9H2Rmo8ysysyqVqxY0eiwIolUWBy+TFVYDBMvgI0fxU4kCbPHB2PNrBXwK+DbDc3r7mPcvdLdK8vLy/d00SLJ0bE7nHMnrJoPky/XwVnJqHSKvhroUWO8e2raDqXAIcCzZvYu8Hlgqg7IijRS7+NgyP/A24/BX38UO40kSGEa80wHDjCz3oSCHw6cv+NFd18DdN0xbmbPAv/u7lWZjSrSAnzuMlg2G174TbgI2mHnxk4kCdDgFr27bwVGA48Dc4D73H22mf3IzIZlO6BIi2IGQ2+CXl+AB0fDkhmxE0kCmEe6S31lZaVXVWmjX6RW61bBbYNg62YY9Sx02Dt2IskTZjbD3Ru1a1zfjBXJR+26wIgJsHktTDgftmyInUiaMRW9SL7q9lk4a0y48NnUf4NIf31L86eiF8lnB50GJ94As+6H538dO400U+mcdSMiMR33bVj+Fvz1h9BpXzjkrNiJpJlR0YvkOzM44xb4aCk88C9Quhf0OjZ2KmlGtOtGpDkoagPD74WyXjB+BKyYGzuRNCMqepHmom1nGDkJClrDuLPh42WxE0kzoaIXaU467QvnT4T1K+Hec2DT2tiJpBlQ0Ys0NxX9wwXQ3p8Fky6BbVtjJ5I8p6IXaY4OHAyn/QreeQIevkbn2Eu9dNaNSHNVeQmsWQx/+2W4zPEJ34mdSPKUil6kOTvxP2FNNTzzEyguhc9fETuR5CEVvUhzZgZn3ByuifPYdVDUFgZcFDuV5Bntoxdp7goK4ezbYf9TYNo34Y37YieSPKOiF0mCwmI4727Y9wvwwDdgzrTYiSSPqOhFkqKoJFzauGIA3H8JvPNk7ESSJ1T0IklS3B4uuB+69YOJI2Hh32InkjygohdJmpIyGPkAdOoN954Hi16JnUgiU9GLJFG7LnDhlHCly7u/AguejZ1IIlLRiyRV6V5wyaPQqRfccw784+HYiSQSFb1IkpV2g4sfhr0OhYlf06mXLZSKXiTp2naGCx8MNyuZPAqmj42dSHJMRS/SEhSXhrNxDhwMD38b/var2Ikkh1T0Ii1FUQmcNw4O+Wq4/+xTN+qqly1EWkVvZqea2Vwzm2dm19Xy+jVm9paZvWFmfzWzXpmPKiJ7rKAIzroNBlwMz/8aHhwNWzfHTiVZ1mDRm1kBcDMwBOgHjDCzfrvMNhOodPfDgEnAzzMdVEQypFUBnP4bOOE/4LVxcNcZsG5l7FSSRels0R8FzHP3Be6+GZgAnFFzBnd/xt3Xp0ZfBrpnNqaIZJQZfPG78NU/QfUMuO1EWD4ndirJknSKvgJYXGN8SWpaXS4FHt2TUCKSI4eeDZc8Als3wthTdH2chMrowVgzGwlUAjfV8fooM6sys6oVK1ZkctEi0lTdK+Hyp6HzvnDvufDSzTpImzDpFH010KPGePfUtE8xs5OB7wHD3H1TbR/k7mPcvdLdK8vLy5uSV0SyoWN3+PrjcNBp8Ph3YdpVsLXWH2NphtIp+unAAWbW28xaA8OBqTVnMLMjgT8SSn555mOKSNa1bgfn3AXH/Tu8ehf86RRYNT92KsmABove3bcCo4HHgTnAfe4+28x+ZGbDUrPdBLQH7jez18xsah0fJyL5rFUrOOk/Yfi9sHoR/PF4eH1C7FSyh8wj7YurrKz0qqqqKMsWkTSsqYbJl8N7L8Bh58FpvwzfsJWozGyGu1c25j36ZqyI1K5jBVw0DQZdD7PuD1v3S2fGTiVNoKIXkbq1KoBB14UrYG7dFE7BfPH3sH1b7GTSCCp6EWlYr2PhG8+Hi6I9cQP86Uvw/puxU0maVPQikp62ncNF0c66DT5cCGNOgCd/AJvXN/xeiUpFLyLpM4PDzoXRVXDYcHjhN3DrMTD/6djJpB4qehFpvLad4cybw8FaKwj3pZ08ShdHy1MqehFput7HwxUvwvHfgTcnw+/6h5uaaHdOXlHRi8ieKWoDJ34PrngBeh0Tbmry+/4w48+wbWvsdIKKXkQypbwvnD8RLn4kXDtn2lVh//2cabpIWmQqehHJrH0HwqVPwnn3hPGJI8N1cxY8p8KPREUvIplnBgefDle8BMN+Hy6ncNcwGDMIZk3SLp0cU9GLSPYUFEL/C+GqmfDl38LmdfCXS+F3R8LLt8Kmj2MnbBFU9CKSfUVtwg3Jr/w7jJgQ9uE/dh38+rPw1I1hi1+yRlevFJE4llTBi78LB2sB+nwRjhwZbn5SWBw3Wx5rytUrC7MVRkSkXt0r4dy74MN34bV7YeY9MOkSKOkEh54TSn/vw2OnTARt0YtIfti+DRY+BzPHwZyHYNsm6HYoHHIWHDwMuu4fO2FeaMoWvYpeRPLPhg/D2Tmv3QtLXw3Tyg+Cg78MB50etvTN4maMREUvIsmzejH842H4x0Phble+HTr2DPvy9z8Jeh4Dxe1jp8wZFb2IJNu6VfD2o2HXzvynw+6dVkXQ/XPQ5wToMwgqBkBBUeykWaOiF5GWY8sGWPRy2K+/4FlY+hrg0Lp92MrvcVQ44LtPfygpi5s1g3TWjYi0HEUlsN8XwwCw/gN49/lQ/O8+D/OeAlIbsl37htLfUfzlB4Vz+1sIFb2IJEPbztBvWBgANq6B6lehuiqcs//2Y/Ba6vo7VgBd9odun4Vu/aDbIeF5xx6JPMiroheRZGrT8dNb/O7hnP2lM2H5W7BsNlTPgNmTd76nqB106QOd9wu/CLqkHjvvF36RNNNfAip6EWkZzKBz7zBw1s7pGz+C5XNg2Zuw8h34YD68/0bq8srbds7XujRcuqGsR3js2CMMZT2gdC9ov1fe7g5S0YtIy9amA/Q8Ogw1bdsCH74Xin/VPFi9KJzquWYxLJkezvXf7bM6hsIv7QbtU0O7rtC2K7TtknreJfx10KYsZ38hpFX0ZnYq8FugABjr7j/b5fVi4C5gALAKOM/d381sVJHkmDKzmpsen8vS1RvYp6yEawf35cwjKwC44LaXeGH+B5/MO3C/ztxz+TFpvbe+126YMovxryxmmzsFZow4ugc/PvPQPc6bzutN/exsZU4rb0FR+DZu1/2Bwbt97uqNH3JEh7Vc2b+YYz+zBda+Dx8v2/m4+BVYuxy2bqw9oBWEXww7hpKy1POy8MundSkUl4bvCBSX7hxvggZPrzSzAuBt4BRgCTAdGOHub9WY51+Bw9z9G2Y2HPiKu59X3+fq9EppqabMrOb6ybPYsGXnboGSogJ+etah3F+16FMlv8OOsq/vvUCdr1W99wHjXl602+eO/HzPBouzvmWeeWRFg6839bOzlRnqXk97krfW97rDlvXhpunrV4Uzg9avgvWp8Y1rwrBhder56p3T6vgFYT/8KPPn0ZvZMcCN7j44NX59yO8/rTHP46l5XjKzQuB9oNzr+XAVvbRUA3/2NNWrN+w2vaKspNbpO7z7s9PqfS9Q52vvr9nItlp+HAvMmP/ToU3O+8J1Jzb4elM/O1uZoe71tCd5G3pvo23bEq7Xv3ktbFqbev4xdsApWTmPvgJYXGN8CXB0XfO4+1YzWwN0AVbWnMnMRgGjAHr27NmYnCKJsbSOMq9r+p6+d+nqDdS1xVVbkTZ2mdn678lm5sa+tief22QFRWFfftvOe/xROb3xiLuPcfdKd68sLy/P5aJF8sY+qa3KdKen+976Xiuo46BfXdPTXWY6rzf1s7OVOVt581k6RV8N9Kgx3j01rdZ5UrtuOhIOyorILq4d3JeSooJPTSspKuDawX0ZuF/tW287ptf33vpeG3F0D2pT1/R086bzelM/O1uZs5U3n6Wz62Y6cICZ9SYU+nDg/F3mmQpcBLwEnA08Xd/+eZGWbMdBu9rO+jjzyIp6z7qp77071PW5QJPOYGlomelkauq6yFbmbOTNZ2ld1MzMhgK/IZxeebu7/8TMfgRUuftUM2sD3A0cCXwADHf3BfV9pg7Giog0XtYuaubujwCP7DLt+zWebwTOacyCRUQkN3J6MFZERHJPRS8iknAqehGRhFPRi4gknIpeRCThVPQiIgmnohcRSTgVvYhIwqnoRUQSTkUvIpJwKnoRkYRL66JmWVmw2cfA3CgLr1tXdrlZSp7Ix1zKlB5lSl8+5srHTH3dvVE3j03romZZMrexV2DLNjOryrdMkJ+5lCk9ypS+fMyVr5ka+x7tuhERSTgVvYhIwsUs+jERl12XfMwE+ZlLmdKjTOnLx1yJyBTtYKyIiOSGdt2IiCScil5EJOFyVvRmVmBmM83sodR4bzN7xczmmdlEM2udqywN5LrTzBaa2Wup4Ygc53nXzGalll2VmtbZzJ40s3dSj53yINONZlZdYz0NzXGmMjObZGb/MLM5ZnZM7PVUT65o68rM+tZY7mtm9pGZXR1zXdWTKfa/qW+Z2Wwze9PMxptZm9g9VUemRndUzvbRm9k1QCXQwd1PN7P7gMnuPsHM/hd43d1vzUmY+nPdCTzk7pNynSWV512g0t1X1pj2c+ADd/+ZmV0HdHL3/4ic6UZgrbv/Ilc5dsn0Z+Bv7j429cPXFvguEddTPbmuJuK6qpGtAKgGjgauJPK6qiXTJURaT2ZWATwP9HP3Dal+egQYSqSeqifTIBrZUTnZojez7sBpwNjUuAEnAjuC/hk4MxdZ6suVx84grCOItK7yiZl1BI4H/gTg7pvdfTWR11M9ufLFScB8d3+P/Pk3VTNTbIVAiZkVEn5B/5P4PbVrpqVN+ZBc7br5DfAdYHtqvAuw2t23psaXABU5ylJfrh1+YmZvmNmvzaw4x5kceMLMZpjZqNS0bu7+z9Tz94FueZAJYHRqPd2e490kvYEVwB0WdruNNbN2xF9PdeWCeOuqpuHA+NTz2Otqh5qZINJ6cvdq4BfAIkLBrwFmELGnasvk7k+kXm5UR2W96M3sdGC5u8/I9rIao55c1wMHAZ8DOgO5/nP2C+7eHxgCXGlmx9d80cO+tlyfE1tbpluB/YAjCP8If5nDPIVAf+BWdz8SWAdcV3OGSOuprlwx1xUAqd1Iw4D7d30t0rqqLVO09ZT6pXIG4Zf1PkA74NRcLT/dTGY2kiZ0VC626AcCw1L7eScQ/hT6LVCW+nMEoDthP10u7ZbLzMa5+z892ATcARyVy1Cp3+K4+3LggdTyl5nZ3gCpx+WxM7n7Mnff5u7bgdvI7XpaAixx91dS45MIBRt1PdWVK/K62mEI8Kq7L0uNx15Xu2WKvJ5OBha6+wp33wJMJnREzJ6qLdOxTemorBe9u1/v7t3dfV/Cn2lPu/sFwDPA2anZLgIezHaWNHKNrPGP3wj7497MVSYza2dmpTueA19KLX8qYR1BjtdVXZl2rKeUr5DD9eTu7wOLzaxvatJJwFtEXE/15Yq5rmoYwad3kURdVymfyhR5PS0CPm9mbVM/+zv+TcXsqdoyzWlSR7l7zgZ2Hi0G6AP8HZhH+NOtOJdZ6sn1NDArtfLGAe1zmKMP8HpqmA18LzW9C/BX4B3gKaBzHmS6O7We3iCUxt45/n92BFCVWv4UoFPM9dRArtjrqh2wCuhYY1rUdVVHptjr6YfAP1I/+3cDxbF7qo5Mje4oXQJBRCTh9M1YEZGEU9GLiCScil5EJOFU9CIiCaeiFxFJOBW9iEjCqehFRBLu/wHKEWw4E3VywwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"w_val = model.get_weights()[0]\n",
"b_val = model.get_weights()[1]\n",
"plt.plot(data[:,0], data[:,1], 'o')\n",
"plt.axis([40, 85, -0.1, 1.2])\n",
"x_pred = np.linspace(40,85)\n",
"x_pred = np.resize(x_pred,[len(x_pred),1])\n",
"y_pred = 1 / (1 + np.exp(-x_pred*w_val - b_val))\n",
"plt.plot(x_pred, y_pred)\n",
"\n",
"p_1 = 1 / (1 + np.exp(-x*w_val - b_val))\n",
"\n",
"# cross-entropy loss function\n",
"cross_entropy = -np.mean(y * np.log(p_1) + (1-y) * np.log(1-p_1))\n",
"print(\"Cross-entropy: \", cross_entropy)\n",
"y_pred = np.round(p_1, decimals=0).astype('int')\n",
"accuracy = np.mean(y==y_pred)\n",
"print(\"Accuracy: \", accuracy)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model.evaluate(x, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The value of the cross-entropy loss function could be decreased from "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise 3 : MNIST and Multinomial Logistic Regression"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this exercise we use multinomial logistic regression to predict the number of the handwritten digits of the MNIST dataset."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## TODO : read MNIST data and compute validation accuracy for the multinomial logistic regression model"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1875/1875 [==============================] - 9s 5ms/step - loss: 321.2496 - accuracy: 0.8419 - val_loss: 243.0990 - val_accuracy: 0.8799\n",
"1875/1875 [==============================] - 9s 5ms/step - loss: 263.8232 - accuracy: 0.8672 - val_loss: 203.2465 - val_accuracy: 0.8986\n",
"1875/1875 [==============================] - 9s 5ms/step - loss: 253.0255 - accuracy: 0.8741 - val_loss: 208.3061 - val_accuracy: 0.8858\n",
"1875/1875 [==============================] - 8s 4ms/step - loss: 247.8131 - accuracy: 0.8757 - val_loss: 522.1936 - val_accuracy: 0.7770\n",
"1875/1875 [==============================] - 8s 4ms/step - loss: 244.3168 - accuracy: 0.8773 - val_loss: 264.0963 - val_accuracy: 0.8677\n",
"1875/1875 [==============================] - 9s 5ms/step - loss: 239.8172 - accuracy: 0.8792 - val_loss: 251.0652 - val_accuracy: 0.8803\n",
"1875/1875 [==============================] - 8s 4ms/step - loss: 243.0666 - accuracy: 0.8798 - val_loss: 245.4965 - val_accuracy: 0.8903\n",
"1875/1875 [==============================] - 8s 4ms/step - loss: 236.9614 - accuracy: 0.8811 - val_loss: 270.9126 - val_accuracy: 0.8773\n",
"1875/1875 [==============================] - 8s 4ms/step - loss: 235.9632 - accuracy: 0.8825 - val_loss: 208.9754 - val_accuracy: 0.8894\n",
"1875/1875 [==============================] - 8s 4ms/step - loss: 229.1799 - accuracy: 0.8818 - val_loss: 288.4063 - val_accuracy: 0.8719\n"
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
]
}
],
"source": [
"from __future__ import absolute_import, division, print_function, unicode_literals\n",
"from tensorflow.keras.datasets import mnist\n",
"from tensorflow.keras.utils import to_categorical\n",
"\n",
"\n",
"# Import TensorFlow and TensorFlow Datasets\n",
"import tensorflow as tf\n",
"\n",
"# Helper libraries\n",
"import math\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# Load MNIST data\n",
"(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
"\n",
"# One-hot-encoded label vector\n",
"y_train_cat = to_categorical(y_train, 10)\n",
"y_test_cat = to_categorical(y_test, 10)\n",
"\n",
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))\n",
"model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax, batch_input_shape=(None, 784)))\n",
"model.compile(loss='categorical_crossentropy',\n",
" optimizer='sgd',\n",
" metrics=['accuracy'])\n",
"\n",
"history = model.fit(X_train, \n",
" y_train_cat, \n",
" epochs=10, \n",
" validation_data=(X_test, y_test_cat))"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"313/313 [==============================] - 1s 4ms/step - loss: 288.4063 - accuracy: 0.8719\n",
"Accuracy on test dataset: 0.8719000220298767\n"