diff --git a/notebooks/Block_2/Exercises Block 2 - Neural Networks.ipynb b/notebooks/Block_2/Exercises Block 2 - Neural Networks.ipynb index 2aa140acdd7279cbe29979573f6928a4b961a63e..b2036ef24a803040c399f134b4dcc9e8a1df7c23 100644 --- a/notebooks/Block_2/Exercises Block 2 - Neural Networks.ipynb +++ b/notebooks/Block_2/Exercises Block 2 - Neural Networks.ipynb @@ -782,6 +782,547 @@ "# Fit Network\n", "history = model.fit(<---- your code here ---->)" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Exercise 4 : Prediction of House Prices" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this exercise, we’ll attempt to predict the median price of homes in a given Boston\n", + "suburb in the mid-1970s, given data points about the suburb at the time, such as the\n", + "crime rate, the local property tax rate, and so on. The dataset has relatively few data points: only\n", + "506, split between 404 training samples and 102 test samples. And each feature in the\n", + "input data (for example, the crime rate) has a different scale. For instance, some values\n", + "are proportions, which take values between 0 and 1, others take values between 1\n", + "and 12, others between 0 and 100, and so on." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Loading the Boston housing dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/boston_housing.npz\n", + "57344/57026 [==============================] - 0s 0us/step\n", + "65536/57026 [==================================] - 0s 0us/step\n" + ] + } + ], + "source": [ + "from tensorflow.keras.datasets import boston_housing\n", + "(train_data, train_targets), (test_data, test_targets) = (boston_housing.load_data())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let’s look at the data:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(404, 13)\n", + "(102, 13)\n" + ] + } + ], + "source": [ + "print(train_data.shape)\n", + "print(test_data.shape)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can see, we have 404 training samples and 102 test samples, each with 13\n", + "numerical features, such as per capita crime rate, average number of rooms per dwelling,\n", + "accessibility to highways, and so on.\n", + "The targets are the median values of owner-occupied homes, in thousands of dollars:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([15.2, 42.3, 50. , 21.1, 17.7, 18.5, 11.3, 15.6, 15.6, 14.4, 12.1,\n", + " 17.9, 23.1, 19.9, 15.7, 8.8, 50. , 22.5, 24.1, 27.5, 10.9, 30.8,\n", + " 32.9, 24. , 18.5, 13.3, 22.9, 34.7, 16.6, 17.5, 22.3, 16.1, 14.9,\n", + " 23.1, 34.9, 25. , 13.9, 13.1, 20.4, 20. , 15.2, 24.7, 22.2, 16.7,\n", + " 12.7, 15.6, 18.4, 21. , 30.1, 15.1, 18.7, 9.6, 31.5, 24.8, 19.1,\n", + " 22. , 14.5, 11. , 32. , 29.4, 20.3, 24.4, 14.6, 19.5, 14.1, 14.3,\n", + " 15.6, 10.5, 6.3, 19.3, 19.3, 13.4, 36.4, 17.8, 13.5, 16.5, 8.3,\n", + " 14.3, 16. , 13.4, 28.6, 43.5, 20.2, 22. , 23. , 20.7, 12.5, 48.5,\n", + " 14.6, 13.4, 23.7, 50. , 21.7, 39.8, 38.7, 22.2, 34.9, 22.5, 31.1,\n", + " 28.7, 46. , 41.7, 21. , 26.6, 15. , 24.4, 13.3, 21.2, 11.7, 21.7,\n", + " 19.4, 50. , 22.8, 19.7, 24.7, 36.2, 14.2, 18.9, 18.3, 20.6, 24.6,\n", + " 18.2, 8.7, 44. , 10.4, 13.2, 21.2, 37. , 30.7, 22.9, 20. , 19.3,\n", + " 31.7, 32. , 23.1, 18.8, 10.9, 50. , 19.6, 5. , 14.4, 19.8, 13.8,\n", + " 19.6, 23.9, 24.5, 25. , 19.9, 17.2, 24.6, 13.5, 26.6, 21.4, 11.9,\n", + " 22.6, 19.6, 8.5, 23.7, 23.1, 22.4, 20.5, 23.6, 18.4, 35.2, 23.1,\n", + " 27.9, 20.6, 23.7, 28. , 13.6, 27.1, 23.6, 20.6, 18.2, 21.7, 17.1,\n", + " 8.4, 25.3, 13.8, 22.2, 18.4, 20.7, 31.6, 30.5, 20.3, 8.8, 19.2,\n", + " 19.4, 23.1, 23. , 14.8, 48.8, 22.6, 33.4, 21.1, 13.6, 32.2, 13.1,\n", + " 23.4, 18.9, 23.9, 11.8, 23.3, 22.8, 19.6, 16.7, 13.4, 22.2, 20.4,\n", + " 21.8, 26.4, 14.9, 24.1, 23.8, 12.3, 29.1, 21. , 19.5, 23.3, 23.8,\n", + " 17.8, 11.5, 21.7, 19.9, 25. , 33.4, 28.5, 21.4, 24.3, 27.5, 33.1,\n", + " 16.2, 23.3, 48.3, 22.9, 22.8, 13.1, 12.7, 22.6, 15. , 15.3, 10.5,\n", + " 24. , 18.5, 21.7, 19.5, 33.2, 23.2, 5. , 19.1, 12.7, 22.3, 10.2,\n", + " 13.9, 16.3, 17. , 20.1, 29.9, 17.2, 37.3, 45.4, 17.8, 23.2, 29. ,\n", + " 22. , 18. , 17.4, 34.6, 20.1, 25. , 15.6, 24.8, 28.2, 21.2, 21.4,\n", + " 23.8, 31. , 26.2, 17.4, 37.9, 17.5, 20. , 8.3, 23.9, 8.4, 13.8,\n", + " 7.2, 11.7, 17.1, 21.6, 50. , 16.1, 20.4, 20.6, 21.4, 20.6, 36.5,\n", + " 8.5, 24.8, 10.8, 21.9, 17.3, 18.9, 36.2, 14.9, 18.2, 33.3, 21.8,\n", + " 19.7, 31.6, 24.8, 19.4, 22.8, 7.5, 44.8, 16.8, 18.7, 50. , 50. ,\n", + " 19.5, 20.1, 50. , 17.2, 20.8, 19.3, 41.3, 20.4, 20.5, 13.8, 16.5,\n", + " 23.9, 20.6, 31.5, 23.3, 16.8, 14. , 33.8, 36.1, 12.8, 18.3, 18.7,\n", + " 19.1, 29. , 30.1, 50. , 50. , 22. , 11.9, 37.6, 50. , 22.7, 20.8,\n", + " 23.5, 27.9, 50. , 19.3, 23.9, 22.6, 15.2, 21.7, 19.2, 43.8, 20.3,\n", + " 33.2, 19.9, 22.5, 32.7, 22. , 17.1, 19. , 15. , 16.1, 25.1, 23.7,\n", + " 28.7, 37.2, 22.6, 16.4, 25. , 29.8, 22.1, 17.4, 18.1, 30.3, 17.5,\n", + " 24.7, 12.6, 26.5, 28.7, 13.3, 10.4, 24.4, 23. , 20. , 17.8, 7. ,\n", + " 11.8, 24.4, 13.8, 19.4, 25.2, 19.4, 19.4, 29.1])" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_targets" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The prices are typically between 10000 and 50000 USD. If that sounds cheap, remember\n", + "that this was the mid-1970s, and these prices aren’t adjusted for inflation." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Preparing the data\n", + "\n", + "It would be problematic to feed into a neural network values that all take wildly different ranges. The model might be able to automatically adapt to such heterogeneous data, but it would definitely make learning more difficult. A widespread best practice for dealing with such data is to do feature-wise normalization: for each feature in the input data (a column in the input data matrix), we subtract the mean of the feature and divide by the standard deviation, so that the feature is centered around 0 and has a unit standard deviation. This is easily done in NumPy." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Normalizing the data" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "mean = train_data.mean(axis=0)\n", + "train_data -= mean\n", + "std = train_data.std(axis=0)\n", + "train_data /= std\n", + "test_data -= mean\n", + "test_data /= std" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that the quantities used for normalizing the test data are computed using the\n", + "training data. You should never use any quantity computed on the test data in your\n", + "workflow, even for something as simple as data normalization." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### TODO: Building your model\n", + "\n", + "Because so few samples are available, we’ll use a very small model with two intermediate layers, each with 64 units, each followed by a `relu` activation function. In general, the less training data you have, the worse overfitting will be, and using a small model is one way to mitigate overfitting." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Model definition" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "def build_model():\n", + " model = keras.Sequential([\n", + " <---- your code here ---->,\n", + " <---- your code here ---->,\n", + " <---- your code here ---->\n", + " ])\n", + " model.compile(optimizer=\"rmsprop\", loss=<---- your code here ---->, metrics=[\"mae\"])\n", + " return model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The model ends with a single unit and no activation (it will be a linear layer). This is a\n", + "typical setup for scalar regression (a regression where you’re trying to predict a single\n", + "continuous value). Applying an activation function would constrain the range the output\n", + "can take; for instance, if you applied a sigmoid activation function to the last layer,\n", + "the model could only learn to predict values between 0 and 1. Here, because the last\n", + "layer is purely linear, the model is free to learn to predict values in any range.\n", + "Note that we compile the model with the `mse` loss function — _mean squared error_, the\n", + "square of the difference between the predictions and the targets. This is a widely used\n", + "loss function for regression problems.\n", + "We’re also monitoring a new metric during training: _mean absolute error_ (`MAE`). It’s the\n", + "absolute value of the difference between the predictions and the targets. For instance, an\n", + "MAE of 0.5 on this problem would mean your predictions are off by 500 on average." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Validating your approach using K-fold validation\n", + "\n", + "To evaluate our model while we keep adjusting its parameters (such as the number of\n", + "epochs used for training), we could split the data into a training set and a validation set, as we did in the previous examples. But because we have so few data points, the validation set would end up being very small (for instance, about 100 examples). As a consequence, the validation scores might change a lot depending on which data points we chose for validation and which we chose for training: the validation scores might have a high variance with regard to the validation split. This would prevent us from reliably evaluating our model.\n", + "\n", + "The best practice in such situations is to use $K$-fold cross-validation. It consists of splitting the available data into K partitions (typically $K = 4$ or $5$), instantiating $K$ identical models, and training each one on $K - 1$ partitions while evaluating on the remaining partition. The validation score for the model used is then the average of the $K$ validation scores obtained. In terms of code, this is straightforward." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### K-fold validation" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Processing fold #0\n", + "Processing fold #1\n", + "Processing fold #2\n", + "Processing fold #3\n" + ] + } + ], + "source": [ + "import numpy as np\n", + "from tensorflow import keras\n", + "from tensorflow.keras import layers\n", + "k = 4\n", + "num_val_samples = len(train_data) // k\n", + "num_epochs = 100\n", + "all_scores = []\n", + "for i in range(k):\n", + " print(f\"Processing fold #{i}\")\n", + " # Prepares the validation data: data from partition k\n", + " val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n", + " val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n", + " # Prepares the training data: data from all other partitions\n", + " partial_train_data = np.concatenate([train_data[:i * num_val_samples], train_data[(i + 1) * num_val_samples:]],axis=0)\n", + " partial_train_targets = np.concatenate([train_targets[:i * num_val_samples], train_targets[(i + 1) * num_val_samples:]],axis=0)\n", + " # Builds the Keras model (already compiled)\n", + " model = build_model()\n", + " # Trains the model (in silent mode, verbose=0)\n", + " history=model.fit(partial_train_data, partial_train_targets, epochs=num_epochs, batch_size=16, verbose=0)\n", + " val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n", + " all_scores.append(val_mae)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Running this with `num_epochs = 100` yields the following results:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[1.9184445142745972,\n", + " 2.4037296772003174,\n", + " 2.4944815635681152,\n", + " 2.4431681632995605]" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "all_scores" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "2.3149559795856476" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "np.mean(all_scores)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The different runs do indeed show rather different validation scores, from 1.9 to 2.49.\n", + "The average (2.3) is a much more reliable metric than any single score—that’s the\n", + "entire point of K-fold cross-validation. In this case, we’re off by 2310 USD on average, which is significant considering that the prices range from 10000 to 50000.\n", + "Let’s try training the model a bit longer: 500 epochs. To keep a record of how well\n", + "the model does at each epoch, we’ll modify the training loop to save the per-epoch\n", + "validation score log for each fold." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Saving the validation logs at each fold" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Processing fold #0\n", + "Processing fold #1\n", + "Processing fold #2\n", + "Processing fold #3\n" + ] + } + ], + "source": [ + "num_epochs = 500\n", + "all_mae_histories = []\n", + "for i in range(k):\n", + " print(f\"Processing fold #{i}\")\n", + " # Prepares the validation data: data from partition #k\n", + " val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n", + " val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n", + " # Prepares the training data: data from all other partitions\n", + " partial_train_data = np.concatenate([train_data[:i * num_val_samples], train_data[(i + 1) * num_val_samples:]],axis=0)\n", + " partial_train_targets = np.concatenate([train_targets[:i * num_val_samples], train_targets[(i + 1) * num_val_samples:]],axis=0)\n", + " # Builds the Keras model (already compiled)\n", + " model = build_model()\n", + " # Trains the model (in silent mode, verbose=0)\n", + " history = model.fit(partial_train_data, partial_train_targets,\n", + " validation_data=(val_data, val_targets), epochs=num_epochs, batch_size=16, verbose=0)\n", + " mae_history = history.history[\"val_mae\"]\n", + " all_mae_histories.append(mae_history)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can then compute the average of the per-epoch MAE scores for all folds." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Building the history of successive mean K-fold validation scores" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "average_mae_history = [\n", + "np.mean([x[i] for x in all_mae_histories]) for i in range(num_epochs)]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Plotting validation scores" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n", + "plt.xlabel(\"Epochs\")\n", + "plt.ylabel(\"Validation MAE\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It may be a little difficult to read the plot, due to a scaling issue: the validation MAE\n", + "for the first few epochs is dramatically higher than the values that follow. Let’s omit\n", + "the first 10 data points, which are on a different scale than the rest of the curve." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Plotting validation scores, excluding the first 10 data points" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "truncated_mae_history = average_mae_history[10:]\n", + "plt.plot(range(1, len(truncated_mae_history) + 1), truncated_mae_history)\n", + "plt.xlabel(\"Epochs\")\n", + "plt.ylabel(\"Validation MAE\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can see in Figure above, validation MAE stops improving significantly after\n", + "120–140 epochs (this number includes the 10 epochs we omitted). Past that point,\n", + "we start overfitting.\n", + "Once you’re finished tuning other parameters of the model (in addition to the\n", + "number of epochs, you could also adjust the size of the intermediate layers), you can\n", + "train a final production model on all of the training data, with the best parameters,\n", + "and then look at its performance on the test data." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Training the final model " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Gets a fresh, compiled model\n", + "model = build_model()\n", + "# Trains it on the entirety of the data\n", + "model.fit(train_data, train_targets,\n", + "epochs=130, batch_size=16, verbose=0)\n", + "test_mse_score, test_mae_score = model.evaluate(<---- your code here ---->)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here’s the final result:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "test_mae_score" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Generating predictions on new data\n", + "When calling `predict()` on our binary classification model, we retrieved a scalar score between 0 and 1 for each input sample. With our multiclass classification model, we retrieved a probability distribution over all classes for each sample. Now, with this scalar regression model, `predict()` returns the model’s guess for the sample’s price in thousands of dollars:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "predictions = <---- your code here ---->\n", + "predictions[0]" + ] } ], "metadata": { diff --git a/notebooks/Block_2/Solutions to Exercises - Block 2.ipynb b/notebooks/Block_2/Solutions to Exercises - Block 2.ipynb index 6f6ac9ff4e490225c9a6294c22b732a08b35a5d7..e8efd9c45cb3dd12ab9cb322992482c419c41321 100644 --- a/notebooks/Block_2/Solutions to Exercises - Block 2.ipynb +++ b/notebooks/Block_2/Solutions to Exercises - Block 2.ipynb @@ -45,7 +45,7 @@ }, { "cell_type": "code", - "execution_count": 101, + "execution_count": 13, "metadata": { "colab": {}, "colab_type": "code", @@ -69,7 +69,7 @@ }, { "cell_type": "code", - "execution_count": 102, + "execution_count": 14, "metadata": { "colab": {}, "colab_type": "code", @@ -1153,8 +1153,7 @@ "source": [ "In this exercise, we’ll attempt to predict the median price of homes in a given Boston\n", "suburb in the mid-1970s, given data points about the suburb at the time, such as the\n", - "crime rate, the local property tax rate, and so on. The dataset we’ll use has an interesting\n", - "difference from the two previous examples. It has relatively few data points: only\n", + "crime rate, the local property tax rate, and so on. The dataset has relatively few data points: only\n", "506, split between 404 training samples and 102 test samples. And each feature in the\n", "input data (for example, the crime rate) has a different scale. For instance, some values\n", "are proportions, which take values between 0 and 1, others take values between 1\n", @@ -1226,7 +1225,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 2, "metadata": {}, "outputs": [ { @@ -1271,7 +1270,7 @@ " 11.8, 24.4, 13.8, 19.4, 25.2, 19.4, 19.4, 29.1])" ] }, - "execution_count": 3, + "execution_count": 2, "metadata": {}, "output_type": "execute_result" } @@ -1284,7 +1283,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The prices are typically between 10,000 and 50,000. If that sounds cheap, remember\n", + "The prices are typically between 10000 and 50000 USD. If that sounds cheap, remember\n", "that this was the mid-1970s, and these prices aren’t adjusted for inflation." ] }, @@ -1306,7 +1305,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -1345,7 +1344,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ @@ -1369,10 +1368,10 @@ "can take; for instance, if you applied a sigmoid activation function to the last layer,\n", "the model could only learn to predict values between 0 and 1. Here, because the last\n", "layer is purely linear, the model is free to learn to predict values in any range.\n", - "Note that we compile the model with the mse loss function—mean squared error, the\n", + "Note that we compile the model with the `mse` loss function — _mean squared error_, the\n", "square of the difference between the predictions and the targets. This is a widely used\n", "loss function for regression problems.\n", - "We’re also monitoring a new metric during training: mean absolute error (MAE). It’s the\n", + "We’re also monitoring a new metric during training: _mean absolute error_ (`MAE`). It’s the\n", "absolute value of the difference between the predictions and the targets. For instance, an\n", "MAE of 0.5 on this problem would mean your predictions are off by 500 on average." ] @@ -1398,7 +1397,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 5, "metadata": {}, "outputs": [ { @@ -1445,16 +1444,19 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "[1.8914849758148193, 2.359283924102783, 2.2852821350097656, 2.6983723640441895]" + "[1.9184445142745972,\n", + " 2.4037296772003174,\n", + " 2.4944815635681152,\n", + " 2.4431681632995605]" ] }, - "execution_count": 12, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } @@ -1465,16 +1467,16 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "2.3086058497428894" + "2.3149559795856476" ] }, - "execution_count": 13, + "execution_count": 7, "metadata": {}, "output_type": "execute_result" } @@ -1487,9 +1489,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The different runs do indeed show rather different validation scores, from 2.1 to 3.1.\n", - "The average (2.6) is a much more reliable metric than any single score—that’s the\n", - "entire point of K-fold cross-validation. In this case, we’re off by 2600 on average, which is significant considering that the prices range from 10000 to 50000.\n", + "The different runs do indeed show rather different validation scores, from 1.9 to 2.49.\n", + "The average (2.3) is a much more reliable metric than any single score—that’s the\n", + "entire point of K-fold cross-validation. In this case, we’re off by 2310 USD on average, which is significant considering that the prices range from 10000 to 50000.\n", "Let’s try training the model a bit longer: 500 epochs. To keep a record of how well\n", "the model does at each epoch, we’ll modify the training loop to save the per-epoch\n", "validation score log for each fold." @@ -1504,7 +1506,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "metadata": {}, "outputs": [ { @@ -1512,7 +1514,9 @@ "output_type": "stream", "text": [ "Processing fold #0\n", - "Processing fold #1\n" + "Processing fold #1\n", + "Processing fold #2\n", + "Processing fold #3\n" ] } ], @@ -1552,7 +1556,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 9, "metadata": {}, "outputs": [], "source": [ @@ -1569,10 +1573,24 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 15, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnk0lEQVR4nO3de3xcdZ3/8ddnLrlfmjZJG5peKC0gtNBC5CZiQUGoKCyyKrouKrtV1/2tPrysuBfZdS8/XRV3+YGwiCywKt4Q8cICXQSKgEALbSmX3lua9JKmSdqkSXObz++PcyadJJNpSDOZkr6fj8c85sz3nDnne9LpvOd7Lt+vuTsiIiKDRXJdAREROTopIEREJC0FhIiIpKWAEBGRtBQQIiKSlgJCRETSylpAmNkMM3vMzF4xs5fN7LNh+WQzW2ZmG8LnimHef224zAYzuzZb9RQRkfQsW/dBmFkNUOPuL5hZKbASuBL4GNDs7l83s+uBCnf/8qD3TgZWAHWAh+89091bslJZEREZImstCHff6e4vhNNtwKvAdOAK4O5wsbsJQmOwdwPL3L05DIVlwKXZqquIiAwVG4+NmNlsYBHwLDDV3XeGs3YBU9O8ZTqwPeV1fViWUWVlpc+ePfuI6ioicixZuXJlk7tXpZuX9YAwsxLgPuBz7r7fzPrnubub2REd4zKzpcBSgJkzZ7JixYojWZ2IyDHFzLYNNy+rVzGZWZwgHH7o7r8Ii3eH5yeS5yka07y1AZiR8ro2LBvC3W939zp3r6uqShuCIiIyCtm8ismA7wOvuvuNKbN+BSSvSroWeCDN2x8GLjGzivAqp0vCMhERGSfZbEG8DfgocJGZrQofS4CvAxeb2QbgXeFrzKzOzO4AcPdm4J+A58PH18IyEREZJ1m7zDUX6urqXOcgRERGzsxWuntdunm6k1pERNJSQIiISFoKCBERSUsBAdz06AaeWL8n19UQETmqKCCAWx/fxFMbm3JdDRGRo4oCAohGjN6+iXM1l4jIWFBAEAREYgJd7isiMhYUEIQtiEQi19UQETmqKCAIAqJP+SAiMoACAoia0acWhIjIAAoI1IIQEUlHAUEyIJQQIiKpFBBALGLoKlcRkYEUEEBELQgRkSEUEIQtiISaECIiqRQQQMQUECIigykggFhUASEiMpgCgqAF0auAEBEZQAFBcA5CfTGJiAykgCC4ikm9uYqIDBTL1orN7E7gcqDR3eeHZT8BTgoXmQS0uvvCNO/dCrQBfUDvcANqj5VYxOjRrdQiIgNkLSCAu4CbgXuSBe7+weS0mX0b2Jfh/Re6+7iM4hONGJ09akGIiKTKWkC4+3Izm51unpkZ8AHgomxt/42IRoyETlKLiAyQq3MQbwd2u/uGYeY78IiZrTSzpdmuTFRXMYmIDJHNQ0yZXAPcm2H++e7eYGbVwDIze83dl6dbMAyQpQAzZ84cVWWiupNaRGSIcW9BmFkMuAr4yXDLuHtD+NwI3A+clWHZ2929zt3rqqqqRlUnBYSIyFC5OMT0LuA1d69PN9PMis2sNDkNXAKszWaFohGjT/dBiIgMkLWAMLN7gWeAk8ys3syuC2d9iEGHl8zsODN7MHw5Ffi9ma0GngN+6+4PZaueoBaEiEg62byK6Zphyj+WpmwHsCSc3gycnq16paOAEBEZSndSkxyTWgEhIpJKAYF6cxURSUcBgcaDEBFJRwFBckxqBYSISCoFBMkxqRUQIiKpFBBoTGoRkXQUEKgFISKSjgICtSBERNJRQBDeB6GT1CIiAygggGgkgjsaE0JEJIUCAoiGfwW1IkREDlFAELQgAJ2HEBFJoYAgpQWhgBAR6aeA4FALQsOOiogcooAAohY86yS1iMghCgggGlULQkRkMAUEwX0QAAldxSQi0k8BQXAnNagFISKSSgFB0BcT6ByEiEgqBQRqQYiIpKOA4FALQvdBiIgckrWAMLM7zazRzNamlP2DmTWY2arwsWSY915qZuvMbKOZXZ+tOibFFBAiIkNkswVxF3BpmvLvuPvC8PHg4JlmFgVuAS4DTgGuMbNTslhPIqaAEBEZLGsB4e7LgeZRvPUsYKO7b3b3buDHwBVjWrlB1IIQERkqF+cg/tLM1oSHoCrSzJ8ObE95XR+WpWVmS81shZmt2LNnz6gqFE0GhO6DEBHpN94BcStwArAQ2Al8+0hX6O63u3udu9dVVVWNah39AZFIHGl1REQmjHENCHff7e597p4AvkdwOGmwBmBGyuvasCxrDgVENrciIvLmMq4BYWY1KS//CFibZrHngXlmdryZ5QEfAn6VzXpF+++DUEKIiCTFsrViM7sXWAxUmlk9cAOw2MwWAg5sBT4ZLnsccIe7L3H3XjP7S+BhIArc6e4vZ6uecCgglA8iIodkLSDc/Zo0xd8fZtkdwJKU1w8CQy6BzRa1IEREhtKd1Kg3VxGRdBQQpLQg+hQQIiJJCghSzkGoBSEi0k8BgXpzFRFJRwGBenMVEUlHAYH6YhIRSUcBgXpzFRFJRwEBxKIKCBGRwRQQHLoPQr25iogcMmxAmNlPU6a/MWjeI9ms1HiL6hyEiMgQmVoQ81KmLx40b3T9ah+lFBAiIkNlCohM35YT6ptUASEiMlSmzvqKzGwRQYgUhtMWPgrHo3LjRQEhIjJUpoDYCdwYTu9KmU6+njCSl7nqTmoRkUOGDQh3v3C4eWYWz051ciPWPx6EAkJEJGnEl7la4J1m9n2gPot1GndR9cUkIjLEYQPCzM4xs5uAbcADwHLg5GxXbDyZGRFTb64iIqky3Qfxr2a2AfgXYA2wCNjj7ne7e8t4VXC8RCOmFoSISIpMJ6n/DFgP3Ar82t27zGzCfoNGI6ZzECIiKTIdYqoB/hl4L7DJzP6b4HLXrI1jnUtRUwtCRCRVpquY+oCHgIfMLB+4nOD+hwYze9TdP5xpxWZ2Z/ieRnefH5Z9kyBwuoFNwMfdvTXNe7cCbUAf0OvudW98196YaMR0H4SISIoRXcXk7l3ufp+7Xw3MJQiOw7kLuHRQ2TJgvrufRnD46isZ3n+huy8cj3CA8BCTTlKLiPQbtgVhZp8/khW7+3Izmz2oLLWTvz8AVx/JNsZSNBLRISYRkRSZWhDfAv4EmAKUAKUpj5Ix2PYngP8ZZp4Dj5jZSjNbmmklZrbUzFaY2Yo9e/aMujLRiG6UExFJlemE8yLgGuA9wErgXuBR9yM/DmNmfwv0Aj8cZpHz3b3BzKqBZWb2mrsvT7egu98O3A5QV1c36rrF1IIQERlg2BaEu6929+vdfSHwfeAK4BUze9+RbNDMPkZw8vojw4WNuzeEz43A/cBZR7LNkYioBSEiMsBI7qSuImhNLCDoYqNxtBszs0uBvwbe5+4dwyxTbGalyWngEmDtaLc5UmpBiIgMlOkk9SeADwAFwM+BD4S/6EfEzO4FFgOVZlYP3EBw1VI+wWEjgD+4+6fM7DjgDndfAkwF7g/nx4AfuftIrpo6IhHTkKMiIqkynYO4g+CX+zbg3cAl4Zc2AO6e8VCTu1+Tpvj7wyy7A1gSTm8GTs9Y6yyIRSL09SkgRESSMgXEsN19T0SRiKkFISKSItOd1E+MZ0VyLaY7qUVEBhjxeBATXUQBISIygAIipBaEiMhACohQ1BQQIiKpDtt1t5mdCHwJmJW6vLtflMV6jTv15ioiMtBIxnb4GXAb8D2C7rcnpGjE6O5L5LoaIiJHjZEERK+735r1muSYhhwVERloJOcgfm1mf2FmNWY2OfnIes3GmYYcFREZaCQtiGvD5y+llDkwZ+yrkztqQYiIDHTYgHD348ejIrkWNbUgRERSjeQqpjjwaeCCsOhx4D/dvSeL9Rp30ajRm9BJahGRpJEcYroViAPfDV9/NCz7s2xVKheiZqgBISJyyEgC4q3untq76u/MbHW2KpQrsYhaECIiqUZyFVOfmZ2QfGFmc5iA90NEIobyQUTkkJG0IL4EPGZmmwEjuKP641mtVQ6oBSEiMtBIrmJ61MzmASeFRevcvSu71Rp/QW+uua6FiMjRI9OQoxe5++/M7KpBs+aaGe7+iyzXbVwFvbkqIUREkjK1IN4B/A54b5p5DkyogIioN1cRkQEyjSh3Qzj5NXffkjrPzCbczXMaD0JEZKCRXMV0X5qyn49k5WZ2p5k1mtnalLLJZrbMzDaEzxXDvPfacJkNZnZtumXGUlRjUouIDDBsQJjZyWb2fqDczK5KeXwMKBjh+u8CLh1Udj3wqLvPAx4NXw/e9mTgBuBs4CzghuGCZKxoPAgRkYEynYM4CbgcmMTA8xBtwJ+PZOXuvtzMZg8qvgJYHE7fTdB1x5cHLfNuYJm7NwOY2TKCoLl3JNsdDQWEiMhAmc5BPAA8YGbnuvszY7jNqe6+M5zeBUxNs8x0YHvK6/qwLGuikaCrDXfHzLK5KRGRN4WR3Cj3opl9BjiVlENL7v6JI924u7uZHdHPdjNbCiwFmDlz5qjXEw1DoS/hxKIKCBGRkZyk/m9gGsFhnyeAWoLDTKO128xqAMLnxjTLNAAzUl7XhmVDuPvt7l7n7nVVVVWjrlQ0DAWNCSEiEhhJQMx1978HDrj73cB7CE4ej9avODQI0bXAA2mWeRi4xMwqwpPTl4RlWZNsQSR0JZOICDCygEiO+9BqZvOBcqB6JCs3s3uBZ4CTzKzezK4Dvg5cbGYbgHeFrzGzOjO7AyA8Of1PwPPh42vJE9bZEo2oBSEikmok5yBuD3/F/z3Br/8S4KsjWbm7XzPMrHemWXYFKWNMuPudwJ0j2c5YSAaERpUTEQmMpLO+O8LJJ5hg41CniqkFISIyQKbO+j6f6Y3ufuPYVyd3ImpBiIgMkKkFURo+nwS8leDwEgQ3zT2XzUrlgloQIiIDZbpR7h8BzGw5cIa7t4Wv/wH47bjUbhxFUu6DEBGRkV3FNBXoTnndTfq7n9/UkjfHKSBERAIjuYrpHuA5M7s/fH0lQSd8E0p/C0L3QYiIACO7iulfzOx/gLeHRR939xezW63xF4sEjSm1IEREApmuYipz9/1h19tbw0dy3uRs37g23qLhwTYFhIhIIFML4kcE3X2vJBhiNMnC1xPqnoioWhAiIgNkuorp8vB5wg0vmo5aECIiA2U6xHRGpje6+wtjX53c6W9B6CS1iAiQ+RDTtzPMc+CiMa5LTkV1H4SIyACZDjFdOJ4VybVkZ30KCBGRwEjugyDs5vsUBo4od0+2KpULCggRkYEOGxBmdgOwmCAgHgQuA35PcAPdhKGAEBEZaCRdbVxNMH7DLnf/OHA6waBBE4oCQkRkoJEERKe7J4BeMysjGEN6xmHe86YTU0CIiAwwknMQK8xsEvA9gpvm2gmGEZ1Qkn0xqbtvEZFApvsgbgF+5O5/ERbdZmYPAWXuvmZcajeOkr25JnQfhIgIkLkFsR74lpnVAD8F7p2InfQlqQUhIjLQsOcg3P0/3P1c4B3AXuBOM3vNzG4wsxNHu0EzO8nMVqU89pvZ5wYts9jM9qUs89XRbm+kYhpyVERkgJF0970N+AbwDTNbBNwJfBWIjmaD7r4OWAhgZlGgAbg/zaJPJvuDGg/Jq5h6+hLjtUkRkaPaYa9iMrOYmb3XzH4I/A+wDrhqjLb/TmBTGEI5lR8L/hQ9fWpBiIhA5pPUFwPXAEuA54AfA0vd/cAYbv9DwL3DzDvXzFYDO4AvuvvLY7jdIfLCgOjq7cvmZkRE3jQyHWL6CsGYEF9w95ax3rCZ5QHvC7cz2AvALHdvN7MlwC+BecOsZymwFGDmzJmjrk8yILp7dYhJRAQyn6S+yN3vyEY4hC4DXnD33Wm2vd/d28PpB4G4mVUOU8/b3b3O3euqqqpGXZm8aLIFoYAQEYGR3UmdLdcwzOElM5tmFlx3amZnEdRzbzYrE4tGiEZMLQgRkdCIenMda2ZWDFwMfDKl7FMA7n4bQf9PnzazXqAT+JB79u9gy49FdA5CRCSUk4AIT3RPGVR2W8r0zcDN412vvFhELQgRkVAuDzEddfKiEZ2DEBEJKSBS5MfVghARSVJApFALQkTkEAVEivxYVAEhIhJSQKTI01VMIiL9FBApdBWTiMghCogU+bEI3erNVUQEUEAMkB+L0NWjgBARAQXEAPmxqFoQIiIhBUQKnaQWETlEAZEiL6qT1CIiSQqIFPlx3SgnIpKkgEihFoSIyCEKiBT58QgHe/oYh57FRUSOegqIFEV5MRKuUeVEREABMUBxXhSAA129Oa6JiEjuKSBSFOcH4yd1dOtSVxERBUSKkjAg2tWCEBFRQKQq6m9BKCBERBQQKUryg3MQ7V06xCQikrOAMLOtZvaSma0ysxVp5puZ3WRmG81sjZmdke06FeUFLQidpBYRgViOt3+huzcNM+8yYF74OBu4NXzOmuQ5CAWEiMjRfYjpCuAeD/wBmGRmNdncYLECQkSkXy4DwoFHzGylmS1NM386sD3ldX1YljVFyfsgdJmriEhODzGd7+4NZlYNLDOz19x9+RtdSRguSwFmzpx5RBXKj0WIRUwtCBERctiCcPeG8LkRuB84a9AiDcCMlNe1Ydng9dzu7nXuXldVVXVEdTIzivKiCggREXIUEGZWbGalyWngEmDtoMV+BfxpeDXTOcA+d9+Z7bpNKsqjpaMn25sRETnq5eoQ01TgfjNL1uFH7v6QmX0KwN1vAx4ElgAbgQ7g4+NRscqSPJrau8ZjUyIiR7WcBIS7bwZOT1N+W8q0A58Zz3oBVJbks3XvgfHerIjIUedovsw1JypL89nb3p3raoiI5JwCYpDKknyaO7rp7dOYECJybFNADFJVkoc7NHeoFSEixzYFxCCVJfkA7GnTiWoRObYpIAY5blIhAPUtnTmuiYhIbikgBpk1pQiA7c0dOa6JiEhuKSAGmVSUR1lBjG17FRAicmxTQKQxa0ox29SCEJFjnAIijTlVxWzc3ZbraoiI5JQCIo0F08vZse+grmQSkWOaAiKNBdPLAXipoTW3FRERySEFRBrzp5eTF4uwfP1wo6GKiEx8Cog0ivNjvPPkan6zZqe63BCRY5YCYhhXLDyOpvYunt60N9dVERHJCQXEMBafVE1pQYz7XqjPdVVERHJCATGMgniU959Ry2/X7GTXvoO5ro6IyLhTQGRw3fnHk3Dnrqe35roqIiLjTgGRwYzJRSxZUMOdv9/Cdx/fSDDInYjIsUEBcRj/fOV85k8v498eWsfDL+/OdXVERMaNAuIwJhXl8dNPnssJVcV86WerWVPfmusqiYiMCwXECMSiEe7+xFmUFcZ5381PceUtT7F8/Z5cV0tEJKvGPSDMbIaZPWZmr5jZy2b22TTLLDazfWa2Knx8dbzrOVhtRRE3f3gRsYixansrf3bPCh55eRe/XbOTf/z1y7zwegvurvMUIjJh2Hh/oZlZDVDj7i+YWSmwErjS3V9JWWYx8EV3v/yNrLuurs5XrFgxltUdYntzBy0d3Xz4e8/S3tXbXx6NGAuml9O4/yC3fOQMTqgu4ZsPreOyBdM4d84UEh4sM5zWjm5KC+IZlxGRI+PumI39/7G97V30uVNdWgDAr1fv4Ow5k6kuLWBvexeTivLS/t/e19lDeWGcHz67jdqKIk49royygjjxqNHVm6AgHk27vZYD3axp2Ed9SwdXLaolFjXi0dH93jezle5el3Zern/xmtkDwM3uviylbDFHaUAk7T/Yw42PrOeUmjLeevxkLvzW4/3zygpiRCJGa0cPAFOK8+jqTfDpxSfw+LpG+hJOS0cPX79qAfFYhBVbm/nXB1/jU+84gesvOxmARMIxY8iHebgPeHtXL109fbz4eivvfEv1gGV6+xL8es0OliyoIT+W/gOXXPfgD2VTexd72rp4S03ZqP5ObyY793Xy4Eu7+MTbZmflS2S0jvRL7cs/X0NRfpQb3nsqQPj566a8MD7gS8Xd2d7cycxwVMVM9nX2sG5XG2cdP7m/bNe+g+THIlQU57G3vYvJxXm0d/VyoKuPaeUFQ9bx+w1N5MUi9PYlOHvOFKIR476V9dzzh218+49Po6a8kIfW7qK1s4c/WjSdgz19VBTlkR+L8HpzB7Mri/vXtXz9Hm5ctp6vXXEqc6pKePH1FuZVl/KLF+u5YF4VRXlRmtq7ue6u57nrE2cxf3oZD6zawbtPnUZZQYwXXm/l2S17mVZWwNzqEr6zbD0VxXnMnlJMWUGM0oI4sagxe0oxP1mxnSsXTufxdY3UTCrkqQ1N/O+ruynOj/Gx82azbe8BfrlqB+WFceZVl7BiWwsAf3xmLe86ZSoXzKviifV7+Jv7X6L5QDcnTyvltV3B8AIF8QgVRXnMqChi7Y59LL1gDq0dPUyfVMiahn2UF8Y4bfokvvv4RramDGp25qwKfvrJc0f1A/OoDQgzmw0sB+a7+/6U8sXAfUA9sIMgLF4eZh1LgaUAM2fOPHPbtm3ZrfQwHl/XCEB+LMrfP7CWrU0H+OtLT2LdrnY6e3p58KVdI1rPnKpiNu85QEl+jNKCGGcdP5llr+ymujSfBbWTeHxdI+9ZUMNp4fTOfQd5+7xKvvv4pv51fPPq06gszef1vR288y3V3PLYJu597nU+es4s3jZ3Ck+sb6K8MM5J00ooK4iTH4syu7KIB1bt4JsPr2P1Vy+hvCgOwKX/vpzXdrWx4u/eRV4swmOvNbK1KWhFXXLKVM6bWzmg/vs6e3jstUYW1JYzp7KYLU0HSDjMrS4B4GBP35BfRWsb9jFzShGl+bHDfhFu2N1Gd1+CU48Letx1d3btP8i0sgK+/ch6aisK+dBZM9O+96G1uyjJjzG1LJ+51SXs3t/F1LJ8Nu05wMbGdm56dAOv7NzPg3/1dk45roy2g0HAl+THeGXnfp7d3ExeLMJHzp5JfUsnj7yye0iY9PQliJpx19NbeXpTEzd/+Az2tHXx2q42Fp9URTwa4ZUd+/niz1bzkXNmMntKMXmxCI++2sjnLz6ReNT41eodbNvbQWlBjNlTivnCz1Zz2fxptB3s5TMXzuXvfvkSJ08r42/f8xYK4lF27z/I2oZ9lBXGueWxjRTnxZgxuYjl6/ewfncbvYng//g7Tqzi4lOm8uPnX2dtQ/9/N75w8YkU58f42m+CRvzb5x36Ny0vjLNgejkXnzKVlxr2cevjm0i4s353OwDvO/045lWX8MT6PazY1kJpfoxzT5jCsld3E49G6O4N+jLLj0X4k3Nm8fDLu/pD5RcvNPRvp6IozoGuPrrDvs+K86Ic6O4b8m8YMaguLWDX/oMsPqmK7t4Eu/cfZNOeA/3L5McidPUO34daYTzKidNKWb29lfxYhCnFeewYdDNsxCDxBr4aT6stZ0drJ03t3YddNh41evre+PduNGL0hZXKj0V4/5m1HD+lmG8+so4P1NXyd+85ZdgWRyZHZUCYWQnwBPAv7v6LQfPKgIS7t5vZEuA/3H3e4dY5ni2Iw+ntSxBL+XX21MYmmg90U1IQY39nD1Wl+aza3kpeNMJj6xp58fVWDvb09X8o82MRygvjNLZ1BV+0ew/g4WGqqFn/f6RUVyw8jgdW7RhR/VI/bINdtWg6J00rpc+df3toXcb1zJ9extyqEuZNLWX97jbW7Wrr/zV05qwKVoa/ni4/rQaAZa/s5pMXzKG0IM663W3c90I9qR/Bq86YzpmzKvjliw2UF8bp6k0Qixibmw4wc3IRT25oIh41pk8qpLq0gNbObtbvbh+wrcnFefxxXS2v7+1gT1sX58+r5LWdbTz08qGQnj2liK17O5g+qZCG1s6M+7hgejlbmw7QFh5SnDWlaMCQtCdUFXPytDIOdPfy3JZmOlK+2E6rLWf97jYO9iT6l039Mks1qSjOlOK8YeenWz7ZSs1kbnUJGxvbR7TOpPLCOPs6D607+YUZixi9Cedtc6fQ0NJJfUsnvQmnrCDGtPICtu3toKs3wbzqEiqK8nhua/OA9RbGo3T29JEXjfR/hi8/rYanN+2l+UA3b6kp45tXn8a3HlnH4+v2UJof40/OncUP/rCNC+ZVUVYY48XXW/s/YzMmF3J8ZQmxiHHi1FJue2ITZ86qoLIkj6b2bq49bzb//r/rcQ9a9jXlhcRjEZ7f0kxxfpTmA9040NrRw1VnTGfJ/Bp27OvkopOreWpjE4tPqqa+pYPntrSQF4tw5++3UFEc5x0nVrFkQQ2PvLybZzbv5QfXnU3bwR5WbW+ltqKIiMEvVzUwa3Ix7z+zlo2N7cypKua5Lc08uaGJ3r4Ef37BHOpbOti9v4tLT53GPc9sJRoxOrr7yItF2LnvIB3dvbx9XhXxqDGvurQ//KrL8vuPBgz+rnmjjrqAMLM48BvgYXe/cQTLbwXq3D1j/9tHU0CMRl/CeXpTE/FohHnVJUQjxsbG4Mtve3MnZYUxygvj1Ld08sT6PUTMKC+M09zRzaIZk5g/vZwv/mw1q7e3MruymNqKQtbvbmPb3g4+es4sXmrYx/L1e/jw2bP44FtncO9zr1OaH+P7T22hs7uPOVUltB3sob7l0BfmiVNLmFNZ0v/l+k9XzufKhccB8L3lm1ldv4/V9a39X1R50QhXLjqOaMRYsbWFaMTYtreDg73Bl8Jwv+wWzpjEqu2tA8rKC+O0HewZ8Etu9pQiTqgqIeHO1r0dbGkKvkyjEeMDdTOob+ngyQ3pPyZvqSnj+MoitjR18OrO4Bf0tLIC3n/mdKZPKuLOp7b0f5EW5UXp6O5j4YxJxCJGnzsfO282P35uO89sPtSB43knTKEv4by8Yz/lhXHeNncK5YVxZk4ppqmti/96agvnnVDJ5qZ2mg/08JaaUhpaOvn2B06ntaOHL/5sNW0He7n2vFm0dPSwp62Lk6eV0tDayczJRUwuzuPS+dN4dWcbNeUFLN+whxOrS4nHIvzfB1/ltV1tXLVoOufPq2RL0wHee3rwt9/adICu3gRlBUGdHnxpF+eeEHypTy3P5wfPbKOiOI+3za1kY2M7JfkxWjq6eevsyZQUxCgriHPHk5tpPtDNBSdW8eSGPXT1JPjE+cezbW8HZx0/mWjEaO/qZV9ncAgEoL6lg6c2NvFHi2rJiwVfWk3tXVQU5dHTFxy+3Np0gMK8KNWl+WxuOsAJVSV09yaIRw13iISHSV6q38fc6hIK86J0hZ+fZGutpy/Bs5ubWTRzEsX5sf5/j9f3dlBbUdi/DoDu3gS9iQRFeYeWS+pL+Bs6LJP8vjyaDkEeqaMqICz4y94NNLv754ZZZhqw293dzM4Cfg7M8sNU9s0eEGNhuHMXb0RrRzddvQncYWpZPgmHDY1tnDwt/XmIju5etu3t4MSppfQmEkPOc/QlnN5EgrxohMa2Lpa9spt3nFjF1LICdrR2UpgXpaokn9ebOyjOj/H0pibOO6GSycV57O8MAqvpQBe1kwqZFR6SSTrYk/ziCPY52Sra0drJf/9hG+87/The2bmfS06ZSnlhHDOjuzdBY9tBaiuGHmvf3txBfjzCpMI84lFL+3fsSzjrdrUxb2rJiE8M9vQFLaHB6+vo7qUv4ZQWxEe0nsHr3NfZQ2VJ/ht+r0jS0RYQ5wNPAi8ByZ+TfwPMBHD328zsL4FPA71AJ/B5d3/6cOtWQIiIvDGZAmJomyvL3P33QMaft+5+M3Dz+NRIRETS0Z3UIiKSlgJCRETSUkCIiEhaCggREUlLASEiImkpIEREJC0FhIiIpJXz3lzHkpntAUbTW18lkLEbjwlI+3xs0D4fG45kn2e5e1W6GRMqIEbLzFYMdyfhRKV9PjZon48N2dpnHWISEZG0FBAiIpKWAiJwe64rkAPa52OD9vnYkJV91jkIERFJSy0IERFJ65gPCDO71MzWmdlGM7s+1/UZK2Z2p5k1mtnalLLJZrbMzDaEzxVhuZnZTeHfYI2ZnZG7mo+Omc0ws8fM7BUze9nMPhuWT9h9BjCzAjN7zsxWh/v9j2H58Wb2bLh/PzGzvLA8P3y9MZw/O6c7MEpmFjWzF83sN+HrCb2/EIysaWYvmdkqM1sRlmX1831MB4SZRYFbgMuAU4BrzOyU3NZqzNwFXDqo7Hrg0XB870fD1xDs/7zwsRS4dZzqOJZ6gS+4+ynAOcBnwn/LibzPAF3ARe5+OrAQuNTMzgG+AXzH3ecCLcB14fLXAS1h+XfC5d6MPgu8mvJ6ou9v0oXuvjDlktbsfr7d/Zh9AOcSjIudfP0V4Cu5rtcY7t9sYG3K63VATThdA6wLp/8TuCbdcm/WB/AAcPExts9FwAvA2QQ3TcXC8v7POfAwcG44HQuXs1zX/Q3uZ234ZXgRwdj2NpH3N2W/twKVg8qy+vk+plsQwHRge8rr+rBsoprq7jvD6V3A1HB6Qv0dwsMIi4BnOQb2OTzcsgpoBJYBm4BWd+8NF0ndt/79DufvA6aMa4WP3L8Df82hIYunMLH3N8mBR8xspZktDcuy+vke9yFH5ejg7m5mE+4SNjMrAe4DPufu+80OjW47UffZ3fuAhWY2CbgfODm3NcoeM7scaHT3lWa2OMfVGW/nu3uDmVUDy8zstdSZ2fh8H+stiAZgRsrr2rBsotptZjUA4XNjWD4h/g5mFicIhx+6+y/C4gm9z6ncvRV4jOAQyyQzS/4ATN23/v0O55cDe8e3pkfkbcD7zGwr8GOCw0z/wcTd337u3hA+NxL8EDiLLH++j/WAeB6YF14BkQd8CPhVjuuUTb8Crg2nryU4Tp8s/9PwyodzgH0pzdY3BQuaCt8HXnX3G1NmTdh9BjCzqrDlgJkVEpx3eZUgKK4OFxu838m/x9XA7zw8SP1m4O5fcfdad59N8P/1d+7+ESbo/iaZWbGZlSangUuAtWT7853rEy+5fgBLgPUEx23/Ntf1GcP9uhfYCfQQHH+8juDY66PABuB/gcnhskZwNdcm4CWgLtf1H8X+nk9wjHYNsCp8LJnI+xzux2nAi+F+rwW+GpbPAZ4DNgI/A/LD8oLw9cZw/pxc78MR7Pti4DfHwv6G+7c6fLyc/K7K9udbd1KLiEhax/ohJhERGYYCQkRE0lJAiIhIWgoIERFJSwEhIiJpKSBEDsPM+sIeNJOPMev118xmW0qPuyJHE3W1IXJ4ne6+MNeVEBlvakGIjFLYP/+/hX30P2dmc8Py2Wb2u7Af/kfNbGZYPtXM7g/HblhtZueFq4qa2ffC8RweCe+Ixsz+yoLxLdaY2Y9ztJtyDFNAiBxe4aBDTB9MmbfP3RcANxP0Mgrw/4C73f004IfATWH5TcATHozdcAbBHbEQ9Nl/i7ufCrQC7w/LrwcWhev5VHZ2TWR4upNa5DDMrN3dS9KUbyUYrGdz2FHgLnefYmZNBH3v94TlO9290sz2ALXu3pWyjtnAMg8GfMHMvgzE3f2fzewhoB34JfBLd2/P8q6KDKAWhMiR8WGm34iulOk+Dp0bfA9BfzpnAM+n9FYqMi4UECJH5oMpz8+E008T9DQK8BHgyXD6UeDT0D/IT/lwKzWzCDDD3R8DvkzQTfWQVoxINukXicjhFYYjtiU95O7JS10rzGwNQSvgmrDs/wD/ZWZfAvYAHw/LPwvcbmbXEbQUPk3Q4246UeAHYYgYcJMH4z2IjBudgxAZpfAcRJ27N+W6LiLZoENMIiKSlloQIiKSlloQIiKSlgJCRETSUkCIiEhaCggREUlLASEiImkpIEREJK3/D7nw54XGVMDuAAAAAElFTkSuQmCC\n", + "text/plain": [ + "<Figure size 432x288 with 1 Axes>" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], "source": [ + "import matplotlib.pyplot as plt\n", "plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n", "plt.xlabel(\"Epochs\")\n", "plt.ylabel(\"Validation MAE\")\n", @@ -1597,9 +1615,22 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 16, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABBgElEQVR4nO2dd3wc1bXHf2f7qkuWe5N7xwU3jGk2xWBqAg9IIIRACEleQiDh0cOjtwQCCSWE5hcgVNMxYIxtbMBFBvfeq2xJVi+rLff9MXNnZ2ZnVytbq5U15/v56KPRzN3dO6vde+7pJIQAwzAMY18c6Z4AwzAMk15YEDAMw9gcFgQMwzA2hwUBwzCMzWFBwDAMY3Nc6Z5ASyksLBRFRUXpngbDMMwxxYoVK8qEEJ2trh1zgqCoqAjFxcXpngbDMMwxBRHtineNTUMMwzA2hwUBwzCMzWFBwDAMY3NYEDAMw9gcFgQMwzA2hwUBwzCMzWFBwDAMY3NsIwg2ldTgr19sQnltIN1TYRiGaVfYRhBsK63F37/ailIWBAzDMAZsIwg8TuVWA8FImmfCMAzTvrCNIPC6lVttCrMgYBiG0WMbQSA1gqYQCwKGYRg99hEELhYEDMMwVthOEARC4TTPhGEYpn1hG0HgdTkBAAHWCBiGYQzYSBCwaYhhGMYK2wgCzUfAUUMMwzAG7CMIOGqIYRjGEtsIAplHwD4ChmEYI7YRBKwRMAzDWGMbQeByOuAgFgQMwzBmbCMIAMVhzM5ihmEYI7YSBF6XE4EgJ5QxDMPosZUgYI2AYRgmFnsJAqeDo4YYhmFM2EoQeF0OdhYzDMOYsJUg8LhYI2AYhjFjK0HAGgHDMEwsKRMEROQjomVEtIqI1hHRPRZjbiKi9US0mojmEVHfVM0HUJ3FLAgYhmEMpFIjCACYJoQYDWAMgBlENNk05gcA44UQxwF4B8CjKZwPRw0xDMNYkDJBIBRq1T/d6o8wjZkvhKhX/1wCoFeq5gPIqCHOI2AYhtGTUh8BETmJaCWAQwDmCiGWJhh+DYA5cZ7nOiIqJqLi0tLSI56P1+Vk0xDDMIyJlAoCIURYCDEGyk5/IhGNtBpHRFcAGA/gsTjP87wQYrwQYnznzp2PeD7sI2AYhomlTaKGhBCVAOYDmGG+RkSnA7gDwPlCiEAq58GCgGEYJpZURg11JqI89dgP4AwAG01jxgL4JxQhcChVc5Gws5hhGCYWVwqfuzuAWUTkhCJw3hJCfExE9wIoFkJ8CMUUlAXgbSICgN1CiPNTNSGvy4FAkAUBwzCMnpQJAiHEagBjLc7/WXd8eqpe3wqPy4EAawQMwzAG7JVZ7FR8BEKI5gczDMPYBFsJAo9Lud1gmAUBwzCMxJaCgJPKGIZhothKEHhdTgDct5hhGEaPrQSB1Ag4hJRhGCaKvQSBUxUErBEwDMNo2EsQaD4CFgQMwzASWwkCr4s1AoZhGDO2EgSsETAMw8RiS0HAGgHDMEwUWwkCL+cRMAzDxGAzQcB5BAzDMGZsJQg4j4BhGCYWewkCziNgGIaJwV6CgKOGGIZhYrCVIPC5FR9BY5CdxQzDMBJbCYJsn9KHp6YxlOaZMAzDtB9sJQjcTgd8bgdqGoPpngrDMEy7wVaCAACyfW7WCBiGYXTYUBC4WBAwDMPosJ0gyPG5Uc2mIYZhGA3bCYJsnwvVrBEwDMNo2E4Q5Pjc7CxmGIbRYTtBwD4ChmEYIzYVBKwRMAzDSGwnCHJ8bjQGI5xdzDAMo2I7QTCwSxYAYN3+6jTPhGEYpn1gO0FwfFE+AGDFrsNpngnDMEz7wHaCoEu2Dz3z/KwRMAzDqKRMEBCRj4iWEdEqIlpHRPdYjPES0ZtEtJWIlhJRUarmoyfH70Z9E/sIGIZhgNRqBAEA04QQowGMATCDiCabxlwDoEIIMRDAEwAeSeF8NLwuB/ckYBiGUUmZIBAKteqfbvVHmIZdAGCWevwOgOlERKmak8TrcnDUEMMwjEpKfQRE5CSilQAOAZgrhFhqGtITwB4AEEKEAFQB6GTxPNcRUTERFZeWlh71vHxuJ2sEDMMwKikVBEKIsBBiDIBeACYS0cgjfJ7nhRDjhRDjO3fufNTz8rocCLBGwDAMA6CNooaEEJUA5gOYYbq0D0BvACAiF4BcAOWpno/X7eQG9gzDMCpxBQERvaU7fsR07YvmnpiIOhNRnnrsB3AGgI2mYR8CuEo9vhjAV0IIsx+h1WEfAcMwTJREGsEg3fEZpmvJ2Ge6A5hPRKsBLIfiI/iYiO4lovPVMS8C6EREWwHcBODWJOd9VHDUEMMwTBRXgmuJdubN7tqFEKsBjLU4/2fdcSOAS5p7rtaGncUMwzBREgmCDCIaC0Vr8KvHpP7422JyqYJNQwzDMFESCYIDAB5Xj0t0x/LvYxavy4lQRCAUjsDltF2VDYZhGANxBYEQ4rR414jInZrptA1et7L4N7EgYBiGST58lBSmE9GLAPamcE4px+dSbjsQZD8BwzBMs4KAiCYT0VMAdgH4AMDXAIamemKpxOt2AgAaQ+wnYBiGSZRH8CARbQHwAAAZAVQqhJglhKhoqwmmAi9rBAzDMBqJnMXXAtgM4FkAHwkhAkSU8mSvtsDrUjQCDiFlGIZJbBrqDuB+AOcB2EZE/4YSRppIeBwTaBoBm4YYhmESRg2FAXwG4DMi8gI4F0r+wD4imieE+EkbzbHV8UkfAZuGGIZhEpqGNIQQAQDvAniXiLIBXJTSWaUYGT7KSWUMwzAJBAER3dSWE2lLsn3Kbdc0htI8E4ZhmPSTSCP4C4CVAOZAaTup7xx2TDuN8zM8AICK+qY0z4RhGCb9JBIEYwFcDmAmgBUA/gNgXluUiU41eRlKYnRFHQsChmGYuFFDQohVQohb1Q5jL0LpL7xeV0L6mMXrciLL68Jh1ggYhmGSyizuDEU7GAWltMShVE+qLcjPdLNGwDAMg8TO4l8A+C8APgDvAPgvIUSHEAIAUJDhweH6YLqnwTAMk3YS+QheALAWSo2hswCcSRT1FwshjmkTUV6Gh53FDMMwSCwI4pah7ggUZHqw9VBtuqfBMAyTdhJlFi9sy4m0NTk+F2oDnEfAMAxj264sGV4X6gIhdIBoWIZhmKPCtoIgy+tCKCLQFOZ6QwzD2BvbCoIMj1J4rj7A9YYYhrE3zRadI6LBAG4G0Fc/XggxLYXzSjmZHuVWagMh5Gd60jwbhmGY9JFM9dG3ATwH4F8AOsz2OdOr3Hp9U4e5JYZhmCMiGUEQEkI8m/KZtDEZXsU0VNfEkUMMw9ibZHwEHxHRb4ioOxEVyJ+UzyzFSNNQHYeQMgxjc5LRCK5Sf9+sOycA9G/96bQdmVIjYGcxwzA2p1lBIITo1xYTaWtYI2AYhlFIJmrIDeDXAE5WTy0A8E8hxDFdsU36COrZR8AwjM1JxkfwLIDjATyj/hyvnksIEfUmovlEtJ6I1hHRDRZjconoIyJapY65uqU3cKRkqVFDdRw1xDCMzUnGRzBBCDFa9/dXRLQqiceFAPxRCPG92vB+BRHNFUKs1435LYD1Qojz1L4Hm4joNSFEysuC+t1OELFpiGEYJhmNIExEA+QfRNQfSeQTCCEOCCG+V49rAGwA0NM8DEA2KfWtswAchiJAUg4RIdPjYmcxwzC2JxmN4GYA84loO5QG9n0BtMiEQ0RFULqcLTVd+geADwHsB5AN4FIhREzxHyK6DsB1ANCnT5+WvHRCMjxO9hEwDGN7kokamkdEgwAMUU9tEkIEkn0BIsoC8C6APwghqk2XzwKwEsA0AAMAzCWiReZxQojnATwPAOPHj2+1cqGZXi5FzTAME9c0RETT1N8/AjATwED1Z6Z6rlnUiKN3AbwmhJhtMeRqALOFwlYAOwAMbdktHDmZXic+Xn0Alz3/XVu9JMMwTLsjkUZwCoCvAJxncU0AsFrYNVS7/4sANgghHo8zbDeA6QAWEVFXKFrH9uYm3VpkqLkES7YfbquXZBiGaXck6lB2t3p4rxBih/4aESWTZHYigCsBrCGileq52wH0UZ//OQD3AXiFiNZA8T/cIoQoa9EdHAWZailqQKlCKkNKGYZh7EQyK9+7AMaZzr0DJZ8gLkKIxVAW90Rj9gM4M4k5pIRM3cK/o7QOo3rlpmsqDMMwaSOuICCioQBGAMg1+QRyAPhSPbG2IEOnEWwvq2VBwDCMLUmkEQwBcC6APBj9BDUAfpnCObUZEV38UXXDMV0xg2EY5ohJ5CP4AMAHRHSCEKJDhtUEdf2KG4Pcu5hhGHuSjI/gByL6LRQzkWYSEkL8ImWzaiOMgoAzjBmGsSfJlJj4N4BuUJK/FgLoBcU8dMwTDEdtQ40hFgQMw9iTZATBQCHEXQDqhBCzoCSXTUrttNqG7rlRnzebhhiGsSvJCALpRa0kopEAcgF0Sd2U2o7bzh6Gpy4fi8IsD5uGGIaxLckIgueJKB/AXVAKxK0H8GhKZ9VG+D1OnD+6B7wuJ2sEDMPYlmSKzr2gHi7EMd6nOB4+t4N9BAzD2JZECWU3JXpggvpBxxw+txMBNg0xDGNTEmkE2ervIQAmQDELAUpy2bJUTqqt8bnZNMQwjH1JlFB2DwAQ0dcAxqldxkBE/wvgkzaZXRvhczvYWcwwjG1JxlncFYC+h3CTeq7D4HM52UfAMIxtSSaz+P8ALCOi99S/LwTwSqomlA7YNMQwjJ1pViMQQjwApZNYhfpztRDioVRPrC3x6kxD5bUBPLNgKyKRVuuIyTAM065JFDWUI4SoJqICADvVH3mtQAjRYdp66TWCG99aha83l+LkQZ0xsieXpWYYpuOTyDT0OpQy1CugtKaUkPp3h8kp8Lmi4aObSqoBAHXc1J5hGJuQKGroXPV3Mm0pj2n0CWWlNQEAQHldU6KHMAzDdBgSmYbM7SkNCCG+b/3ppIf8DA+CYYHFW8q0ZjUsCBiGsQuJTEN/TXBNAJjWynNJG5dN7I3nFm7DLe+u1s6V1wbSOCOGYZi2I5Fp6LS2nEg6yfa50Svfj1V7q7Rz5bWsETAMYw+SySOAWn56OIwdyv4vVZNKB9k+NwCgT0EGXE7CYTYNMQxjE5oVBER0N4BToQiCTwGcDWAxlESzDkOOX3krCjI9cDkI5XVsGmIYxh4kU2LiYgDTAZQIIa4GMBpKc5oORY6qERRkeuD3cKYxwzD2IRlB0CCEiAAIEVEOgEMAeqd2Wm1Pjl8RBHkZbnhdTgRCEby2dBeWbC9P88wYhmFSSzI+gmIiygPwLyjJZbUAvkvlpNJBjk95K7K8LjSFIggEw7jjvbUAgJ0Pz0zn1BiGYVJKojyCpwG8LoT4jXrqOSL6DECOEGJ1vMcdq3hdTgCAy+HQNAKGYRg7kEgj2AzgL0TUHcBbAP4jhPihbabV9oSFkknmdChF6OqauMQEwzD2IK6PQAjxpBDiBACnACgH8BIRbSSiu4locHNPTES9iWg+Ea0nonVEdEOccacS0Up1zMIjvpOjJByRgsABn8uJyvqgdq2hiXsVMAzTcUmmDPUuIcQjQoixAC6H0o9gQxLPHQLwRyHEcACTAfyWiIbrB6i+h2cAnC+EGAHgkpZNv/WYMqATAGDa0C7wuo1vy77KBkQiAs8t3IYqnYBgGIbpCDQrCIjIRUTnEdFrAOYA2ATgR809TghxQNYjUttcbgDQ0zTsJwBmCyF2q+MOtXD+rcbYPvnY/uA5mNivAF6X8W0prQng6y2leHjORtz3yfo0zZBhmGOZxmAYB6oa0j0NS+IKAiI6g4heArAXwC+h9CkeIIS4TAjxQUtehIiKAIwFsNR0aTCAfCJaQEQriOhnLZp9K+NwEAClP4GexlAYtWpZai5PzTDMkXDtrGKc8NBX6Z6GJYmcxbdB6UnwRyFExZG+ABFlAXgXwB+EENUWr388lIQ1P4DviGiJEGKz6TmuA3AdAPTp0+dIp5I0Zo2gsSmMYFiJInI7k0m9YBiGMbJ4axkAQAgBIkrzbIwkchZPE0K8cJRCwA1FCLwmhJhtMWQvgM+FEHVCiDIAX0PJXDbP5XkhxHghxPjOnTsf6XSSRoaSShqCYTQ0sSBgGCbK0/O3Yo2uUGWyBMMta4N77t8X4U9vr2rx67SElK1qpIi8FwFsEEI8HmfYBwCmqn6IDACTkJwjOqX4TM7ixmAEFfVKETq3s31JcoZh2h4hBB77fBPO+8fiFj9WWheSZe2+aryzYm+LX6clJFV99Ag5EcCVANYQ0Ur13O0A+gCAEOI5IcQGNUltNYAIgBeEEGtTOKeksNIIqhqUaKEQN7VnGNvT0l29nqZQBJne5MYK0TbrTcoEgRBiMZT+xs2NewzAY6max5EQ4yMIhlGhlqVmZzHDME0t3NXraYlG8NI3O7XjVPoW2OBtQUzUUDCMCjV/oJYFAcPYkgc+WY873lsDQNnV69lZVpd0V8PyuiatN3oiSqoacd/H0XD1mhSuPSwILNAnlGV4nGgMhlHV0DKN4JkFW7Fi1+GUzI9hmLbnX4t24LWluwEAgZCx2sCpf1mAU/+yIOYxQgj8+tUV+G5btIrx2U8uwoQHvgSQuGpBvanMzcGqxiOderOwILBAbxryuZ1oCIZR0yjzCJIrN/HoZ5vw42c7XJFWhmEQqxEA0NYIPdWNIcxZW4Lr/q845lpJVSNG3/MFindabxjNa83B6tQ1y2JBYIFHFQQ9cn3wu5UmNbIIXV1TCLe+uxrXzlqezikyDGPB4i1lWLi5tNWft6bRWFpGX504kiCARJqSzWVrAGDd/io0hSPYV2mdbWw2Q1c3pq68TSqjho5ZXA7ln3by4M5YtvMwGoJh1KrSvjYQwhvL9yR8fJgjixgmLVzxolK8oLV7iOgX61A4YtAIElUqrlajDc2RiACw+WAtACAQpxui2TRUa6FxtBasEVgwsEsWXr92Eu69YKSiETSFNTWt0qLo3NLt5Si69RPsragH0PI4YYY5Uh76dAMmPfhluqeRNFUNQcz+/shi4tfvr46xzbcV3++q1I5rAyGDRlCdYIGWgsDjil1qNx+sAQAE4qwXUiN447rJANhZnBamDCyEx+WAz+1ETWMITeEI+nfONIwJRwSenr8Vj89VKmKs3adkGR5NaBnDtIR/fr09pbbj1uaT1Qdw01ur4ppD4lFaE8A5Ty3C7bNbJ80oGI7gkc824rAaFt4cf/1ik3Zc3RAyCCSz2UhPIiEhBYGVvwGI+gh65fsBAK8t2YV1+1ueyZwMLAiawe92oqxO+aJN6FugnXcQ8OWGg3js801YukNx9mR5lb7HQe5uxjCWSDt3RZILsKQxqCyKi7a0jv1//f5qPLtgGxZubr7gsRACFfVNGNI1G4ByD/rF++XFOy0ft7eiHr9UncRW5mIpDM2CYOuhGgghtAjFXL8bXpcD28vq8MnqA83f3BHAgqAZfG4HytSY39G987TzEQEt21giP6xHk3XIMB0Zaee2MrEmQmrZrZXQebC6UX2+WFNTWW0At81eo32fQxGBiAA6ZyvpwNWNQYNp6M3iqM/w3L8v0o4/XLVfO7YSBPI90AuCBZsO4fTHv8YHK/drpqFMjwsRNcNYzqG1YUHQDD63U1Pv8jLcuOvc4Th9WBcAQL3pQ9mgCQLWCBjGCrm4mTdRzSEdqnWt1C3woLq5s4rj/8vnm/CfZbvx4ar9KKlqxCF1bGGWB4BiGopnzlm7r1oTIF2yfdr5sgTJZk3hMIQQiEQE5q4/CEDRFuoCIWR4nHA4SNtcFmaxIEgLUwYUaseZXheumdoPpw1VBIH5QykFATe+b19EIgI7y+rSPY2UkiiEsT0hBUFlQ8tMQy11ElvV6NlzuB4z/va1srhLjcAi4kdWcQiGI5j80DxM/+sCAEAndRGubgjGFQQAsPWQEg2k/58kWhOaQhFc8eJSjPzfz7FF99i6pjAyPMbATtYI0sSlE3prx1leJQTMr5agqDbtagKsEbRLnlmwFaf+ZQG2qM65jkgwcmx85qxMQ3sO1ze70DfGCbHccKAaj8/dHLPwW5lnN5bUYGNJDVbuqcAh1cFeWR/Ex6v3Gx4vw8dlGQj52l3URbiqIZhwYd9UonzOzEKmf2Gm1XAU76rAN1vLUd8U7WBWVhtAXSCkrTkSFgRpwumIFnnK9CrSWQoCc72QhmAYjcEwbn13dcrnVdMYTLq2id2Rzvz9KUzRTzft0S9VGwjF2Mbl4ihNQ3WBEE56dD7ueC9xNFA8QXH2k4vw1LwtMclXDcEwFm0pxaxvd2rnZFz+nsMNOFijfBZe+XYn/vv1H/Czl5Zh0ZZSHKhqgEstNV+809iKpSDTA5eDUF7XhMVbjU5rfTUCuauvN1kMuuf5YMUPuyu14z2HVUFQ14TyugCyfW7DWBYEaWRsnzwAgE9NCvF5lN97TSFwDU0RvP/DPqw6gmYV8fhw1X4tLFXP1Efm4/j7j5348XQiN3sduZNEqJ1poUIIjLz7c9w+e43hvCzDUFUfxP7KBkxTzS7zNyaO3tFrBPJe9aYXc3mHQDCMK19chrs/XKd7DmVh3ltRHxNyu2hLGa58cRlOeOgreNTmU2tNoZpetxN5GR48t3AbPl1TYrj2o3G9tONd5YoZsr4pBKeDcM6obgCAgmRrTwPYXV6P5TsrMLFfgeF8tjc1OcCcWZwEL/xsPD5YuR99O2UAiGoEy3YYa4Q0BMMxu4Cj5ff/+QFAbKZkS51tepbvPIxwRGBy/05HNbdjjXbWHbBVaW2NYPnOw/C7nRjZM/eIHi+/B28W78HkAQW4aKyyUNbpfASzvtupLch6zdsKvUZQ1xTG819vxpBuOdq5qoYguuZEd9x6wRGJCDQEw1i/X+mUu6eiQfMRWCE1AnNkk8fpQH6G29Lxm+GJmnB2lSuJpfVNYWS4ndoGslOmJ+E96lmjbv7OHN4VAPDEpaOxaHNZyspQsyBIgk5ZXvxiaj/tb787Nl0cUHYcrdmvIFW1RS55TimG19pp+O0VgfZnNmltQkn4CLYcrEHPfH+MA9KKZD8jew7XI8Pj1BypEv334MY3V2mCQHMW1wcNu9tmBYFuYa8LhPD0/G2G62YHbqNOcFQ3BvHrV7/Hd9uVCqA7yupQniCPwRFnsfW6HcjyWb93+kfsKq+DEAL1gTAyvErUD6CYlpKhc7YXpTUBnD+6h6YRXDS2l/YepgIWBEeA32MtCP6zbHerRgztV01PzXxHmGaImoY67hsZ0mkEmw/WYFCXLMPuMRwROOOJr3HSoEL8+5pJrfa6Jz06H16XA5vuP9tw3lwOYV9lA3rm+Q3ho1k6QaBffCMRgQc+3YBLJ/TGYDWJS68RHLDw9dz94TqtnSxgDAstr2vShACgCIJE/P2rrZbnvU5HwrLRgOIrqGsKY19lA+qDStSPFFA5cYSImTtnDkOPPD/G9clvsyb37CM4AuJpBK0dNnqgUvnAp8pBZBc0QdDB5IDeLyAj1eauP4gzn/gaH5kyUOVitGhLWbPPay52BijC5dpZxfho1X5EIgKr9lRq16w+92bN+PtdFYZs2aqGIDy6Qmx1TSGc/4/FmLfhIHYfrseLi3fgN699r13Xm3r2W5Sn2FhSY7D7S38AgGbLSPzpzMFxr+mdwB6XI67pd4K6c7/17KEgAt5cvgcNTSH43U7tvc8yOX7j0a8wExOKCprVkloTFgRHgLmDGdC82rexpBqfrkk+PfzMJxbi6leUUtf6xJSWsmR7OeZvaj6Nnjn2aNQtwLKXtrQtP/b5Rs1pCbQsDt+qdtHc9Qfx5YaDeKt4D55duA0XPP0NVuqEgRlzpcwDVQ1oCIYREYqGW1kfNIRZV9YHsXpvFa6ZVYxK1f+lXwf180+mTpH+vSmvTSwIZozsFje0s5/uvNfljGv6nT60C4rvPB1Xn9gPUwcW4rO1JagLhJHpdWr3mZ2kRpDnT96X0FqwIDgCrExDzeUOzPjbIsMORyKEsEx+kSVqASA/gZBpLlrksueX4OqXuXcCAC1Nv71Q0xg0lCFoKXozhdx1NuhCJM/7++KY68lw0MKRKqNyguEIlquNVEoShOOaTUOHqgOacOiW41ObPVn7wL5SI4j0mndzGoGZlmgEhVneuF6kPgUZ2rHH5cBd5w5Hrt+N4jtPx8e/m6pdczkdWtbv+L4F2Fpai9LaAPwel1YeQ7+BtBIKsqhlQRYLgmOCLK8Lf71kNJbePl07Z9WdCFASQ/SaQG0ghMc+34h/f7cTQgj0u+1TPPDJhoSvZyUoJJzF3DzSWRxqZ9m3t85eg9//5wdsLKk+osfrFzt5b3rTRXVjNI6/JZ8TK0FQGwhqzy8FUKkueuY2NUx0waZDaGiKDZo4VBPQ/AM91Wqah+L07f1w5T4A0Q3XnsP1+Mf8qN0+WUEgBcnhukCMWXB49xyM7qVEROX63XG/YzJSEFAEwYVje2LV3WeiMMuL4d1zLB9zXO9cCKFkGGe4ndp773E6kJehmIdyLMxE158yAJvvP9vgO2krWBAcIT8+vhe65vjw+rWT8KtT+scdd82sYoMm8P2uCjw9fxvu+mAdKtTwtBcW7wCgRDf8zzurYnZKibQN/WLAWCO/4+E2SLpav78aH69Obpe/93A0zBBQnKTXzlqOb7c1b8cHjOYSqRk2mD4Pkx+aFzM2HrO+3Yml28vx5JdbtHMyVl9udOqbwprZ5a73o0lg/1m2GxtLqvHzl5fj3o/XxyR4HappxN4KZQHvmacKgjjls3eq4ZdyIX94zkbDdfk8Eqv6O6U1AYTVf3xVQ9DgjP74d1Mx+zdT8H+/mIQ5N5wEIsIINUz2trOHGp5nYJcs7djcU8ARx4Y/rk++dpzhifoIPC6HVlI6x2/tL7DqW9AWcNTQUTJlYCGmDCzEwM5Z2F/ZiEsn9Na+fACw7VCtYfxmXZkD887mpcU78FbxXvTOzzCcDyVYwI5GIxBCtFlUQjqR715raASRiMCdH6zFlZP7YpjFjvCcp5Tqk+ce1yP5+anTqgmE8OWGQxjcNdtQ48pMYzCMCfd/aTC/yDwCc1SLzH5P5nPy1y82YVzffGwvq0OXbC8O1QRQHwwjy+uKCoJAKO7uea+aFburvA69C/yGa0u2H8aS7csARDWCgzWN8LkduOmMwXjwU2Wx757r06KC5GfTPPeNJcZSIYVZnpjY/iXbD2sLcGV9EA4C5DsjcyN8bidy1R36oz8+Dj+d1Af1umqkH/9uKoZ1z8Et7yrajjfJRTrX78aPx/XCu9/vxXmje6BTlgf3f7IBI3rkoGee31CYTnLWiK44f3Tyn5nWhjWCVuKS8b1xw+mD0C3X6Ng122Y3lVgLgkhEaLsv83oVTLCAHY0gaG+mkpQhNYJWuN+d5XV4feluS3+P4SWT8EfIEXLxlr+bs2nvrWiIscHLPAKzRiDRf06emLsZO8vqYj6bjcGIFlopF2tp4pFaan0wHNc3sLVU2fT43c6EbRV75ikbnUPVAXhdTuTqdsd6U4yMXrLyJcgKwECsmaVnnh9f6/oWVzUEm93wZHpdmDKgEBm62j4je+YaIndaslt/4KKRmHvjyThtaBcc1ysPb/3qBPjcTlx9opKPdHzffMP4v18+zjIIpa1gQZBizN3K9BqBPh56f1WDFt9oToBK5BA+mtZ9x4p/Yd3+qlaprplM0lVzyP9nc6F9yZRLlrLig5X78OtXV2iLuFWy046yOhTd+om6wEXfi5E9Fa3kyS+3IBiOxA1v1C/6T87bglP/sgDX/btYOx8KR9AUjmCPaq6SBdaigkDt2d0YintvMnPX5zFG13TLMW6OhnbPhstB2FfZAK/LYSi9UNQpGqVTVtuEX/27WKsVpWfqwKjGlOM3GjbOGtHN8L37Yv3BpJ3liezzsvREMvjcTgxScyD0TO7fCTsfnomh3YzX0mUSkrAgaGP00UD7q6IaQUlVo5bupN+5ju6Vm9g0FIxgZ1kdPltbEndM/MdGv9BCCLy6ZFfKspmPlNV7KzHzqcV4duG25gc3Q6L3MVnkTre5RUFWpj3ria9x+3vGejtr9lYZtMG3V+zFnLUlWtkQK41guboYvr9yHxqaooua3A0X76rA28V7LX1G+ysbcNnzS2LOL9hUisF3zgGg7PSBqDYqQ5brm8K4+e1VmjkmkRYp2ygGgmGDHd9sJuqdn4HzxyhmEI/LYQi97qITGlsP1eLzdUp9/mHdcwy76KmDOkffA5O9/awRXePOsTkSZV1bmYZG98rVOpe1hHF98zF1YCFeu3YSbpg+qMWPb23YR9DG6FV3mTAGACW6SA25U7/17KFYs68KGw7EjyoJhCK45d3VWLrjML648WQtEzMe4Tg10r/bXo4731+LNXur8MjFxyV/QylG2rhlyOKRIDWs1jANSUHpbmYHV90YRA/4selgDTYdrMGDF43Srp33DyWsc5Spjs8BVTgkMg3N/n6fIa9ERqEAwLfbyixNMgs3l8acM2P2LUiNoDYQwtsrYpvN/+LEfnjpmx2Gc9tKFbPSgapGbCuNbnh6F2Rgua6SZ7bPpX1OwxFhqMFjpf2+ff0JmFBUgNpACCPv/hwAkJ/hxi9P6geXRbZvjt+Nbjk+lFQ3wu92Gr5z9184MsG7AGR645tnrMxLH/z3VIuRzTOuTz5evVbJ8D5xYHx/UFvBGkEKmDLAupibuXKg3kxUUtWoKfxyV+dzOeB2UDPO4rBmY33Z9MXUI23W+ggkKQiEEJpTu6UNQ1KNW915H010lDTBxNvNVtQ14aY3V6KmMYjaQAgXPP2N4X+jp7pBagSJTUNynJlEtahkopRVefGwzufwnE470tvXN5bUxJRGBxD3XiTBcCRWEOQoguCbrdYRTKN65cTVitbtrzbE/etj8QFlZy3n3RgMG+Lme6gRRTK6BogWa9PvyLN9btwxczhumTE0RiNoCkUwqKsS7aPPHfnRuJ64YnJfyzlLMtMQutkeYEGQAl6+egLunDks5nwv9QshKxXqox92ltdhkxpPLk0EPrcTLqdD2yXd8d4aPDRng8ERGQhGtHjrz9cdjOtPkDbTJoMgUL78bxfvxV0fKOV6M5MoSNaWSGEVrzEJAHy7tSzhAivfrXAcH8GT87Zg9g/7MPv7ffhuWzlW7anEo59tshwr/zfuJE1DevZW1GOEuqMFYhPcpM+oujEUEzIcr9qs3lG6+3B9jBMZiBUE+RluQ8RTRX1TjG9hRI9c5PrdcevuZHhcePBHoyyvSbqqwsQsCIgIeeri3RAMGzZIP5nYB2/96gRcMCYaQSOFhiuO41bW8DlnVDdkeJwoKszECepmTK/1/va0gQnnCwAZFg7bn08pMlQX7YikTBAQUW8imk9E64loHRHdkGDsBCIKEdHFqZpPW+J1OXHF5L6GGGQA6Kk2pjCHh/btlIFXl+zW7KGy1Z3P7YTbSVrU0IpdFViyrdxQcrgmENQWwcN1TRh4xxz8e8kuDLz9U81mC0SdhXqnmazo+OnaaMKbVeu+QzWN+NW/i1HVwobjrYGMHImnEZRUNeInLyzFze+ssry+rbRWE5xWGsHqvZV4RW1ekul1Qa7v8bKQq02C4Cf/WoJ3LEwnVr6WLQeNocTmXfgBnc+owmQeitfsXb8bjucQ3VRifN38TI/WfxdQegE0BI3/9x55ftyqi6mfUJRv2Nxkely4+PheBpOXHCeRJiyrWlkybLMxGDGYXBwOwsR+BQZbvbzHeJE/8vrpw7pi/b0zkOt341cnD8CTl43BGWoZ5zevm4wBnbMsH6/HZSHg//f8EVh/74xmH3ssk0qNIATgj0KI4QAmA/gtEQ03DyIiJ4BHAHyRwrm0OT63E5fp2lwCQH6G8uUrKjQKAnNUxToZfeF2wOVQNIK3ivdgy6FalNUaKyne+OYqfLnBWEvorvfXIhRRnL8SKTzMpqFQOGIIabUyLby4aAc+X3cQry/bDUCxOd/z0bqYcUdDZX2TpTYjd6rxIpxk4tIPuytjIqjmrj+I6X9diO/VDlBWPoJfvBItv+F1ObTFJq4gUBf4cESgMRjGt9vKsWZv7PMv2V6OFbuMHa7M8zMnXe3T+YzMkUNVcUx28RKT9Jhj7M8e2c0QqnjLu2uwo6zeMCbD4zTU33nuiuMxVFf/X9rSpZY1oSgfv5s20JA/ISNwZDKXx+nAyz+fAMBo0rJC7/toTvuSWpG+JIXTQbhgTE88cOFI/PGMwZhQVBDv4TH892kD8WorVmg9FkiZHUAIcQDAAfW4hog2AOgJYL1p6O8AvAtgQqrmki7kwi+RX9q+uhC5X53c3+Ao1uN1O+FyEirqg/ifd5T2l6U1AVz10rKYsdleV2xsuU5zkLvFYEjvLA7j4TkbcaCqET+Z1Ad7Kxqwo8y4ewSiX2i5CMrX//O5w+Pu0rYeqkWW1xWTV2FFYzCMMffOxTVT++Guc417BantBOJoBFJTOFDViJlPLcaXN52iXVu0xegktWreoq8b1RgMa/+jeI5laaJpCIZjetrqnZJvFe/FW8VRTSESEagLGO/BbI7RRxJJh/G3W8vwyrc7sSFOGYrmFlQzd84chqtP7Ic/vLnScH63rkCdy0Hwuhzor9tBd8rywu+JCgtpS5da1vDuOfjjmUOwVLdJkU56ArDyz2fA53ZqAijP9N345PdT4dVVI9WHkTZHvio0rOz7XXJ8+F0Lo3L+dNaQFo3vCLSJj4CIigCMBbDUdL4ngIsAPNvM468jomIiKi4tbT4Cor1gLh4lF2N9J6XbzhkWoxFIfC5nzG7InJcgkc49PXpTSNDKRxCMaCGsd5wzDEO6ZqG0JhCTDCWbcZjt3onyEE5/fKEhwzoR69WoqC83HNTOHapWIk+0ZKs4gkBvgtlqyuI216238hH0yI06JRuDYU0TiKcRaIKgKazVypFzS+SnqG0KGerlA7FmOL02JjWC577eji/WH9R62ZppqSD4+ZQiOB0Enynqadfh6CKf4XGCiDTzkYzZ15uTpM1c1tuZ2E+xyetj52UTeJCy8Ou1EPO8R/TINZhS9YllzTG+qAAPXjRK8wswLSflnkEiyoKy4/+DEMK8rfkbgFuEEJFEmX9CiOcBPA8A48ePP2bSYc0fdrlo5frdeO83U7T46a7xBIHbYXCQJaJLtk8L35OELEJFDT6CUAR1gTBG98pFpteFztleNAYjqA2EDE2zpWbx2tLdGNM7TztfFwi1SjbkGrXHs34XOPFBRYhcf8oA9bWsBYE5XHLxljJc8eJSfHfbNC05SmLlI9CbVuqbwtr7Ey/3rEwtadwYDKNUbYDemIQgqG4Ixph7EiUfH1bNOZX1iaO49M1OXv/lJJTWBHDDGyvjjpc2cPP/bbf6XnXL8Wnln4kIK+48Xdtpy4geIBpUMKl/Jyy9fbr2GdbnBDxy8XF4bsE2TLQwy2Q243ztnuu3PH/akM4xtn6ng/CTSX0SPh+TmJRqBETkhiIEXhNCzLYYMh7AG0S0E8DFAJ4hogtTOae2xNzA5uyR3QEo9tSxffI1E1E884nP7UxaEBRaOOQiFhqB3kfQFA6jvimkOeakU6+sNv7O9a3iPdHz6uI8Z80BfLetHEeKzEi1yuqUZZWbwhHL4ntmO7uMbZ/17a6YmjThiMCmkhoU3foJrnppGVbtqTQ4bBuCUUEQjrNKy8qcetNQVCNQfstm5XqqG0I4bFEX3xxqnOlxgihqGtLX0j9pUCGe/ek4w3i9iWXKgEJcMKan5bzN+NzKV/+XJyklD3arhd6Gdc82fJY6ZXk1oaHXTvWlGMwbmdeunYS5N56Mnnl+3HfhSEsHrNz4DehsbQKKl7n98tUTcee5Ma5G5ihJmUZAyn/6RQAbhBCPW40RQvTTjX8FwMdCiPdTNae2ZnDXbPx0Uh+8tnQ3JvYrwIyR3bD9wXNiqhZalaQFFCeb1ZfICr87dpxh0Q/FCoKGJkUj6JGnvH7nLOULXVoTMDTk0NuyB3bJ1pKDatTSxL9W6+5Y9bf9zWsr8MxPj084d2lusYq00ZczqG8KI9dvvE9z+e9D6i59nmpmKszyas7SUERo5qeFm0vx7bYyjOyZi5MGFWLZjsMGQWA2j63ZW4XiXYe1ipkNwahpaNGWMry6ZJfmXP3ZCUXI8bnxxvKo0Kysb7IsHXHvBSMx69udOFjdiC/WH0Sm1wWv24nyuiYIIQyO3qkDC3H2qO6GxyfaWV9/ygBDzoEep2q2yfG5keNzaXN7+MfHJdW3IVFmdbIJUotvOS2hs3v+n07VBBaTWlL5Lp8I4EoA04hopfpzDhFdT0TXp/B12w1OB+GBi0Zh58Mz8davTgBgXbq2f5xdUWGWB65mEpckViYa/SJp5SNoCFprBObIIX3rQr35I565Rs+na5ovfVGnFReLNa3o4+etWiiaNYKSKmXuZbUBZHqcKNLZmsMRYVjAgmG1wbjHCb/HiYamsPb+mJ3Fv5i1HPd8tB5N4Qg8ajarvozyne+v1eaa63fHOEM3lNRopiQ9/Qszcd+FI3HSoELtfgoyPSivbcLY++Ya/DB604wkXjN1ALjxjPhOUunIdThI+787SMkojmeW0dMaVWt75WfE3QQBSnewZObCHD2pjBpaDCTfLVwI8fNUzaW90yPPj3X3nIWlO8rRI8+Pod1yEIkIOBwEtyM5We2yGKdfRKMaQXSBa2gKoTYQ1mzAUUFgXLDqA2H0yvejINNjeM54NnFzGGi8ctehsKKRSPNMeV0At7+3xpD4o29eIgXPQ3M2ID/Dg+tPGRCjRcg5VdQH0SPXZ+juFgqLmOJetYEQMj0u+N1ObD1Ui3fVnIDvd1fiD2/8gPsuHIlsnxv1unvt2ykDWw7VxkR7yczgXL87Zif7w+4Krc6+ZGi3bG1jMLl/JxRmeXHzWYPx8eoD2FfZEJM7IAXBK1dPQHVjCKcN6YwMjwv3nD/CMjzS43Tg1rOH4uE5G3Hucd0NFTn19O+chW2ldeiZ7292gf/65tOws7wu4Rjm2KN9pZHamEyvC9OGRotlyQUiWY3ASp1fr6tRpGUW63aYa/ZVaTtnAMjzu+FykKHzFKCYZDI8SrlgfeRLbSBkWf1U7mK9LgcCIaWqpT40UHLb7DV4e8VerRLjnsMNeH3pbi3RDQBKdYut1Aj+uXA7AMX0YXYW66OLcvxuFOh25i99swMzjzOaVuqbQsj0uuD3OPGtyc/x/sr9uGBMT0zoV6AVZQOUTNkth2o1B6tE5g3k+t0G/1CPXB/mbThkmNtjFx+nFV4DlGib4jtPBwBsOFCj5WzokfV/Th3SxXD+qilFMWMBZdd+/SkDNIe7ASHHAJP6FWDu+oMxGcBW9OmUgT4tiOhhjg3YANfOMfsI4jnRwhGBS47vhfGmOucSKx+BzGTOkIk/DkKvfD8+WnXAUPisTjUf5fjdhqqStYGQpTlHljSQnahk6OOt767GMwuiJQtm/6C0JDS3RvTqdtP7qxo1J7JVQTWzaUhPXoY7Jqz2k9UHDH/XNIaQ4XXGOPYl28vqsOVgDYRQasxfd3J/nDpUWYhl7X7JnLUlcDlIMzVJThvaJSb8dfqwrpbCEVBMhXqB/eo1k/CXS0ajdxILdUshkCZYLp3AkTd2hQVBO8dtWvi7WEQHAUqUy2OXjMY7v55ieV2ahKzKEOgdjo9ePBp7Kurx4mJl172/sgH1TWFkehWNQC8gKuqbYhbxg9WNuOiZbwFAM8uc/vhCbCypxhvL9+DRzzZh+c7DWLHrMNyqtlNRb23ekUjH9U9eWBrT39eqpo8k1++OG5orCUWEZhqyYuOBaryvCqyJRQW4/Zxh6GVhq9c/HxHBp1vkZZkDPfFeDwD6F0bDI28/ZyimDirExcf3SngfLUWvPw7skoV195yV1g5ZTHphQdDOMTuX9VUZ9TTXuKUprOxIZVRNvi6FP0MXtjmxXwHOGNYVbxXvxcaSakx5+Cus2FUBv9sVkxfx6GebMPOpxYZz+iQufUy5vgzxJc99hx8/+13cQnLmnXaRLoJpic580xSKoLyuCcf1ysVzV4yLKQyWjCAAoJmGrHh7xV7M+k4p1SF35PraOdL5rK+xAyjNWSQDOmfhzpnDcN+FI3GmKhQSRcPoTTTmulTJ8LtpAy3baOqRUVHSJWDXqpuMAguCdo60t583ugeeu2Iczh/TE24noWeeH9/cOg2/UFvf6aNc7rtgRMzzSE1gz+EG5PhchoQxc8ONEwcWorQmgIWbos5FqREAigZh1VEpFI7g0zVR04vePn+g0jozNvqanTCoSxamDOiEtfuqDNf0oaz6qKfyugDKa5swsEsWZozsHrOY5WV4tAqYVsh6Nhkea9OQuV+AjMzSa2X5mcpzjDSP1b0/PrcT157UH1dO7ounLh+L726bltAp20mXwZsoKigefzxzCObccFLCMRcfr9TBmjEiNueBsR8sCNo5jWpETecsL2aM7I4rJ/fFlgfOwTe3TkPPPD9G91YWIL1d/coTirTjf/1sPABgn2rb31NRj94FGQbBYTbFyOxhfcmHTK8LJw4oxMSiAjx68Wh0t0iCe2bBNjz/9Xbtb32JjTWmxd3MiQMLMfemU3DSoM4xPZv76Yr06TWOm99ejX2VDSjMUhZmc7+HxmA4bvmObQ+egzevOwE/GtsTJw0qtIxnP21IZ4tHKklWkiFqITZzbRy9hqH/3/jczmZDIvWaTap26kO6ZWPnwzMN2hZjX1gQtHO0JjVxTAnnjOqO/z5tIG4+a6jl9VMGd8ZJgwrx1FdbcfaTi7B4Sxl652cY+veaF5uh3bPhIKM5Z2DnLIzqlYu3rj8BM4/rrjmC9Zirbeo7T8ld/vShxogXbQ5qLsM1U/vFXNMvsvN0lVYXq01TZA0cuXuWDvPe+RmGRVuP00EY0i0bj186Br3yM2J6yALGdohvXDfZ8FhJV1U78Lgc6JHrw4VqJJBew7BqcZgIvbZgFm4MkwpYELRzZLRJPOei2+nAn84aEmO/lwuux+XAeaoTcMOBaoQiAn06RTWCyyf2wcXjjI5Ir8sZY1s3mz5kEppeQJkT0fTVV6VDuF+cHajcQXtcDozrk2e4phcE5pBNAOikNj+XJRAm9++ERf9zGq4+USmw9vkfTrZ8TT3m+5t748kY3kPZ7d8wfRAm9zeWghjYJQtTBxZq9YvcTge+vW06/nbZWADGBL940UHJcCSmIYZpKfwpa+ecMrgznlmwDafEMVPE47krj9f8CyN7GBe5KQM6aTWDLhnfyzLbuUee32CGGdbduGOW5ov7LhiJqoYg7v9kAzaZOmGZq68CMJQ21qPvjDapfyd8v7sSz10xDqN65RmSwqyQZh1ZZG5EjxxDqOUQ026/0EJLkIu+RFbRXPO/Z1p2bfvyplMghMA9HylV1XNMC3ZrFOMDrOsvMUxrw5+yds6k/p0sa/g0h9vp0HbIsn+rZMqAQoTVcFLzAiaRUUUXje2JkwcXGpzLAHDL2UNR3xTCWSO7YY7qIA5HBGYe112L1e9ksYDH0wj0dvEbTx+Mfp0yccbwbpoZZsGfTkV+pgej74ntXySdvjILebSuQqqZr28+zXKXneNz48ubTsHpjy80nDfftx4iws1nDUHfThk4fZgxRFSag2TpiCOlvbUOZTombBqyAW6nAxeO6YFfntQPX950CjwuB4KqjyDeQicTtc4a0RUXjY2NYe+Z58cLV01Ajs9t2P3+5tQB2uJtbswDKCGlr1w9ATeYmoUYQjJdDvzXhN4GW3xRYaZl7f3fTx+klVe4RI21t3JkS/p0yjCEteoxtxZNhkyvC1ef2C9Gq+pdkIEnLxuDZ0zVQpPl8olKcpeVtsYwrQ1vN2yCtF1L3E4HGoMRZMfRCAZ3zcaS7YfjmnL0SBu4x+nAoC7ZSlG2SBh+jxODu2ZhRI9cvKcmZWX5XDi1WxetZtH0oV3w0I9GoUsS8f5W6MMfH734ODxw0ahWKYjWGiRbEtqKBy8aiQcvGtmKs2GY+LAgsClvXncC5qw9ENcJffs5wzBjZDcM7hobTWNGOowHd8uCx+VA/86ZWLe/Gh6XA1/ceAp2l9dHBYFq85Ymj1BEHLEQAIy9bYkIHpe1EPj99EFaBFZztAc50l6EGWMPWBDYlOE9cmIcpHp8biemDEjOvi1NQ9Ip/fLVE/DdtnKtxHDvAj+mD+2CdfurNUHgVm3oydS+1+N2kqGCarKtGm86Y3BS41b++QxehBnbwYKAOWqkIBihhmB2yfYZzCJEhBd/PkErrQ0ATnWxjdckPh4epwPBcHRnby4rcbSY+wgwjB1gQcAcNUO7ZePyiX2aLVegd3zKejonD25ZWKzb5QCawrj7vOEoqWrk3TvDtAIsCJijxud24qEfjWrRY/p0ysCy26dbxvQnQobEnj6sa0rKMjOMHeHwUSZtdMnxtTg8soBNNwzT6rBGwBxTvHDVeMz+fl/cctwMw7QcFgTMMUXvggzccHr8puwMw7QcNg0xDMPYHBYEDMMwNocFAcMwjM1hQcAwDGNzWBAwDMPYHBYEDMMwNocFAcMwjM1hQcAwDGNzSLSwDHC6IaJSALuO8OGFAMpacTrHCnzf9sOu9873HZ++QgjLKo/HnCA4GoioWAgxPt3zaGv4vu2HXe+d7/vIYNMQwzCMzWFBwDAMY3PsJgieT/cE0gTft/2w673zfR8BtvIRMAzDMLHYTSNgGIZhTLAgYBiGsTm2EARENIOINhHRViK6Nd3zaW2I6CUiOkREa3XnCohoLhFtUX/nq+eJiJ5S34vVRDQufTM/OoioNxHNJ6L1RLSOiG5Qz3foeyciHxEtI6JV6n3fo57vR0RL1ft7k4g86nmv+vdW9XpRWm/gKCEiJxH9QEQfq393+Psmop1EtIaIVhJRsXqu1T7nHV4QEJETwNMAzgYwHMDlRDQ8vbNqdV4BMMN07lYA84QQgwDMU/8GlPdhkPpzHYBn22iOqSAE4I9CiOEAJgP4rfq/7ej3HgAwTQgxGsAYADOIaDKARwA8IYQYCKACwDXq+GsAVKjnn1DHHcvcAGCD7m+73PdpQogxunyB1vucCyE69A+AEwB8rvv7NgC3pXteKbjPIgBrdX9vAtBdPe4OYJN6/E8Al1uNO9Z/AHwA4Aw73TuADADfA5gEJbPUpZ7XPvcAPgdwgnrsUsdRuud+hPfbS130pgH4GADZ5L53Aig0nWu1z3mH1wgA9ASwR/f3XvVcR6erEOKAelwCoKt63CHfD1XtHwtgKWxw76p5ZCWAQwDmAtgGoFIIEVKH6O9Nu2/1ehWATm064dbjbwD+B0BE/bsT7HHfAsAXRLSCiK5Tz7Xa55yb19sAIYQgog4bJ0xEWQDeBfAHIUQ1EWnXOuq9CyHCAMYQUR6A9wAMTe+MUg8RnQvgkBBiBRGdmubptDVThRD7iKgLgLlEtFF/8Wg/53bQCPYB6K37u5d6rqNzkIi6A4D6+5B6vkO9H0TkhiIEXhNCzFZP2+LeAUAIUQlgPhSTSB4Ryc2d/t60+1av5wIob9uZtgonAjifiHYCeAOKeehJdPz7hhBin/r7EBTBPxGt+Dm3gyBYDmCQGlngAXAZgA/TPKe24EMAV6nHV0Gxn8vzP1MjCyYDqNKpl8cUpGz9XwSwQQjxuO5Sh753IuqsagIgIj8Uv8gGKALhYnWY+b7l+3ExgK+Eajw+lhBC3CaE6CWEKILyPf5KCPFTdPD7JqJMIsqWxwDOBLAWrfk5T7cTpI0cLecA2AzFjnpHuueTgvv7D4ADAIJQ7IHXQLGFzgOwBcCXAArUsQQlimobgDUAxqd7/kdx31Oh2E5XA1ip/pzT0e8dwHEAflDvey2AP6vn+wNYBmArgLcBeNXzPvXvrer1/um+h1Z4D04F8LEd7lu9v1Xqzzq5hrXm55xLTDAMw9gcO5iGGIZhmASwIGAYhrE5LAgYhmFsDgsChmEYm8OCgGEYxuawIGAYFSIKq9Ud5U+rVaoloiLSVYdlmPYEl5hgmCgNQogx6Z4Ew7Q1rBEwTDOoteAfVevBLyOiger5IiL6Sq35Po+I+qjnuxLRe2q/gFVENEV9KicR/UvtIfCFmhUMIvo9KT0VVhPRG2m6TcbGsCBgmCh+k2noUt21KiHEKAD/gFIBEwD+DmCWEOI4AK8BeEo9/xSAhULpFzAOSjYooNSHf1oIMQJAJYAfq+dvBTBWfZ7rU3NrDBMfzixmGBUiqhVCZFmc3wmlEcx2tchdiRCiExGVQanzHlTPHxBCFBJRKYBeQoiA7jmKAMwVShMRENEtANxCiPuJ6DMAtQDeB/C+EKI2xbfKMAZYI2CY5BBxjltCQHccRtRHNxNKbZhxAJbrKmkyTJvAgoBhkuNS3e/v1ONvoVTBBICfAlikHs8D8GtAayCTG+9JicgBoLcQYj6AW6CUSo7RShgmlfDOg2Gi+NWuX5LPhBAyhDSfiFZD2dVfrp77HYCXiehmAKUArlbP3wDgeSK6BsrO/9dQqsNa4QTwqiosCMBTQukxwDBtBvsIGKYZVB/BeCFEWbrnwjCpgE1DDMMwNoc1AoZhGJvDGgHDMIzNYUHAMAxjc1gQMAzD2BwWBAzDMDaHBQHDMIzN+X+PC3i7cu1EaAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "<Figure size 432x288 with 1 Axes>" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], "source": [ "truncated_mae_history = average_mae_history[10:]\n", "plt.plot(range(1, len(truncated_mae_history) + 1), truncated_mae_history)\n", @@ -1630,9 +1661,17 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 18, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4/4 [==============================] - 0s 3ms/step - loss: 17.8865 - mae: 2.7644\n" + ] + } + ], "source": [ "# Gets a fresh, compiled model\n", "model = build_model()\n", @@ -1651,9 +1690,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 19, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "2.7643771171569824" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "test_mae_score" ] @@ -1662,7 +1712,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We’re still off by a bit under 2500. It’s an improvement! Just like with the two previous tasks, you can try varying the number of layers in the model, or the number of units per layer, to see if you can squeeze out a lower test error." + "We’re still off by a bit under 2800 USD. It’s an improvement! Just like with the two previous tasks, you can try varying the number of layers in the model, or the number of units per layer, to see if you can squeeze out a lower test error." ] }, { @@ -1675,9 +1725,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 20, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "array([8.708372], dtype=float32)" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "predictions = model.predict(test_data)\n", "predictions[0]" @@ -1687,15 +1748,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The first house in the test set is predicted to have a price of about 10000." + "The first house in the test set is predicted to have a price of about 8700 USD." ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": {