{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "dWyPGNkCGhIX"
},
"source": [
"# Part I : Create Your Own Dataset and Train it with ConvNets\n",
"\n",
"In this part of the notebook, you will set up your own dataset for image classification. Please specify \n",
"under `queries` the image categories you are interested in. Under `limit` specify the number of images \n",
"you want to download for each image category. \n",
"\n",
"You do not need to understand the class `simple_image_download`, just execute the cell after you have specified \n",
"the download folder.\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "8rckz3ZuGhIc",
"outputId": "6f615f06-759a-4eea-839e-658155df8d36"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 2 image links\n",
"Saved 2 images\n",
"Found 2 image links\n",
"Saved 2 images\n",
"Found 2 image links\n",
"Saved 2 images\n",
"Found 2 image links\n",
"ERROR - Could not save https://upload.wikimedia.org/wikipedia/commons/thumb/c/c0/Robert_De_Niro_KVIFF_portrait.jpg/1200px-Robert_De_Niro_KVIFF_portrait.jpg - cannot identify image file <_io.BytesIO object at 0x7fae297b5770>\n",
"Saved 1 images\n",
"Found 2 image links\n",
"Saved 2 images\n",
"Found 2 image links\n",
"Saved 2 images\n",
"Found 2 image links\n",
"Saved 2 images\n",
"Found 2 image links\n",
"Saved 2 images\n"
]
}
],
"source": [
"from selenium import webdriver\n",
"from selenium.webdriver.firefox.options import Options\n",
"from Image_crawling import Image_crawling\n",
"\n",
"# Specifiy the queries\n",
"queries = [\"brad pitt\",\"johnny depp\", \"leonardo dicaprio\", \"robert de niro\", \"angelina jolie\", \"sandra bullock\", \"catherine deneuve\", \"marion cotillard\"]\n",
"#queries = [\"Bart Simpson\",\"Homer Simpson\"]\n",
"limit = 2\n",
"download_folder = \"./brandnew_images/train/\"\n",
"waittime = 0.1 # Time to wait between actions, depends on the number of pictures you want to crawl. More pictures means you need to wait longer for them to load. \n",
"\n",
"# Set options\n",
"options = webdriver.FirefoxOptions()\n",
"options.add_argument('--headless')\n",
"\n",
"# Create Driver\n",
"driver = webdriver.Firefox(options=options, executable_path=\"/usr/bin/geckodriver\")\n",
"\n",
"# create instance of crawler\n",
"image_crawling = Image_crawling(driver, waittime=waittime)\n",
"\n",
"# Find urls and download images\n",
"for query in queries:\n",
" # Craws image urls:\n",
" image_urls = image_crawling.fetch_image_urls(query, limit)\n",
" \n",
" # download images\n",
" image_crawling.download_image(download_folder + query)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "CRHl9UX6GhIs"
},
"source": [
"Please check carefully the downloaded images, there may be a lot of garbage! You definitely need to \n",
"clean the data.\n",
"\n",
"In the following, you will apply data augmentation to your data set."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "3SX21FtcGhIu"
},
"outputs": [],
"source": [
"# General imports\n",
"import tensorflow as tf\n",
"tf.compat.v1.enable_eager_execution(\n",
" config=None, device_policy=None, execution_mode=None\n",
")\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import os, datetime\n",
"\n",
"# Shortcuts to keras if (however from tensorflow)\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"from tensorflow.keras.models import Sequential\n",
"from tensorflow.keras.layers import Conv2D, MaxPooling2D\n",
"from tensorflow.keras.layers import Activation, Dropout, Flatten, Dense\n",
"from tensorflow.keras.callbacks import TensorBoard \n",
"\n",
"# Shortcut for displaying images\n",
"def plot_img(img):\n",
" plt.imshow(img, cmap='gray')\n",
" plt.axis(\"off\")\n",
" plt.show()\n",
" \n",
"# The target image size can be fixed here (quadratic)\n",
"# the ImageDataGenerator() automatically scales the images accordingly (aspect ratio is changed)\n",
"image_size = 150"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "rN_Mp1rmGhI1",
"outputId": "6417b1f9-e7d4-4d56-a213-191f9d17524a"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 480 images belonging to 8 classes.\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAADU8klEQVR4nOz92ZMsS5Lmh/1scfdYcjnrvbeququ6a/YVoGAoAgyHFD7yDyWfuUCEQqHwgY8QEpiBCMFBA9NTS9d6t7NkZkS4u218UDVzjzx5Ms+tqu7pppRXxT2RER4eHu6mZqqffvqpKaXwx+2P2x+3v32b/U99An/c/rj9cXt4+6Nx/nH74/a3dPujcf5x++P2t3T7o3H+cfvj9rd0+6Nx/nH74/a3dPOPvfn5n7wu1lqMMQCUUsg5k0Iip0RFep13WGMw1mCtxVrb9pd/5XnJpb1mjCHnTIwR7xxd5xn6nufPn1EKpBi5u7kjxwSl4LqOkjPOOq52e7abPZ3vcF1HTIGUEjlFQoyklIgpkXPG5ILFsBsGLi72XOx3fP7qNZ9/9hlX11c8f/6Mq6sr9pstl9sdQ9fjvcdaS04Tbugx1nA4HJnHiZwyGIMx+ptKYZ5nwjwTwtx+v3OOvu8xxmCMIZUCxpCBlAumG8gFQgiEEMg5U1JiOk3EKK8djgdOp5FxGrm9ueUURnIpeN/x7t1bTuPI6XTCOYfvOnzfUfS6FwDv6LoO33UMw8C72/ecTieOx2M7N4CUEpt+w27YknJk6D1937EZBrb9gLeWkCOvXr3Cdx1ziPyH//hT7o5HTmHiN7/5DTFGSoHt9oIQA3MMnOaJaZpJKWFSwnuwtmCMjJMClGyY50hOMj7Q5MF5EsGgv4iCXPOI0ftg6D30vacfOva7jt12oB88zhkMhZwhTDDPkfvZiXoN7m/f9fXf/TPwb//bv3hwh0eNM6cMgBgotCtnzr/QGHmtbotRLv+WQrvAADlnrLXs93uctXTe0ntHnAMhRuY5Ms4zFiPvdx05Z7yx9F3PMHQ45zHWkFKhlESMM5SCN+C9IxuHs47eeZ7t9jx/ds3V5SWfv37Ny5evuLy84Pr6mu1uy6Yb2PQDwzDQeY9zDmMGrHcUY7DGE3sxIvnthVIyuWTCPBNTJqWknzNYa5qRG2PIyDXIpchg7TZinDEQY6KUTMmFeZqIIRJCYLPdMI4j0zyzvdhzOt7JRBASYSeT0zAM5JzlkTLFQIgySeWS8V2HtY5E5nA4MIeZeQ5t0Fhr8V2HyeB0sA+bC/b7PcfDkXdvvyKEINe5ZLbbLV3n2G16us5xWfZM08TxcGQcJ7kmeXlAwVBk4nbIJF7HS5FHWT3OrXIZWGZloO2zdY+2MBicNThv8d5irZFFJBdiTKQUyXlZHB7b/lO/D08ZZ05qiIAxcj3072INpl5QzNmX3TdKKJQMlMU8cylYYxg2A84YOieP8TQxzYFpDswx0jkvF9zK6uytw3eerpOBn3KmFFk1YwwykzpP5xzZWzb9wHYY+OzimtevX/Hs+prPXr3i+tlz9rs9l1dX9H2Hd57eeYZhwHuP957OW4qV3935nhTkRsusn+R7c2aeZ2TSNzjngAIGnLM4azHGqnGW5i0YL8YZUySlrNcqE+ZAjJEYoxjnNDGHme00crwdGI8nbm+PlJSZ+4FNCkyTrFDTPAEQY2SeZ+I84ztPwXCcR6ZpIqVESokQI8YYnPPs93tma3FWbm7O1xhjub078NVXX3M8Hem6Dmzh8mLH5cWW3juGocd4x93hiMGQUiblxcMqOUMWCzRGJnm7Msw2YX9iqr2aaaGs1wI9tpHr7SzOyXNjIEX0midCWK+cHxqHefBl88T79/Z51ObMJ+yzbE+vnAYwGWdcO7B1FowMNgM4J4Yj75tmkDnpTSpqnMgMCJCMuK7hcMNVP5CMZcqF29uj3GDAeTm9nDPz6ch2s6X3XtzElJjnwOF0EJcqy8pjTMH2ls12w7AbeHn9jBeXV/zJq8/44ntfcH19zfXlNcN2R9/3DNsdu91Ozr+I2+ycGGc/7ElZVsft3lD0RzjnIM9QMjknUoxkDBkrhqcTmPdLSJBSar8lpUQpnlLkmHMIMqBLoVws+0zTxDTPzGHm9nRk3u8YjyfeDjfcbXccxxOHkxhqTnIuc05sNgPd0DMejnJM0BWvE8MMgZw1RMmZZ8+eMc8z3759x5e//S0/+9nP6fuer77+mt1+h+86Yox88+Zr+t5xud/y53/699jt9/RDx26zJcZEzoW7wxGb5HenmIgxUErGdBaDw1iLNVavQSbFQs73Qp57g/mDwY3F6bJrTcE6S9cZhsGw2Xq8dxo2FaYxM50id3cn5nkJxcw96zb3vsasvm/5e/3+8t7ZmRpz7+/VuF8d5OOGvmyPGqcGMC1mNOqPGAt2dWTn3L3vKZTywDfrTGmMwSErksmFkjIxZ+aQSMXIvGjqd8oMGOZA5zyd93Q6WGKM5CxupQG8s2yHDfth4GKz4fPXr/ni1StePn/Gq+cvePnyJfv9BfvdBd2wwfuObhjkZlqLVQ/Ae4/vOoo3mOJw8k7zsqwxlAylZGzJFOvkuhiL7XvqSmHMsiRY6ykFXCkYXyCbtsJgbYtf1yusc56u7wkxYp1nMpbeD+Rscd7TDT3WWbLG7b7rMNOJlDM2JUo/EGIQ7+IsDDF479otHseROUg4sd1fkHLicDrh+56YE2muM6slzZFpvuF0+ks2Xcd20/HZFz9g0w24ay/73NyAruISc0scbu7hFzIhZV1BP74Zs7i0RvcVV1ZXTOtwzuG8x7rlO7JOrJlClgOdDe3zcXn+N6AjdH0iD+z00FsPvmg+/tZHtkeNs4IL1QVR71ZmGruslNYuplqy7tgGqGkfLqvfVL0bi6GkTEqFOURKsW0SQN2X9h0aE1lrmaeJEMVNsdbgjKH3novNhqv9nmeXl3zx4iVfvBbjvLq65Orygs12x3azxfcD1okRWuewVgzUALYToKno3+tZsbnvpkAxUCwUMVisY7WrrrSl7V80xrIAWQamUYBJQBxdPYusJhhxk72XGd8BznkBlJzFeUcpmTjNWOtAgRtihFLEtVfPJa9ne5Y4zWAIQVzpXEqLc2PKdH1PSpGkcbYpUGIhhpnj4Q2dNWw6x/7iGZvdjs57NsOGUz/ip6l9h1wfWTGtuqWloKvlalDo08KZHZ1vBkwxLbwS8M22EMLqZN5ca+Q2YQzGWUwNrR6yxse2R3YpD779gMV/R6bs48ZpJGSwUWMDC1hDMWCckRhwZaiUeuFloBVbgEwx9izgr0itNZah7winiZgKij9hjTycsxgrA/Tl82sZqM6J+zbPxJjAGl68eMHgLb21vLi45geffcaffPEFL1+95PmLa/aX6n5tL+iHLZthh/UClFiNL411YF1bOZ1Ai804S6Gt5vL7xChLzou75t0yiawRxiLxe0NR64SnoMkcZoq6mUaPb0wmF3GxO2QQbocNIUY22x2bu1sOxz39doOzjtvbW7i9IYQAuVBM1hjdUIolpVjXHoyRlTpHWbUqyt73nbjzvbi/4zgSgm1xakl1gnE474gp8u4w8tO/+isuLy+5vLxkf3HBs8sLvLec5iN5EhS3c57OdaCAVS6Qzkb1fSDofDtzPI2R8MUafOcYho6+E9zA6gQpbrteb2vwfYfNHxnnjxmB3q8P9rln4ed/lnv71affzUAfd2t1tgSJP41eIms9ThFJ69SS2okYmdmKuBwNqV25MJSCQ1ydznfYAQiJVGrAXmRiyIaixy650G96rLXcHQ5kCr53DM6z947dZuBit+XFs+e8+uwlL1+/4PLygs12y9AP4sL6nt53dM5jvcO41cPWh8U4WYWcpoTEOJeYqP2rD+sc1omruU4jlUSLP/HLqloK5BhBZ3mXPcXktsqUXMg2ixHpZ7quY44BnxJ+0+O3A9vTiWE74Kxjs9nQ+05ChBQ5TSOQKCSMKWy2A3mEUN3KmFAfnTFFiZ1zZhzHZbXVFdM5p2i5eAMxWrz35Cxu7OE0MoXI+7s7Li729F2Hc44ffu/7nKaJoKmhHKLGmkXRXDWg6teeDdz7BlrqnRD0V2P6vncMfS8Iu3FQrKL3hZygpAL5d0/om3YDn9hK/edx63twLvrI9qhxVo9URpOspMYULItbJEuceHkyUo2ClUYBj2VAm1WcZYyR/KbvsMWQSyBEQT8rmpdXd0zcV6vgk8X3Dm8tg3ds+46L3Zbn19e8fPmC62dX7C/3bHcb+l4GSuc93jlxfZxVN1aMEGubFa3zuuvrWN3zOkTOrpNVF7Hex+rJGtnbaJxaP1hKIVujIYC6ytZgs5Wba8BiwdZrIBOAN4g761xzeTGQggBiYZrZ7XbcTSesMSSznIekqzy6/JNqSkEvdtb70nKuq5SG1ZjRWkPOdokhjcF5iaVjzuQ50E0zBokDt/0G7xxTCNzEG5JO0NUwS/13Fep8OAbXry4hgtXV3juneXbxcnLWVbNkYsya/84th7pgJ5+2fdf9n9qzDY9POOSTKyfktprlIj67RWI8RYYwdVrKBeOoTj6mrGAjNchqmNYYuq5js92QXGjuTgiZUv+XC6UsiG0pBeccV1dXdM7gDHSmcHm558Wz53zx+ef84Ac/4Ppiz+V+y2bY4DuP8+qydeKuOuswzoMTV7ZY+RF2ZZyl5iP1xjQ0ul0XnTiq+20MpaSVYapDYYpMVKubkdTIC4LW6tWiICkm0MmornIUck6N4NB58M4xdD1912Ox5ALHaeIqJw7TxOl4YiJiUyQnMDmz6Xo6V5isJVAnToNLgWwyiYUUkVISI3PiWlvn1N2Xm52iuMneiytc8QDf9ZI6SoUUIn3fYTB8oymilOIKzCurScDc+1cnjroiGb1CRhB57x1d5/C9TLoVnc1zbm74rMSQlAslG1bzzSND/tMM8TsZ7HfYd709CQjVoKtoHJWxZCPEgPoQ3kuRnGAxS3wpp6bxmmk3sAIeORXJ8WnAbh2UObfPumJkBTGGKczEFOnp2A4bMUxn2XWez15/xssXL/jss9dcXuzZbAZ85/CdMI+87+i6Tt1OdV3tyjDFFwAktmsrZD1nfX7fra2uhfydIWVSTvVnr+B2QymCaOdSlF21uMr15snqu0xgEoOu7sXqnPq+bxNJjJE5BuYUGYMQFvZhgltDTgZKIMdCRkAfUwrbzYaUMlOY6bxHIxFxj/te0GLrGpJaCsSUMFkCt4VokADPMPRstltePH/RAMKcZs3jJrabDSOZXCTlIhN30YU81wsm810F/mjhuVwDI9cIZ3FDh+8FvTdYWTHJxByZpkiYJddbclI8cmX892zlQdMxD7/7kZc/2MM8us+nbY+vnKDunjy9P/FUsGT5+xx8XoMf61i4xhiSZ0uSp6vIZl2V9IJW5DakSExRmUWGzjmGzrPfbXl2/Yzr6yv2Fxf0Q4/v3ArBk5nfucUwFzdWb5qu5HWlkli7rC1CDes89lx+f5bfacRVE6M1yx3Sn1VBg4ramjb7LS7XB8XvdcDq88o+MhiKc2019V7SLtvdju1ux+Z0Yh5H5tkTYyZnyTcWnTkzWQkcBescjkJSw6uIuLOaTdQcbLt3cLZa7naSM+77nmEYsM7J742WnIU1dXu4a65xm31L9UAWw2kI/9n4K22hMNbgvZVJt3OCeZgl/TTPiWkSFlQMCcl50cZRvaaP2o2po/mx99dPH7f2D47ziSvpE24tDbaWY34Yi1WXtzlgOvjKahDKv8IYabm9YkhGmBs5VaBAVq6akLaKLGIE7heqW4CS6fzAbiOG+erVK66vr7i8vGDolvjSWofzHd71WNthrKCyxSzpGmPAUrDGfpCvlYyr0d9l2uAW8oW8bu0CUIC6pWtXTF/XUHz5fDGrb5HPlXuxnrFG8sA6hq0R9LrymF0xCrqBMWKgl1dXHI9HTqcTp+MBP82Scpp0pdOZcpomksZmru/IFFyj3NWbv54gV/e9FIZBqI6bzYbr6+vmWbjq/jqHGzqss4zjyDfffqsrvW2I/r2h1jyUCqvkujAYK+6/NVhv6QdPP3g673Cuhl6VuBE5nSbmeaa6A5pRv2cTHxrIwzZjnnh//d7Hd/jws08b6BOAkBGPT0ZPG1gpJWxaZj3NwesCUN3aQoqL+1ZTBPXzGEtOkKcR7yRB3fc9EEkxk0l444gxElKks4ZpmnAYHIXBd2y3W370wx/y2WefsdkM9L3csK7GI3WQ6GOdK20gjhIA5AYjucc6IAvIqrgYVmXvyFbUFb3n7tLeXm7M2jc7m5WLMNxYrk89VnU1DQq+6Vh1ir7ltmIbrHF0tmPwPRe7C8JVICaZ0KY4C+ptLFaX8ZyirJ5kTE7C5kqWzz/7fkOcT+OJeZ7IOTN0PZdXlzjvyaVwdXXFZhjYbLcATNPEOE5MU+BwOFJyZreVIoKYgvx7yIRQY01xZZtLvzaes5BIfqvzBt87+s6y2Xi6rtL0KgCUiCkSUwWZhChi6xRrno45PyUm/eDmPvna77497dbe//qVkS3cWaiMIHETC5R7bkFdRZWqlXLUuCphuk5Wg7OdASc5tlwym2EgxcRkZnpvGbqOi4s9z54/Y7+XBLi1CA/XyYrpmttnJC+rxng/cFiS/hlXnFqTPJpN5ZXbWYre8PreQzdlGXjV6HNRBhBFYyFFLKu7WcGjdlEXb2XlYS9hRvVCoBHtO98x9EOLHbu+1zyuEdRSz1/4wQvF0iAMqzBNSsqwek3lfLyxdF1Pp+g3pXA6nTgcj23CDSG1kzPGCE9YAaCkVUIlLzFlqVFMu5bVH1nfH/FKnIPOG7rO0nmDcwVnJR9cMKQCCVp4dO4Bce49398eitee3NY7/WGNsm6fRN9rbpqCF2vjPFv2EVesZEM2i+tXY9HGNspF0D51g5zwsFr1S407ijWyiqaExRFjwlAIwbHbbnl2ecmz6yt22y3WAiVjUdfKdVjnBUlVw5S0iWZr9bcZPR9MJpsix0FiRmNciz9yycstKGjiTH5fzuVDt2b1VHJumQxKJcuUHJthSvWPwWg2rl2rVUzM6lpXA25pKcBa8RS6rmuxX98PLRZ01i05xmqcBmz1hIzFecvhcNTJzbHd7+lsJz8i5kaVG4aBw+HA8XTi7u5utSoZttt9+75pOlIoxFbKpzGusTI+zgzzfDPIHC+urqycfScPWTXFMDFWyvCAWCCbTNFjryLZ5b59yvadbe3TYsjv+pHH3doos/J9mmxNhTjNO64J3hQnSXRN/racWVlmzObKGKOMlL4hfNvtVkqm5sg0KT/TWgKZroC3jhdXz/jeF1/w2Wefsd/v6ToPCLnelKREJotrlDFd9VLCaL6WrGVMGn8WI3FLzFkAEuvAe4yRGBUn7B1Ji5SFFaWI6gJ2oaMqN6BCtoU/W3KCHDElY7I8ZHU1YD0YdbfJaMiNPFmMN6XcaiCdtQz9gLGejGUzR0LKXE4Se8aYGE8jMSVijEzzTHGS7x2GgXEccdbhnae7vhIPw1iM94QYpTY2RMoBqWsNQTyZknHOtRAmxkyKR6wboRSOt2/wvfBdk06aXde1tEdZeQbUe1TR61IwNos35B197+h6r6VgVjAM9URytjrOaABZMYWnCAH/SbdPOLXHUyn3g3Z1CYsxjVHjXUfvbUMjS5ZA3lAoSapVci6knM6Mc22gXdcJFD8MWOs4Ho7EcNdiuzrLd0PP5eUlP/zTP+X1q1dcXF4ux9KFpc22RRMkMuohZYxT8AYQ+ghCOq/ziqlwiaEYdWk1VjPGCGADkK2yeeTLpFilnBvoB6BHdWOrceYGMFXXtLSD1TipAhm1OHnxQJLGVjUXmTHYQqMeVmCm6zr6rpdUUs3HImVlJFoZlTVaYeSdVOLkRDjNpCjMoZwSUXnTKSbCHMRNL4mu60FX3s2wATRH7DvxFpKkUApCQE8lt/j5bHzpmMjI+9aA7QzOg++Q1dIWRc1Nc9FTSoKSl6Sou9UVlL8uj/NvZHvarW2+wSqmVGaQtU5n3MU40wqkcNYuYIfNlLKwaBrDRGfTzWbLxcUeCsQ5NPe5MkGwktvbX1zwvc8/59mzZ+y222XmNWCKuKyVZrh2X8lFLVbAn2ZY5DZgZbYVwyjZSiVHBY+qYVCgaJVKzmSUQ1vkvbOqi3YdC4a8GGfJy8x/D6HFZI3f9bv12tdzpCyxZiN0WCGH1DxpBbvqaljrU63THK8aWCXZe+fIRmpjjYWYRUlimualtCwlYlzCmpQSRbFfaztNXQkvd4mDnexTf6dev7wEm83lPL8GSPxtLdYZrBcDNVaOkHIhFZ0Ec82Zyz21KOhnrbi9//9qnLaCJ+L4t7KfSqGTBP2SwDerR70qdVjX/FwpcpxhkFl9v9+z223ZbDYMQ888zmq0prkw3nv6zrPbbri6vODFyxfs93s2mw2u8+KC6YpTQaAiYePykGAXUiEXUQLAGKx3uK5DSASJYqwweHKROcgLm6jkLPFqsdB5fIEcIiEGTaqv0i7od5bFEE1W9QFd2UuL5+vEoV6ryS19VP9FAaus8W0jopeqLCBpnhhnxulICCMpRYAWg0oc7tvKWua5GXit8RwxhJzE5Iyh7wdKSY0EX8ok1EnNcVpn8d5pobMT9YkiCgwpi/vcdcI7tq4jBRkHlX1UCwRo40X9FlswFiGSeHCelh+tShJzrNcvtyPYgqpWyBjLRifkv6MG+mQqpUL4i3GuEU6lSkWrq4uimvWxSg2sy76stSrNUYTdEkIr/an5xpoGaStjymyHDZf7C66vrhh0wNlaJiT+5jlKV92kXIgkXNFYqqVT5LtyydUXla3QuKamFEzJxHkWhpExZGsIp4mSMr7zwo/KoqtELpUvJfGTTgolphXKWlfB0h5nqzwVuV2QxkLWypVCjkldY9nXO3BRwJ2SE3OYGeexHR4M1ncN6nVWcIEU4sJGqiCc3muRf1E2jzFcXl4ux7JOr5G4q+utcnNFsqUaMswxNJUHWemXCR1WCwEFnAGLag5V1pR6LrkQU2IOcr8sRatRjBp+XlblNSr0B9w+SJn9NW2PG2cjdJvmWlX3LhdhlMQUMdEtrJU6y6/dNXVjq/tnlMvakN/VvnVGds6vcmDgrGW32XC537PdbNQwa/y6oJpmnSdcXcSSi6CxLN/T3qsGcnYjy4KatsHm2ySU1fCcc3r+mg/NjRkMdbUuWYypThZrhK2sYP+6eFT9l/UbhRVZXMkCiiAbjfEpWXOqyzVNqnZQC50X4K6ivZr+UIMsRgxDLon8LmNUw6kUtPoBgxUgK62qjvLCa628ZBk7pSk1QKHz3dnvLWQN7Y14DlbOQfn9GIkg9BpkYizEpEQOa8SYtcqpIud1XPx1xJ2/K1f2u26Px5yurnjy4wEk1MrMehGCsdgowI8zAkpUo6hInhCifXu9lMLlZgNI8rrve7bbLZcXV4ynk+j59H3b12C4vrjge69f88Xr12w2A86aGh1qxVqN8eodESDKmmUGLVnSJfWnW/0tFZwwiqhaJF6yxjaXkuq6ZhnQnfcKOEmMXAdAVikTQ6GUdBZf1t+/Ns6W80OMwRTbAKX1PtVY1gaAxpnCUZY8oveeYTOwiYHxdGSeR6Z5pJTUvJE5BHHRnRBBYlTPBSfVJUbcWmucaiCJWoLEqACWzWanKRer8WjQf9PK7c4qUyIekjWWrvdstlthaulkM4cRWKtHFEW7o9y3AjELsJSSiHXFrNUyVOaXhADOZbyTlZ1cSCUJOLSe657cnt7z0471+xnxJ5EQKi5UB5jMWjLjZ2MxqhnjjaFz54dczzJOAQmQ1dNay77vKSkzzTO3d3cMXUcpRRTf9LPWGp5dXfP82TOury/pnGsGKfkT24ATa2xTNqBKnlQXszF5FPzRFEBd2Y0zktqwaoSp4CwS/3YdOSYSUWRKUtJi4UKcZ0oKoLlLSTXV76hmJmSIAkLJayvXwmeVoNNKXKteRfUuYlwGfoxR2FcUTJJyMg0TySkJwhoiOUZqereVgulqH0smUbDei2dQCjmExiQy1IJ4mZwrZ7YUNN+s6RCk+qOWkXnftQknpUIIoqm03+/BSOpss93Qdb0gwmHm7q4yhooCYpUIb8gaYpCE9SO/07VifO8Fn7BO3d9oEY6EoXi5ByZ/RyMpH/uzPPj+Y58vn/bGg9uTqZRCEb0b1gMqLcCGESSyur5Vh6eCFMa49nfNQYlMomv1mSGrwFVKBGiyGZWkMPSDAkc7NsNGdWIW4EnCJLMYpw6uZWK458Kur4/S9RYrqv8aUMVAhZzFLStF3adaI1hd1xrgyaPUY67c6AX8yG2/+n6pjKrKZS6chQZ5lXpZM4iq620wTaajQXSlfkbJDmWROrVW3VJTcN6Rk1D42rVADD3qarmO5XLOWLd4HN53GvsLM0sE1yBW4A3O73/1xozBZdEzKuretnww4q1J1kWF2zQ/Vm+PswsOUsfDMtaUFYalWP1daxt9wDjKU8ZX7u33wHv3j2fOXjf3d3h0e0IaUwWUtdK6kbMVFm9GobN2MQZbaszoqKmTWiGyrqAw3rX41ZhFpOnm5obj8cQ0TcQY2e12XD+75vr6uiG050LXugrrc7sGfFar9seen10rXama4edMSVlXi6gxk2wtt6aACWb5LYIq5hbXnd+Wc/rjck7tJNTolioRahrnbH8tuQLIEpMPXUffeXrf0XuPo5BzJKXAejKwxghpAcM4jmz6QTjMiHsrbqmWXBUBzmLIuG5Redjv93it9tnvtzjrMUYqYEoZiSETQ8F5j7EivG2dxeZELokQJr0WSVa6GqSUWmRgyDX1VgpOC6yLEW3abASp9qqFi4bDuejFNJpZMKzi8se3iok8ssP5vh957zsd85Ht8ZUzC92surRtsDUKZRvFMjitujwpEW1UhTcJ/jebARAXdbPZgN6IFBO+E5mJvu8XBTUngtO7zYbBdzgKm75js+lx3mIc4n4ivE8DZJ1V14gjyA22zq/Q0oKu8RTULTRGFdpse88URUbNIvoss7LDWllRMqkGrEvpiZCDlmuEsHhkYdZKlrpYlxUJ3BrJB+hlXSRvZOTdd4PrJCP6PpGcZpwpeGekMqff0PuBzg04jpW0hDe2KfZ775uHYrUIoCLYboXwFkW6K4dXvreSzCeMCYhEiHhRu92W169fMoeREEYOx/cSNhjxvOYQWozpnBpmRWWTxOUpFZVGMZCrbIzuVy85kOLC8445tSJuMVTxsiQk/esBcj7JAB8z3o9sj8eceTlA86TqsdrgOz+BepFMKWRjiSZjTKbkhexcKkKpz9fVImvU1qkraAz0fUfXd1Jc++A1roNV4b1qoPceoJ6j7mv0s2JTkpAvRryEKh4pYMIaVRXLWi5FVkI7y/e2c1LDAh0wuqIaua65JF3+1QXKRpk3heyW42QFROrAszJj6vVOCiYtWECNwY2RQb3pe0Lfk1NiLhMKiErutNAmHSHlCxJa2v1CXmMZ4NVNbR6RcfJdTmL+mvdMWVbAvvdLDF5yG1wySZVGtysUUinaUsMoacPINbRnt3YhIFQEu163RhldUoEfjJnvbKfNtXn4naeO94CtyMsf/+DjqZSsboxZuVTlYUtfx0ZiZDLzgRhnygWnJIQYFvmPmkoBueH1wq7rOinCuR2GAd91wDrtITfQVD7oyggXo5Qc2lo31aBURFjlJUUWs2RIOTfjrL+78o5KWdIaGK3FNCI/KSsgja1Uq02KpjRqBYqpZU5x1v2rEJMIU8eUMJuuTSKCyKogljEanyVSTmrwWVbGumrrqZsCzjh2m52wglLi9uaGTj2dFMUYsU5V+BzFZHIxWL0OwmFdhMVTStKDxdd+MFLIXkvzlhU9UErEmMJm1wv/NqUGEtWHUPtkhZT0nIJfYSHU15lPLodc61I1jxXsqnXCpcX6WtxtUSUPHrDS+4P+Yy+U72jQ5vH9P+FYT3NrKz5QGszx6LbEU9WlFD2Z8TS1FTLMga7rNEUDYVUjudsJRD+OI8e7Q0sNDMMg8odVpGr1C0uxWNOpIlzWsjF7r4bTnQESSYuYja1ixzQwxFqHETEkLaQvpBhETUHJ5845VQiM6g6u3Kasxrn4ruQSMc6oUrmsxLnUSUqAEIrozDrnsN6RVqqGIhEq/xZYlZpJ/rDlWil0nWe33XJxccHd3V1rXhS1BcNut+P07gg544xhioGQZhKFi4tLWfWMhxQgizRJLknHgpz/4aD3ZhjYbi+bFMw6brdO9JJEgjNSSsIa2AyeYpIY9yCfibEwxwI5MM6RcYzM8xJzL/eu6tIiChqtHK16bItioTGpFQ6keTGW+0Z6vvLVif3ewK7j4+zvD/84+1hbCD481qdsT6ZSSjn3i2u83b7E0Ipz5XzWWjvariBBCHMbvCEI7WtdCO3LqiJBv1hi0Jp+0Xivea336V+oC2dafHEfBDojHugPMCvWkwBbyyCwrsp0iPjV2vWupApx19u3yD+V5mfMokNbJD6uLnJ1Q611bWDVfOh6tNQwoRSoVLSa7/yAkFBECrNeq0rdq9exjAv1z9XqjZWIWc4K3DRUXfO9OqEN/bYh1SlpTnXY0He9CH1pXF+NM8QoYmpaoG1tat5QiImUMnOIOGeJKRNmWVVDSMSQSHE17mxRt7aml1BWllTnCFi5HveluRCCQ6w8vjNDumcp5sGnD7z4EYf0qc9/0puyfUKe8wE3dmWctbvT2k2tn5NBJM9DWF22sshc9n3PdrNpqYL6fgGGzUZ4oGiRsmkmJcZjROKyzhKVbmj0vNa+U0VU2/HbW6bpxcKykjrrmnHmrLFvmyEX40w1djV1ApGVXIrNZR9jWrZVjFVlGuV1R1U4XLvxFWBaPJEaURQVyUqNkCCGmqRGtCQklChqPD3DZsPQDxy4a3lS5x0uZ5iDuv1yDuM4tliyG3ptGCUT6G63xVq3SnN1DMOWvhukEkZ5tlYn0xBHvO8oxVKwbVKJMTKFmRwTISX6vifM0vtGFPNEDUPEDPViVG5zvY8aB0uBz8JSEu9uNdbyR8bwY+P7D7Lv7799ZyUEEGKyyBEubuL99MDaWKvLuigTSFwVQpA+HdPIZrsh56BAg+fi4kLYLNPE4XTiNI6EOZBSpvcd3nZ0rqfrNmyG4Z4Lq4ZXebc17pQTk8FXNHYrYiDWe5zr8UOHqeygXGPogu96KpXPOqWvWUffbyi5a6uY0Nbke5aJylCcURCpgEkK4sjAosgKhbdM44RacivXQiczKmSiZPS8osrlFMlJurOFKEwaa2AYei72O9LLF5xOR6ZpppQipHZgmgMlnBt69RDmGFT+pefq8kpJ90Fjzp6+H9hud/TdFuetkh3m9ru9G8B4jMk4N5CLqCJM04SZO+JcmObA7fEgq1/TmUXuiRpgNdFa7t5AJL22FGUE3TOeSlr4TsHik7s+Hdp92nGe3j7JOE2bv4yWItlGIrDrGKC5pdVIlyV2cYfSWXLdGKOxV8F5y3ga8Z24e+M4EkMA70kpSw/MrGkH57G+w3c9rsqcmIXsYO6tlvpl7WlFTw11ItF6SU2W5aodpL+jNANZ4uoFeKo1oQuqXb+qUgcNpslSZlMoqmxHKW2wpSyTQEqJmDMhrmQ2y0qOcmWYiwRI0tSGKBrKPoHOG3bbjpJ3XF9fUUphDoHbww05C4HA2UCnrQunEHQKEHR4nqTqxtkj6xEnsd+EMVZcYaMnyXJdYkoYk7C20PVSZeS8w2dHjYRkTBTtlFYkXq//UxphG0mlTvgLLrlg00tVULtoqxH84faRVfDJxdE8vUs7ziN7/mHc2nos04jwlWRQ1eDEHksDRdYXTz4s+6RYq1jSUntYL6hKS6QUcU6UDUKQYl9nzLlRGx0czqkUiaZXFDhpK99Z0H7vQtW4uP06+WORHFnYHfc9gvXfLS20+pazYaHGWV9vtYwC0YobjWnK5N57yc/lKn8iOdyqvVSUGHFOMo/Udn61UDqnSMkBZ6DvHGU7cHV5Qc6ZcZq4Pd5BWcgfvhTAEVLUXqNiOFWQy9npbNJbJiraqg5K0VxN1qhxGutwvtfxU72bWvhe2u9DwUd1PpSBBjUd1b703oVeYx3rUfvx7Q+wtD24nQ38T9vtI9uTxqkRlXRocpqEdhZfKzQMEufUEidrqGLDOddUMaS85KmWelDERR4suWQOd0e8k87SnYfoHVnVwec4N3kQa6SK1BYEFXWdDuCMKUK+t1VkTMEVmT9kTaiQe710tqLROZNCaPu361iKELhzWsrM1N1sKgYtJaTJ70oLVAg/p7WQlxFShHOY7LAmY0Jq6K33PdZ1GNu1CamkQElJHwsYVNXZa1FzdXGLyqCQMiTp9P3y5TOGzYCxlsN44vbulsPhKN2xgSkI3c45QTzDnDAmn6HcQPvemuZoOWNjcK5r5+ScI6UZYwuu6zHzSDFF0kcl4E1m4y1zhrCuZKqGZwThqS+duZT1yRnOY+699DcbI3769mkTw6etnNZgO9cq051VOcwG+kRxhOqKYsB5S2e89sYoZ8ibbMIqcs6QUtSeF4beiybRmUtsTKuosFYEnbKxFCOq7dM046x0vc5FKHXZyCCrYFB1Y0G5pc0dgpKQEeAKpEVxfg1uGSqnWMjg66ZMdWXEskJvqxsrBd4lxHZbilWN/LLQIVMW/SJvO1BFhtr1DFVPyMmSSibmQsyCeM6a49O7scT+VJdQ/nZW9JeGXqReri4um9s6hSD0v2EgG8ljJi3vylmKEA6HA5vNhq7rVAdI0V7tSWIQcC57QbBLMWLcWmKXYuKYZgpJxouGMtaK6l9OhWSXPjqmInbVLEsF1epVfGBbTcR/W7YPFvPvsD1tnCtIvarYWbsM2lxSY7nIgpFbA1PrjHDqyGpwGlvF2s5eBpC0x5NaxFpdcEa1MuJ+LTGkJcbMbCPGzhgyzll632E1hpPCedfcr0pNk8MZDCqkxaJFQx0YrAynGrRbCBZLOwW9CmYNgNWKCiGOA1JjmpLkVilks2IB5SVOTDnR1VWilLMbK8QDI7IoxVA59+IJSpFB0WOWts7U94X0YKkGOrDbbhnHib7r4Cj6rr62WCwZkxfBajAttq0ou2kq66trqhRFQdILcwrSU0dd3JACuUQoiUobNoaG8ForZNG14PbqCiyvmOWVheix7LMaOB+O59/Vcn9HI/sgX/odtscZQrXjs8aZtuUQofWbbJUPK7S20v5KvfiWZxcXYISPeXe4a9KYORWwtMG5FDXbxk6hxrIIAmdcx83dgZzvMMYydKJjJJ2tejov3af6filfcm5pwWetpe/2S/V9hds1/SDnvkhqtO7LzZiKkhHUzUtSSG2NWZHVpVZVXHhDyYlRG/7OMSpbaJkAYsmEnFVaVFbq/bBdUGigYMEIgwcSxnm6fkPOiRgCMU5NAsQ6Jy33VAld0iyiGOFdx367J8yBaZx4f3NLRCZWX2twMaICb4X0EWIixJkYA6fTeCbBud1etHx1zhnjHaWIR1OyBSdIbomKTlOEQ1uU9+rFaelwlKnqAxkoNZbVWk8NRq1Rrm8F8Gq5GaX+/4HtMev6BAtaY01/Q9vTMacxypip8WVpPUvEla2s0gUkSSmTYlmlRizWSqfjnLP0sDAG57yqwokBjWMgztKYue9lXrRO2onHXJhD4jBOTPN7fv2rX3M8nUgp8fL5C2EcecuLq2t22y27zcDgXSsr6ocBrznZzncUnxov01gpuzfFqUqg1ZpNIQo4IIe04IIlkuTiiG1rfjLm0kSUpSBYPIRUMsfjkbvjkWmSXF7rS2ItJSemmBhjpBTRh7XG4lXUbOh7nl1ds9tu8M4zbDwlWIhB+MvjqbVVN8YQo5TcSUuC6sZbQsjNOPq+Z7fdcXkxMwwDIUfSnAjapxOqdMiy8lWtoM4LY6vremka5buV5yCpHoMwkSR8yaQ0NxK7obrEOi+mRaisxphLa8BlZa5GCmCd5JJLWQoKIGvZW1mvszTD/ej2iVb3N+wvP90816zdzNJu1IJYyoqzRi+rFH7S1sXWFuYpNOOuujSlxgjGtodoC03EmJTiJ2kbMIzzzO3hQJrvuL07cDqNzPNEKbqPEwPfb8Q4N72n86Jcvt3ttY+KY+iSxIZWylis99jsIHt13a1o1SatWy2IzIhOn9JzJDfgQvq9KFqaEqfjyDhOhJIIKqh8dzxwOI7Ms+QiS1Fwy0uOLqTEHGUgS12mSHF2vaxQMWZePHvGdrth6LvGYpI84Pl9KkXb4TXmUaEWntdYtO87+qHXxk8eM1oVz8qtaVGt2yk17teJste4U6qOZDKoW0qVJ23xrpNhbwqVNAKuTfji7Zc2YVSyxTrOXFJTpj1Hwwjp0K0Gaqq64zoc+E+w3P0Bt09cOVfGSa0dQJ/Tfv9Cjodak1ddweNxXFbf1SxrrMRoQ+8Yhh13t3eM44GcM69evRJ2itZ/3h7uCHPg9s2B3W6P9x3TOPPVt++IpZBK5iv/hr7rpAPZdmA7yOz+/NkzLrcbNkPHbjPg3Z26vj197YzlPN12kJtckBm9QoVlAVwkRowtZhzH6Uye4927G969u+UQJ+6OR8ZpZDpNjEHAkxiSINfe4fqOwXfUNoTOC+BmjLjFxkpu8P3NgRATz64ueXZ9KWh5hjgFAbeMoxgnJX42Ky5QO4sp26ZJbhaG7UDMif40SGmYlTZ6MVVPSLCFCpx2nVyrznfSlsGLjEycI+N0aq6w1eMYY9ntLxSvEA2igqgfGFvofE8uhaipmjBLUytx83VsrTCCuhXFFKR8zLG0nCzks3TLMobPRNH/Dtnq49KYRrmsisqW+m/RJD1oSYdTQKj6/HI1JF6pU99yXFMWYa+QZ+JkiF0iTol5mun7gYsLkcuUln9ScxlipneFF8+fsx3kvaHrCDjmIK3Wx+OJ40nICxYhgQ9dx9Xlnl3vGTrHdjvw+sUz9rsdF6rm55wXTZ0SW5fkFGJLJTUqnaYRYq4kAGncczqduLs78ObNG969v+P97ZFTCsxxJqRIipEQJdZySNNX6z2297x6/oLdsJFO0J1tk+HUd1qbasF5vnnzjvfvb/jNbx2vnl03jaa+7yUWUzDFpKzJf7eUUhk0FWZxMRHLLJS8/Z5hs2E7bZlD4KS55UKh84N2axNQTTwZpw1wUxsLF/sdL58/57NXr/jRD3/MOE7c3tzxP//kJxzHkRwTdrAi7GW8dArwg+ogBQY8MRyhiMuvWaoPONuoYRpryLYqfklYUO4rKOrgrIvHR1HTT3RVP9mj/QO6vp9IQli5GZSGFFa1jdZnsnzwESpyuUx+C4+jesW5ZGKIUNAYVAZdCKG185unGVOg73qut3v6rqfkQuctc4I5BDpbMDFgySJnGWbmeSaGmZQix05SNZvNQMmJy/1IDJHeS62o82KgTsneNT9SDC3xn2tJU9V6HU+8e/+e29tb3r2/4eb2jve3B24OR0nor6tHirh2nfdsBtGP9V3H1js23rHxVlsOCK93M/Ra92koxksv05yYxsD79zTDyfudxPPUAmMUoFrcv4a4671KyeNVvrI1FlZQJylXt+oOVYQ2xSgE9CJuqrOWru948fIZ3//sc/7ki+/xwx98nxAid3dH5mniZ7/8JbeHO+KcwRf1VgY2270Q7aeZt+/eNaUXCW8WBb01saBVDxlDasjPkuP9KBT0WD7jE1fSR3f7HVfmp+z46bbzNV4pS0xQ81hSbUGbltYVJetzNNRBUsuoavdmGhorTW4Cl5dXlFwIc+BkCv3lJX3fczocsJ8ZtsOG51eXAvnnwnbw0iw1GHau4HJg7h1T6DgcYJonYojczLPouzrDZuiJIXDYH4lzkFVru6XfDBSDKqR3dG7AGKHZxTyTY9aCZ2E7zXPkcBj55pu3vHn3lm/fvCUWw+3xwN3phEm50QqtNfTaHmG73bDf7+i9p3eeXY2NnaH3ln7T0w09WK/eiqEYxzRNzGFiPM68f/cOA8ooykJh9K51M1OIW2JXK4r53nooBmek6DulLJOFluJVCZmcM6GU1qvGOWleJKuZa25qP3RcXV7wgx98jx9+8X1+9MX3ef7sGc464qtMZwzv373n7vaWcZzpNirwtRm4unoGxTDNM99886bFqoJOL8HlWlO4KSFIkkhWbhENrsPufOjC34wb+zt+x1MfezyVsoqx5GilQmlnhGTQlbHm+8pKB14RzbO4wZy7LFVBHETcq+86hn5LIbHdbrm6uMAUSCGR5kBJkXEal5VsDqQQiGHGlcxgoessftMRO0OKiXGOnOaJOUSmMZJCZjxNhGlm0/dcX11ydXlJnxK578g+k70S5o0oHBcDmdhaE6SYSTFrDlDO5eb2gOs8L66uMQWGvmfo+qa1671nu+1lFdLrlk3hcDoyv3vPMMiqaZ3Fd7Kie9/Rdzug0JWMGzpmllXlcHuH9VIDaiyQA1KdUmSSMRbjDSULUGTIDNmT+4HdNtK5rj0MaENaA9FqntuoFEqm7zo2e2m9eLW/4Nn1NV9cvaKL8NUvfsH/9f/wv+cf/MO/z5//+Mf8+Z/+kLf/5O+x2ff8D//T/4jte0LM3NwdmcI3yAquvVgUTGurnHpbnXFtDMFax4r2nIoLnK1F5m/GMP8atyfc2mqcNYWgq6fm4dqmblR9SE5qtVouSYj2gTXythaQBqm+N0kq6Hvv2HYdcQrackGI28fDgTkEZbhI49bT6UhJRZsibbjY7+R4KfH29laaxSZx0ZyVWXkKM4fxSNdL4bDrPDZakklYlxcnPGeSKtilHIlxJoSZEGfAqFK9aLLWGkpvhRjhnYqZFQjzzDwKSJRLoVgR2YpZ1CK2g/Qd8c7RD9Jj03cdnT9q6wOL8wZWco9Zc7Ek5bEauU7i9mqaw6xaGBrtwqYrZU2LiIqewagskvce33dY74VS6TxD33NxccHl5SW7QfSdbIbbN284vv0Wezxx88tf8ps482zX05dZKHox4KPIoMwxcDzNOg6KlI+VrCm1aoym9ajRwSeNlLQtRTXIJpe58mob/5lHYs2/A9sTjYxqiVLRxK/MUnWgtZXScOa2fvBQ41xmxXO2Ts2j6puaS014J/D94B3T4a7xdacw8+7ujuPxxOFwZJpGxmnkeDyy6XquL6+EBbPb4Z24fWNamvU65ylZOlKFnDiOJzZDL+VOaSC7jK2ooZ5WqwjJmZQiIU6EOBHD3IgW3nku9pKiGIaB3nc4Yyk5czqcJLaOkcPdLYfDHbFkooH3NzdSzO0cu00v3GXr6TdbVX/o6HzPZiPdwjZDJzWURuonRddVPJKqkm4dmK6Xa2ut9ofRa866gMFrC/lem+Uqh9aYxXD7ns1mw6Yf2AwiU7oZBgbf4a0lh8jNmzd888tfsAdO33zD18dbfvjFc8oItkjLjT54UjGkHAixqOdtROWPjPMWR02rqUuu4zAX7ZSe16k8GrLeSEIfDuI2zn6f7aM2/tdo/E9KY8rAo0HlGLmpcG6YrFzVdZv3+gPEG67dnesKq+ydWnmhyON2u2W73ZDihPOeOWVOIfH+cMB3G0wy/Oqrb/n2/Tu+fvuWkiJX+ws+f/GKz199hu+0Kt8P2L6n7zx/78VLTocDh7sDX331JdvtTlFRkaCcY5T27JrottZKkyTrxK3NiUQi5Zl5lhrUMI3EMEOxONvR9xvcRuond7stxlrubm64Oxz45a9+yasvvmDYSzonDD1hnLg7nrgzg/TBDJnbMBHmiWmeOc0zWZUKLvdbXlw9Yzds2HQDFy+u2W82XG+2bDe1b4EoTMhK7YWdY2rht6eq/lkjcaxLBe8C283AfrdlvNjz7k6U+M1sxPVWt7HrOrabDZ2Xpk7jccQOhb0fePvmW0KObJ9dksLIxns2neHNX/0lX46Wb24nYTAlnaw0FZJSbtIporhYNXVl0oGqKaUkf41LqxKD6ISZpqH8gSdrKnNI3/w9to9mTE37zx98ewIQSjI4FSWrp7EWjq7K7euVsiqvLZQ5owlnaTsXciHmxc1Kuop5K/1QfKdACIbbg7BqptPMzWEEc8u3b2752S9/wc3hwHGeCdPEtzcHvnp/y0+/fCONbZzT+EnOZb8bGLqezlqc7YlZRDu9c8Q5EuZ5UUVXxo/TkjQJDAspQMu/6WweYuR0GkVdwDku9nt2+x3DMHB3d8ft4cjN3YFiLTenE2kcefPulnGaOZyOvHn/nsPxRExRUwu0wSgThVzT0zhyOAaGvqfvBl7OM1f7HfN+z9XF0Ajpzm0llVU9kpXwWc0HFiVQGAzOiDEPOiluhkG4w6kI39Z3dL5ju9myGTYYYwQXcL0AhTkrXXLHpuuYbm4ZXKH3EMNMnKGUxNWlNDkuSJye9BobNFVmjKpPiLttrMNgCSlJ7rUkSgmr0Kq2SmTppXo2dmm1obDOFvzui93HTfAjR3zQp/50Q36iy1iRskNdAWscaRHI26pa21pxzWq81PpBsqRNrPJlU0pYSmOiQL05VrtUVxHnzN1xxDJhiuXuNBHLHbfvD/zyq28khwZM44h1lv5wYjNM8r3GEmdRtnPOst8NXO8vuNjt+Oz5c7zr8Ebcx5ylxjSGVeqD0tILGCODVcvVGiids4BN00jBiPu33dJ1gvJOU1AkOYB13I0Tp5j49vbEOE0cDnd8++YdcxgVBZZBW2Ou+n0YOJ4yxzGokNZGOp3NEZMKhshWe89s9N/m8jTvphbFa7pFB5TQGYX1sxkGhn4jMiEpk43RtoFioNK0SdvU+9o53GibeQtDpgwbehPxrlbHSF5yu9mAkcobKRiXINEYIeQ7lUbxfmk1aXCkUrQb+cJvXlQQ0NVS4o9FK5jGPKj/PUc8ntqeNt+P73EPi/kdjl23R41Tepu45oJaffjVaunc0qrPmFratai7i2GW1vJP2upJdYTM7pZtJ631DIZxmpjCzBREaMoZj3c9z/bPePP+QHpzx1dffyvt1EMgRClX213s2V48409++A+gGGJMHG7e0nVCup7nW3797VeUrzJffb3nX/2Lf87VfkvfOTpl65ymmZgzHRlsxg+DdrYGHwxms5XC7xAYx6Pm6SKH44n95SUvXr+i6zrCPHM3zxxPR/qu43K/53g88u4wQr/n7/+Tf8VPfvpTgvmWq5LoXNb0TU/AM8+ReYrE05HD8Y5pPBGmI0eOwhjqB3ztd5kLzsi17bxravxVe9dAu28NKEkio+kKeOPYDBtmJeRfXV02wGgcJ7qux7mOaQqkkNtg320GrvZ7nl9ecbXditZTKcy//iWeKD01dx1Db9iMEXN6yxyCEDJSUhaRDla7pJsWnEKHcilL2WGrEYba63U95qttNjyjrZprQ/2U7ek9/3oc2fPtCeNcYi6rfM9qnE01T9uafygMXRq5uZQC3jRyQVYXN6VMiIHj8dhSB0m7KrfyLERvZxg8U5wYx5lxHLXAWNTCX7x8yfd+8EN+/A/+Of/kn/0v+O1vf8PPf/5TnEu8fvWc/W7g3be/4d3NJSFEejeQcWQc3bDjYtNhtE2fdZ2IX2ktqKyehuI8YwrNPxLnsJAMXF5esr3Y0/Xa7ChFSoxshoEpJeaUCWPgR3/2Y1784Ef8/X/2X/D1/+n/zFwi+71nPrzh4uKCZ89fcDOKTGSO0JfCt2+/5ubmHbdvv+E4nkglY0uBGLEpMVjYdR27vmc7KCF9EIR33eO0cZ9BJltvkb5ihd3QY9jjrOGrr7+h9z15EPGszveSZvEdhoIzlt4PXF9e8uLqOa+evWY3dGALIU302wFfHN4Z7G6PnRImTxjnKFHiYmc90iaiVjTVPLolF4teeiTmhEoLrCv3WjKmFVgXVPBLf+c9l/L3BYT+U2xPdLYWw6xxZe04JXFcpxUKMhDazAxNQ3Xh457XaFr1c4tC45KpKMusqpX+xS7qddvtQBcT3lpO25nT6URSvujzZ9d88b3v82c//gf82Z/9PaY58Ktf/Yqu79hf7Li+2hPH9yQ8IRa8G+h3F7h+A67D9xtMTtIpzAicXzWSvHNYbykO3OT1db3R1uC6jsurgWG7oet7EjMpSB+TMI1CATSG62fP+OGP/ozPfvRjfvjDH/L85UtCGLF54K7MbPeX7C+vCTZQshjOVddJMbK3lDAqcSDh+57L7Yar3Yar/VYAqO1OZCqHHt/JPVmLnrVctZ631YnSFUvvxbUPUQAbcWUzvktLRRLCOHLW0jvPpnYav7pmu+mJecZMiHFGpSD2O0qayCa29oGlSH4yJy0g0AqW+mahCO/YiuHVkjrxAOyZkdYsC9Bah1Q1ivvh3qMsob+l2xPcWteS0LbWdRqD846uF95lBX1yBUm02Y9Be3iAGl9eJDdKaTGlMYXrZ8+YQuD97a1eZSGcFxJ4kUV59fo5r6+eY7H86jff8rO/+gV3d3dMYeb7X3zBn/7pn/LDH/8Z/WZDionD3R0pJ6YwcZosIRb6zSWbbsv11XO++JPX9BZOd++5dB1D39OZLSFDQmpGu66jG1TzFcR4SUzHHuMc3TBw9bJjv7kQvidgNxtOzkAc+Yuf/4R+2HF1/Zx//b/9L/nBP/6n7J69pBjLP/qHf4/Lyy1ff/MVxnd01jOHAVMKQ9+x6Ts+u7zCd4XdzkMcpUNqKTx//pwff/8HvLi+4uXza2mL2IvY2Wa/o+t8a+p7JnaGTJwGwPvmAnrvKfNECEFzwK6xgqqHE+fIZvD01rIx0JeJXQ/XL67YXV1zOt6Q3wUunj2H0wGTEmbYE46BKURO6u1U9b0YF6X65LSNn7W46HCuhsur6qeylKwBqi1Uc51FJWyE0LHk0P92qSJ81+1R4+y6jqouWGPOysUc+l7dJr8ADTmTkigFLMXY6gLmc70dY6Tn48XFBbd3B2KIWGtI6TxGquLH0zjx6s9f8vL6Od9/+T2+9+Il3755w2++/JKtMeS7G25//XN8CLy43PAv/9k/5S/+4t9zus2cbm559+493/veJVcXe7beMN7cMZOY5yNHB/5iz9XllaRfui2+36kki6CHOYsioHUe03mwDus6BmvYbKRCI+WMBb49jnz97XuevfiC7//gBzx//gJfErdf/4aSJj77sz/jf/O/+s/5d//O8+/+2/+OZ9fPicUIIDUG+o20u5tPt4x3N5xuRW/26uqai/2OH//wz/jRD/6Ei52gq6JqJw/xYpb75xSUa3laozd0VVQ+DL0UZefMdrslUTDO0vel3fvOWrbOYVNkvvmWv/rm59x9+Rvmu3d88aMf47sNne95/ac/Irx7w3w8cPQ9N+PI+8Ndm4xrnWmMiriSZJWtE79zOJ8aTRBWMaNZlBVrI6banlF+C81bq0T5v8vbo8bZ973mJSu30bRGNdY7AYO0sVBupT6yd666N1TDrW6LbNU96bpO4whh49yf6epquxmEVHB1ecne7xoRfjyNpDlwePuWb3/1Cym3KobnVwN/8sVngpbOM9695vnlJfvNwOA8nZE0Qr/dSV51s2Wz2erg6LC156TWTRrykjaykrz3vmDK4vZ7nVB2l5dcTYEr53n92edcXlyIxGdIzHdHbr/6Bu8HLqzj5XbPy+trcs7M48hhPJDDzFRm3s1HYT3lxH6QKp3ryyu+eP0ZL168YLuRFAolY50g5c5XcvhCjK9bdQ+BVRpMuld3zili2xNiICdpnWAV3OuMpacggktSqna6O/Du26/ZX16x2V/RbXZ0m4G8GSDOHObAYRwFVdf0kFF0t+QorKiYKWqc2RgFfgrW5gYsLj1uFqS83pPax6dG1G2E/d22S+AJ49xu98rKkfxkTUu4Kral1RMtXWKSVkYosTrn1iQoV5I82uhV2SgVbQWNH8pixNUwU0q8fv2ai91ORKY2nve3N0zzxPPTc371q98w3o2c3t5gGNlfXrLZ7/lf/mf/iPfvbjmeRnZXl8TDHSUGvLEM/cAwdOwuBq72Gwb1Boom3L3yW2uRsDQNlB4qlfidizQLzBh8J3lCYy3fv7jisx/9eWPngCHPUhI3vR35Dz/9/7LfX2BPR/71P/nH7C/3HE8n3r5/y89ObzmejtzdjozjCe+l3+afvPyMy2fPeHZ9zZ98/j0uri/pB2EPlZwkV7iSkalEitq2oTJtlsVE02AezIwAPc5zsdsRYtAStygxrHd0Bsw8Q7J0wyWpCCZx9/7Iu6+/YTtObC4uuHp+Sekc0Tm+vnnLu9s7DodTUwt0znFxccH7dzecjpk5TcLyyaaBbdaKMFjXyeQv3bmloqbhFla4wosx1pYM5e/8ilm3R43z+vkzDEsNo3SDKqrCp8yZCgKxiE5VoalYFNanJpwlR1pvUM6Zd2/fElOCLLlPfy+hnAX14fuffcbr5y+4vrgkpMIXP/iCy2dXXL98zquXr4lTgJgox8L743ve8R77Pc/bN2+ZQ+SHn30ffC/f41B0ddAeloXeG3pvyGnGO2mSULJRr8lIPWddNa3HWaOVJB0p26WyPxZigpBgmg+kMFNSprOO3veSO+4Tt6e33N4duTve8u27N0zjzHgcMbHnovdcby7oXy6CygOWz7/3PZ6/eM7Ll6/oBidurPfKk1WWjfa/FLkTEXxOZWl/UePPrM1/AIoVRXqjImmX2x3eOk7TyGa3pfceoQ8A1uH6nrwTfnLfd8RxIijFsOu2bF9dsL16yV98fUOIUvOKIvTzPHN7e8vpKLldqWxaVAJFtwlVHEzYDDYXXCf3pObaK8hYm/3msopPUTDp7/jy+bhbOwxSgVKSEghSS9Cvk9qw+heduFrCWI1Tq1ionEmNURsrp1SkjXa8mgIwxvDs6lrjKw+msNtumpvtjWU6TYRxwndeUzEFm2YGB74Y0njXSsa6oZMVuB9w3UBnC50H7wpzmnXlkcFq1K3FCcIpzBUdGE5kRizSTwQkXu68rLoYw1yk3CylRNQmvKhAl3dwse3pDPQGegfdYNXFL3hn6Qfl03rPi5cvuLi8YNj0+M6cdVNrLqAp5Lz0JXFOC+GFP8mCbxqWXoELe8gi7vCm6zAWdtstXecxMckEiJEJ1Ato4522uc8C7jjryVimNHMYR0KMTUi76tkaY1pKZD12WKV8KEb742She9q85EF1X+4ZYM2p1/6pstvfDCT014EGPw4I9Z2KGBsVCE6kIiU+doX+YT48uWqQa7oVQDEW08uNiioYXQ11PfPByoUxhufPnzNstDLfFErf0TnLdjPQ947xdGI8nkR6VjV9Bpvw+45cPNPhW+xui98MdN1Gm/H2eN/jXcG7grMZRr2pmm6oD4oIQTcjtAaHSIgY1zdPylondaC2o99sGJ1jsidOxwNRm9xaU8AWdoOnv74k7gIhR0IKHKexddjKGa5fPOfiYs/Fftt0Y4ehp3PLYO262rBW4svKYZ7nGW9FvEzkR1ZeianhQ15yhFF4tN5aTN/R2Z6L/Z7Oe+I0CUHdaWyNE+qd7u+sNLRwvuduHHl3d+Td3S1zDO2ehjBLDe52e0YiqGSJds+VUJBzoUQhTGAsvlchc1tV8M9JBuuYc8l/ViN9bKT//ttfx/EfXzk7T3aWkqWyw6SEzRnTZC2lP2KVbvQGFXxGLnBKZwaaciZpYv/t27etwdHaMKurUmsfKYW+6/ji1SscRdvmWaAjZ1HLs+6KuN8J6KIFxFb7gnaa8+v6DuNFGqQbtnjfC5fTWJxTRVhTWhWG7Tpc14FzMqFU1TjlqQ6DktVzZuh9m/Wd9VJX6cSN7YeOcLmnP+7ouh5jhJoYVKFhOp1IcfEeKNqiwTkt4xIliF2nNMma0rJrUM23NoYhiOKhJoFlgOdlhaqPWj8ZY2SeZsIsk6XoFzk64xl2Wy4vLui7jhIjp85TYsTXpkwxkucJbwz7Z1dcvHjBdnfBL7/6mp/+4uf89svfCLWxFF01Ba2NIUgNr9VGyiolJicpj4Y/SG8KUkyE5Cmr1pFLxsRQU6VC55NwypYFDPub4fT8Ybenm+dCo+SvIe2c14x/FteJ0p5Xvuh6Zayfravm2ffwgG5MPVFrsY2wbTFIxb5JRrp9dR152IBqylak1TuR4Oi6HuGU6XvOqvykwxhRiS8JOisVLbYWgJflZheW1ubUFoRFOnZX99eo+LZxYDuL9Y4uV+BGQoKu7+k7KbgWLaQVpzfG1oaw8524jdbiTMZZ18bYegJbizwblrisCqqtPZuKB6S0GGdWPaR1gynR9vV4a0RSdOiw+63087RgshHj9B5vLdv9JZuLS4qzTFoMbzUNFY2IUotSvBLn+54Yk7Z8WDNgOXNVF5JB7Va3uKvr8eiso/iatpPihVpKdnag33f7GwxjnywZg/MZ9/xv3a8sBrmeldc3vObZWgB/v/Kd5YKf1XrWRxE1eBCBLIHdBeFzZinKFSaLDOi+71sJkvcDRfm8uWSMka7OXaUUBkMqIkrmXKfNlOp0XH9bVkOtqu5GXDtTcM5iVAzbWAu2toMQEv5mGBhHFX12luI7iQHzVqVFtIX8LNIgNefnrXTkSim0iatAKzjodBIRQGXpBL2kpWr4sQozsnQqi1G7lGXpbVNzkZVs0nnbBN467/DbjXyHBZuttA50Hu8sm/0lw+6CmDNzmKXFQ0XzV8Yp7R+DpoDqr3lqk32yGudavb+NEyc6v6DVTyaLC6dj83cyzodO7W9wAX7UOOd5bs/Xxcat7VxK5BWhvRpkCKKiNk3TmctaV06ggQM19wXr2X5ZXfe7HS+unxFOI5utaKx23dAGknMVnBJ3r0Ucxiy6t9biOwdmKX+TBIiEXjEWrUyJeGUE1WZLoJ6WdGlViZZMipGoKYfUTdrPxYmSgnKSJd1gsVZW7r7riUnYMlY1Z3OMOOs19hMJEmo3sVxWDB91501FKYXJ0zmn7f4yJUWIWbV2RXZymmdCDMQ56jlXlzoyxyDSLfPMOI2cxpNcEGswtlDyTJojNlkS0kelHwaur59BMITjyFhu8Pstw9Uzuv0VP/mP/4Gvv/2G2+OdFMGPp6ZF1FhKBeZpIqZMLWc72z5iAJUQI2m6tOpyp6QV/Q7RQSpNOuYDPOS7gDcP7vrE5z/p8E/v9KhxBu06tTa+rMaXoq5+Oa0axy6uUghBleGX3FNBAmdBaBf3aX2xqrHWvhzbQRLvtaSslqPVfUuSQMMYKTnKKzSvilaJcJXEjuhxajxTjbKu4q3Nuq01rEa0axEF+BQiKUTiPMsqgyHFoHFgBSu0TrKTTmHa9UkGjpPGwFk1WklZOphpRtgUQ45JFdyXhsNlxU8Wd1EmD4shaomHM5ZJ70vOcq5FazNrvjjFKIYZpJQtqGFO09zCjFohUnECC5iS8UbEwcDgNxtZeQ1cXwnx33U9X3/zNV9+/SVff/O1pHGaFu056JdybrHwg5v58A9x0Y0aaAEj12tt2zUWdaVgu6L11mVlC/eBy/sv3TfkD997eMEvH3UECuWD1z/Ffh81zoqk1hsbo+qvzsuqdy7fsbirVfRqaaW+UMFSOm+ee3bSZaH5VX7n5X6v7q3UEDprKbXaQjMCtVV81gx8LWerDBfnjIpZy3OSEe3d1XnXlX2JZ5Y6QAPa4HVpXltywTpLSZGUDDY6rO9wVlblzgnNr6hIWJ14fOdJJsrgdBq7134fRZQiag6wTjB1BNZidmdcUz5sSgClrBrrZjHyJHKa1auJ68kzLG0bYlza9tVQIudMQlDYHANdZ3H0kmrZ7cgF3PGA70WZbzydePvuLW/fveP27lYnvPP02Np7+th29ra590ZbPeVa2dZElTZx1c8ZA64KhH9ktfzYKvph9uHsP/fsuXxg3/f3Wg73mIGfb48a5/F4bBd0cYek+U3SXGLVdBH1c32tKKRdlgu35BrQPp1WjV5Q0moQdRBVt/T68pLPX77GRelZYg34LKtqcZ6ibmP7fKnATE3DSD0nZInfTIXvLag8ZI6iLAdo86QKMrD8Pk3PkAUMAksmUftwmpSJIdD18net4il1peu61uh2mhJD14MphJQY+gFqyZqxLX5eWhsg/TrrqC1IW8GSKUnc7RwTKQRyjFI4npLWu0rN6zirhq/ex2kcRZolyPM6OdUCB4NhHiesM8KDHe94th/YbAZePH/Fi8/+hDkF3n/+Gb/56c/56ue/5Ddff82vv/6K2+Oh1e9W4Krvu/Z7qnJ8KPHpEXpvW+Ma9fm62N+0yVlBM6N52N8TyHnKFf5UI//U48ETxjmOYztQM845ME2LcZLtAkjk0nRG61e3dvCLnBA554bcieDWUjlRXStA3dotF7u9oIRlkUjxncdYL+kRW7kjqtyn06YMjnoOwiEVwEmIADkn5ijxV+P21vOtU3RZVteaq3VeUi05QcoBkyruUAjzhPMdhUGAkzoycmLoHFn7kRptFeh9p+CNumlmEUgW9T+9nmbV5r5olzfEPZQVUOLfqHIrUn6nMXGK4pKnSIiBcZ44TdMijVKvqa6alCKNglGVPiPMrq7b4PsN1jmm08jN3S2//fI33N3ecTgduTsd+MWvfilc2pwbJ7bRHfPcJt0F/Pv0HGGhisst4OTa4zlr8qt9QdtYfMyD/oQTeGyf9Xt/SDLCJwFCYpxJ3FntoFX1bgS4FOevuq/LSUO98bU1eePZmlphgOTpzCKXuZ4dYWEKoakCYdMpAV8RUoq4Oq52zDZCvbOGpcV9jUdrYqQkzTHKb6mlcfeT2g0J1dhTyPFOEdZaJqfUMz1eZflU9XJUK2cx/KKoq9VJoagrVgefiKsZnSByi5uKesFLX88GTMWgHcZCu0c5R3V1Y1u5Q47MKarLW3uSVsU7/b4sQmDeOnrvGeyA6zYUhPCRsufu5pa79+9FcmU88f5wx7ub92RdMb1xy30uMkmiK2kKK/rgQwN6uQVtLNW+nWesIL0nVhqykqqBskyyFfh7+DvMMq4+9v7qdB7c5w9nj2fbo8Z5Opyafx9j1HgrkxOUoh26siUZKf2BjKsphpWRGGs1b6guYpibIJb3gr4CVOnLmncLIXB3OHB7uCPV2bLQYiyZ2aWjWXWjnaHN2KYsaGCrbKgpkSI9K0Ocm/K5zOqLesBa7Curtk/VevXey4qUC4UICNc4xUCOInxtSsIah3XopCSxqPdCkKirrYa3GJ0w4P7qXbCpNPBtjQFM0yTIZ5QYcp5naUOo10/c2sQ8TYQwM8dAyJmgfU4sCNmiKMGjZErUFgcWNlfPuLy45PLikjwHjmPi5uc/p99eM00Tx7s7DnHiy5s3/OyrX3E4HfDOK+FCQLlSJL3RdV0D3Q63J2KUdov3t4dXOrMYWKnTq/xhjAh8S5vVsogD+EwqDxnWveon89F3ltfNsscDH3vakM3Hjvzx7XG0dp5amkhyYeq6Vg6sSLdrd+sKakDtzSG2VCBLf5H6I5uRqrt4PB6BKnkYm1tnjOHd7Q2/+eYroubhYgh453GKdBYipeY4dVIQN9CItGUd+WWJR8gWEhATLswcj0d8P9APV/hexJW9rULQmtIwVUdV2C7zaWQ8Hjjc3Wie09F1M5utdNCKXU/Z7jC+oHoO2soQ6V5moeriYOpMz9l3lpyRmVBW7axIbMmZoMXR8yRSmjGIyyriWblNbjXmDLH2DZX9rLMCkOWlk5rJGRNTW4VDnNntLnDeM50m7t6+J8UkbQCfz2DERf/ZL/6K//DTv+Qvf/YTTElo2cAZwOacayBhjPPj7l+DKh5ATut71RiMlJlZHVu1/DCnTA5La8Iax6+3Zkqf8PqZSf6Or59/zYffe397Aq2tUo1Q+1KcM/9pKXm5biKn2D7EMgGuOY4Vpr/Ppa3u3HraPBwPvHv/jjnMxDiQo6doXk80C2w9aMt1yRoEbUmCNkjEPUXzliJNEsOM892S31ytsu1SGm0Vn4RKFmMihMg0ThhUTSCDNRNDP5EGoefVJHxV8VuOVx86qdVO2fW6ZOGtCmEdzXsKaFTBnzRPpHkSLKAaX01hpdhCkBhlcgza+oBclG1URAqmZIzKfhDlvk/TzDge6N++Yx5nqVK5PVBSwXUDR/eemBOn04G/+vWv+ObNtxxPRzad1/MvjXSyDteqAbXbU+/2A2jmfWT3Y/FpKaVx3deZgbRCitf38bHtU2PLv47372+PG2eIdYpqCGxdDYUbA6ByE7I0UfNx9zeJDGguzv2TXtIXS/fiGCM3N7d8+823nE4j++1A9I4Ue4yNjchrV/C/GOWyOtdzbbFcLlASJUVynElR3MABmvxj0whq54euEjWdlElBpDumcaKUQtepjmqWTmibvmc+HqEYugz9ZnMGism0UbCGldehzXCTrJjVrV0XD+SYBPSZRkIYidPINAYtzRI3t+RIylGBIUGEpxSZNcVigN55UeijYHIQckWCKRimGDmcTty+u+V4N9J3HRf7Hc52AsIlwymOvL19z6++/A0/+cVfcZqOou2bRS3f5IxbEUySdq9ecrX31PPqOvkd47cW8ulFLWqYkktdxtl3NYynvvS7O6nffXucWytOPC3CK+u6BnmeEdBFS6j1HUWKjMSmcsEWf9zd9/GX5ZX6jfKe5XA48fb2ju1+L5UW00znpJ9IMaDCOlBjKDuo+6pNeKr7UIroscZAnk7MxzvGw4HD3ZGuF4GuzWYjyg52kfTMWeKZWWO7MAdBhK3FuY7Ob8gl0/cD282W0+17TtbSGdjt90TrMVh2e2l5V1YAV12h42lqKShrrZbpZXKhgTYxx5a3DDEyz9PyCHWFlHrJlKTl4RQTU8rMOTPFQCxyT0T7VrpOR1nuKVHAo5gkz2p9z9XzF63lAgYOc5D+pT7y23dv+e23X/GzX/5CtGhBa1pLA6qWlXOZsCXtlhcqoqbhit7/5rqujHQx6pUnow8UaKrPsQuw1tv+fOL/iLv84Osf7FIefL3+hoe2B1//2Dk8sD3ZyAiWmsq6rQGq+yU6dQVte5oFZauf5R6rQ3/Jg6ebi8Rmc0702Sy9LlPG2kTJVsAddVWLyVCspCqKaScrMVykRHEH4yytFFJG+pr0A50ThfRilnigrM7PAtYW0RbyDj/0DPsdJheGzcB2u+F0d0dKhTBHwjiCFZ2lFKLUh6qqQorCMKoD0eg1NJq/NBp3VpQ1plnc2RiJQcCdGDSnqfq9IURCmCWFUgkHupolNXi5pln5xWZRFFCvJxfpTGZ3lqHrBOADYioETtydjnzz2y/5erzh7e175nkS+dS6uqPu5apeE2hpDuEEF7Iv2CzEjDYZcT6g10a9IP8fGar1ZmnYVKVkpM5VPad7BnbuSZcHXlu9et9u7y3zH3PNzz93fpyntieqUtbPP/Woi2EKYrrYyPJuWR3/8ePWCe92PDLYHUMlqudETtq3pQ4uA9lEUQ3EqkuuLJ+cIAZKnEnziTCKKHWxnmEj9Z1u1VCpRiv17GyR1ILo6oDtHF3p2LLH4RiGnmE7YL99Sy6GEDPj6YhxHd51hGmk2+50xTBKbYuYbPC+b7N/TlG8gJomCYGUIikqIhtr64h5+VupeHMITGFe2EBF2l5ETffU39JU953DWieKdfX6laIkAc/V1VWLW6eQmErm9P4df/mzn3KbT4QUIQvSnRopYB0C1djSKE2ztueA2qO1KjRmBaSWscPZ59vHViNsvdWx5FZSrn3X0fVWq4bMPRtbj8H7x1n2efj98pH9z+3kvvE/bPgf3x6vSql91QoK9MD9S2NXX1LziObeDHf+ifuntKzD7YeZ5fUQA2/eveH/+F//X/jf/Zt/wz/40Y+WmkWg95IGKClr/jRKOiIbrPGK3sJ8PBFPd4RpZD7ecTydMK7n5evX3B1vsMUy2IGEoVQk0wAhkkNgPh45He84nU68v32/MKFywXUbjPJ+N5sNJYnBhHnGT6M0k+1VrtJnYirMp5EUI2RDd6WKeabgHKSgNLwYpe9oCuQUWrokhEn/VYMcp8aVnZMgoilnIqb1vqxF8cJekmtnDXhryNaSUiSmSImzVHgA3hs2uwvGOfCTn/yE/+b//d9wc7hld7knHwRQspK7EoPQkr6KjqM1stZKtV7lPzvnsSY2/jQ2k7LBJKTSqHoTZplc27ioK+j9UWT0+NrIyXnPZtMzDNLi4YGMTXOlH379w/H64Up4/uLH3/9wBX7g8B9sT7advx8FroBmeccgUHw1zPv5nLUPfPbJh89u+SaZtWqlyVdv3nAYT1JhEUOLZeYw44prpVoWpKbSOqT+UhLw4XQkTTNplkFdjFSpSBwbKemEKe/Y5tyKmnMpxHEkThPj4Y7b23eMxxOHuzu6odfY1MBWiPVB5Unu7k7M08iz62uZNMLMdDqC8/huANdLAbN1goCnRNbYOUyTxH9JSq/CLG5qziv6nSoKhiBsrRACISiCrNU6pUA2tjWnLSVrg6BF7tNqr5OUlzTW8XRHtJYURIHwxRffxxT42S9+wc1BdIKHtGnxYUVQa057KSUs9yZb2v1ByQKVaywPGuXyoUFYbd5gFvbPvTeN5qGdd3Te0XWWrnM4b7APGKeeysNf+Oj7q70e8P4eDDUfdYMf3p7oMnbvAAu2slxwaDFmS5zDhwbKesmv+xdF7NZfIgeX613E0AzcnI6M86TGGRuqGoIhZ9dimlIK1nlEOscS4yyrz+nYnoeQdCq3svJMgTBF0izt1ntFbVNMTKcj8+nE4e4979+9YTyemI8ju6sLrJfibWelKNpZiWtOpxO372/gR1YqX2JkHkeM85Rc6DZS6oWxlAwxRciCUs/T1KpVqvuaYiCnuaVMZiUbhBCkJCzUHGYmBOGrChKrpX5pJcxmTCseqP1pKogTY+BwvGXGMHtRx798/TlzKfziN79mmmcKS0FEu2Nlcf+WFNlqkOq/Vl3nsmJstXFVn67i/Xa89dJpaixa2lisIFvNc0tDJCvdzb00G65u7cPb0yHbJ0d1q/P+tGN9fL9PjjmhOhNmnT48f3+Nmn3H7Sy+WANKerBu01OcIZTMPM+ifVqWIu4a1wxoHxYzg+0UOJkJpzvG4x0hRpJxbC62hARvv33L8e07nLFs+oHjeMIbKcdKKTKPE/M0cnPzlsPNe+Z5FFQ0R7yqrNsC4XTgzjliSNy8v+N4HMVFNlBMkT6ex4PkJ2NVPpDz960/KZoumSkpiMJ8TlDiUkOrxlZrZsM8M4aTxJElEfKsdY+GmMVAjYpwKX+wgT8VAMoaPpxOR7765o2sxjExJcOPTyeKc7y9fY/tPM4gBPtS+5xkUlyAIPmaSkyvFUsFq3nUQjXg3JDbOts3gzV8YPx1NNT9rNXX2uotVUullOa6ey/sLDlkXoVLD2wPvnwOZH4ci3oMpfrdtyfc2moccmpnwXj1WMxiVLK3znR6QSstrtLUSqFB7LT4Z0VKWH1L0aT/NM+8v3nPr7/8kuf7C378xfeJql1UyexAo7aRk7ibBVKYifPEOJ7EnQW6YWC729MXS7Ydd9+84XQ68D684frFC7nhuVIWtZLkNHNzc8u7t2/49a9+Sd/37PZ7Xr1+xWeff87Q9zgjxplzZrvf4zY9xnswkhJJU0VcsyqcC5KYU2gphVSyoMhawlWNsgmipaTVJEGpeFoClpJwZsOsLTQsWN+u5VkP1dVqCVZtw2KsZwyZb97fcnt3x1wMYxHgKKaEL6jhSzOqKhRdhRW1vTI5K6UuZVIqakh2NZ4qY3kx4KWcbBW2niH4Rr+8KgWu/FSjBfdaU2urhq/RgoKSV8f5yMrxCQtKZaI9tX2qqf5egFAp69C7ol3VGM9PZI3CSjS/vNGYIM2g6781Gf3xMxZeb+L27o6vvvmG19fP+OHrz8+Ms/VrQfOCDY+QgR6URSOEbKeqBAPFOAqiPzQeDpyOR3b7HVl7g8zzLJNJycyT0Pxub2/59utv8M5xeXXJ0HVcXV2RQ8AWKDi6/Y7d9TXddtOI5CBGbnT1c15qPZ13y0qaM5kiLJ8wN07z2kiT0u9mpeVJuiQTU5RKlJyXWd6WdgPO3chVxCbBqQx6pFj79njizc0tyTpijtodLpOLkR42Wdo6SJqLJTdJXRlpOeKFBbYesuXsIatofc+AqWSX1XgrbWaQe3I2Mpf0VzXK1kdVv/oPUy1SPm5R5myvP8j2dMxJjTML5qk5oU0s5zd/3ST3PF+13MD1Vv+u31mAb9++5ae/+iW77YZ/+vf/PnZyzcXbbDZtZbhTJLcS1+dpZJ4nQoh0mw19v2G72WMyeCfMnR/98Ie8u9zz7ddf0js4TjPT8cTxODJNor2KyRyOB5y3/NN//I8YfEfXD2wu9uy3e9I0MR4OXH3v+3zxp3/K5z/4AdfPnpOmiRSkuifPs5RiRSHeC0pqCfNinBjUGBPzLOSEtSBaiFFqMWdxPXNOjeh+n4WTVSHh/L5J5y6jDYaNMVqMXQgxcTtNnOZASIm+HxprSwgGWjydM1lX3LwyzLWFlhWzrN3Ls3ClvldYH8JWOqE5V2WspWzLY/W1aoQF8F67qvtKcjBLXuyvY1sVe/+ht8c7W6++ezmDDxPBNQZdO8FnBykPz1z381eUcjbDV7EpjCEZeHt7w6+//pJf/PpX/ODVZ1xu900wbF3fV7eoTJqUEq4f2Gz3dMMgtaArxsqw3XCZr8BkptNRO4h1WO8o7yMpSU7xe599j812w9WzS0wW1lM2hv3lBV3n6TrP5fULLq+uudhfYBMY77UZkxEXNGTmOeB9Zc/YJvlSB2NUVzrF1OLSVoEyS39SkYFJS34xi0zpGU3NFHDnxmn0uqtUEMZYYoGbwx2/+fpLvnrzhrvTkUzh+nLP4XhiOk3MMdD3IspVm+ouSomr468W5/U1/hAMrM1yV+OggTu0R40n2yBR4yxnxBbTGiFV91Y+L6PxzAU+2/4AFnV/Ylpvv2co+rhxnrXuriewXOT1RWww9wdw8f1VkXPjrr7HereV24u6J85ZDuOJtzc3fPPuHc8vrhi6nqECJXWGXs22VUSsFLRnZU/na6Pf5Sudd/SbgV3aU3LCOU839ORsyDliLMQQePHiFRcXl1w8v9QKkUIshc12Iwyh3Yb97ophGKRzt8bWxYFNwo4xNmmcVePwBTRZi6Q18KcsomrzPEsBQFjlMlfSMClGoR+q1mtR4yvOfjBOWnRvJEYcp4mbuxvGecJYy2azZb/dMoXIFCOGoKwfqEnDc4NbTdr33Oa6QnJmeLrCW6klXRzYojFkWQy3AkYPDUOg1j4sRrk+BXN/CH440D5pe8qQP5ygfl/bfzLP2eKXttXVU1c5u8xsFQWvOR0xu8qbtGf5r0anqnZpPvwtEmKIwFTvPcfTiW/evePXb97w+cvX0izWd434XqfaugrN8yyVJt7TbwZpoe5FV1a+QM/FiiLBZrvTwmu5+7vdnovLfVMRvL56Tj8MGG81tjUY54WJ0nn6vqPveww0RQe3yn5La/ZM0BwfiBtXxa5qe4oaX0Jl88SmZtjKwNSdDZqmqW5vbb+A1rgWIzKgjXSvXl7rcGOE0H8aT9zc3lAKXD97xtX+govdnmw6+s3EeDpwPB4wBp5dXzOrwJlIyqzDlXMdJqPuVyG1WdxaaYtRiiVn1Q3WNh+L5eXVxG+r+YuJ5sJ6MbTGKQhEYzphqkZynSCeGumrMf97rnif9l1PUBH5BOP84JBnF31tmNUdlRtUQYFlPS0fHEPP8ezHrGsAjTEC8lmWbmXOsrm6wG8HjHeEkumLlq1lkTKpKoEFVBbTN4VxMJp6ARTsqd/pfUe/3eE1UR+mmb4fSFoU3HUiUO29I6SoCnlGhKu16axwZ2k/qkSNq7Il4ckOXF+YpxMlJaLGd5lCNhCKyItk7YgdlAk0TpNUluTIlCPjODapyxikIsbaDozVOFBWTkt1IVVXyWiTxrqilcRxPnFzd8f797dcXl7wX/2X/xX//J/+c0qGMSTujgd++Vd/yb/9t/+O9zfvm8rifc71cm85u4eyRy0it1gnxAeb5e/Hx9vSXv6xsSzUTUNCKIs2gY/1NpwrPD61fbjnJ372aUjm7JhPHfWJmHORBakuUP2GsnYdVq9/WJpTWvcxWNza0p6wvHHv86Ue1wigaCzMKfCrr77kh68+Y9N19NYSkgzkUgomrwWsjbBJVm5YBV2MWaEEpbpYtpWNeWsx2WCKJfnc6jzF2C0maaMdpDrFGquNj0yLsXN15xQ4EUTRakMk2xTgc04kNc7aLCor8X0NBEUls9fC6aSpnlI5xNZI64iKgGgMVplA9Qau629TLhzHE8fxxDRPXF8/4/Xrz/j8i+/z7t0dg/UMuz2377+k66SIOuf44GAXqZV7yLCOgfo/YwSkqvIxFcj7ANpYgYY8ZJgVAa5DSK9BafcZlUFZD9IPvuQTt09cSp845Adv/z4rp7HLTW7/ri+kYRGeWq+kGsQvgr4VsePsX9rMu1C+ztZ6XRFr3Gmt5W488f/67/8t33/+Qrst+8WthaaiB9C5rvnapRQZzKZgnMPaQtXtwXgkcW3puqFR23BAb/GV0qcghu8MRtsDZmNlxaoxuLZ7L1laIGoFk+qs6irmpbepIJ9WVs1cSEi1TY2bq2pB0KqTaqizrqZFC6dFScxgnPwL0m3M6oRTU06VNVRvgPSzjLy/u+H2cMc0B3786hVXV9f4fuDNzdfsLy7l+rhMMYlSIuls0Leb1e5TW6nNokQv40ALEqBW9K0qRliNl/Nhdzas703m8rTUwSjtBLOhpEJ2BWPX+kBnYAeftv2ePu5j2xPG/LhxrlxP4Iz72GLe+zOoOW/dJ/t88KkziH39vPZMF5e2MkaWWTTEyNvDiVgEKZ3GEWdVOc46jPUN0avSFVAIaaaYjMXhoiM7B04S78nI6maQ9nYOg8sFgpSgFQcX+w05iiKd9z2+hKVvinNkCknj1yVFUUSVQAdnrdRIWUq5Ckb4tRitpRSF9ppAnCdVa1dVgznMzLpatutpwA891nttfyCyMSWLq+y1pYM3juJMbSRBbVx8OJ745S9/yd3dHcMw8P79e377m1+zGbbEMXEX7whx5Hi8k/aJnWMcpzYhrZldlZlTDU5vta5mWiVUvRQjLuvaq23pn/XYa+NlGSuSWqn7GAXN9J5JuCpubi7I1Lcat59sa38Yo/x94tcnGULtGn/gtpYnLf+Doz1womeGKU/urZ6cJaANclO/fvMtz3c7nn/uSbFbUgRWj5VFnsNFGSwuewFZtDsXBaR/0eJ6mgK+60QKshSK900YmwwW+Vyao2jWGtNW7OpVWHVXa1eVapCldvouRdsoloZ+StrOtN+fcyJHAYKkGZQeqxo4KLlfPuU7MUqr/UTJhSztWkR8WgewxamOrMUhRQPj6cTd4Y6YIt47ackAXF5c8IMffI9cLO/efcN/+Muj6M8O0oCoqTawip7MEiOuDalKIWUn3oowbdQbMtWg7UdiWDlwbiJu5+OmXrMK+izjwGFNzeU+Mig/9Kef3u0T39FTe2L7+A6fREKo7lgDgtTRX9fsrb+r6P6lnBvwgydqzhGh+w6TKUViP1ul9w228/zi179iayx/9vw5cRgUaCg4I7SwXCIhJEFmrcXV1aaAMR0uFW1TkBQcEtkM21nVxkWI6jGKovuccEYS5PM0iTvvLLbrgCQaRpqyyOuYU0O9rF2yUjXSWEW8FmCggh5ZGT+13CuXTF5MQPZ1kqt1TqRF17FbUdFroA3QnDMGKSb3nceqLtHpdORwPJByxPeecTrR9x2vX7/mX/7L/4Jxjvz8r37C/+3/PuK9ZzMMhDmRUyGbTE66cp6tT/V+1xjfaFpKjNPaslpla9c3jYLP2EL3FoNstDdR+cA41x8SjWKHtV3Lky+ji3N7MB8DZh5afB7a894Yf2zfuhA9+H0fbo/Xc1LagDlTQjBrI7of/H93h2Dh4yKSGfq0rpimgNGmn3UI/Pu/+J8xY+B//Z/954vafLJ0G0FSYwzSjAcZ3FOM2FKgRHhzkgLtgoApUWPeDLZoRy3vhL3kLdkUvvzVN3ROWv7lODJHlREpSDMi7+iHgf2zK7rtgB96/FYkTGq79TmGtnrmGOuSeZZW6XqNkzXFgg7WyjutPWOM7bX6Qjql6YUUiRG3mErV8c0GjKPlFa1ye5OS12tXtmmadMBGvvr6V7x5/46vvv6SH/7wz7h5/56DOdL3AzEGMThreDhOXBknsl8uEVucOke+1Xn6zjdgDHN+jCWyKopRyHVa3F+RvTHGtKJ+i3C6fb/CHM4H3L3x+hFz+SQresDgP2H3T/nQE2htBXpqHLh+794L668rS9xZD1FYVt1y/wSNcDZps5w5s/BSxP2sAKuGLvhh4NmLF4TpRDHgvPBsc23+Y3Pj5oYwCgIRM/HdCVukXCsV8NZr6z/hkqYghAEylN5RLEzThO0HjSkdWdHTQsb7AWs8KRnu3rxXZQHo9hv84LHekkohVVe3LPxXY2o6ZwHfrHMCnSiIs8ZAqiFsei9pHWe0MqNe6yp0ZnQqk4c1VvAACi4jZ6Kfqa44QNd1/OxnPyWGwH7/jLvxjpvb9/zsP/4Hbm5uyClJmKDXuNEOdVuLcq9XvYYLmgUMqgUprZ38UwFaYTkIi5tbq1us6lU14M3a1Zh6YisfPPn413/8w5/0cjv3J7Yn3doKYldUdf2e2NV5fEiRWsLFAk2bDSvlqgFFZ8dbwJ/zi1njtuVLC0Y6VG8GLq+uePPNKDk9bb2enQg/x5KUeTMzHoIM7pAptydcceRM61XSdz1935NzJGhVh0mWEoXhE6ZZJFKMGG1S+RBjMr7fUYwlJRgPt0zzyJwifrdhuBjwQ0e2Rlp/69TeOa9Fz45U0jLmjMU6wTWNCmstl7E0g+67js5LTWuONbpFV4l6fVUKFEWmkUnQqMstMawM4vq3955f/eqX/PrXvwYMc5qY5om337xlf7HRvqWSDqKURp88GzZrQ2uhdD3D6gGIfLDJ5ZMMcwHUqlu7eHBL9/RVgyyW9Njvgso8mCr6jvs/8QVP7vI0CcHcM0CDoKh6cSzCc6wx1hpta2mUuhhWQaF6IHXX22JZ71ozwtXkQGl9V6IRto3tPalkshVJlWQSxWV877F2x5wkprq7uePnf/Ez3n97g7eef/D3fkyPpYTAdHfgdh41fvP02w3dboffbEhJ3E9pr1AoV1eSznl/Q7g7Ya1hs90w+QMkEX0+loQbOnzfE+eZN3/1LbfHO75+8y2ms3S95/LyghcvX7Ld79jtd81Vq65laxJVY/bV+x757X1nJReLEde7FHVRRcS6TpwJNcS8uuw5UayARKlIK4lSWzBowXjOwuU1RpoVP3/+XPvFyP0sRhAnA3QKQjUQyADNEBcW2cJKymCSTjQFQ8IoE6xiGefjuNx76PhQsdpsskicGFYeiBhAjXn/U2yPTThPegl8B4bQMhMuMLZBQBq5oBXMULeiIm9GGENoukIRC535VsFrNVBlsbQbgM6QRsAAmRvkyZfffsN//f/8f/Bv/tm/oN9upEqjFPquw288fc74ztH5jvJjy+H1iTglxhlCDPiS2fRb8IOs+LngkoHjRBxVClP5ucY6XIh4a8k3B+LphPWeaKDpihfDOM74eSAOibvTkZvDHadJ2TxjYjRwvLkjxsTF1SUhJq6vL1QxXuh/s3ZzW6oylntgtU+L15WXsupGhmkTYLv3xrbrWZRqFXOE3gn90Fp8151VDq2L2Cs3uZbl1c2ynjiXMETccDVQs1DwjK1GWgdBPT/9XeqGLlNxaedyf1tK0BS404nerNoo1uv2hykV01P9jkb++373k3nOc5S2Xsh2acQg1RhLPSG1qLWXWla1eBRaO8rly6DY0oyzTb71vdV+1TjfH2747/+nf8+//sf/DGssISdKBtd7hmGgM4hYleuwZcPlVWA8TLz/6gbijHWwHXo6lQ9JURT6cg6kMGPGkTxNxJhwriN3vbRZT5lUkWOjuUPtwWlMAuMoWMa7kfkkMiI1lqy/TVoxFJWi9HRd1x4xhGWVYEFBa59Ov+YTr4wTqqGU5VpraiNRkeGiMigLG6m2/auNhZZQYmm1tzZOU0pLIZn2nfWTVRiOxVu6NxbaZFzqa0Z72eQ6WB4blssvXYOU1DFzf5X90Ei+q5HVz/whDf1Ttu9gnAbNSsiMXeOa5v8r+zXr63oXVKqHqvBdCsIwicuNN9aIi2SN9AMyYvYVXGpb0V6eVhTd5jjz5ZtvOYaJXdmpJL/sb61l2PRSYeIDNge220K6LFxurjFzpDOGfefZXT0DJHYLd7fMB310Pe9u7jidJrz1XHSXbHZb3DNxqXGW7B3DVmPOXLgYA3boKdZyPEac37ENM7enO559/xVXL6959b3XGEUph+3As4stIKit9154s2v5FWfxiKqc3AuL156kOaU2oE2RRk5FV8ucJDebgTkHUpIeJQZDmCdub2+5fX/LMvVJv9KKEThvpY8pSLsJ1E2sRqWfqtq2mMqEElcTo4oPFpwrZ4ygylgypuA95KQdwoyVzz5gB3USqMDiQ2N0Ib5o46fyKC7zwXj/fbc/xDHq9qhxOq+OixqlV4FeSeLrzJSgWKhdx2qcWTFGY6QCwXnXqudTFCoYWehW1tnm0hoPDdIoAvvr5cYYpy6SrCDGWkzvMYPIgZgk1DbvZqy1xFXPTYwUTOcSGeOJPosigt8MDFcXONdhCoTthvLyOSVHjsc7dtNIDIF+NuyfPaMberQpCtkYImCc03i78PyzAeMciSLi1Skxp8Db92/YXG4YdgOWQuc9/aZnv9u2622tFTe6sYDugRv68K7T7l01Li1tEqsTWrsLel4hhFb7aTC8u7nh2/dv+ObN11IWV0MP5eXWRdd5iWt97ykmkxLap0a+QTKqa/J6UoJG3Uf5vWt2Wev8Jve86xvoQEiWGIW4ULdSQUank5DVw1NpnYuQNBhF98U7yQ2IWtzw5UQeG/0P7/dR+v0nHuu7GO+jxmnd4rpYFeat9KyaFyxmcWc/XPXrzTEYp7S2hg/lWmqy9M+0Biqfd8GRdHDq+RhqECquorXMSTR0XKkqAiKEXIyRusqWRJXZHJuleiEZ5hCIc4BO0jBm04PpgMx2v8FME2mOuDFx+flrXN8xHe6EWI/BY7W2MkFJ6tGLu7vZ75hzwCTLhbnAD56ud7qSWHx1i1c9PdZCXvXaV0Co3tzqYq1d33Uk10L7wlKsrc2fssbWh9ORw+nAcTzINTHVK5H8bUV+ZQzI9Vu7ptV3ai3vJegTrGE5A13l7sUv+k8Nl7zX8rFi8c4qGKaTRIVlqxtd3eF1xGvW16XKpyyVM4+unDVmfWKfD18yj77/sWM8+V2r7cmVs/6yhc1hzoEczKqRa3tRTmS5ps346kW1aN1hqTOp+vU2NxS3KbbrILFeZtyqXWMw2AzfvnvP1nY83+yYQ8L7TJczhKh4SBtKGGvpd55pPhDHI9PhlnSK9MOGftjCix0jiWAyXzx/Qbw7EY8jIxP+T17j+45v/vKGlEYGPFduz3g4EsPENI98Ox7pNgPdpmfMgdN0R0gz3UYM03khDngLlMw8nqQdXymtD0oDodSFxRQhULSbW5jD3GLCmFXAyizMmaLXKBfTSBDyHZkwR94fb7gdb5nTEUwSho4R0mESZU5hQVlJv8QokimGgrMG1+iUNU2zmvxW5lAn7HZeKyS5iY711R01zF0hFwMsExRFyBRS6mYopigbazHYyvsupbRrZ+3D1TP3t09ZzZ7a5/d9/6HtUeP0vW8onjUGu9J9zyVrK/NavY9yLd095Nqc/22WtEBtlLvEL6Wp8VEUYJG/8M40UnxFlVJKjKeR/89f/HvSn/2Y5//wH+vKGYjB02+2Wtu5CC0bI1zUtOlIRKZT4O34nj6eGMLIYCPJW5KFn3zzM7ZXF/S9J31z5Ov/4S9FbjFlOmPJMfDN6Uvevn/HGGZCmtld7xnHkfdH6R6NyVhn6PuBfvC4ztH14pKmGClZUj9rKZJqnHJhyiqPV9q1ryiqTFZZgRga4buUTEiReZZWheF0JObMnCN388jd/J7IyGbbQelIORNyIoeASblyEGVaK5AjFG0W7IrBGY+zBmucqNOrK2m0m7UmZMi1SiQjCg1Z6i2dEjFqxUwNU0Io6hEZQkjtt4sSpgJd6latgag6+GvOta6cH9s+Zizf9fU/9LHW2+NurV/yV6aiaGVppVelNZoCW3Mx6mPFxWVxUWv8IZPlufqSnLhd/lWDqi5ti6z0JuWUeH93y2E8KcK4TBgVoFgYLDrROEM3dADElJjCRAqJmBPJgus7jHeYGDEpgjN4b5mnE/Mos7OzkJIUPd9NB6Y4CehylLK0BNBpF+zOCTldq0bWN6iWhtXBtFZB+KBiZ/Xa/RWhurGpVLmTLOLTMRFCEvddjXNOMyFPZJIIYWkVh0UAHIXfWcgfi4tMzZnaGrLofUNpgmfOdf2NhlwMGdN4x0sOPINyfkVr1pKSaRhGSnVy0haLWshfxa3q91fnNSsIZGzRjuXL+HvYE33ivd/zM8t7H75+7xAfbI+7tU7gbXFngAKZrIjr4srmZpg1QDcNpTujZplaTqSVCxhKOqd0rx1fo3kvIQiYRn1rm8a5x2nkNE/ELE1gsw7yClDVlaVeDectbj9gO0cic7w5MIcZO50IYWY3bNhutlxd7jExCQp6uWE8jczTxHh3ENJ3kdXukI8EAjOBm2+/lXaCux3b3SWbzbaJYTnVuKmobN2q5EeVKamG12Kme8bYnq/yEjW2Sqlo9+jINM2EWJhDYpwCkUwsiUQipEAuCestJbLkpUENs7a3F9qk9BoW4zRZUjPUxr/t3gsnT2C8WpWj55zVQItRd1tiW3FONcVkBd3vstx3ZzwxVv3bANTUyBp4NOKOVxAsW3GLi5AdvLPYBy3gU+LMj0SI5uE/zBPvr//8lLXzSeNsoA4646kYlQwCxJ3VRpnGWTpf8F5l8N2CyOHM4nYZ1d9BeK1rF61k125oU+s2sjrHJHFUBTKNIni/efsN/c8cvTH8q3/6LyQeiYGAoKrSHShhnMWrmJRBFMF9J3L9YQrEMRDzyDFH5uOR2ze/xFiPxeKjMG8MgC3EIiDUGAJTCtjOsdn0vHz9QhT+hk4qVTT57rxtKGqdcIBzMS9NndRVtK74H6ySK5R2GexiMCll5knaNBwOJ22emxnnSHFQPPSDJ76PhBIElLLC5c3G0BW5T7lAjIY8S/kaWRcrrS5JKWFLEn2nvhfNYKBoHFmZh3VSdlbK1IQFZJurm5NU1hizqLRTZOyV7IlBvitEmr5vSueVK3UFb95ciYDBrxYIqlu8XMVH06lF939wl4987uno9qHt42b6yXlOnZYaCJH1Qiy0sII1RXtUOEUlbYPokxGEL5vVhK8z7po4vZxwabFFoa7UdQZX5omz2N7hOgfOEGv5F9Wd1VjZGWJKrc2eab9N4qO8TaIL1IlinlGXyuLF5StZKIN10FmDoaPDY3NPXzKuk36d291O2haoFElDN61TT2xBF+vs37BNo8LYGlMaavyo18GwwKhGorpURAEwFenDGUNiDpF5DkxzkG7WOWNNwXSOZDNjCpQmViKuYk1JOGex2WKsXONSF8dzh6VNNAXhJ5uqsCUXV2/vMn6E/bPWPl7y3msjqUaqSymgxkWS8Ue+HwnV4bKMVz0z7500MrJrdb/6I+o/5YHXPvrC/XceeeGB4z+y30PbpxtnvZ06CLSaqc0wgpjpBek9fe8UQauxSyLdPzG9adZYMuvYVeOJ9lRc1ZKMduXK2ogWXG/pN73ozHZaYaxxSc4SN0vqZ27GYlmS1d6LLk6XOon9SkZ1k9kNnRRWxyzNd7RM0hoaGOKd0OCss6JG0IgCpj1fyNgqdWIMKYczF73oatXqPbO4h0V7deJUuAsN95zmjMlERPU9xSwGOc9M88w4i5JCoTB0A67zzEY6k5WW5mi3opWjOQXsjFliX8k5V/e5NEUFMJLG4pycIse0zdikLUeNm8XbKs1ASztWjT2bx2Ys2VoKgZQNtoiG8b1hRI3pbJ3NDXSdYxg69QBZwKTV9pBX8tjfD75W7tvc8jsfeHnBTZ7YHjXOy92WaZrlhoeomQ/haGZlgSSydq0yTR6y81Idgim6EhTiktYk6/JpTFlEmIqVVav+LaFEm50pdtVtw+B0dg4pEIvn7nTHl99+yZt33/Ls8gX7bc/N7ZHryz3dphflPKtdtu4FIdWIAGLKWj7Wnc/spUgBr5EbnHPWIZqFcF5/W0rtrtRjLjO6UOBEuGtqLq11TvoVldTAIbm5VWDNCHPHycDLuRBybh2rpxgU9Ekc706M08g0zxynEYPUp15uN+ReKkimOUiheS4tTy1Th0yowWZRSzCSDxZvyRC1AICyVJIkCmQjzYWzpl5aPrwoYUXivmqg2RaiSe170X+NkuklnJGSviqBKS6vpxSIMWOsAH7WygS78LitutGO/W5H3wkHeT3pl/P/rIyonHm/56vquVmfF3uf77v+rgf3f+Dvh7bHUynOEZ3HuYyNeYXEgQQhCvIoDF71c+qUXNFDcblUUKrOMmYJ5kH/yUiCzdRZmuay2GzajzbIDO+dZehEH2eaA795+46f/PZL/jR7Or+lYAlxoMteGwrpiub9CmUUl1PaoTtKkVpO6Y6tGqgYVUivA0l+UyniPsRcVQsSKSyaQWecZARQEbZLJYbL6Kqr41LRI2BYjql9OJfSagcoRbWFEiGJ6PM8R6YpcDqdRAAsRawxdM63AZqLhAQpRJI2XCrqBbQkPgrKlQo1yQk0YEpzqmtOUMlQjIwPsyqOF0kUJ1UyygZr2EUdC2W11pYlxy2rs2nP16VgzRXXWN53utLb2hzK45xnuxlUW8o+uGrqrfwdXtcn65dXBv7w/ufvfXT/1fZ4KkX1ZrxzzCaJW1Lk1hg1LjHM2jxG17ZSE+C5xUQxLjWEUIECGQrWGo1rhA5YY9K6GhlQzq76vUVIfd5aBieAzRwS03THz377JUO343p/zd5YkftIgo4awFirxrnEJ973eOfxvseYrnVGk4tQ91tW21oeJwl1IQ4kla9Mdm4xuak1UBTNG0pPkvardHLLNVccU9PlwZhlMiycDWYRzBZFvhADY4zM08w4BqZplDYNpUjNayekemvERc4pEeYo3dBSwphM8UvuuWU4qEjqYlBZ446i51cR0gyLnMzikWKLEAccIuRti8W2cq/cPBOJFwtrblGmur8qrbq6ZkbDUetMEx2zTu6Zsw7nOrzr6PvurAvdg9tHbOPBlz/BoMpH/zh/4xMWzifQWmPonKWobEcSEAypv9OZ0Tp659qAjjFRS+iisldiLgRtK1Dvhq3gQclauIu4S3mp/E+rRi3C6quEaCcSGcrDdBKEQbH8j3/5U9IkoME/+fMfE8KOGKMYZ4ufF1fWGuGqWiuyjfV31VmfBuxUTaSy5Pt0xTPGkazHushkjcqhyBBLOYoSXsqag9O2d7r6xqqyp3KX0zyTcmouq6kEBGukH2dMzGNgPJ6IITLHwFGbLYUQyIiGr7GOXiU4befwO084nTjOgZu7E4fDJFIpGRXdllysoej3gscwo8DTqh9NXWHbUDPLSmjWLp0Bmx2uaPV4ZyUEMlaVLYqWAdrmZspvF08LuntAmMTJzlk63wv42Avo03jH3uNsL1rCznwkjbLaPvL+hy+bR9/969geNc7OiFuXLXgLQbsJW6QwGXX1mruns3pFGGO90Cvov7FctJmpxBS5zdCNg1RoebKCuMeiI6Oxqr4WUyEHdQtjIU2R33z9NRbD5f4CYzzeyqronOThItoZ21jtnF3ZSBUIUQNV48SuWS8FmxJEZb+0hU4nK2250FDWLDFm7bFZB3nttRm0B0pthnuap2aYOSdQ9b9UCpN29Q6nyDRP0pMzipzmusJnmVy8SmMuLJxMJqo0Z4p6zbIIcFEsxmp+UsXHhJ5ncN6qG1+9nvUALQ3YOYNf2zJawBaMk3+LpennmqLq70YAN99ZrNCfJTeaZH/TG1yxlGIZil0hzDWEkskhJZmsDZJJSAtqoefU/vPg9vF3yncw5I+8Yz76zoPb424tsnp6i95gQzLSC9EZ21DPMwJ2Li0VIN2WVwTtlf9d3ZTiZNBbRQsrQlvav2vDlhctNZ1jRa0gR1GDC4UUEu9u7iDDrz7/hv32gt1mx7DbyzGtlL4VyYBr3Ctp84YeV2kLY8QwK+pKVtGx0gYHrNpHqJufi/ymhUH1YYOiFKWTWIyRoKmP2k4+pgWxJSdKysxacRNjIo6JKYmhh0q4YH0u4tF45+Q+1d9iqsso5JGYi+QwS8aSSVbI/HXkiPtocW5Bvz82CBvBPENTu1BwyFhEXEwfjX2EVo6kIoURVOKLhAQ5GbnvirovMbwTbd6C1qhmzRyIN+Os/B7s/Xzo6owfcXXNB08e2Wd1vT58796kcH+OeGJ7MpXinAhc9dGSgsEkobk566jQetK+JAZRGGgDMVdjFWur7c7PviMZii+an9ABpMYs/U7KmfFjDNlkbLKYkHFGaGol6Q3KhThHTqeJ/+4v/gJjPM71bC+u6J0je4s3gqpmBbJsEUN1VtzARhDIWWKZvpNzV7K5MQbjLTYVcrbNgHKUf0uKokU0j0rHk9WxIrFVuT3GSJyDgDlzZJwDh+PUKHy2CB0t58RxmpjC3Fa70No2SMtAAbYMTtM73nt6Lwp3xoiXUWS2xThLzIWYNJbP/7/23qzJluRID/s8IjNPbXetur2vAAYNTGMlCZAEZsGAtBmKNi8ykiY9UD9Ef0AvepXxL+iBNKOZZFpGJo5kI5HCiOSAmgHGBmgsjUYvd+nbd6uqc05mRrge3D0i8uynqm737caNtr511sw8keHh7p+7fy7NjpxnNL4GUYB3rKCYR6jETxVhkusvq5CkgnDGRycAXmKfzjtUI49q5BVfiOCgrBksLg/DoXZeCMuchIuCYhzsCBUJAwQpyt+20jU7REiiQq+FFxzQtxF11aOuS99uJscbywWUlkjn4o/Tivc2OcfysVI4AwlyWMMhNg4cCA4BLfdZYArtECOj74Lu+gZgkLYSZBXMcs9RCL74LxlCzFlLQV+k3AsDDEUGK3Ta1If1ZhM5dF3AOzdvAZFx78E9vPbqq6rZI7ouKkAiu7enCkQsxFq1tGoXPysitlOg76UPSRT8sFYfOWqq4HSaBarve4Qo5F/T6ViIoXVB26bVdR0mkwnarsN02uLhySkm0ykm7RSTts3tHLo2Zb1Muw5BCcv6PpdCSbzSmRxI7NXVqH2NWltKUCUOg5jR0qUsQPiDQugRNXuHXQXSMIowI5j2BmKlDaAAuNonkusICeuYDOSu0moVVUJ/6RuCuJa28UKBLqAn1q7gAJz6iSzgjihPB2KXyuag6y6EiK6PmEwkub/vZH5GTYe6qtE0pjLyels7lpmeRDMadfZ5RrZXP0eSzHVXs1I4+xiTaVR7QqwJpJSS0iOSk38l6XyiLaNqzFkn2n6OvWP/O7gEKInlWuZQ5m/bBLGaOjGIlgydkBtbuIFINMrJ6TFu3SXUFeG9mx/g+cNDuJ0ddFG0EgfxoTxVcOwQnBd/UgW373rEThiCUp6vc3ANwCEgKgjTdS36XgSvD1Is3YcOXdvqnOQemn0IasZqosBkitPxWLqItS2mfZd8827aJuS77XtJ1DDU1BaMainnCN4J0OW8U5QSKWTjHEkbiNChD5KYICEdRXEcQxWlJFSoL+ccCXu8yQWRul+k6HscFFtbuqKkVgJUAa6CcAgXG3oI+sNYKl+8NxJuNYVtxXivqX6U3Rz9ft9HtG1A20ooqdcqFrHoHIh6wRPWwDtLxxKhSq/N+K/Dp/PCO3j1vJqz7Vp4kswS1zg48qirCCKHU21VnqspZNFZ1cHgolPc0/Z65EVFlBoHOUgZmTSikVS9RI7svWhWM3s7Rtt2mIRpApkgETUwNM9v0uEBRQTq8b/9X3+KP/zu7+G1516QkjJqUVcVeEdN0roDxx4WGmdmNUUFYfW+gte0PO5HiL2UpnWTVrpnK4DTdlM1WTtMJxOEmMvVuk47hXUdxpMpptMpxuOxNBFStLbTsqzIjNB1IJJi8baXWCoApX1xOb4MsSArT6hqYUlwKbFCAR1fIUTtt9JOAA5ynFqAL9d4uNqBKjF7ncarXQhw0cFxVHwBiYrGwlplUocz0MdHwEu7eiIxnaPmUMfg0LXSjJfUCnIeqIKVjgkw6NjBoRLGiWSdib/eTiMm04DxpMP4VAQzBCtDM2b5KPndC8zZDSIZ86GQwoFd6Fde8FgtnG2LiqQRTl3V8A2h9hLuaCdTdGQmWygqKJYfz36QUxfTGfRdZOik3F0DGJCRUGOT9l6FuKxQUc2c8V1ONajcMd6/eQd/9oM/x9vPv4Bvf/0b2Ku8IKHTCXow2tBh0k1BynKwiCzZuSqhoEHDI6FgLogc0bZjbYArGjUBQBGYtgFdL2btw4cPRTinEzyanBQdqjPDewgxAR+ggpnCOVROKVtIkskrjUd7X2m8WZBw8gZwQdBoOHhUqCpZuGgcKl9hZ6fGaLfBqJLvExPI6IlIw0ZWpmXUqDrjpC6HKGknRdrs5F6YL2rxW2170fcahmKBGkIvbA3QYzgSKw0Qt8iwjRAkf3gy7jGedDgZtwjTmJBx2AbCBbH5zNgkO+dxjlnzeNlY7XN2QO8ieh8xqiSXlB3AlZAid65PeZB6fzCUzixwKZVNvUqniGJiCE8+oKVQcTqC3HQxgPMrs8OEmRPcLzu2mL+nkwkenJziwckJjsdjuGaEpnIIUcJFIQb0vgex+lohM6IT1NSiSjNQvAhmzOgraxx3Oh2ntn3mb4YQ0UdgPO00Kb3F8elpatHQqYCbQKbgu/VRUSEkRcedgiwSpxVE3TvZRMllFyGbvfk+WD60tHQQLVx5yYeuKyecQZqBT3lqbd0n1JGYhN3EWvoVnzWBJKXoHPh6jMy8GOWuhUgCWBUF3hKeiwAZI4R+nZUMTC0rKCNGgvcN2SjU5ScljMvAIB48WH5tq3ulTMVZ7yimHVboQgg7VS0onu8BtHIxzoFsp9ezl3FA28kMBfZeGeWMcRyc/BjLK03aQXB9AJTin/M/WL5L+kXvBLgJXcS07RGI0DLjzr37oEsH2K1r1LX0C6kq7X7NguSWVfRlbixBzHvRhhJnTOgyB4zHJ2I26+tBu4S1fcTJdIpp22E8GePk5ESbFUlmkGneEDJXbarQ0PM7DZNUzgOVpasRauck5KV9DixpItGA6CIROk7xz+vGw9cE3wjPUlN71JVD5UgKDCjn66QtM4WPIOl5uvlxKDK/rLSm0K7pRhqWEBguFV8zenBK86QY4VlL2JyGeZjgqJLQHWtMFjlkRJ61cGKeeX4bwdyUoWDTcd5NYbVwRqDvIlonMTbvHMh7+FGF0d5Ig8mMk8kEEaHgTzW/zY409D+99xiNRmLOmimqmiIoUsngVB1hmUhgSon0xZLJgglLmZNzkquwt7eH3Z1d7Ozu4oNbt3H7zh389Gdv4ZtvfBHPHx7i2aPraJxixUSo63qAPqfmrgx4V2verUtx3D5oQYBWk7TtNKXzZc0ZMJm2OO16tJp00E6nKZbZhy4nLejmQoD6fRqOMIFTHx1VpdUeIpxO487sshvgvIerhC/XOZ/rRDkKSbZ3qCuHuvJJazpYfSgXzPOy8TJLaEshdkhlCcD9MJYNmMbU2lkWQCnqhgsPuEYzEQIntgKOhK6TOKVUtChXrhNXpieSskRfYX+3xqju0Y06xTs6AfCYUSs2YOttUyH5pM3d2bG2P6cFn4Pu7EwSq6prjxgqjEYV6tonRNJGyoWcGdI9WkELJzQkvTGtKcxupqQlpOf2diQAAoZI7uCKKZt+VVUJM7reqD4EtF3EeDzGW7/8JdrpBEfXrkiKnn4/Iay6MMtmuM5FCB2HS35xicQakZZ1oY4xpIT2rm0xnkwF2FHUNjG6J7o7pL47yYJUrWOCKUInNapEojGdbnJk1h1lbSXXLkKuVqO6FLkFH2nqol2D3YegXbYL80d3Dah5rLHFFDbLGUVMYvYimMAWZVuWtE7iKhlibxlBITJcZK0mYa0ZlpCRU+DLwevmbXfebGVIE+H8xqd2rG1kxFBwIkqXrIokm6OuK3CM6EONuqkk2ToWpEqF4CRTSBdXUzcpfQ5gwBrxlGhYqi5wg4lOoZbsnSbH2nwNCSV4NcOVsU+5ZUKMGJ+O8dav3gaB8bfe/G0pJyPhS2onwlPbKzpKqrm8y8n+dpnS2ToqADRsCW8JB6ZReuUbajWbx94zJNd+gVP/SuaMza6XTccqf5zkpxobYpofRUWLSREPTAXUSLhSA12v2tV8VEZu0suW4VSYrMXOYS5IiEFK+/Q7RGJ6SoaPaq+kTTNKKqASAcSW0QwwyToL0sPTgVKcl1XonRPgCy4ieAkfiVKIaQrIeRQm1IWbqx/XWF0yVle6GAldH6VTtA+ovENTeXiMQCBcvnQglR6nY8Q4SQvOqZYxFKGqKtTagBXwYGLhs0FEx0rxmOgwSSfZJyQiQqotIjtEHwCvfhAr8gstUSLRuHVTa6G1ZZNECXH0LQIBHRF2Dw7wzI0bIGbRbtMdTCYTTKYT8LSFkVqHPoC5VUtCEOqul8LlyWSSzOAYoyQUTCcYTzvRPlDGdFWHpalVtswD7H19aIKZTHv5v/ZeAZ1hu71FfiYlYSfN5+31dbViFJ0Fy/wgAtxbeZsdU3JyFakBESXmwFSpohumEbKBCZ4jEJyUiwWWXGUVcHMjnJOQCUG0cR8YrgcAhlelEKPkUDNLgoikYNbwqMD1CEwMXxH6vgNgRdxaN8wDPfExj8fpcwaokx3UnACIGLUjaQdQOTRcYWd3JG3WWQQ4psY6LoEYRIS6kjq7upZC5sAz5FuYXWS20DQUwFKWVjmnaV4M7xV5ZMsFliSJqLty10tYI5vGwshWIq0hBlSOUNUel+p97B3sJUYE0uY/XdtKJpDmw07GE4zHwLQd4+T0EbpWfR/VuEGzZqQjt/ozJObjoNZz1vrXuGVml9PFrHWJzrvUGxNzQmjAmWg3qa+UJAFyYj0ETbhvewZVDjXLcpZE9yjgTk+InaKhai47QMzZVCmUtWn0OWkkg+na/UvNL3MDiBzYCXuEbVbmixIDfa+ZTgCi0yQXZvSR1MQmgLOVJKRgHjVXCU02nxk9KynZtmJRLshzfPecY3UoJXBawM4x6krq5/o6a6UKQgURVfNNJ73cLJjmFIHy3qtwenhXSdgrRoSQgXab7CScLj8nApiEh8aRk+B5JVUMlfOyeDQ7JKiWB7OAMiEKMqwgSl17dF1M5pkIp9eyqQa+ruGqSnl/RKt0kxYnp8doO6kgcZ4Ax5i2YwijQ4++bzGdtLLonABfFnoQpyCDFG4ghJyKulGEDRJZDuXC5UwhQklw09yl1+xYSK6JWBBalBAj2i7CVQ6jyAqkaklbBGJPiEoJk4RAdz9xERSGI6Q+N/o0X7v8KoP7hBMpsm5WRidjwQ/biIDQC2dwoMK8ZknS5xhEMKMlKri0tjhfrJjaIUpDpUS0fUYpWyLYtOLZRY3V6XsD9vGIuhYwoqoIVaNV5gzsNLWgrcyYjjtoVhyMk9s5h92dXYG9yanvJ8EzB/EP2cvnmfOCyHmaUB+JpHrBVfANw1neJrkkBGBg2vbAVBwsM8sYEfVI2uxdurSHru9w5coljJpGFrw23q0qo+TUpAdtebizUwO0h6aXFn11XePypUu4cXSEF154Aaenp3h0fIwH9+9jMpX2gZPJNCG/fR+AqpJUGP19aTGbLUgEa31oSxskZnrjK1SaFeTs46DcN4ay7ykCI5k+vnLwlQh1XdXwrgZHh8m0B1xA0wR4L/MDDoiRwNFJYrVWi3jnAO9AXtL9KIZkrpcmufwemTcrkwsc1G/UzwRpaBR8RNSYsfXZBJAqhQhArIUSB5HBPaPrI5h7CXe5KGwYahX4qgIZJ3AgRNIcby5LuHWs0KRrQo+bHGLDsV6gVwrnVOsMY4zwDui6gKoSrpquD4BmVWbGBFkMHDRmBSi/kPiAAi7FtBtK9ouEKzyTpu+VP1sY16DgB5JWFVJnK2TJeZ8Roc/2FUOE2fmc9ECR0YcO+zu72N/dReUlPETew1WVUDva4qbcyME5McuTYuKI4AghODh3gL29XVy5egVHR9cxmUwwnUzw6OFJQm6n0yl6dtImNko2DGeGtKQF46CviwO5CkQeNRy8mnOW+QRDRcklC2MAYScTU8zqq1eu49GkxXu37qDtevReWjMEn88pBGry1JGUCjpPoIrBzniTrLQLWlhPer3ipyqVrfaBkd1WCtPl2kIMEgp1rBuNTz6rC4wQpJCeGcl/FN6lXjQ6RwQSKy54h7rOZi4U8AKAqpI1WPYeNelbKlxc/uGF7y14Z+VnF7+0XrzXmLWZTzVo1kzoNeNF6RBrBUwSXYmTOyYdkA3YUeSQC1NFj8uICk5YOVk5OxGqTzHo82jAg/X8jBm9DdqsR4aFH5ByUCWeGnD10iVcvXRJEUsTUJV4c9tMuyELqGcH9g7BV0nzSahGzrm3t5vimDujXcmn7QWpnQZhPuhDkALpEAtCMEoMfAlcIQZpXLVSaCo1mdVrZZ83LbNKjABIrluT7Jhx+dJlXDkZg9gh9kBoI7ouwMdMYWp0MUhJIOrXea+doyFIrJ8r/gMQRfNqiQnpvWOVdCsRM/NICLwEqU0sE0peHYOQipUWUQxRuItDBDsnefMhx8LNpJcwnFPWPZ/6zKyLY656/zzflQ9sr23XcgiZbySxPQlQd50kucvrtuvZ7i+Lv+87IWQmYXUrwwqsvqAAJz2AkAQw8f/qzkuixMAxyM02gSSSHbaXiYlgYQSMnBKznVZTiKCJxiSS5rpvvvEGPvfii2jUD3YJ/RyCUg7qFsaY/CpHhLqSeC8FGpBij5oGo6YBHxzg8uUrBbrbY9pKYvu0U2qRaYt20iJ0ZvpFZdUT31A2B+FuIpJaTcvxhSGfjpRjVkpK+uAh7Q57hDCFpBx6EDyev3EDfd/DdRFhHIFAaOsOTVPD+p4yizmLyNJ3M8VVHYLOO+DAMSaWATbhAoEpqjmeN2fxqcUHlL1DUzUpQsq7U7kBXA9NRTSjiTSipBZHHxE6sbo9Ab0jRBbU2eKbJqh1Xas/yuuFBxcggBd0DBtr2PccHNUAhPOHQegD4DqH7nQKqnug8kK0hIjGQZr+cEREBEWXwIBucozI8jzGINw6sqJlkSdUIefWigkMiOg5UBQmBoC117uZgmwgnhwimb+ijc0kja6X4xJwdPUKrl++kjNvzHdzOXSR/DoS5gcrfAayT5w6PuviNP46APAjn4Lzfd+jbhqEyGhDj51+D13boZ1OETvJMCcQJiGjvTEEeF2E3nvUXkEqcghkcUYGwxLChaYzs8dXRYaVw43DI4ynraDHQVgQ+s42YdL2ffJbKP0MSkLmoYkM2lzFLAcyYCcCYJfmgEh4dbMVQkkji7vAAEWlLIGEcYqYdoxBATAhc6tdJb4ka4IHyf2NPSGgR81S4pjj2/n6Sh9vuXycH9i5yCSjDZgQSP1CmTQp9o3oOoHpwYyaxKzLwI3T7A6CxcBi3yPACxFy6FRjyi4JbYoL0tuqJoDtMgy2GLNOAKeFw5AcTQADjtOcdqdeY4kMhh67oxF2R6OMnOrikYqI/Bz6rwiuJr2z5ICKT0qg1JKditubM1MjK7dPZLgosT/SChLvfabABMDWN4WlkLtywtFb1zXqqk4hpl5R1y5ExCjMfxwDKExhSY3lbwBRYkdo6lqM0MgIPSNWMuGOFQMz87P8NWbSQgtM7CO2GO25bmak90ni3DEh7iACsSZ0KBhW1B/NCRSRwIreE7xXChInpq+BZn0MoCCpfY4I0UkpGjnd7Gcl5swyuP6Lho9tdorVn1qTvicj0ybm0EpFXpIBCFJpr+fxjlB5h+Cl4gMQikxAzc8I9ErjIRkxOW0sVU+YkVNMaubJAaynCIPA5JNw2oGc8nIRec3p1P4ezOA+InQtmqrCqKkHwinT5SFtGpQjSS8hggU0ItnFETj/ZgkAo+Ts1QvVxk+MLnqg64GovUiUXMz7OpWdhb6HU7YFTwSqqxTiGY12pP2gCmenDH11lAwlcS86cDuBLdoEpshOh24yBYeIqweXcPf+fXQhousIXsvHWKtIEBkOMXUsl2NQmmHJO6BCMPUkkWXuYNlADKYAdgr+uDLElv3i5C+Q9v70UFO6gndV8iXrWiwFHzpYCEp6igYwe8Towc4jWrYQR0QsbwO4bNAZpHf2O+sUKJX4ypKxPn1PbMzksJvJOdW85YgAqjy8F201qgXR9OQw5l44b2LUY8R0HEkgp2w+2o4zc87so2ZzEsjUlklIC7/C4nAEwOtiAYArO/u4cniAL778Cp45uoHRzg4QA0ofc3aY/rCEfeec3O6qAgVCHzNDO7mYQTAiVLVMLzPDsfAuhRDh+4CqkoXW9wE9EToAHKMY9/r7HUmqWu0rzQpSojESH1CyKUgQdY7oQgCUkNqpy8CaoeXIIXQdrhwc4B/+wfdw+1//a3z06B6i8u7GKN+DFkATCFQ3cD2SNQHNf+17S5mTmW2cZHtJP+RksCKwFF97jiDqAPRaYK1JDYUrYrau1yIH4Z+t4Cvt0MbASAIEYAgpmlXykGbWEwk6L50GApj75Brp6dYoK1rwaN1Y/snF72x+5JXCKf4ep8WiBpLEmSKjD9LPsQ8G+xtdhhA1eRfV6TeQxyrU1XxxNFA0yU5iyVaRhOqYslfM7DXtITEUKeSSXdoEgxVQssUGXLtyFS/euIFnD6/jjVdfw6X9fdReuHCMENulVMFiKgu/0ybXkUN0ggI6ON04ZHPIRMyU8lwJSNdhcygqiBNDgLKTwcIhzJwqUBJgZZsRoeBl0rBU5LQJ2iokmye99hACmrrBa6++gsoLSAdNNkcU5oCypjI4Qea9E2YK26zNfBWgxv7XAu+oKyWZuOpjOi8sfqTnYmhZmllKMjPOZpAEh7CcYkAygaqoHMrRgSGJDXCCyDvFyEyrCgFakbhv876NZqRFD7c1Ws9mR68WzkJ7wUHZugWtLIPQfd+LCWms2+QRHaHykmki/TEgLG9OTTZSf44gOcocdU7Fn4NytiKatgXYFTSUydTWn28gkPcZQyQScmRyeO3ll/G1L34RLz/7HF5+5hlUToVAP+ctIK6bEPSmW1V+mmRWc5kC2EmrAZkjEzDWsA2leKGYgrqD60YTe61k6VqEXiphEIV4LOr5UmVNlYGdwgsXhDelIKqQQ04q4QSvm44ITggBO3t7eOG5Z2H5sdRXKW+WiVDW0lZO4sbBsXXJEL+z9DVtK9INIWo4hvW9MofYOw8miYcKWTYp1gttCakCaekrBeJEpLF0JTjvg5jhErETq02wDkXuwQkcWxHVXD+WfJU2fbbgyaaiuqaesywBg6RDsfp8HHX3DnDTCGYPIgUtPFCD0NUVcsWRcdDoDp/8SxQM6rJwraSKLfyiW3TOtc1mqKClhKapsbe3iwcPHkhoI0Z0fY+/962/g29+/ev4w+//AaaPjsFdJxSZfQdi6YpmmUxpQSBvTAZbygZhGTxqgxVAh6GyaXETwfmYBKRxlXQCg2giidVqNY7OQR96Ra5t4xKhZ9YkDjIfMioFZwRCFK6iEKUfTOjkJzjAkU8pf6SC3XYd7t+/h66bIvQBvvW6uWa/XTZdoKaAUNUIJORnxsfTOChtKVIYKy8Uq0qy1LuQ7qFTkFDoZTS+G9Vi0CL2shVGCAFOw1QoCMLMDnbOwUMJzbzT0JnFwpG8zW1E82xizMufLXiy6TnWorXFk8EOxMxwUROaY9AYqO6XWq9X5sqWlRKmAdL/evNYF2swk6Qw0XKSg7GZO+2/KJT93nsENQtHVY2mrvH6q6/iW1//Bt780hsYMQAvSKscy4NMUFSLM6BxvOxnCs5RFpEnuy4JERfvDUqnnDMdl/3nwS+XeS1N1jJfNsXsXHkfzK9WUuquE22sxzLzkkgaEycqGEsE0Zjra6++Aibgw3sfoejnp+bk0NS3OKlRklgD5bSxUP5NxrQQNMwTOYim9GnX0+vP/V1jtLAMI1YRIfToOmA67aS203kwucxq2IdUlK8pMHJ1hWDrhcn/WwwaPDiHxl164M3HauF0BqUQ2OU6RjHVLA3PwXhykgmhAe3sj8hWZgFl+81JMG2XVQGNzNpdOYcDjKgqmXq1T71AS4b2ndEIBzu7OLxyDb//97+Lb3zlt/Hy88/h4d27AlRpfip7rSIsNg0RnnTRstB1IQIqM+Z/ayiA0m/I2tOO5Zh0nsTQND7adM/1+2YBhBA0BOBS+Cql5eUZAyCJ9tY8CckHlYPanJeJ8k6zh2wz/fKXvgQmwp27d5PgpqoW5BiuVxYFEVCHlIgecggkowqS/RRC1LpMApSMyNUOGnGDsbwL84XMm2OGdDAP6Pu8wXV9SH5zUN6mPkhljUMUoWTSHiyyVhicOp1Jd7NFi9vmc5UALJOo4ntLPjJ35DPI+UrhHO2QmnKQnEYS4WJisMaZYmB0rfpiiHC+g0+U+z3gIqjKHcbSDdXdnpmlAoE0b5MgDj5LQrRTcKDyIoSVr7C7u4tmVINjxGQ8xuG1Q1y/fojnnnsOl3f38Z1v/11851vfxuTkGKGbou9aUC2+sOXLTrpWzFprgqPZNpyAKvWAfAWmiLZtU7aRgS4inLL4I1uVSxbQGFX7q0VgwE0qXoYlP0Arf1xKNQMgv9mAFp27wAFd30rzXZKgew1CN5WSNgAZQFJgR8teEbteTL0Q8Q//4A+wt7+Pf/8Xf5E2B9n4vMaqPfb29jAa5TZ6kSWFs+ulbYS1OozooWXaA+7iGJWpwClhl1qyHIDYBRV0M45EBwrFJcNpnjQlhNrKziICd4gIqIhQe7mvnoq2jWbcRKeF4MtW+AVqxpmRZXb+HBdj1lZkLkHegSx1xIvgguS2hCglYyGEtHChtZPOyoMGAWe5zLSrkWWQIO1+4MybU1V12sXZAfv7+7hy6TKeObqBo6uHeOWll/ClL76BmgjPHh6iAYO9w7SXZjaVE1Z3M0udh1y/ZtyklvQDLW/J8x71aGQOMiIFiV+qD55aU2ghsTD3GXWn+qeGQBtAQg7RQeotKVsFrjSbbSOY2disfydFKwHLFDEDf9x5LTxQ89hXqOoao6bB6ckpxuOJXEthVhv1Z9k6ENDNIXKqgxWmiDAnnBk5NnyMdUPX0A/DgOlsQFBhoTGkppQYMWr4hSj5qhY7dRWBKg9H1cD9EsNGk+8tbWypNFDx53EJ6rxqvRBAyDLqhgJFKqAu+T9i0CjBdJQdUrQMJzS2YI3QwaaAhOzAuBe5OA9y2VhV1zg4uISdZoS6qvDs0Q08/+zz+MobX8bVg8t47eWX8ZUvfxnc9+inE/TTMajklXEegLbgQ1RTTi4ul29xNnOpXDxiSnMfVcOz4UG6oGV3N4uVEcABYDagpbDl7ZdRTmKPHIfCWcRp03d0ncFM/OgRKDP2WcjAqDGTUJJL5/GVF6GrKpyMJ6irCi++8ALqpkkE1wDDqE+8d+n6jYys7PUSU5JFb96n/BvtJ5C2sABSWw62NaTPSTO6FKu1krmoHEViiejxbHPyEeQrnRCXrM+U32tuEyMhwotGml8ePDv7uIBDlGM1WssQOz8wXHT5/Gp6DhBXkKSTdQKRey/vu4pQRcAHSYhO+C8hVT94Ek1c+gYMArlKwgtM2Nndxfd+5/fx5S++gddfegWNb3DtyhV84fXX0Y9PEWPAdHIqPyj2QIzo21bDCwpCqSZjRkJ5HeW4mqb46+9SLV4ANVXTgJgRgmS4xBA0JKLkUyo44B6Se2rIpSxCa3/OUSwKdk56N2kieOUl1GB9P5mhlCuizZynVE8ZGeAQ0HFEpw2CWa0OCQ1pewanVgAcXEWAE0bFwyvX8cd/9I/xz//5f4VJ1+LP/uzP8L/8yZ/grZ+9BaII7yXFhEnCNJPxFM5XCEH6iYbQpQ2hrAJKpWEMAOJnEiQvtqq0MTGcIM06NdKaUdZD23bJfLWeMMklYmiTI4KrGAHSztH5nMFGelADgouAw9y4KF05gAQucKwWzo4QeiD0AukbRb/VzYFz6RcBmpTuEKNHXUulAMSVw04DdJoP2seAHmaiaBaPklDt7e5hOp6Ce8Z/8U/+S7zy8su4dvUqukmL5599DlcuX8G1S5dBAOpKQiJQsefQC/VJkMyQAtuBlaaRXqNOawEIaYIDsuVuWSfiE4YEAAHCCRSJUnpeiEELe62AGCB0Wsxii1UOLGBNATbpTY0sCfLC6ACM6gb1aISqqlH5Bj0iepbMF6lttM1Gjutd5q+tnBck2mWqGGMyFO3G6LsO7ckYV69dxfd//3v42le+iv/9T/8Uf/OTv8F777+Hum7QtlNhZwcwnU4S8AedIvtZ9hvN/GXFIMg5VOTQaHMl6SCuy07NKmmbyIghgtxEaS47EUpDhU04o1g46CWUExyETYPs/mq/0cCJmnS5zzk/5jGg1epw+PnNVCfNn2ThWM8hpP8TINot+YbaMiEwQm+kUaTtACRLiCo1ORxQQ/JRCVLZYiwFo2aEph7JFhoJLz77Ejx57DU7+N1vfwefe/11HF0/xMnJcUrtGlW1VIdwFOoKqBNjNaIxxxpTEjZEq4gZlYUzxU4LyzNhPj5rUYvF2nCkVBseknRdUEtKtpT4zJbFb2EZuQT9zzmNdSJz8VIOP1V1jbpuROO4CjF2oIAUK45saZHiPiQ/05ufmRM2hO0hlwCCpZVEO5nCg/Dic8/jtVdfw+npBPt7+zi8dojRbo233/kV7t27l5BejowAW5TDtMkSebepsgSPynk0lVCiWimXsQKCKJnLua0Hg9o2+fUysoaWmmEpThdgSsimiex1DcetUp0LxmJBHgpTKVv582mrWjhsvuQ7m+0Wa2hKxGTjKGZg5YSDxzQFx4gYevRdn3ZvpyEU572kaSkTgSQoK7gUgNgydvd28fLzr+DK5avopwGnDyb4o9/9Q/zdb30L3/zGNxCmUw22M67uH8DCNxSzL0msHoXYiwl0EctbzDs4hx6UFGaZimf5qtBd2ma+zESyRAOKWTsQ0saPqtb2XIDkuXa6cUCcaXKEiio1z1RezZmCzKVYE4xmdw8Oov1GOzuykNUcpmi/0xI2RNuY7+7JYVTXqY28r+y3uCTozluObETsenSTKY4fPEToe+zu7eH3vvM7+N7v/T585bG3v4f/7l/8C/zbf/dvcfvOTUQOmLZT3L93XzqKF/fBEjByZ3MzqyU3eFQ32BmNUNc1mtFI2PWdV87cTImTQkhECylGU+G9BnCCosIimNDuAS4JgCW+nG8Mv7/4cKvPcZZLWI3WsksIrYEMRBJY72MnLcuDNC7lKMLXeeMWjcCu+m4geFejdoTd/R288cWXcbB7Cc8ePYuvf+2b+NznPo/G14h9wLWrl3FwsA8KfeKEyZAxa2aSJGdbRpE0zrBfb4tEFrFT7Vg1DfrYC6cNsybOS5A93UDSpG8Fa6TizRaFmb9WPseDAHfyS9Vkc84hBCdVEbqICWogRNaYnV6naklfaUYPiSlYN02ROshJMPvUnFhiE0RSKePUr/OVg6vEx9QtU/A7zbVN8V2INpxOp8Lu5z2a0RQ+VuBYYcyn+Md/9I/wu9/9LkKM+NWvf4W3334bf/7nP8C7776LB48e4OGjh5Be4woJRaM3VSDQCKRrh6ZpUDcN6lGDqq7FRbJr6iuQ+rRG0u29198IhNCjqRtp1lQ5dNMJPIQDmcGAJ/imgnfCkGAd8CRktxixLTCmJW/OPLxgn3LdWCmc3lUgmE9pglb6bZmq0N7r+wCaitl36eAKrl2/ioODfYAYfU8Yjfbw+c//Fg6vHuLo+hE+//oX8NKLL0k1PgBL/2KOyQwFFZrGzM/0nBOwNHifDEnVbKRY1jciBeYlOM/5XOn3mUWqx1QmQIBSvWN00HhsvmvpnMyI0cMRl9ZtsYWaZtb4K0uWqSOXzHevTPQEZa8zp5eL3Fo128o8XO+dcicVFoKWu1nc1FgmwAJs9b1kG/Vdp5uTQ0CPG4dHuHF0A77yODi4hCuXr4BjxMH+AW7fuY1bd25jPDnFeDzW0Izm9trx1ek3nqOESnuF8B2ByAvIE1nDZVX632vreKKIqhrp+4Tnnj9C3VSoKsK9Bx+KFlaSbMmLjnChWA/l1M8OXvB0zWc3ktM1H1p3jJXCudPsyO4TpN1dZEXKFD1kzTV1znw8oOum0li3j3j1lS/gG9/8W/jc65+XjT8ydkYjvPraa3jm6AbqqhJTJkrGi/OUhQEMb3NLhJDiVuYTckL4TFBJy50MtLK0r0gRgYUQ1am/5i0pHILiWoJN0r8slRMmCMQsDHTkJEYapWVAtJQ2K4eDCqj3qoU9jHyHImvIiEWA4GRhxqAx4j4Jd6VNnsqr6mNIPrJdo9S5ym9tmhFGIzFdqcJAOKuqli5kipJZmAUQRL7vO0xbj9G0hdO2j9CNCyTpeM8/9yyefeYGvvrmm/jBD/4fvP2rX+GXv3obv37313jv/ffxwfvvI4Y+1VOmnRUurRWbHxNWlCAjyXx4LxZAVdUAE4hqEDdo6n2ACIEj/t53/wHq2uPh8QP8xV/8O6Fq8Q6uJjiOUjccJfNoofDNjk0FaSOppMUbwaw2XnOslcL57I3ncHp6grZtcfXadezs7KBWFviq8phMJvjgg5u4c+c2Dg+P8OZX3sQf//F/hsPr17G/v4e90S6uXr2Gg4MDOHDBZOfgmcF9QO082r4V+XKFrzBz4Va8CyD1zihEKd3gpPUKrz2lspkgqlYmaLaoCsGsb5IKihVJZCfaKLfYk2uwBU8KbEijXxawiCvRihqSisygEBFYFhkK2hNPhFopR6XbV7QaqMJY0KT3TsJFXmk56lpMRaqUSVAF3I4tfVV8zlFWzeadaCVJB2zRtmM4xwACdnf2inkSBNSB0VQVvv2tb+Pv/O1vIYLx6/fexU/+5if40Y9/jB/9zV/hvXffxUf3PkLlKzTNDprRCL4SM1b4wwSscRAGDGZG13UYn07x8N5DHB8f4/T0FMfHx3Dw8FWD/f2reP3zX8HJ6Sn+8q/+E967dYyr166j2XkeX3rj+whxDFCLw+sN7t79AA8f3AXh4aBkLMU7C0kbCh0P30YW7ME7PPPd4rOD7w4OschOXi2fK4Xzd3/ndwDIDnzp0hXheK2krZ5zhK7tcO/+Pdy//wCXL1/Ga6++im9+9Rs4uHSAndEIsY9oRg3qqgaBYcVc1o1MjUQN0QzPLQtxthKhHPptUnpILqxrNWuTPUwohNYjxuxz5cZIvAQ8sOoUQ24hi1uRykillijMWzWTBQkWIfdQzWnIbtQ8UEAQRspoa8oXBgS8gdUn8sy8aK2kllKV5WVlFY/5sq7YxFIihGFlCuqEvkfvHPrQwVpb6P4nv8w57O3tpawj7yvsjHZxeHiEa0dX8fOf/xy3bt2SBIsgbSsenZyke2ZctQxZRzEQJpMWDx88wr2799B2Lfqu1/soHLjT6RQnJ8eYTFsQVfjgvQ9w/94DVHWFo2vPYG/vKpqRAzBFDCPEWGM02i+QVUOQ8/0u3hkK3Kr3bJ5sla55b7ghzH7vHML5T//JP8PR0TVcvXoZnmrtbEWprMsWtFXFO4NDbfE0wytg1TJCTaILHZI3GzW/Nv9ATgnnCZoXGzcdMBltpuD0VU6SWkwaKTDCoowq6yzGMflzhjinyVNTPCGIQF7QWnHiogpRISxpmBVgm4jLPnROpvdSv+mUJY4s1c7D1R7oxZ1IlSB6HkMwzZ+svGT+GKs+aezQ6l7ruoZxQqTqINXWljTuAb23PdBBUyYlN9foOGUzlPskaXodrl69hqOjG3jzzTfxhS98Dj996y28/8H7AAh37tzGrTu38P/96EfCxugqVL5Gr+EPcg4IhPHpBB/euYu7t24LcFTXGNUjQMNYxycP8e67b4Nchf3dfbzzy7fRhym6forvfPf7eP7S87h8+SpufvA+jk8I02mNS5d2UNfDrumzwmmvmeDmezj/2vC9Zd/HzDmWHXt+k5gdK4Xz1u07AhZMW+xVNXZ2dgQCh6TWykaaFzUpImhpZrmNuPhg0EY0DKGRyAtaY5EpF9IImqSBkGhaTtoOBt6Y0JssmmBa2IQljGDHTKTMnG8Y9H0zf5xW3zAYHCVnVHjHSRrGQuORrhJajcrBc4XQd7BuV9bpGsoQF0047Vqj+cfSBg/QxDc2tgnZAJqqEl+ZGa6qQKEH+sw2SJp0UCsKWtWN+pXGKGjmNhKoZ26FpUV6/d9KnG3hROUm4giwz9ScZGCO/hYHShu1cw6fe+kVvPzs88Jv1DT48N493PnwDv7yxz/Ce++9h5s3b+FnP/sZ2tCJZ+qBUXUJD+4e46Pb93B0dISmrgEAH969ixgl+2q0s4vT01Ncv36Ir37la7h79570Qg0tfvjDH+Kdd97B0dER9ncv4eQRgfkKnn/uDezs7Ch/bZGfrBbC3GuyGy94zT4Xh68Vxypfy0KpxwNgxQ/D9wzjPoNw/r//4Yf46m9/CTtNDaparX0UU0QWfs5DlZsVkpkoC800oSYiJ1M0M73ZcSQq4bLEmUnK9r0lP6LYDcmJAM0O04qOKWcApa/LpuIiJRM1WbAFiCFhm5CeOwVXyDu44BAdAQFqSso1EzS729BmOVIylQzNtAqSSEj+ondZkKKirxIRkLioXKciu75KjY7K2s+UxA8kfz/Hb02LW3xX/GKK0JsDXXhBE9XF5yS4RHkKIpAQB6X/BZiSzaGqaxxeJ+zsiN/50gsv4fat27hxeEPiuizlX7H3eOb6IV558SU8+8wRHEnHsF//+h2MxxNpZQGHj05OAQD37t1H10oHt9BL3m+nTYkdTnHlymWMRiMcn4xxcjopFkJpcS0TCi5M4eHrg3lNdhvDYtzp2FR8nvJrzj6/8PjzY6Vw/uDf/0fs7+3imcProL0RmkZiaCDxHmW3NWNJdwkzRZNdKM+jMsSTCmGZCJD/NQuxMC1VYGnBL0pGgs1DAQjZSC4o0UA42QqUVUtxwUpv33TODQLgKRyjZqzF6BKnUfKptHaTjLlu5obn+5aEk/W6yIiRNexj6K3VqwlXrXUwU4Y647Mtaz+zi5wFEkN/kyhNhhBxK+eSwvKA1aKyFkQHuesgn9pAwGWuW/O3DIllAHu7e9jb28fR0RFefP5F3Lt3Hy+9+BIihIVx0k5w/GiMRw8e4fTkGDcODxFjwGR8il/8/Bncf/AA4/EYk9MJ3nrvA5yMW9y+dRuj0S6YI7rQpSwqcg5d3+K5K8/g4NIBfvXOu2g7Yda3jdDmnFRJDNdNMS8YzlM5h1CrA7RsboFcULFgzmdeWzZWCueP/+YtEDPu3LqJP/7+76WGN3v7e4IEkiQ6R9VwiZRYhzXHtQUqqp0E9k4CSoV610XsoGCL8L1W0asW0zxWkh1cNIkyoHPIXahs4jkObkAqX1LHk0Bwmk3CMGQ0MwRG57U2VcJJjqz42SWFTSTZUzF49OSRiLDZgaPLi0J3U+cY7ApzJlkfujgM0PGEighBteWgdMt8YA2fWMZNavaUdvZhpowxHgKQVD5z1ssFGoTJixBA0h8wmbkGbjlmwJPwBBe8sMwMn0r7lM0vWVWMw2vXcHj1Gj7/2muAd+m4k8lUKFdDAPc9unaKfjrF8d//Nu7f/wjHDx/gwQcf4r//N/8Hfv7OB7h58yEuXbqCiIjYMl585jlcu3YdV65cQTtt8eG9+/j1+x9gPDnJm2vklKGV83/VhLfNlzPXU0o+MeFEthJLIaXB81JYh8qitDKzwAKr1OhK4Wy7Hr9+/yYIjG997ctgYrTtFERI/meqBmBWuotCEyYkE4UAGtFzaWKoMMOMQb3ZhdmbjILCqdYZSE9nh5mY2VNlPaYBTEqrqe/mwmY5IAerRaV8o0BIscYFN8IWAVhTGLMiyxuQj1Bdqcxy0GM5uKpRv44gVwfxT0Gp2sOEM8UDlaDa+aw97T87uSNrPiTn82oCOxPasqbTTLEoHcGSEaebEZWGMuc7mIRYzfSUWmkjym+2vNqIgMhRmBrJg52El8h7+KYC7+4ghgPUzmGHK4kp+wpHzzwDJqRsogf37iMEyXTa39nFyfEjTKdjXL12OWm48kK4+DcZaDOLKFtjSBp3EKYrrFpzU1i/yOXh08aFtALK76waa0mlP7p/Hxw6vPPe+3AOcGDs7e3qBBMqvQ5SusNM4z/80SlvlaA0lwBY6DQ5v5VuewpQiAqEUGQAUn3gYJlEtmgW/c7BroYs7IX3kC9WHwuptO2aslCgCdX2uXLpl+aKc0i7sJnFeeTNKPl8yNlEYgoK6krWB4W1pyUpmZpucIPwiKX8Ga+tyxuGsUjImorZ/CZou3nS2k3rMTLkMzLzQBQk5+IAVgFVAK/0pyNLcgYvYFrnaGViBFBADD1C36UML1LXyDYl7xyaugbv7AB7jGkX0QVgf38Pp6fHqT3ldHwMS+X0DLTTCULXYX9XiLhnzceksUzG0l8q3td7TUu+Y1OxQgvzwsfzz5eNlcK5t9Pg+Pgh3r/1EP/yf/gT/Of/6HvY32lw/34N5z12QXA7PgmBU19ObgBgzAMSOwupfIkdwBDkL6bop9xwKR/KYmM7df4NUkvJCEnQkoDO/NBAxeTaZBogYxPN+i47UGrkS3LtXo5nC3aW2HrWl+BEZGwauXgfSEDULHm1+ZW+8pJI4C3fl8TUY2XaL0wt5zyqukJTC1LrnOTGlj6QAyUeV4mb5thnpe3mSXmGk4DXOU7qCFpkABVU3eScUwFVcIiU5MtBQDMIg0atCQ56cxCjFLt3rSbIB1awx+43g7sOvbaenEymiNMe3aTDrfsPcfP2Q9z84B72d04xnU4kZ1cZORrvsVvXuPvhbdTVCPt7V8DBowtId97GcJ3wYHMpH6efPfe80L287LhAKfRZtrOASw3w8rFSOB89OkFV1djZ2cXxeIKfv/M+6maEr9UjMDz2x1NcvXo1VU6IyaqLJ/k7tkNExD5I305ikJOqBFdVqiVkQcUEZevPL8xW+YGsqhYKExmFhSQ7z+a5YtAeT81KQDRM8oVZjkGEwAEgZRFnTS1j9fmSNpEJtzMZ6bFUrmXhF/opswZUM2tFhdPvSPc27btClNgApU289T2V5ALLuZWyqwZ1M0oxQQGEFEjS1LyKvJTWQcJSBlI4J1UqKY/XEGLv4QpNU5JZiw8JKYUjLS0CgcgXO6kCvVGaIEgmlvyPKHW2VuqWmNjjkKlgthvdpOvxwZ27+J//zf+JnZ0d3Di6jvHxCQ4Pr6GuK1Te4f69h1KoPZ2inU4wGU8Q+l28+eYbaSMt/eLhYxRr9HyPF8U+S/M1vb+BSQts0NnauQYAYTJt8e7N22jqBs/duJH6OO6MRnJfqgrR58MxQX0MjfnpTZObEiC9M4ICMYZs5X2EYRrKqkPkdUkgt5Qs8x2hk0ODCXKOxSR12a+kdBwzl4fzVLhQGagxMzEJ8tBLMc1o2luSCQhgn8whBon/CvXPOWvpTIEpPrLTHVZCTA7BOc3uMVpQqYmstQysrioFX4bC6ZXujpll81ShN0EsE+yTtnTZn84mrkua1zZLMa+ziWPuh607hvqfrFU5IWgRvMUMGWXBhGy288JzMmlx594D/PVbv0AIhFFToasrXLt2FQCjU1Kztm3BHFPXcecIJycnOftsRhh4xbN5/zStiHQ/k+lrAooNRiGTpem8bKyuSqkaTCZTnJ6e4GD/AH/901/i5u27qB3hb3/1t0EhStX9oYPfFaRRCLPk+7neb6j6+76H415VitPdWpKe5be79Dz9fnCi6iRmdKmw2pIZRD8lHwCyc1uXNGbTs3YdRc6l/qWEU9mK4yEI4L1m6sQMfqRFLJowqvkZAXgvXantfe8cCNqC3ZIC9DcmsxIKtZC8F0kYY739hVSf1M0Io9Eudnb3U5maIdLZzy7a7xHBFf0rq0qaJHnVyrbBVDBOYcshVuCoqgDnUts9BrQ/iiDstqkJd5DNq7BTxL5DPxkDeg+s6NvmmogSUJc6m7Osn5t37+EX79/Cz9+/BccRTd3g8uUrePnlF3D7zm384pc/RwyUCrV3dvbwW7/1Ai5dOsD/9D/+r+n3eq1YkRhythact8demyhV6fq8d4ONK1sYC17X71RGJVp8jhZ8pyoen0k4+76H9x5NVYH7gGnb4WQ8xZ2PHuDR8RiXDybYPRnjdHQMsHDgUNUkM5Zi1M3WC48tI+VyMgdQjPBdj9h1ChI4OC+NYcl5CTvoDXSUTUZSU5OSeQtdnE65eQQciBykAxdzyoc1QMY0abFrpG0gC1RIPkICC3ym+rBjiWA7DWd4CeYXplkCb5z4yxQU1CJKHEFJW3mlEyEBpkokFRB/dWe0g729PTTNDuqmSel6AFK2ji771D9UFmKhIcml1L4knKRZQqZhK2/9usS6UdMZRKlCh3T+U9tGqPnaB3Rti66dIvQd+ukYo1pCPhXqwldjIU9j21DFAoosTZr/01/+GP/hr/4aHYARO9RVg8uXL+Hu3XuYTjtcv3aEyWSafvO061CPRtg7uITXvvB6BoK40G72uNR6+v3AQm+CAHCrH0oar3i8wEQdWFT2mNMZBufNrwH/7X/zX8/JHrC2kVHOzYRRVISYGLdDUJ7SENTZtx1T49jQeGSGWczjQ7ZjIhA0xEEMdgIyCOFosSUTMluT+ot2VJFf0XRSnykJAFFpTGIMejgFSpz1HCnMKmjPjgiQV1JiVplM869PNFRQQnmk5qZRVqZJEBWThQyqobWVg1Oh1fsk1oH9D8nGovI83qHxNZpmJCZtXc/tvmlT4GyaG4VJNmszZaYlPUDnJ53fe3E0yIRTBbMwb1GY+gxBq0PfI3Q92ukYbTtF6HupQPIZcCnjr+W6YZ2yECPu3r+Hm7fv4M7djyAEZQ6+qdE0jSC1fVDLhbC7Kxy7dz76CH0ImLatFGmUazmdl9KJsqBRNreLORw8VqFKj/Xzy95b9vnZx8vGRv05kfSEbZoFdUchO0P7eSMrfOYUabZmjqBxPugkJ82ALF+6EMVaUvIscoi9UKkAjGjawRrqqlnJrLmzTqooxMdiNcNkJB4ug+X1r73mvIeva5W8aCtMbFQnTBBSwwkwvPjdnhJjvcHwEpCktAGZtmN9z1cVmtEOmtFOyoxZOJ1q3ieNqBrSQiZVVSfzLvmjKphQYjC4sigaA0zAYrlsdDVqLQUEqeftOozHJ+i6FhwZFaq0PspECnuuD9KxJn2Hn/7yF7h5+0M8ejSGh0e126DZHSXh7LoO47Gk9F29ehk3btzA8ekpTk6kzPHhw4eyTmbQ9UWPyzGb9TP73rrH53mvHBsKJwq5o8RnSmmBQxerfWZLwRwc/wxj7nSLz1+aoRd6/pkz5sfFzlXMX2lBDBZJ+V0qLD+UfrFp2+wvzu7y9peLJGLpwuXTxlqZaavaSnMtpIgewsQeQ0CtIZUYhDza2Bc4iM/syKFjJZuOQQoVlNe2n3ZybpIwkZCPB7ioCf8MQDdHSt5DxIOHD/Drmzfxf//HH+Ld2x9iPO1QOY/Do0OM6goPHz7E7q7UFgOMF198CV3X4he/+AWcI3z+c69jf/8A/+pf/UtTigXaABjAl29RoVqWCNI6wV733qrXlo3NhbMYxvVq7tjClX0G+Ty3hJz39Oc8/8KAMi19Mlww5duLplP96wRYURbu2fDRrMlI5kcWnxmcpDTR7QLsfOUlJYeJxf7XGh1YaESZD8twiAlGCmnMnG+wQPUk03aKk5NTHJ+OpdscAI6M3dEIVeVxOmlxfPwQp6cn6PsezEDb9hiPxyDncef2bXzkP8J0MknCOZzW2XMObwLNvVegtOn1WeTW3pv9TLY4F31n1ZLbWjjF/Sj9oG2PsNFZBs+2EjRe+XTlec54xo3GgjWw4SfXCfzykYR22efPfO94xbPhucsNyNIX7Rktmed22uJ0PEYXNCRFgoKPmgZ15TEmwt2P7mIymaLrekwmLdq2Q9f1cC7ipz/5CcbjsXRg+1gHzcjbUPhoThjPoTmXad2y5bt9buFH16jtJWfd8HNnMZ0fy06SxiDvtzxt8e+CNxa8Vmo1+2OCRsXxNv89qz45a14tj9qViOdWp9/oqky3ilnNKQkdQMpBZgNTDD0qtTLz4PHycSEXv/i4A4xnCPhsu2JXZQ8BGGh2/ZP5Z9SwQjazznIJG4yNDrnteedv4rlu14LFQFTMz8KzrxgDq3MW1bOQ0PaXufH5N/xMOVbdgXn/asnRC+ErfWlasL7sEak5vFIeNzn3EzbWCieA7OfoMKLl5Dcs+RoteVwcdPhosI63mcDHsCGccSzdsTe3azcby9b20hV6hvMtk36eezD/VnEty1DRZcO0Y0nOlUrm0r9JNAvT+clZBxcxNhPOchAKtBabCRTnP7NJx8vOseaFDcfmN+u8e+n8b5s/eFpCG2/x2YzMS3HBpzZXGXPXdGEjXejqSov1h8nVHTHGQjtSkkkTXhhmNTBlNzVrn/yxRjgXmWOkzO/ZqF3s/5xzAugcR1FU8eO4BR+fgVSI5zld+U012MrLeKxDBTQOzdqsNQsvgjDjc25y/M+SWTszSFkEElyMoQX0pO9LK3fVFGbYEDQgsq18yfuDP2cfc4d/zODWlp+Z/fy5tGcJ+ujIAliivgoInflMT/ZYKZyLUN9BhhCKtbxybHKjNp/iRX7N+cdq/3mTMX8tlP+mdLfipfK7i65HbGVwaQVQ6WVdDFo7NxIwvOVsbHQr1hzTgKAiXiopl4VfaaatHVFN3tlEjE/7OJPmzOl7xeIr/gz89UXjIkGLTc53xnExkZcVBzF5nZ2PWcuV7R8afGbucCsv+AnULwsuSW6l+pzMOcOHxJphzp8qEzJmff7PgoCuFs7BWsiCSI4GCzdlO6wdhddwgXO39FBzIYjZsVp3neUactijQLdXXcHWO0Bx3HPL28wBaPV8LJuhbbz7tIpWXHtKgOe865bzVJ6tdKlmTeFP+9hac5ZmLWg+U6h02ufuwJp5o7l/l39q7XpYAfcPjrPB2Oh2r1oYWwrRrIiUZu1mIr/FuTaQ8FUL/uyisPy8szFOAJmPd4HpaorjkzNrH8/5NhfOYi6t4Wy5UBZN9YW4olsfdPOvPuaEocE5Z2do7alz3GXebF9z4YsW5+P5reVGuf5mLsL+54b52MxJgwKAsQayfcaAoRRv50/QrH08C+nsaG2ZhHDh1/YJ+0cXdXoqHiw85jYnslV6rivaamxvci8XiEGZ1NIPZTkv83CT6ToIkZVIbnEFv9FmLUjpNjKrgL2zzIdbNF3Lkp4/rvE4gMhkVq080fbI9QCtRelnDc25TccwRj1/nee9M+f1/SwJIccyqSDV1itU2S3X4G8UWju8Xdm3NELigWCeA2hdcMLNvrsk0Xz+fTv+KhTCPrJZ3IyG2/WSBbEeqV1wCYNXBnm1A+DNrJb1V7tt2GX12Fzw5vNil49yrVmhOBd2bSLftlBKMSmDkrTPiGAC2+TWFriPUWjYWFqVcp4xxJku7rAz9Y8XPdbXdG7wejpY/nO+a95sFp+IYAubpTC0QjIXUGn62psLDvMZENKt0/dgZu2cv7nprS1RtscoJJ/A0ZYtiLPKZnHg4Xcem6+/waV8DMeXWu5hlcmgWLs09GnIJgF8NgQT2CLOWb42oOzXIPDcZ5fMz+p5W68un4jdfcNxvrDHEjPtjIeaFeQ55HrR17ZJatjQlV61mZv4peZYBSDkkuZUs7Y0ebesevm0jM3rOZGt21JznsPd3HhsF5LZZtecAV6W7RxrDrnIVzzXmFv3XCzTc2hN2nwBz5mTaz67YTRls3OXhdYMQBNf8rlmLIniOj9LoNCZQynAPD57MYHExd8501QXu++m59lkLFywa0ChZXpiozGz8pfM/vnHBWqei0h8B5JspvS9PA8FIDTL5PAZEExgqyQESpNkSQgozNp5jJO3Wzo083d2bD3hH+8NWroFLPk9a+eGZiJ6a2Dt9QvyogV5cPbi31Vf2UALMxA5FuVi2XRN85CSEM46Ph3Cu3lVSjF8mVtrZu0A5l/w4y9wN1t2YzZ0ex/bmIvvnUcedF3aAj1PyGn2K+tN1c1DILNzvPGcLz20FVvHeUAohzcHIaZZtsHytS1P/kSNM5q1RRKCvLLlETbZ9mjhw00OvfL5urNehGm37pzMC6zSFV8qVOci7XORZty5j7RVAsISF6Y8RgJlC0BocIj50NhvnFlbWp058V1fGWhNbHiHZ32Hixyzx159jnN7uUuLrQ0023ZuVlxJOiRt5yI+8cpCLlDoSTILgoGQrgSEOJu8BMxbLGceT5ZQnynxPT+nJJsr7/2Gv3k+9LAdsrjujWR0r7uRZ1zI28Y5lx8o/ZPN2oWH3vbIFzufW7y5eCzE1SzpPRaHpIWak+cenFdrPlk72Po459xvzebswKz9hH/XJ23KLD/78olZB5AsWnybfG/b6zjfWD/vGyW929G4KLQ2D3NhCMhco9x8twynfNLr4SLGFj5n6QMaSgtsJphLnPTZ+bvA9bPQoJ25wcOni05OK59uNM4FCmUUdFFF5/zHL3ZBLg4ZLTrx/Jur8mpXFecbRUnKEJrxOS3MwnPa8tMvjLNjs/S9OR88C2R+ODPZvOC1uXFR0nnWG7PafN7KpysKhC/SsCozYYaXu91crXU91oxZwS+PdT74Z/imac2o7R7t2K5MfMdQ+Eth/SxoTBtbE3wloURpXi1f3NubYOdXnwu8kgs63vZjnRzNaZY1OwIt+NgqhHnTKptPfCTfep7tHZj9javDVb+BZi2QdSQNX6NFpsrMbr/wSINPnn0sOQCvehMXCOCcZSwyNJa8kOJ6My7EWdHa2f6QORVzW0tn5q1z3kj7nTFabi3M5Rww7KWaTsha/CwI4qKxoXDKj3dW9EoZyh/kvG+0WJZN5GMQiS1RYiElPsN1qCk7Z/qZb77gt21qmQ7M2uG3NztA+tyaz85o2JWu5Zr3FtOkbHDNXABC1kbQrs2V6CQvMGt58PizMLYChIhIW6cvMsXUYb+4a3ss47FVLixbEHN42eY+6eIo7factWcdwzKtuXfPcMB1H+CiXGxGQ84ltts1frb8zHJsZdY6lzWnjGTPrh2zGR/p9W0uYNn4VDhVZxhcaIpzHup8U7T8pq1zHVaNYe/ObNaa9hx8dlCvWZ5zOer7aR9r45ylHDpySXMmsxY6ycUcrVpKm4A1i2oPz5Y38PHeoLULYi5euf7DOYd0CIKcOTtofURm2aUserL1yO7t4jBN1LxaM2vtdHMcQgnrMJ8zz9d6wXxc6+Jij7uV5iRH2tV69o2zoLLbjmWoz7IJWaypl42LuvoB548dd8uDz4VT0j+LgLezjyevOHm+gZGKYXGtBYKLbNYyb/N7HtfvPstxly/QrapSRFs6DETxrDf4zLjQkNxkpex9YpbNJuppAxi7OB6v/sAnNNaEfRZQiCwfGVTL/w+PlT6ZXjesY/5Gf3rM2uVzs53PSaS9OYHE3YJ5sxbAajvUkLa56/yEFt78hWwtAwvNKdvaMb/rr72YQgEPfayZz204Zpn5V420tWyXhTF8tgix3eAQc2atXAjIlf1QMkg0S9j2G4rWFoDQrBE7G0t5AuZGbuH2wbcy++l8gxY8mnl73eLnoXm38ngXPnj+2TLP4gLPaWZtjOaW5DBeORJmTcs2r0//2Cp9j2i+idGg6nrNmN/RNgFQtl2KK4CmLY90rrFMxs+0hnipdM5qjU96nLnImbHerNW9NmtU+Wd7n/OTHpvdpw00Zz6QIwfvxOccmLWfeIrYhhp7yUZybqQ3xd6WfGfBCZbN16LXBwjlJgkFZx2bLO5V3soFXEJZlWK2DwGJVDpTYkKjBsMzL0r9e/LGZvdvO7SWCI7c4qygZee76Dla97s2Ros2O+gmhxDZWSSg84JUfnaTMUtTsnGm3aKx2BdZ8OyMY8PfNbd2iu9bVcqQfc+0Ig81J0qf80kWxrONrXhrzawdvvE4dvEZf21VbGzh2Dbscv6RwYrhuMjZOa/Vtrl9s2HSQ4lzFV9YpbVWeeI2h9E03yDxff7KDJh8opXkOcZWoRTnjO0dw7zaQX3nFmPhpF7kcl4Aanwsg2a0w+LftBQo0jFEa5+ksSB0cSHHzIIZo8F5rC6USx9LRQD6b+lzfjrM2s3GerO22K7dILc2C+QmXlCerNm/dp5l5197hcPzbPCZZRooJVNvd8oVJzrPl7lAa1eVT51vbFxQvWScTwTyuUVroqhKGbLvZQ6h4TxkPtvP3tjc5+RiMgYhs83jZ+VY+vELWHfrb9XZT7JUMOZ2601CKfrVhcdb+HDZ0TYe5/v2spndDDHe5NwmoMmPHPicmPE5AfzG+pw2Cp/Pqc85iGnPlkbRWXf3M6jJVehheQ/PcDVnHrOIx1YnX6PJlqIp5zj+mY642aeXZggtiTMln7NoZGQRAf3E4CBi1s4c5Ykwac9/DVv1SnHOJZ+ztGdX36YNmN9p4cMNBhfHX6bR7MCbLcrz1HMuPO45TdCM1i42ubdeiMv2CzrzTVieibmogmH5JYG5pMYs6jlRNjLKPmeZIZSKntaFtT62cX51sEUohVNubSaUntGYwGAbWzhBZ5izZT9z6aG4fPD4btLKHh1ns/YXjnVW27oNZfbd9de0wVU/JlPEaDHjzFwOE9+LDT/5nGcslP/Ex/Ibu6Fwqm4qmBAGi4/OsPHOXdP8twYhyy13wo9939wyM2Yj/2upPv70LcKNk99nM4SK/Fr5xPAlQ2sHR/lUlYytsOg+efX/dDwdT8eicaZeKU/H0/F0PP7xVDifjqfjCR1PhfPpeDqe0PFUOJ+Op+MJHU+F8+l4Op7Q8VQ4n46n4wkd/z99zyCUkCwrFAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
" 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Class ImageDataGenerator() returns an iterator holding one batch of images\n",
"# the constructor takes arguments defining the different image transformations\n",
"# for augmentation purposes (rotation, x-/y-shift, intensity scaling - here 1./255 \n",
"# to scale range to [0, 1], shear, zoom, flip, ... )\n",
"\n",
"class_names = [\"angelina jolie\", \"brad pitt\",\"catherine deneuve\" , \"johnny depp\",\"leonardo dicaprio\", \"marion cotillard\", \"robert de niro\",\"sandra bullock\"]\n",
"\n",
"\n",
"\n",
"train_datagen = ImageDataGenerator(\n",
" rotation_range=10,\n",
" width_shift_range=0.2,\n",
" height_shift_range=0.2,\n",
" rescale=1./255,\n",
" shear_range=0.2,\n",
" zoom_range=0.2,\n",
" horizontal_flip=True,\n",
" fill_mode='nearest')\n",
"\n",
"\n",
"dir_iter = train_datagen.flow_from_directory('./train/', \n",
" target_size=(image_size, image_size),\n",
" classes=class_names,\n",
" batch_size=25, class_mode='sparse', shuffle=False)\n",
"\n",
"plot_img(dir_iter[0][0][1,...])\n",
"dir_iter[0][1]"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "V2fYccc8GhJF"
},
"source": [
"Before you continue, you need to split the downloaded images into a `train` folder and into a `validation` folder."
]
},
{
"cell_type": "raw",
"metadata": {
"colab_type": "raw",
"id": "VamXG4FoGhJH"
},
"source": [
"./\n",
"├── train\n",
"│ ├── brad pitt\n",
"│ └── johnny deep\n",
"| ├── leonardo di caprio\n",
"| └── ...\n",
"│ \n",
"└── validation\n",
" ├── brad pitt\n",
" ├── johnny deep\n",
" ├── leonardo di caprio\n",
" └── ..."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "9322su6vGhJJ"
},
"source": [
"If you want to use the example of this jupyter notebook, you can use the images provided in the ./train and ./validation folders."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "xPqJWgeAGhJL"
},
"source": [
"## Define a ConvNet Model"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "UuJV4JBKGhJO"
},
"outputs": [],
"source": [
"batch_size = 20\n",
"num_train_images = 480\n",
"num_valid_images = 80\n",
"num_classes = 8\n",
"\n",
"model_scratch = Sequential()\n",
"model_scratch.add(Conv2D(32, (3, 3), input_shape=(image_size, image_size, 3)))\n",
"model_scratch.add(Activation('relu'))\n",
"model_scratch.add(MaxPooling2D(pool_size=(2, 2)))\n",
"\n",
"model_scratch.add(Conv2D(32, (3, 3)))\n",
"model_scratch.add(Activation('relu'))\n",
"model_scratch.add(MaxPooling2D(pool_size=(2, 2)))\n",
"\n",
"model_scratch.add(Conv2D(64, (3, 3)))\n",
"model_scratch.add(Activation('relu'))\n",
"model_scratch.add(MaxPooling2D(pool_size=(2, 2)))\n",
"\n",
"# this converts our 3D feature maps to 1D feature vectors\n",
"model_scratch.add(Flatten()) \n",
"model_scratch.add(Dense(64))\n",
"model_scratch.add(Activation('relu'))\n",
"model_scratch.add(Dropout(0.5))\n",
"model_scratch.add(Dense(num_classes))\n",
"model_scratch.add(Activation('softmax'))\n",
"\n",
"model_scratch.compile(loss='categorical_crossentropy',\n",
" optimizer='adam',\n",
" metrics=['accuracy'])\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "JFdkIokMGhJT",
"outputId": "63e7d032-4083-4fe0-d970-c10bf0c39a94"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 480 images belonging to 8 classes.\n",
"Found 83 images belonging to 8 classes.\n"
]
}
],
"source": [
"# This is the augmentation configuration we will use for training\n",
"train_datagen = ImageDataGenerator(\n",
" rescale=1./255,\n",
" shear_range=0.2,\n",
" zoom_range=0.2,\n",
" horizontal_flip=True)\n",
"\n",
"# This is the augmentation configuration we will use for validation:\n",
"# only rescaling\n",
"validation_datagen = ImageDataGenerator(rescale=1./255)\n",
"\n",
"# This is a generator that will read pictures found in\n",
"# subfolers of './train', and indefinitely generate\n",
"# batches of augmented image data\n",
"train_generator = train_datagen.flow_from_directory(\n",
" './train', # this is the target directory\n",
" target_size=(image_size, image_size), # all images will be resized to 150x150\n",
" classes=class_names,\n",
" batch_size=batch_size) \n",
"\n",
"# This is a similar generator, for validation data\n",
"validation_generator = validation_datagen.flow_from_directory(\n",
" './validation',\n",
" target_size = (image_size, image_size),\n",
" classes = class_names,\n",
" batch_size = batch_size)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "cytHiQUTGhJb"
},
"outputs": [],
"source": [
"logdir = os.path.join(\"logs\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n",
"tensorboard_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "C7dCbyXPGhJg",
"outputId": "98b4085e-ed6d-43e2-831f-aec32161583f"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/20\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/opt/conda/lib/python3.7/site-packages/PIL/Image.py:952: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images\n",
" \"Palette images with Transparency expressed in bytes should be \"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"24/24 [==============================] - 30s 1s/step - loss: 2.1493 - accuracy: 0.1146 - val_loss: 2.0780 - val_accuracy: 0.1625\n",
"Epoch 2/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 2.0772 - accuracy: 0.1437 - val_loss: 2.0687 - val_accuracy: 0.2250\n",
"Epoch 3/20\n",
"24/24 [==============================] - 29s 1s/step - loss: 2.0690 - accuracy: 0.1667 - val_loss: 2.0434 - val_accuracy: 0.2125\n",
"Epoch 4/20\n",
"24/24 [==============================] - 29s 1s/step - loss: 2.0251 - accuracy: 0.2229 - val_loss: 1.9642 - val_accuracy: 0.3500\n",
"Epoch 5/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 1.9674 - accuracy: 0.2438 - val_loss: 1.8757 - val_accuracy: 0.3000\n",
"Epoch 6/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 1.8926 - accuracy: 0.2750 - val_loss: 1.7863 - val_accuracy: 0.3625\n",
"Epoch 7/20\n",
"24/24 [==============================] - 29s 1s/step - loss: 1.8776 - accuracy: 0.2979 - val_loss: 1.7791 - val_accuracy: 0.3125\n",
"Epoch 8/20\n",
"24/24 [==============================] - 35s 1s/step - loss: 1.7989 - accuracy: 0.3146 - val_loss: 1.7242 - val_accuracy: 0.3125\n",
"Epoch 9/20\n",
"24/24 [==============================] - 29s 1s/step - loss: 1.7280 - accuracy: 0.3438 - val_loss: 1.6277 - val_accuracy: 0.3750\n",
"Epoch 10/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 1.6853 - accuracy: 0.3667 - val_loss: 1.6015 - val_accuracy: 0.4375\n",
"Epoch 11/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 1.5953 - accuracy: 0.3750 - val_loss: 1.5688 - val_accuracy: 0.3875\n",
"Epoch 12/20\n",
"24/24 [==============================] - 29s 1s/step - loss: 1.5313 - accuracy: 0.4062 - val_loss: 1.5065 - val_accuracy: 0.5375\n",
"Epoch 13/20\n",
"24/24 [==============================] - 29s 1s/step - loss: 1.5017 - accuracy: 0.4271 - val_loss: 1.5153 - val_accuracy: 0.4625\n",
"Epoch 14/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 1.4794 - accuracy: 0.4437 - val_loss: 1.5480 - val_accuracy: 0.4625\n",
"Epoch 15/20\n",
"24/24 [==============================] - 29s 1s/step - loss: 1.4285 - accuracy: 0.4563 - val_loss: 1.4185 - val_accuracy: 0.4500\n",
"Epoch 16/20\n",
"24/24 [==============================] - 29s 1s/step - loss: 1.4327 - accuracy: 0.4437 - val_loss: 1.4999 - val_accuracy: 0.4375\n",
"Epoch 17/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 1.2907 - accuracy: 0.5083 - val_loss: 1.4782 - val_accuracy: 0.4125\n",
"Epoch 18/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 1.2766 - accuracy: 0.5292 - val_loss: 1.4866 - val_accuracy: 0.4500\n",
"Epoch 19/20\n",
"24/24 [==============================] - 27s 1s/step - loss: 1.2222 - accuracy: 0.5375 - val_loss: 1.5021 - val_accuracy: 0.4500\n",
"Epoch 20/20\n",
"24/24 [==============================] - 28s 1s/step - loss: 1.2967 - accuracy: 0.4854 - val_loss: 1.4795 - val_accuracy: 0.4625\n"
]
}
],
"source": [
"history = model_scratch.fit(\n",
" train_generator,\n",
" steps_per_epoch = num_train_images // batch_size,\n",
" epochs = 20,\n",
" validation_data = validation_generator,\n",
" validation_steps = num_valid_images // batch_size,\n",
" callbacks = [tensorboard_callback])"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "wt_ONw5PGhJm",
"outputId": "e75d8a73-da49-4dbe-ffcf-7cb316be39a2"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABD00lEQVR4nO3dd1zV9f7A8debobhFcAIq7okLZ5pmVmqaudLWr3WzeduZ3duwedv3ZplZXRu3YaY508xtbtEUcaOi4FYUBEEZn98f34MhAR7gLDjv5+NxHh6+6/PmeDjv8/1MMcaglFLKe/m4OwCllFLupYlAKaW8nCYCpZTycpoIlFLKy2kiUEopL6eJQCmlvJwmAuVVROQrEXndzmPjRKSfs2NSyt00ESillJfTRKBUKSQifu6OQZUdmgiUx7FVyTwrItEikioi/xWR2iKyQETOichiEQnMdfxNIrJdRM6KyHIRaZlrXwcR2Ww770cgIE9Zg0Rki+3cNSISYWeMN4rIHyKSLCLxIjI+z/6etuudte2/27a9goi8LyIHRSRJRFbZtvURkYR8Xod+tufjRWS6iHwrIsnA3SLSRUTW2so4KiIfi0i5XOe3FpFFIpIoIsdF5B8iUkdEzotIUK7jOorISRHxt+d3V2WPJgLlqYYD1wHNgMHAAuAfQE2s9+1jACLSDPgBeMK2bz4wV0TK2T4UZwH/A2oAP9mui+3cDsAU4AEgCJgMzBGR8nbElwr8H1AduBF4SERutl23gS3ej2wxtQe22M57D+gE9LDFNBbItvM1GQJMt5X5HZAFPAkEA92Ba4GHbTFUARYDvwL1gCbAEmPMMWA5cEuu694JTDXGZNgZhypjNBEoT/WRMea4MeYw8Duw3hjzhzEmHZgJdLAdNwr4xRizyPZB9h5QAeuDthvgD/zHGJNhjJkObMxVxhhgsjFmvTEmyxjzNXDBdl6hjDHLjTHbjDHZxphorGTU27b7NmCxMeYHW7mnjTFbRMQHuBd43Bhz2FbmGmPMBTtfk7XGmFm2MtOMMZuMMeuMMZnGmDisRJYTwyDgmDHmfWNMujHmnDFmvW3f18AdACLiC9yKlSyVl9JEoDzV8VzP0/L5ubLteT3gYM4OY0w2EA+E2PYdNpfPrHgw1/MGwNO2qpWzInIWCLOdVygR6Soiy2xVKknAg1jfzLFdY18+pwVjVU3lt88e8XliaCYi80TkmK266E07YgCYDbQSkXCsu64kY8yGYsakygBNBKq0O4L1gQ6AiAjWh+Bh4CgQYtuWo36u5/HAG8aY6rkeFY0xP9hR7vfAHCDMGFMN+BTIKSceaJzPOaeA9AL2pQIVc/0evljVSrnlnSp4ErALaGqMqYpVdZY7hkb5BW67q5qGdVdwJ3o34PU0EajSbhpwo4hca2vsfBqremcNsBbIBB4TEX8RGQZ0yXXu58CDtm/3IiKVbI3AVewotwqQaIxJF5EuWNVBOb4D+onILSLiJyJBItLedrcyBfhAROqJiK+IdLe1SewBAmzl+wMvAFdqq6gCJAMpItICeCjXvnlAXRF5QkTKi0gVEemaa/83wN3ATWgi8HqaCFSpZozZjfXN9iOsb9yDgcHGmIvGmIvAMKwPvESs9oSfc50bBdwPfAycAWJtx9rjYeBVETkHvISVkHKuewgYiJWUErEaitvZdj8DbMNqq0gE3gZ8jDFJtmt+gXU3kwpc1osoH89gJaBzWEntx1wxnMOq9hkMHAP2Atfk2r8aq5F6szEmd3WZ8kKiC9Mo5Z1EZCnwvTHmC3fHotxLE4FSXkhEOgOLsNo4zrk7HuVeWjWklJcRka+xxhg8oUlAgd4RKKWU19M7AqWU8nKlbuKq4OBg07BhQ3eHoZRSpcqmTZtOGWPyjk0BSmEiaNiwIVFRUe4OQymlShURKbCbsFYNKaWUl9NEoJRSXk4TgVJKeTlNBEop5eU0ESillJfTRKCUUl5OE4FSSnk5TQRKlQaZFyFqCmSkuzsSVURpF7NYs+8Un6/cz6kUe1clda1SN6BMKa+0ZwHMexIunocej7o7GlWIc+kZbDp4hg0HEll/IJHohLNkZFlzuq3Zd4opd3fm8kXz3E8TgVKlwSHbuvPrJkHXB8DX373xqEvOpF5kY1zipQ/+7UeSyDbg5yO0Da3GvT3D6RYexI6jyby7cDcz/zjMsI6h7g77MpoIlCoN4tdD+WqQnAAxP0O7Ue6OyGudSE5nQ84H//5Edh+3ZvIu5+dDh7DqPHpNE7o2CqJD/epULPfnR+zVzWqydNcJXpm7g55Ng6lVJcBdv8JfaCJQytNlpMHRrdD9YdizENZMgIhbwMOqF8qyI2fT+GR5LKtjT3PgVCoAFcv50qlBIIPb1aVroyAiQqtR3s+3wGv4+ghvD49g4ITfeXFWDJ/e0cljqog0ESjl6Y78AdkZUL87BDeD2Y/AvqXQ5Fp3R1bmZWRlM2XVAT5cspesbEOvpsHc2iWMruFBtK5XFT/fovW3aVKrMk/2a8bbv+7il21HGRRRz0mRF40mAqU83aF11r+hXaB8ZVjymnVXoInAqdbvP82Ls2PYczyFfi1r8fLg1oTVqFji697fK5wFMUd5efZ2ujcKIqhyeQdEWzLafVQpTxe/AYKaQqUg8CsP3R6E/cut6iLlcCfPXeCpH7cw6rN1pF7I4vP/i+SLuzo7JAkA+Pn68O6IdiSnZzB+7g6HXLOkNBEo5cmMsRqK63f9c1une6BcZVjzkfviKoOysg3/WxtH3/eXMzf6CI9c05jFT/Xmula1HV5W8zpV+HvfpszdeoSF2485/PpFpYlAKU92OhbSEiEsVyKoUB063W31Hjp7yF2RlSlb4s9y88TVvDh7OxGh1fj1iat59oYWVChXcONvST3UpzEt61blhVkxnD1/0Wnl2EMTgVKeLKd9IKzb5du7PWT1Glo3yfUxlSFnz1/knzO3MfST1RxPTuejWzvw7X1daVyzstPL9vf14d0RESSmXuS1eTudXl5hNBEo5cni10OFQAhqcvn2aqHQZjhs+hrSzrgntlIsO9swLSqevu+vYOrGeO7pEc6Sp3szuF09l3bpbBNSjYd6N2bG5gSW7T7hsnLz0kSglCeLX29VC/nk86fa4++QkWrNQaTstvNoMrdMXsvY6dGEB1di7qM9eWlwK6oEuGe09t+vbULTWpX5x8/bSE7PcEsMmgiU8lTnE+HUHgjrkv/+Om2hcV9YPxkyPXMyM0+SciGT1+btYNBHq9h/KpV3hkfw0wPdaVWvqlvjKu/ny7sj23E8OZ1/zXdPFZEmAqU8VfwG69+87QO59XgMUo5D9I+uiamUOnEunes/WMGU1QcY1TmMpU/35pbOYfj4eMbI3vZh1bm/VyN+2BDPqr2nXF6+JgKlPFX8evDxg3odCj6mUR/rzmDNR5Cd7bLQSpuPlsRy4twFfnqgO28ObUv1iuXcHdJfPHldMxoFV2Lcz9GkXsh0adlOTQQi0l9EdotIrIiMy2f/3SJyUkS22B5/c2Y8SpUq8euhbjsoV8hAJhHo8bhVhbR3oetiK0UOnk7lhw2HGN0ljMiGNdwdToEC/H15Z0QEh8+m8c6vu1xattMSgYj4AhOBAUAr4FYRaZXPoT8aY9rbHl84Kx6lSpWsDDi86fLxAwVpfTNUC4PVE5weVmn0waI9+PkKj/Vt6u5QriiyYQ3u6t6Qr9ceZMOBRJeV68w7gi5ArDFmvzHmIjAVGOLE8pQqO45GQ2a6fYnA1x+6PQyH1kBClPNjK0W2H0li9pYj3HtVOLWqes60z4UZ2785YTUqMHb6VtIuZrmkTGcmghAgPtfPCbZteQ0XkWgRmS4iYfldSETGiEiUiESdPHnSGbEq5VnibQvR2JMIADreCQHVYPWHzoupFHpv4W6qVfDngd6N3R2K3SqW8+PtYRHEnT7PB4t2u6RMdzcWzwUaGmMigEXA1/kdZIz5zBgTaYyJrFmzpksDVMot4tdB9fpQta59x5evApH3ws65cHqfc2MrJTYcSGTZ7pM81Kcx1SqUrhXdejQJ5rau9fnvqgNsPuT8AYPOTASHgdzf8ENt2y4xxpw2xuR0gP4C6OTEeJQqHYyxuo7aezeQo+uDVjXR2onOiasUMcbwzq+7qF21PHd1b+jucIrl+QEtqFM1gLHTo0nPcG4VkTMTwUagqYiEi0g5YDQwJ/cBIpL7685NgHsn3FDKE5w9BOeOFj0RVKljrVy25TtIdX1fdE+ydNcJog6e4fFrmzl14jhnqhLgz7+GRxB7IoUJS/Y6tSynJQJjTCbwKLAQ6wN+mjFmu4i8KiI32Q57TES2i8hW4DHgbmfFo1SpcWkgWRETAVgDzDLTYaP3dsDLyja88+tuGgZVZGSkZy0SX1S9m9VkZKdQJq/cz7aEJKeV49Q2AmPMfGNMM2NMY2PMG7ZtLxlj5tieP2+MaW2MaWeMucYY49rOs0p5ovh11noDtVsX/dyazaFZf9jwGVw87/jYSoE5Ww+z+/g5nr6+Of5FXErSE71wYyuCKpXj2elbuZjpnEGDpf9VUqqsiV8PoZHgU8wqjR6PwfnTsPV7x8ZVClzMzOb93/bQul5VbmxrZ0O7h6tW0Z83hrZl17FzfLn6gFPK0ESglCe5cA6Oby98fqEradADQjrBmo8h2zX90D3FDxsOkXAmjbH9W3jMPEKOcF2r2rw7IoLbuzVwyvU1ESjlSRKiwGQXPOOoPUSsu4IzB2DXPMfF5uFSL2Ty0dK9dGtUg6ubBrs7HIcbGRlG5fJ+Trm2JgKlPEn8ekAgtHPJrtNyMASGW9NOGOOQ0DzdlFUHOJVykbH9W7h0cZmyQBOBUp4kfr3VSBxQwjnyfXyh+yNwOAoOrXVMbB7sTOpFPlu5n+tb1aZj/UB3h1PqaCJQylNkZ0H8xuJ1G81P+9uhYpBXTEY3acU+Ui5m8swNzd0dSqmkiUApT3FiJ1w857hEUK4idL4f9iyAk66Zs8Ydjial8dWaOIZ1CKVZ7SruDqdU0kSglKeIX2f9W99BiQCgy/3gF2AtXFNGfbh4Lxh4op/nTzPtqTQRKOUp4jdA5dpQ3YFdBCsFW1VE0T/CuWOOu66HiD2RwrSoeG7vVp+wGoUs4KMKpYlAKU9xaJ1VLeToHi/dH7EWulk/2bHX9QAfLNpNBX9fHrmmibtDKdWc0ylVKVU0547B2YPQZYzjrx3U2OpOGvVfqBBY/ETj4w+N+0LNZo6Nr5i2xp9l/rZjPH5tU4Irl3d3OM517hhsm25NHxLs+KSniUApT5CzEE39EowoLkyvp2D3Alj0YsmvVa8jtLsV2gyHSkElv14xvbtwN4EV/flbr3C3xeBUF1Nh1y+wdSrsX2YNNPTx00SgVJkVv8Fq1K0T4Zzr1+sAzydAdkbxr5F2FnbMgq0/woJnYeHz0OQ6aDfa+qbq77qlIFfHnmJV7CleuLElVQJK16IzhcrOhrjfrQ//nXPgYoq1HnXPpyBilNPuxjQRKOUJDq2zvmn7lXNeGf4BQAk+rMtXgR5/tx7HYiB6KkT/ZHVPLV8N2gyFiNHWXU2u6idjDNuPJPP73lN0ahBIZIPAEs0DlLPoTL1qAdzhpLl3XO7ELtvrOQ2SD0O5KtD6ZuvOq34P8HFuc64mAqXcLSMNjm61GnVLizptoM7r0O8VOLDC+gYbPQ02fWX1emo3mhONbmbGgfLM/COBPcdTLp0aGliBYR1CGNoxlPDgSkUueuH2Y2xNSOKdEREE+JfORWcASDkJMdNh6w/W/7/4QpNr4frXoPlA8K/gslA0ESjlbkf+sKpsnNU+4Ew+vlYDcuO+cCGF9G2zSFr3LTVXvEOtFW/TJbspvtWup/rA0fSMaMr6A6f5efNhPl4Wy4SlsbQPq87wjiEMiqhHYKUr3w1lZmXz7sLdNKlVmWEdQlzwCzpYRhrsnm8lztglYLKgbju44V/QdgRUruWWsDQRKOVuOQ3FoSWYcdSNsrINq2JPMXNzAgu3B5OW8Xc6Vb+PJ2pvpeu5RXRKnAjLP4MjNzC03WiG3nU9x1INs7ccZuYfh3lx9nZenbeDa5rXYljHEK5pUYvyfvl/0/9582H2nUzl0zs64efKRWcupsLK9yD1ZMmuEbsYLiRD1RCriq3daKjV0nFxFpMmAqXc7dB6CGrq1h44xbHjSDIz/0hg9pYjnDh3gaoBfgztGMKwDiF0ahCIyEgwr8GxbdY34G0/WdNiVwikTuthPNBuNA9c3YsdR8/x8+YEZm89wm87jlOtgj+DIuoyrGMIHesHXppJND0ji38v3kO7sOrc0Lq2637RrAyYdpf1IV6lBIvd+PhCi0HQbhQ07FX8hYecQBOBUu5kjHVH0HyguyOxy/HkdGZvOczPmw+z69g5/H2FPs1rMaxDCH1b5vNNXgTqRliP6161ukFunQpbvrfGNdRoRKuI0bTqcQvjBvRl9b7TzNycwIzNCXy3/hANgioytEMIQzuEsGjHcY4mpfP+yHaum2Y6OxtmPwKxi2Dwh9DpbteU62JiStlc5ZGRkSYqKsrdYSjlGKf2wseRcNNH0PH/3B1Nvoyxqn4+//0Aq/aeJNtA+7DqDLPV7dewo27/L9KTre6RW6da3SUB6ne3uki2vpkUnyr8GnOMmX8ksGbfaYwBf1+hW6Mg/nefA+diKowxsPCfsG4i9H0Rrn7GNeU6iYhsMsZE5rdP7wiUcqec9gFHzTjqQMYYlu46wUdLY9kSf5baVcvzyDVNGNohhEY1K5fs4gFVocMd1uNsPGybZo1PmPcELHiOys37M6LdrYy4px9HUzKZ9ccRVsWe5IUbWznkd7PL6v9YSaDrQ9DradeV6wZ6R6CUO81+1Ko3f3a/0/uK2ys727Bw+zE+WhrLjqPJhAZW4KE+jRnRKbTARlyHMAaObrG1J0yH86es9RTaDLcaVet1dPw8TAXZ/A3M+Tu0HQlDP/OY/5uS0DsCpUrqzEFY/i+riqCaA7stxm+wegt5wAdNVrZhXvQRPl4ay94TKYQHV+LdERHc3CEEf1f00BGxRkDX6wDXvw77llp97Dd9DRs+g+BmVjfLpv2cG8fOeTD3cWjSD4Z84hH/N86miUApe/z2T9g515r8686Zjvlmej4RTu22epG4UUZWNjP/OMyk5fs4cCqVZrUr8+Ho9gyKqIdvCUYAl4ivPzS7wXqkJ8GO2bB2Inw33Op22fcl54zCjlsN0++17j5u+ca5I709iCYCpa7k0HorCdTraPV62fiFteBLSSVstP51U/vAhcwspm9KYNLyfSScSaN1vap8ekdHrm9Vp0RTQDhcQDWrIb3tSPjtBWuRnbhVMPy/1syqjnJsG/wwGgIbwu0/Qbmij3ourTQRKFUYY6wZOyvXgbvmwrT/g99etEbSlvRDKH69NZtkvY6OidVOaRezmLrxEJNX7OdYcjrtw6rz6pDWXNO8luu6ZRaHfwW48X1o1MdqW5l8NQz6N0TcUvJrJx6A/w2D8lXhzp+hYo2SX7MU0USgVGF2zrU+sAdPgPKVYcjH8Ek3mPkg3LMAfEvwJ3RovTXbaDnXrKyVciGTb9cd5Ivf93Mq5SJdw2vw3sh2XNUkyLMTQF4tB0Pd9vDz/dZj31IY+J71/1Mc547D/4ZCdibc/QtUC3VouKWBJgKlCpKVAYvHQ80W1nKPAFXrwcD34ee/wZoPi9+tMCsDDm+CyHscFm6BRWUbvt9wiA9+282Z8xn0ahrM3/s2pUt4Kf7WWz0M7poHK9+Fle9Yje4jpkC99kW7TnqS1e6QcsK64/OQRXdcTROBUgXZ9BUk7oPbpl3+zb/tCNg1F5b9C5peD3XaFv3ax6IhMw3CnDu/UHTCWV6YFUN0QhLdGtXguf4t6FA/0KlluoyvH1zzPIT3ghn3wxf9rNHL3R6yrzE/Ix1+uM2aAvq2HyG0k/Nj9lBlv1+UUsWRngzL37LmhGl6/eX7RODGf1vLPv78AGReKPr14zdY/4Y5Z8bRpPMZvDBrG0MmruZoUjofjm7PD/d3KztJILeGPeGh1db/08Ln4ftbIPVU4edkZcKM++Dgahj6qTX9sxfTRKBUftZMsAY0Xfdq/t8uKwVZ00Kc2G6NLyiqQ+ugWn2oWoJJzPJhjGHGpgT6vr+c79cf4q7uDVnydG+GtA8pXe0ARVWxBoz+zmor2L8CJl1l/ZsfY+CXJ62BfAPetu7wvJwmAqXySj4Caz6GNiMgpJAePc37Q4c7YfWHVsOvvXImmqvv2G6ju4+dY9TkdTz901bqB1VkzqM9GX9Ta6qWpaUcCyNideu9f4k1hcU3Q2DxK1Z7TG5LX7NGDl89Fro+4J5YPYwmAqXyWvamtWDItXYs9H7Dm1A1FGY+ABdSrnw8QFI8nDvqsPEDqRcyeXP+TgZO+J09J87x1rC2zHiwB21Cqjnk+qVOnbYwZjl0vBNWfQBfDrBGhgOs/QR+f9+aRfSaf7gzSo+ijcVK5XZ8B2z5Dro9bA0supKAqjB0Enw1CBa9BIM+uPI5l9oHSpYIjDHM33aM1+bt4FhyOqM7hzG2f4vizQZa1pSrZFXdNeoDc5+AT3tB+9tg/SSr++mNH7hu3qJSQBOBUrktftlapL0o3UIb9rQSx7qJ0GKgNUdNYQ6tg3KVoVbxZ9I8cCqVl2bH8PveU7SsW5WJt3ekU4My2BBcUm2GQ0gnmPE3Kwk07AXDvvCoRWE8gSYCpXLsXwF7f7MaiIs6svTaF60VrGY/Cg+vtXoUFSR+PYRGFmswWnpGFp8si+XTFfsp5+fDy4NbcWe3Bq5dtrG0CWxoDf7bvcAaEe4f4O6IPI6+e5QCayWqRS9CtTDoUowGRP8KVjfElBMwf2zBx104B8djilUttGzXCa779womLI2lf5s6LH26N/dcFa5JwB6+/tDqpuKPPi7jnPoOEpH+IrJbRGJFZFwhxw0XESMi+c6VrZTTxcyAo1utaaaL+40xpCP0HmstsrJ9Vv7HHN4EJrtIicAYw/u/7eaerzbi7+vD93/ryoRbO1Crqn6zVY7htEQgIr7ARGAA0Aq4VUT+UikqIlWAx4Ei9L9TyoEyL8CSV615f9qOLNm1ej1tzYMz70lrDpu8Dq0HxKoaskNWtuEfM2P4aGksoyLD+PXxq+nRJLhkMSqVhzPvCLoAscaY/caYi8BUYEg+x70GvA2kOzEWpQq24TNIOgTXv1byRUh8/WHYZ3AxFeY+Zo0ZyC1+PdRubU2tfAXpGVk8/N0mfthwiEeuacxbw9tSzk+rgZTjOfNdFQLE5/o5wbbtEhHpCIQZY35xYhxKFex8ojVxWZN+VldDR6jZHPqNhz2/wh/f/rk9O8tag8CO+YWS0zO4a8oGFm4/zkuDWvHsDS3K9shg5VZu+3ohIj7AB8AV++mJyBgRiRKRqJMnTzo/OOU9fn/fmleo3yuOvW7XB62uir+O+3Mw08ldcCH5ivMLnTiXzqjJ69h08Awfjm7PvT3DHRubUnk4MxEcBsJy/Rxq25ajCtAGWC4icUA3YE5+DcbGmM+MMZHGmMiaNWs6MWTlVc4ctKqF2t8Oddo49to+PjBkIiAw62GrV9Khdda+Qu4IDp5OZcSktcSdSuWLuyIZ0t6B6yMrVQBnJoKNQFMRCReRcsBoYE7OTmNMkjEm2BjT0BjTEFgH3GSMiXJiTEr9aenrID7Om2ogsAH0/xccXGUNZorfAJVrFzhiOeZwEsMnreVcegbf39+VPs1rOScupfJw2oAyY0ymiDwKLAR8gSnGmO0i8ioQZYyZU/gVlHKiI39Y3Tx7PgXVnPitu8MdsOsXa/Kz8lWgQfd8pzZYu+80938TRdUAP74Z04MmtbS/u3Idp44sNsbMB+bn2fZSAcf2cWYsSl1ijLXucMUg6PmEc8sSgcEfWstbnj+Vb/vArzFHeeyHLTQIqsg393WhbrUKzo1JqTy0L5ryPnsXQdzv0Ps5u7pxlliV2lYy8PGHxtdctuv79Yd4+LvNtAmpyk8PdtckoNxC5xpS3iU7y5oltEYj6OT89YIvaXUT/OMw+JUHrNHCHy+N5f1Fe+jTvCaf3N6RiuX0z1G5h77zlHfZ8j2c3AkjvwY/F0/XbEsC2dmGV+Zu5+u1BxnWIYS3R0Tgr/MFKTfSRKC8x8VUWPYGhERCq/wGubsghMxsnv5pK3O3HuH+XuE8P6AlPj46UEy5lyYC5T3WfWKtDDbiS7csSpJyIZOHvt3E73tP8fyAFjzQu7HLY1AqP5oIlHdIOQmrPoQWg6wunC50Ijmd9QcS+WzlfnYcTebdERGMjAy78olKuYgmAuUdVrwNGefh2pedXlR84nk2HEi0HnGJHDiVCkDVAD8m39GJfq1qOz0GpYpCE4Eq+07FwqYvodNdULOZQy9tjGH/qdQ/P/gPJHL4bBoA1Sr407lhDW7rUp8u4TVoXa+qLiKjPJImAlX2LRkPvuWhz/MlvlR2tmHPiXOs32996K8/kMiplAsABFcuR9fwIMZc3Ygu4TVoXruKNgSrUsGuRCAiPwP/BRYYY7KdG5JSDnRoPeycC33+AZWLP3fP9iNJfLh4L+sPJJKUlgFAvWoB9GoaTJfwGnQJr0Gj4Eo6VbQqley9I/gEuAeYICI/AV8aY3Y7LyylHMAYax3iyrWhx6PFusTFzGw+XhbLJ8tiqVbBn/6t61z64A+rUdHBASvlHnYlAmPMYmCxiFQDbrU9jwc+B741xmQ4MUblCU7vg/WToe8LEFDV3dHYZ+dca0WwwR9CuUpFPn3HkWSe/mkrO48mc3P7eoy/qTXVK7p4EJpSLmB3G4GIBAF3AHcCfwDfAT2Bu4A+zghOeZD5z8C+pZCRaptn38NlZcDi8RDcHNrfUaRTM7Kymbgslo+XxlK9Yjk+u7MT17eu45w4lfIA9rYRzASaA/8DBhtjjtp2/Sgiun5AWRe7xEoCNVtaSy+2GATNB7g7qsJt+goS98GtP4Kv/X0idh5N5pmftrL9SDJD2tdj/ODWBFbSuwBVttn7FzLBGLMsvx3GmL+sKKbKkOwsWPQyVK8Pf1sEUwbAnL/Dw+ugUrC7o8tfejIsfwsa9IRmN9h1SkZWNpOW7+OjpXupVsGfyXd24ga9C1Bewt5Oza1EpHrODyISKCIPOyck5VGip8HxbdZArPJVYOinkJ4E8560GmM90ZoJ1tz/179q11QSu44lM/ST1XywaA8D2tRl0ZO9NQkor2JvIrjfGHM25wdjzBngfqdEpDxHRpq1nGO9DtB6mLWtThtracedc2DbT+6NLz/JR2DNx9BmOIR0KvTQzKxsPl66l8EfreJYUjqf3tGRCbd20Kog5XXsTQS+kquDtIj4AvrX4krxG+DLG605c1xl/aeQnADXvWYtxp6jx2MQ1hV+eQaSDrsuHnssexOyM6Hvi4UetvvYOYZ+sob3ftvDDa3r8NuTvenfpq6LglTKs9ibCH7Fahi+VkSuBX6wbVOuEv2jtQj6vCdcUyWTehp+/wCa9YfwXpfv8/GFmydBdgbMfsRzqoiO74At30GXMVAjPN9DMm09ggZ/tIojZ9OYdHtHPr6tIzX0LkB5MXsTwXPAMuAh22MJMNZZQal8xK0G/0qwax5sner88la+CxdToN/4/PcHNYbrX4f9y2DjF86Pxx6LX4ZyVeDqZ/Ldvef4OYZNWsO7C3dzXeva/Pbk1Qxoq3cBStk7oCwbmGR7KFdLOWmtqtX3BYhdCgvGQsOeUN1JUxkn7rc+3DvcAbVaFnxc5L2w6xdr6cfGfa3k4C77V8De3+C6V6Fijb/snrYxnhdmxVA5wI+Jt3XkxghNAErlsOuOQESaish0EdkhIvtzHs4OTtkcXG39G94Hbv4ETLZVJZPtpGmflrwGvv7W/DyFEYEhH1vHznzQ6mrqDtnZVjKqFgZdHsizy/DWgl2MnRFN10Y1+O3JqzUJKJWHvVVDX2LdDWQC1wDfAN86KyiVR9wqq1qoXnur7vuGN+DACtj4uePLStgE23+G7o9CVTs+MKvWg4HvQ8IGWP2h4+OxR8wMOLrFumPyD7i0Oe1iFg9/t5lPV+zjtq71mXJ3Z4Irl3dPjEp5MHsTQQVjzBJAjDEHjTHjgRudF5a6TNwqqN/V+uYN0PEuaHKdNdDr1F7HlZMzSVulmnDVY/af13aEtQbwsjfh2DbHxWOPzAuw5FWo0xba3nJp84nkdEZ9tpaFO47xwo0teePmNrpAvFIFsPcv44KI+AB7ReRRERkKVHZiXCpH6imrfaBhzz+35VTJ+AfAzAcgK9MxZe1eYFVD9RlnDR6zlwjc+G+oEGhVEWVecEw89tjwGSQduqyL686jydw8cTWxJ1L47M5I/tarkU4PrVQh7E0EjwMVgceATliTz93lrKBULjntAw3zdOGsUgdufB8Ob4LV/y55OVmZVq+boCbWHUdRVQqCmz6C4zHW9A6ucD7R6t3UpB80vgaAZbtOMGLSGrKMYdoD3blOl4VU6oqumAhsg8dGGWNSjDEJxph7jDHDjTHrXBCfilsF/hWt0b15tRluPZa/BUe3lqycP76BU3ug3yt/VkEVVfP+0OFOWP0fa0EYZ1v1gTWvUL9XAPhq9QHu+3ojDYMrMfuRnrQJqeb8GJQqA66YCIwxWVjTTSt3iFtljeIt6MN54HtQMbhkVTIXUmDZvyCsG7QoYdPPDW9C1VCY9SBcTC3ZtQpz5qC1PkL728is2YqXZ8cwfu4O+raozbQHulOnWsCVr6GUAuyvGvpDROaIyJ0iMizn4dTIlNU+cGLH5e0DeVWsYbUXnNgBy94oXjlrP4bUE3D9a3ZN0laogKowdBIkHrC6dDrL0tdBfEjp8Rx/+yaKr9ce5P5e4Uy+sxOVyutS3EoVhb1/MQHAaaBvrm0G+NnhEak/FdQ+kFfT66DT3bB6AjQbAA2621/GuePWea2GQFiXYod6mYY9odvDsG4iNB8ITa51zHVzHNkC26ZxrvNjjPz+IHtPpPDG0Dbc3rWBY8tRykvYO7L4HmcHovIRt7rg9oG8rn8D9i+3qmQeXA3l7ezUtfxfkHXBmmbaka59EWIXw+xH4eE1Vo8iR7B1cc0MqMGgzZEkZqbx1T2d6dW0pmOur5QXsndk8ZciMiXvw9nBeb2c9gE/OyZEK1/ZmgjuzEH47QX7rn9yN2z+BiLvc/z0EP4VrLULUo7Dguccd93YxXBgJW+dv4msclX4+eEemgSUKiF72wjmAb/YHkuAqkCKs4JSWLN/ntheePtAXg16QI9HYdOXsHfRlY9fPN664+jtpPkDQzpa147+EXbMLvHlTFYmp2eN40B2baJrD2XWI1fRtHYRxjsopfJlVyIwxszI9fgOuAXQJSqd6VL7QBE7bF3zgrW28OxHrX72BYlbDbvnQ88nnLvkZK+noW57mPuE1R5RTGkXs/hpyjsEpcayNPQhvhnTS6eLUMpBitu9oilQy5GBqDziVoFfBajXsWjn+QfAsMnweV+Y/yyM+O9fj8mZSqJKPatR15l8/WHYZ/BpL6v9osMdRTo9PSOblXtPsmzncR7P/pqjVdpwz32P46PTRSjlMHYlAhE5h9VLKMcxrDUKlLMcXG3NL2RP+0BeddtB73Gw7HVrXECbPD19t8+0RiQPmQjlKjom3sLUbG51TV0wFvYtLdKpAcD1tke2bzl8bnkfNAko5VD29hrSilhXOp9oTdXQ185G3/z0fBL2LIBfnrLaDqrYFmPPvABLXoFaraHdrY6J1x5dH4DmA6x1kAtxNi2DGZvimb3lCKkXs+jROIjbu9anRZ2q+FQIhMp6I6qUo9l7RzAUWGqMSbL9XB3oY4yZ5bzQvJi94wcK4+sHQyfDpz1hzt/htmnWYLGoKXAmDm6fYS056UrV6xe460RyOp//vp9v1yWQnpnNwDYdeOSaJrSqV9WFASrlnextI3jZGDMz5wdjzFkReRmYVdhJItIf+BDwBb4wxryVZ/+DwCNAFlYvpDHGmB32h19GFbd9IK/gptY8PL8+Z3UTbTUEVrwD4b0dP8irmI6cTWPyin38sDGezKxshrQP4ZFrGtOklt6EKuUq9iaC/CplCz3XNlndROA6IAHYKCJz8nzQf2+M+dR2/E3AB0B/O2Mqu+JWWaN8i9M+kFeXMbD7F1j4D+tOIy3RMVNJlNCh0+eZtCKW6ZsSMAaGdwzloT6NaRhcya1xKeWN7E0EUSLyAdYHO1jf4jdd4ZwuQKwxZj+AiEwFhgCXEoExJjnX8ZW4vEHaO51PhOPb4Zp/OuZ6Pj4w5BOY1MPqzx8xympMdpPYEyl8sjyW2VuO4OsjjO5cnwd6NyI00AWN1kqpfNmbCP4OvAj8iPVhvQgrGRQmBIjP9XMC0DXvQSLyCPAUUI7L5zLKfcwYYAxA/foF1zOXCQfXAKbo4wcKUz0MBv3bWkGsJA3QJRCfeJ63f93FL9uOUt7Ph7t7NGTM1Y2oXVVnCVXK3eztNZQKjHNGAMaYicBEEbkNeIF8FrwxxnwGfAYQGRlZtu8actoHQkrYPpBX2xHWw8WMMXy3/hD/mr8TAzzYuzH39QzXwWBKeRB7ew0tAkYaY87afg4EphpjbijktMNAWK6fQ23bCjIVmGRPPGVa3CoI6wx+pf+DMuHMeZ6bEc3q2NP0bBLMW8PbahWQUh7I3qqh4JwkAGCMOSMiV+rQvRFoKiLhWAlgNHBb7gNEpKkxJmf19RsBB67EXgrljB+45h/ujqREjDF8v+EQb/6yE4A3hrbhti71dd1gpTyUvYkgW0TqG2MOAYhIQ67QsGuMyRSRR4GFWN1HpxhjtovIq0CUMWYO8KiI9AMygDN4+zrIh9bi8PYBFzt8No1xM6L5fe8prmoSxFvDIgiroXcBSnkyexPBP4FVIrICEKAXtsbbwhhj5gPz82x7Kdfzx+0P1QvErQK/AAjp5O5IiswYw9SN8bzxy06yjeH1m9twe1e9C1CqNLC3sfhXEYnE+vD/A2sgWeFzBaiii/vdNn6gdLUPHDmbxrift7Fyz0l6NA7i7eF6F6BUaWJvY/HfgMexGny3AN2AtRTQ3VMVw/lEOFa62geMMUyLiuf1eTvJMobXhrTm9q4N8PHRuwClShN7q4YeBzoD64wx14hIC+BN54XlhXLaBxpc5e5I7HI0KY1xM7axYs9JujWqwbsj2uldgFKllL2JIN0Yky4iiEh5Y8wuEWnu1Mi8TdzqUtE+YIzhp6gEXpu3g8xsw6tDWnOH3gUoVarZmwgSbDOOzgIWicgZ4KCzgvJKcb9DaGdrYRkPdTQpjed/3sby3SfpGm7dBdQP0rsApUo7exuLh9qejheRZUA14FenReVt0s7AsW3Q53l3R1KgZbtO8NjUP8jMMrxyU2vu7KZ3AUqVFUVeqtIYs8IZgXi1g549fmDGpgTGzoimZd0qTLytIw2CdIZQpcqS4q5ZrBwpbhX4lvfI9oHPV+7njfk7uapJEJPvjKRyeX3LKFXW6F+1JzhoW3/Ag9oHjDG8tWAXk1fu58aIunxwSzvK+7l4RTOllEvoKuDulnYWjkZ7VLVQZlY2z06PZvLK/dzZrQETRnfQJKBUGaZ3BO7mYfMLpV3M4tHvN7Nk1wme6NeUx69tqtNEKFXGaSJwt0vtA5HujoSk8xnc9/VGNh06w+s3t+GObg3cHZJSygU0Ebibh4wfOJaUzl1TNnDgVCoTb+vIwLZ13RqPUsp1tI3AndLOWuMH3FwttO9kCsMnrSHhzHm+uqezJgGlvIzeEbjToXVgst2aCKITznL3lxsRYOqY7rQNrea2WJRS7qGJwJ3ifrfaB0I7u6X4VXtP8cD/ogisVI7/3deV8GAdKKaUN9JE4E5xq9zWPjB36xGemraFxjUr8829XahV1XPGMCilXEvbCNwlPQmOuWf8wDdr43hs6h90CAvkxwe6axJQysvpHYG7XGofcN36A8YY/r14LxOW7OW6VrX56NYOBPjrQDGlvJ0mAneJ+x18y7msfSAr2/Di7Bi+X3+IWyJDeXNoW/x89YZQKaWJwH0utQ9UcHpRW+LP8uKsGLYdTuKhPo0Ze0NzHS2slLpEE4E7pCfB0a1w9bNOLebs+Yu8s3A3P2w4RM3K5fn4tg4Miqjn1DKVUqWPJgJ3cPL4gexsw/TNCby1YBdJaRnce1U4T/RrSpUAf6eUp5Qq3TQRuEPcKqe1D+w8msyLs2KIOniGTg0Cef3mNrSsW9Xh5Silyg5NBO4Qt8qaZM6B7QMpFzL596I9fLUmjmoV/HlnRAQjOobqcpJKqSvSROBq6clwdAv0esYhlzPG8Mu2o7w2bwcnzl3g1i71GXtDc6pXLOeQ6yulyj5NBK7mwPaB/SdTeGn2dlbFnqJNSFUm3xlJ+7DqJY9RKeVVNBG4mgPGD6RdzGLislg+W7mf8v4+vDqkNbd3bYCvVgMppYpBE4G9zifC3t+gST+oFFz86xxcbS1SX65isU5fvOM44+duJ+FMGsM6hPD8wJbUrFK++PEopbyeJgJ7zX8GYmaAjx80uQ7ajYJmA4o2YVx6MhzZAr2eKnLxiakXGTs9msU7j9O0VmWmjulGt0ZBRb6OUkrlpYnAHoc3WUmg410QUA2ip8GeBVC+GrS+GdqNhvrd4UqjdePXg8kqVvvAi7NiWLnnJM8PaMG9PcPx1+khlFIOoongSoyB316CisFw/esQUBX6jYcDK2Drj7DtJ9j8NVRvABGjrKQQ1Dj/a8X9Dj7+ENqlSCGs2HOSX7Yd5enrmvFA7wKurZRSxaSJ4Er2LISDq2Dge1YSAPDxhcZ9rceF92HXPNj6A6x8F1a+YzUER4yCNsOhYo0/rxW3CkIji9Q+kJ6RxcuzYwgPrsSY3o0c/MsppZSuR1C4rExY/DLUaAyd7s7/mPKVrbuA/5sNT+2A616Fi6lWm8J7zWDq7bBzrtXYfGQLNCjatNOTV+wn7vR5Xh3SmvJ+OmW0Usrx9I6gMFu+hZO74Jb/ga8d8/RUrQdXPQ49HrMWpY/+0WpP2DUP/AKK3D5w8HQqE5fHMiiiLr2a1izBL6KUUgXTRFCQi6mw7E0I6wotBxftXBGoG2E9+r0C+5dD9FQ4G29dzw7GGF6avZ1yvj68OKhV0eNXSik7aSIoyNqJkHLcuhsoydz9vn7QtJ/1KIKF24+xYs9JXhzUitq6lKRSyom0jSA/KSdg9YfWnUB9+77BO1LqhUxembuDFnWqcFf3Bi4vXynlXZyaCESkv4jsFpFYERmXz/6nRGSHiESLyBIR8YxPveVvQWY6XDveLcVPWLKXo0npvDG0jS4nqZRyOqd9yoiILzARGAC0Am4VkbyV3X8AkcaYCGA68I6z4rHbyT2w6SvodA8EN3F58buPneO/qw4wKjKMTg1qXPkEpZQqIWd+3ewCxBpj9htjLgJTgSG5DzDGLDPGnLf9uA4IdWI89lnyCvhXhN7PubxoY6wF5isH+PHcgBYuL18p5Z2cmQhCgPhcPyfYthXkPmBBfjtEZIyIRIlI1MmTJx0YYh4H11pdPXs+DpVd311z5h+H2XAgkXH9W1Cjkq4noJRyDY+ogBaRO4BI4N389htjPjPGRBpjImvWdNIHtDGw6EWoUhe6PeKcMgqRdD6DN+fvpEP96twSGeby8pVS3suZ3UcPA7k/0UJt2y4jIv2AfwK9jTEXnBhP4XbMhoSNcNPHxZ4iuiTe+203iakX+freLrq8pFLKpZx5R7ARaCoi4SJSDhgNzMl9gIh0ACYDNxljTjgxlsJlXoTF46FmS2h/m8uLj044y7frD3JXj4a0rlfN5eUrpbyb0+4IjDGZIvIosBDwBaYYY7aLyKtAlDFmDlZVUGXgJ7EGbR0yxtzkrJgKtOlLOHMAbvvJmlDOhbKyDS/MiqFm5fI8dV0zl5atlFLg5JHFxpj5wPw8217K9bxow22dIT0JVrwN4VdD0+tcXvz3Gw4RnZDEhFs7UCXAjvmMlFLKwTyisditVv0Hzp+2Zg0tyVQSxXDy3AXe+XUXVzUJYnBEXZeWrZRSObw7ESQdhnWfQNtboF4Hlxf/rwU7Sc/I4tUhbRAXJyGllMrh3Ylg2RtgsqHvCy4vet3+0/y8+TAPXN2YxjUru7x8pZTK4b2J4FgMbPkeuoyBQNdOcZSRlc2Ls2IIDazAI9e4fhoLpZTKzXunoV78srUQ/dXPuLzoKasOsPdECv+9K5IK5XTVMaWUe3nnHcG+ZRC72EoCFQJdWvSRs2n8Z/FermtVm2tb1nZp2UoplR/vuyPIzramkqhWHzrf7/LiX527A4Ph5cG66phSrpKRkUFCQgLp6enuDsXpAgICCA0Nxd/f/u7o3pcItk2z1hMe9gX4u3blr2W7TvDr9mOM7d+c0EDXT2OhlLdKSEigSpUqNGzYsEz30DPGcPr0aRISEggPD7f7PO+qGspIhyWvQd120Ga4S4tOz8ji5TnbaVyzEn/r2cilZSvl7dLT0wkKCirTSQBARAgKCirynY933RFsmAzJCTB0Evi4LgfuP5nCs9OjOZR4nu/v70o5P+/Kv0p5grKeBHIU5/f0nkRwPhFWvg9Nr7emk3CBrGzDl6sP8O7C3ZT38+E/o9rTo3GwS8pWSil7ec9X07UT4eI56PeKS4o7cCqVUZPX8vovO+nZJJhFT/Xm5g6FrcujlCqrzp49yyeffFLk8wYOHMjZs2cdH1Ae3nNH0OspCOkEtZ3bWyc72/DlmjjeXbiLcr4+vD+yHcM6hnjNbalS6q9yEsHDDz982fbMzEz8/Ar+GJ4/f36B+xzJexJBuUrQYqBTi4g7lcqz07eyMe4MfVvU4s2hbalTzbU9k5RShXtl7nZ2HEl26DVb1avKy4NbF7h/3Lhx7Nu3j/bt2+Pv709AQACBgYHs2rWLPXv2cPPNNxMfH096ejqPP/44Y8aMAaBhw4ZERUWRkpLCgAED6NmzJ2vWrCEkJITZs2dToUIFh8TvPYnAibKzDV+vjePtX3fh7+vDuyMiGNEpVO8ClFIAvPXWW8TExLBlyxaWL1/OjTfeSExMzKUunlOmTKFGjRqkpaXRuXNnhg8fTlBQ0GXX2Lt3Lz/88AOff/45t9xyCzNmzOCOO+5wSHyaCEro4OlUnp0ezYYDifRpXpO3hkXoXYBSHqywb+6u0qVLl8v6+U+YMIGZM2cCEB8fz969e/+SCMLDw2nfvj0AnTp1Ii4uzmHxaCIopuxswzdr43j71934+QjvjIhgpN4FKKXsUKlSpUvPly9fzuLFi1m7di0VK1akT58++Y4DKF++/KXnvr6+pKWlOSweTQTFcOj0eZ6dvpX1BxLp3awmbw1vS91qjqmrU0qVPVWqVOHcuXP57ktKSiIwMJCKFSuya9cu1q1b5+LoNBEUSXa24dv1B3lrwS58RXhneAQjI/UuQClVuKCgIK666iratGlDhQoVqF37zwkn+/fvz6effkrLli1p3rw53bp1c3l8YoxxeaElERkZaaKiolxe7pGzaTw1bQvr9ifSq2kwbw+PoF51vQtQqjTYuXMnLVu2dHcYLpPf7ysim4wxkfkdr3cEdohOOMt9X0dx/kImbw1ry6jOYXoXoJQqMzQRXMGvMUd54sctBFcuz3d/u4pmtau4OySllHIoTQQFMMYweeV+3lqwiw71q/PZnZHUrFL+yicqpVQpo4kgHxczrTWFf4yKZ1BEXd4b2Y4Af11SUilVNmkiyCPpfAYPfruJtftP8/e+TXiyXzN8fLQ9QClVdmkiyOXg6VTu+Woj8YnneX9kO4Z3CnV3SEop5XTeMw31FWyMS+TmiatJTL3It/d11SSglHKbypUrA3DkyBFGjBiR7zF9+vTBUV3p9Y4AmPlHAs9N30ZoYAWm3N2ZhsGVrnySUko5Wb169Zg+fbrTy/HqRGCM4d+L9zJhyV66Nwpi0h0dqV6xnLvDUko504JxcGybY69Zpy0MeKvA3ePGjSMsLIxHHnkEgPHjx+Pn58eyZcs4c+YMGRkZvP766wwZMuSy8+Li4hg0aBAxMTGkpaVxzz33sHXrVlq0aKFzDTlCekYWY6dHM2frEUZ2CuWNoW11LWGllFOMGjWKJ5544lIimDZtGgsXLuSxxx6jatWqnDp1im7dunHTTTcVOFh10qRJVKxYkZ07dxIdHU3Hjh0dFp9XJoJTKRcY800Umw+dZWz/5jzUu7GOFFbKWxTyzd1ZOnTowIkTJzhy5AgnT54kMDCQOnXq8OSTT7Jy5Up8fHw4fPgwx48fp06dOvleY+XKlTz22GMAREREEBER4bD4vC4R7D1+jnu/3siJ5At8cntHBrat6+6QlFJeYOTIkUyfPp1jx44xatQovvvuO06ePMmmTZvw9/enYcOG+U4/7QpeVReyau8phk1aQ9rFbH58oLsmAaWUy4waNYqpU6cyffp0Ro4cSVJSErVq1cLf359ly5Zx8ODBQs+/+uqr+f777wGIiYkhOjraYbF5zR3BjE0JjJ0RTdNalfnv3Z0J0ZlDlVIu1Lp1a86dO0dISAh169bl9ttvZ/DgwbRt25bIyEhatGhR6PkPPfQQ99xzDy1btqRly5Z06tTJYbF5TSJoEFSRfi1r8d7IdlQJ8Hd3OEopL7Rt25+9lYKDg1m7dm2+x6WkpADW4vUxMTEAVKhQgalTpzolLq9JBJENaxDZsIa7w1BKKY/jVW0ESiml/koTgVLKK5S21RiLqzi/p1MTgYj0F5HdIhIrIuPy2X+1iGwWkUwRyX9CDaWUKqGAgABOnz5d5pOBMYbTp08TEBBQpPOc1kYgIr7AROA6IAHYKCJzjDE7ch12CLgbeMZZcSilVGhoKAkJCZw8edLdoThdQEAAoaFFmzTTmY3FXYBYY8x+ABGZCgwBLiUCY0ycbV+2E+NQSnk5f39/wsPD3R2Gx3Jm1VAIEJ/r5wTbtiITkTEiEiUiUd6Q0ZVSypVKRWOxMeYzY0ykMSayZs2a7g5HKaXKFGcmgsNAWK6fQ23blFJKeRBnthFsBJqKSDhWAhgN3FbSi27atOmUiBQ+KUfBgoFTJY3BiTS+ktH4Ss7TY9T4iq9BQTvEmd2pRGQg8B/AF5hijHlDRF4Foowxc0SkMzATCATSgWPGmNZOjCfKGBPprOuXlMZXMhpfyXl6jBqfczh1igljzHxgfp5tL+V6vhGrykgppZSblIrGYqWUUs7jbYngM3cHcAUaX8lofCXn6TFqfE7g1DYCpZRSns/b7giUUkrloYlAKaW8XJlMBHbMelpeRH607V8vIg1dGFuYiCwTkR0isl1EHs/nmD4ikiQiW2yPl/K7lhNjjBORbbayo/LZLyIywfb6RYtIRxfG1jzX67JFRJJF5Ik8x7j89RORKSJyQkRicm2rISKLRGSv7d/AAs69y3bMXhG5y0WxvSsiu2z/fzNFpHoB5xb6XnByjONF5HCu/8eBBZxb6N+7E+P7MVdscSKypYBzXfIalogxpkw9sMYs7AMaAeWArUCrPMc8DHxqez4a+NGF8dUFOtqeVwH25BNfH2CeG1/DOCC4kP0DgQWAAN2A9W78vz4GNHD36wdcDXQEYnJtewcYZ3s+Dng7n/NqAPtt/wbange6ILbrAT/b87fzi82e94KTYxwPPGPHe6DQv3dnxZdn//vAS+58DUvyKIt3BJdmPTXGXARyZj3NbQjwte35dOBaERFXBGeMOWqM2Wx7fg7YSTEn43OjIcA3xrIOqC4idd0Qx7XAPmNMcUeaO4wxZiWQmGdz7vfZ18DN+Zx6A7DIGJNojDkDLAL6Ozs2Y8xvxphM24/rcPN4ngJeP3vY8/deYoXFZ/vsuAX4wdHlukpZTAT2zHp66RjbH0MSEOSS6HKxVUl1ANbns7u7iGwVkQUi4rTR1gUwwG8isklExuSz32Ezy5bQaAr+43Pn65ejtjHmqO35MaB2Psd4wmt5L9YdXn6u9F5wtkdt1VdTCqha84TXrxdw3Bizt4D97n4Nr6gsJoJSQUQqAzOAJ4wxyXl2b8aq7mgHfATMcnF4PY0xHYEBwCMicrWLy78iESkH3AT8lM9ud79+f2GsOgKP66stIv8EMoHvCjjEne+FSUBjoD1wFKv6xRPdSuF3Ax7/91QWE4E9s55eOkZE/IBqwGmXRGeV6Y+VBL4zxvycd78xJtkYk2J7Ph/wF5FgV8VnjDls+/cE1lxQXfIc4gkzyw4ANhtjjufd4e7XL5fjOVVmtn9P5HOM215LEbkbGATcbktUf2HHe8FpjDHHjTFZxphs4PMCynbre9H2+TEM+LGgY9z5GtqrLCaCS7Oe2r41jgbm5DlmDpDTO2MEsLSgPwRHs9Un/hfYaYz5oIBj6uS0WYhIF6z/J5ckKhGpJCJVcp5jNSrG5DlsDvB/tt5D3YCkXFUgrlLgtzB3vn555H6f3QXMzueYhcD1IhJoq/q43rbNqUSkPzAWuMkYc76AY+x5LzgzxtztTkMLKNuev3dn6gfsMsYk5LfT3a+h3dzdWu2MB1avlj1YvQn+adv2KtabHiAAq0ohFtgANHJhbD2xqgiigS22x0DgQeBB2zGPAtuxekCsA3q4ML5GtnK32mLIef1yxydY61HvA7YBkS7+/62E9cFeLdc2t75+WEnpKJCBVU99H1a70xJgL7AYqGE7NhL4Ite599rei7HAPS6KLRarbj3nPZjTi64eML+w94ILX7//2d5f0Vgf7nXzxmj7+S9/766Iz7b9q5z3Xa5j3fIaluShU0wopZSXK4tVQ0oppYpAE4FSSnk5TQRKKeXlNBEopZSX00SglFJeThOBUi5kmxl1nrvjUCo3TQRKKeXlNBEolQ8RuUNENtjmkJ8sIr4ikiIi/xZrHYklIlLTdmx7EVmXa27/QNv2JiKy2Db53WYRaWy7fGURmW5bD+A7V818q1RBNBEolYeItARGAVcZY9oDWcDtWCOao4wxrYEVwMu2U74BnjPGRGCNhM3Z/h0w0ViT3/XAGpkK1oyzTwCtsEaeXuXkX0mpQvm5OwClPNC1QCdgo+3LegWsCeOy+XNysW+Bn0WkGlDdGLPCtv1r4Cfb/DIhxpiZAMaYdADb9TYY29w0tlWtGgKrnP5bKVUATQRK/ZUAXxtjnr9so8iLeY4r7vwsF3I9z0L/DpWbadWQUn+1BBghIrXg0trDDbD+XkbYjrkNWGWMSQLOiEgv2/Y7gRXGWn0uQURutl2jvIhUdOUvoZS99JuIUnkYY3aIyAtYq0r5YM04+QiQCnSx7TuB1Y4A1hTTn9o+6PcD99i23wlMFpFXbdcY6cJfQym76eyjStlJRFKMMZXdHYdSjqZVQ0op5eX0jkAppbyc3hEopZSX00SglFJeThOBUkp5OU0ESinl5TQRKKWUl/t/33hqQN50TlUAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA5rklEQVR4nO3dd3hUZdrH8e+dTjoQQiCAoYfeAgQRBFGaKNI7ihRdsa7ldZtucXfdtaxdBEFAqWKvIAICUkM19BYgQEgoCQmQ/rx/nAEDJDFtZpLM/bmuuTKZ0+6ZTOY35znnPI8YY1BKKeW63JxdgFJKKefSIFBKKRenQaCUUi5Og0AppVycBoFSSrk4DQKllHJxGgRKFZGIzBaRF4o4b5yI3F7a9SjlCBoESinl4jQIlFLKxWkQqErF1iTztIjsFJGLIjJTRGqKyHcikioiy0Wkap757xaRXSKSLCKrRKRZnmntRGSrbblFgM912xogIttty64TkdYlrHmyiBwUkXMi8qWI1LY9LiLyPxFJFJELIvKLiLS0TesvIrtttZ0QkadK9IIphQaBqpyGAHcATYC7gO+APwI1sN7zjwKISBNgAfC4bdq3wFci4iUiXsDnwIdANeBj23qxLdsOmAU8AFQH3gO+FBHv4hQqIrcB/waGA7WAo8BC2+TeQHfb8wiyzXPWNm0m8IAxJgBoCawoznaVykuDQFVGbxpjThtjTgBrgI3GmG3GmHTgM6Cdbb4RwDfGmB+MMVnAy0AV4GYgGvAEXjPGZBljlgCb82xjCvCeMWajMSbHGDMHyLAtVxxjgFnGmK3GmAzgD0AXEYkAsoAAIBIQY8weY8wp23JZQHMRCTTGnDfGbC3mdpW6SoNAVUan89y/nM/v/rb7tbG+gQNgjMkFjgPhtmknzLW9Mh7Nc/8m4Elbs1CyiCQDdW3LFcf1NaRhfesPN8asAN4C3gYSRWS6iATaZh0C9AeOishPItKlmNtV6ioNAuXKTmJ9oANWmzzWh/kJ4BQQbnvsinp57h8H/mmMCc5z8zXGLChlDX5YTU0nAIwxbxhjOgDNsZqInrY9vtkYMxAIxWrCWlzM7Sp1lQaBcmWLgTtFpJeIeAJPYjXvrAPWA9nAoyLiKSKDgU55lp0BPCginW0Hdf1E5E4RCShmDQuACSLS1nZ84V9YTVlxItLRtn5P4CKQDuTajmGMEZEgW5PWBSC3FK+DcnEaBMplGWP2AWOBN4EzWAeW7zLGZBpjMoHBwH3AOazjCZ/mWTYGmIzVdHMeOGibt7g1LAf+AnyCtRfSEBhpmxyIFTjnsZqPzgIv2aaNA+JE5ALwINaxBqVKRHRgGqWUcm26R6CUUi5Og0AppVycBoFSSrk4DQKllHJxHs4uoLhCQkJMRESEs8tQSqkKZcuWLWeMMTXym1bhgiAiIoKYmBhnl6GUUhWKiBwtaJo2DSmllIvTIFBKKRenQaCUUi6uwh0jUEqp4srKyiI+Pp709HRnl2J3Pj4+1KlTB09PzyIvo0GglKr04uPjCQgIICIigms7lK1cjDGcPXuW+Ph46tevX+TltGlIKVXppaenU7169UodAgAiQvXq1Yu956NBoJRyCZU9BK4oyfN0mSA4dzGTv321i/SsHGeXopRS5YrLBMG6Q2f44Oc4Js2J4VJmtrPLUUq5kOTkZN55551iL9e/f3+Sk5PLvqDruEwQDGhdm5eHtWHdoTPcN2szqelZzi5JKeUiCgqC7OzCv5R+++23BAcH26mqX7lMEAAM7VCHN0a1Y+ux84yduYnkS5nOLkkp5QKeffZZDh06RNu2benYsSPdunXj7rvvpnnz5gDcc889dOjQgRYtWjB9+vSry0VERHDmzBni4uJo1qwZkydPpkWLFvTu3ZvLly+XWX0ud/rogNa18fZwZ+q8rYyasZEPJ3YixN/b2WUppRzkb1/tYvfJC2W6zua1A3n+rhYFTn/xxReJjY1l+/btrFq1ijvvvJPY2Nirp3jOmjWLatWqcfnyZTp27MiQIUOoXr36Nes4cOAACxYsYMaMGQwfPpxPPvmEsWPHlkn9LrVHcMUdzWvy/r1RHDmTxsjpGzh9ofJfZKKUKj86dep0zXn+b7zxBm3atCE6Oprjx49z4MCBG5apX78+bdu2BaBDhw7ExcWVWT0ut0dwRfcmNZg9oRMTZ29m+HvrmTepM3Wq+jq7LKWUnRX2zd1R/Pz8rt5ftWoVy5cvZ/369fj6+tKjR498rwPw9v615cLd3b1Mm4Zcco/giugG1flwUmfOX8xk+LT1xJ256OySlFKVUEBAAKmpqflOS0lJoWrVqvj6+rJ37142bNjg4OpcPAgA2teryvzJ0VzOymH4e+s5cDr/P5ZSSpVU9erV6dq1Ky1btuTpp5++Zlrfvn3Jzs6mWbNmPPvss0RHRzu8PjHGOHyjpREVFWXsMTDN/tOpjHl/Izm5hg8ndqJF7aAy34ZSyjn27NlDs2bNnF2Gw+T3fEVkizEmKr/5XX6P4IomNQNY/EAXfDzcGDV9A9uPJzu7JKWUcggNgjzqh/ix6IEuBPt6Mfb9jWw6cs7ZJSmllN1pEFynbjVfFj/QhdBAb8bP2siaA0nOLkkppexKgyAfYUE+LJrShYjqfkycHcPy3aedXZJSStmNBkEBagR4s3BKNJG1Anjwoy18s/OUs0tSSim7cJ0gSE2A/Uvh9G7ISCvSIsG+Xnw0qTNt6wbzyIKtLI45TmZ2rp0LVUopx3KdK4vj1sInE3/9vUo1CK4HwXUh+Cbb/XoQVNf66RMIQKCPJ3MndmLSnBieWbKTZ5bsJMTfm1pBPoQF+Vz7M7DK1d99PN2d9ESVUhWdv78/aWlpnDx5kkcffZQlS5bcME+PHj14+eWXiYrK94zQYnGdIGjcGyYuh+SjkHIcko9Zt6R9cGA5ZF93ubZP8NVw8A2ux5zmddgUHsw291acuAinUtI5fu4Sm46cI+XyjV1aV/X1JCzICoaagVZQ3FTdl74tw/D20JBQSv222rVr5xsCZc11gsAnEOp2tG7XMwYunrGFw3VBcfYgHFqBZ9YlugJdPX2hSV/oPAga3wGeVbiUmU1CSjoJKemcSkkn4UI6p1IuX/19x/Fkzl60ury+qbovf7mzOb2ahbrM0HlKubpnn32WunXrMnXqVAD++te/4uHhwcqVKzl//jxZWVm88MILDBw48Jrl4uLiGDBgALGxsVy+fJkJEyawY8cOIiMjtRvqMicC/jWsW50ON043Bi6dg9O/wO4vYPeXsOtT8PKHJn3xbTmYBg170aBGSIGbSM/KYcPhs7zwzR4mzY2he5MaPDegOY1C/e34xJRSN/juWUj4pWzXGdYK+r1Y4OQRI0bw+OOPXw2CxYsXs3TpUh599FECAwM5c+YM0dHR3H333QV+QXz33Xfx9fVlz5497Ny5k/bt25dZ+RoERSECftWhQQ/r1u8lOLoWdn1mhULsEvAKgMj+0GIQNLwNPK4d48DH050eTUPp2iiEueuP8toP++n72momdI3gkV6NCfTxdMpTU0rZX7t27UhMTOTkyZMkJSVRtWpVwsLCeOKJJ1i9ejVubm6cOHGC06dPExYWlu86Vq9ezaOPPgpA69atad26dZnVp0FQEu4ev4ZC/5fhyGorFPZ8BTsXgXcQRN5phUKDHuDhdXVRT3c3Jt5Sn4Fta/PS9/t4f+0RPtt2gmf6RjK0fR3c3LS5SCm7KuSbuz0NGzaMJUuWkJCQwIgRI5g3bx5JSUls2bIFT09PIiIi8u1+2hFc5/RRe3H3hEa9YOBb8PRBGPMJNLsL9n0D84fBy43g86lwcDnk/HpQOcTfm/8Mbc0XU7tSr5ovzyzZyaB317Ht2HknPhmllL2MGDGChQsXsmTJEoYNG0ZKSgqhoaF4enqycuVKjh49Wujy3bt3Z/78+QDExsayc+fOMqtNg6AsuXtC49vhnrfhqYMwejE07Q97voSPhsCrzeD4pmsWaV0nmCUP3syrw9twMvkyg95Zx5OLd5CYqqOmKVWZtGjRgtTUVMLDw6lVqxZjxowhJiaGVq1aMXfuXCIjIwtd/ne/+x1paWk0a9aM5557jg4d8jmeWULaDbUjZGfAoRWw9I9w8Szc9zXUurF9Ly0jm7dWHGTm2sN4e7jzaK9G3Hdzfbw8NK+VKg3thlq7oXY+D29o2g/GfwHeAfDhIEjaf8Ns/t4ePNsvkmVP3Eqn+tX417d76fvaalbuS3RC0UopV6FB4EjB9eDeL0HcYO5AOB+X72z1Q/yYdV9HPrivIwaY8MFmJs7erENpKqXsQoPA0ao3hPGfW1cyz7kbLpwscNaekaEsfbw7f+gXyYbDZ+n9v9XMXR9HRWvOU6o8cJX/m5I8Tw0CZ6jZAsZ+Yl2kNnegdVVzAbw83Hjg1oasfKoHXRtV57kvdvHowu1czMh2YMFKVWw+Pj6cPXu20oeBMYazZ8/i4+NTrOX0YLEzxf1snU0U0gju/RqqBBc6e26u4d2fDvHKsn3UD/Fj2tgONK4Z4JhalarAsrKyiI+Pd9p5+o7k4+NDnTp18PS89iLVwg4W2y0IRKQuMBeoCRhgujHm9evmEeB1oD9wCbjPGLO1sPVWqiAA6/qC+SOhdjsY9xl4/3aXE+sOnuHRhdu4mJHDi0NaMbBtuAMKVUpVZM46aygbeNIY0xyIBqaKSPPr5ukHNLbdpgDv2rGe8qnR7TB0FpzYAgtHQdZvf2O5uVEIXz/SjZbhgTy2cDt/+TyWjOwcBxSrlKqM7BYExphTV77dG2NSgT3A9V9dBwJzjWUDECwitexVU7nV/G645x2rq4qP77vmCuSChAX5MH9yNFO6N+DDDUcZNm09x89dsn+tSqlKxyEHi0UkAmgHbLxuUjhwPM/v8dwYFojIFBGJEZGYpKRKOph8m5Fw5yuw/zv4dArk/vY3fE93N/7YvxnTxnbgSNJFBry5lhV7dXxlpVTx2D0IRMQf+AR43BhzoSTrMMZMN8ZEGWOiatSoUbYFlicdJ8Edf7e6uP7qMcgt2rCYfVuG8dUjt1A7uAr3z47h5aX7yMmtWCcBKKWcx65BICKeWCEwzxjzaT6znADq5vm9ju0x19X1Mej+DGz70OqSoogH8yNC/PjsoZsZEVWXt1YeZNzMjZxJy7BzsUqpysBuQWA7I2gmsMcY82oBs30JjBdLNJBijDllr5oqjJ5/hOiHYOO7sPKfRV7Mx9Od/wxtzX+HtmbL0fPc+cYaNseds2OhSqnKwJ57BF2BccBtIrLddusvIg+KyIO2eb4FDgMHgRnAQ3asp+IQgT7/gvbjYfVLsPa1Yi0+PKounz3UlSqe7oycvoEZqw9X+gtplFIlpxeUlWe5OdaB49gl1gA4nSYXa/EL6Vk88/FOvt+VQJ8WNXlpWBsdCU0pF6W9j1ZUbu4waJo1psG3T8H2BcVaPNDHk3fHtufPdzZj+Z5E7npzrXZcp5S6gQZBeefuCUM/gPq3whcPwf6lxVpcRJjUrQELp0STmp7NyOkbOKJhoJTKQ4OgIvD0gVELIKw1LLkfEmKLvYqOEdWYP7kzmTm5jJy+XsNAKXWVBkFF4eUHoxaCdyDMHwGpCcVeRWRYIPMndyYrxzBy+noOJ6XZoVClVEWjQVCRBNaC0Qvh8jlYMAoyi9+lRGRYIAsmR5OdYxg5fQOHNAyUcnkaBBVNrTYwZCac3AafPVDkq4/zahoWwPzJ0eTkGkZpGCjl8jQIKqLI/tD7BdjzJaz4R4lW0TQsgAVTosk11p7BwUQNA6VclQZBRdVlKnS4D9a+CtvmlWgVTWoGsGByNMbAqBkaBkq5Kg2CikrEusisQQ+rg7q4tSVaTeOaASyc0hljsO0ZpJZtnUqpck+DoCJz94Rhc6BafVg0Fs4eKtFqGoVaYQAwcvpGDpzWMFDKlWgQVHRVgmH0IkBg3jC4VLJO5qwwiEbEaibSMFDKdWgQVAbVGsDI+ZByHBaPh+zMEq2mUag/CyZHIyKMmrGB/RoGSrkEDYLK4qYuMPBtiFsD3zxR5HEMrtco1J+FU6JxE2HU9A3sS9AwUKqy0yCoTFoPtw1q8xH8/HqJV9OwhhUGHu7C6BkaBkpVdhoElU3PP0KLwbD8r7D7yxKvpkENfxZO6YKHu9VMtDehRKOMKqUqAA2CykYE7nkH6kRZYxmc2FriVdUP8WPhlC54ubsxesZG9pzSMFCqMtIgqIw8q1gHj/1qWH0SpZR8GGgrDKLxcndj1IwN/PGzX1iw6RixJ1LIzC5+9xZKqfJHRyirzE7vhpm9oVoETPgevP1LvKq4Mxd5/stdbD12ntT0bAC83N2IrBVAy/AgWocH0TI8iCY1A/Dy0O8XSpU3hY1QpkFQ2R1YDvOHQeM+MHKeNepZKRhjOHbuEjvjU4g9kcIvttv14dAqPIhWtnBoGhaAp7uGg1LOpEHg6jbNsIa67PIw9Plnma8+N9cKhyuh8Et8CrEn84SDhxvNwgJoV68qj9zWiOr+3mVeg1KqcIUFgYeji1FO0GkynDkA69+C6o0gakKZrt7NTYgI8SMixI+72tQGrHA4aguHWFs4zN94jE1HzrFgSjRBVTzLtAalVMnpHoGryMmGBSPh8EoY+yk0uNXhJazal8jkuTG0DA/iw4md8ffW7yFKOUphewTacOsq3D1g6Cyo3hgWj4MzBx1eQo+mobw5qj0741OYNGcz6Vk5Dq9BKXUjDQJX4hNoDXXp5gHzh5e4g7rS6NsyjFeGtWHjkXM8+NEWPQVVqXJAg8DVVI24toO6nCyHl3BPu3D+NagVq/Yl8djCbWTnaBgo5UwaBK6oXjTc/Zatg7onS9xBXWmM6lSPvwxoznexCTy9ZCe5uRXrWJVSlYkerXNVbUbAmX2w5hWo0dQa+tLBJt5Sn8uZ2by8bD9VvNz55z0tERGH16GUq9MgcGU9/2ydVrr0T1CtITTt6/ASHr6tMZcyc3hn1SGqeLrz5zubaRgo5WAaBK7MzQ0GTYPkY/DJRLh/KYS1dHgZT/dpyqXMHGauPYKflzu/793U4TUo5cr0GIGr8/KDUQvAO8C6ziAt0eEliAjPDWjOiKi6vLHiIO+uKtnYy0qpktEgUBBY2wqDi2dg4WjISnd4CW5uwr8Gt+LuNrX5z/d7mbs+zuE1KOWqNAiUpXY7GDwd4jfDF1OdciaRu5vwyvA23NG8Js99sYvFMccdXoNSrkiDQP2q+d3Q6zmIXQKrX3JKCZ7ubrw1uh3dGofw7Cc7+WrHSafUoZQr0SBQ17rl99BmFKz8J8R+6pQSvD3cmT4uiqibqvHEou0s333aKXUo5SrsFgQiMktEEkUktoDpQSLylYjsEJFdIlK2XWKqkhGBu16HutHw+e/gxBanlFHFy52Z90XRonYgD83bytoDZ5xSh1KuwJ57BLOBwk5MnwrsNsa0AXoAr4iIlx3rUUXl4W0NYuNf0zbUZbxTygjw8WTO/Z1oUMOPyXNj2Bzn+L6RlHIFdgsCY8xqoLD/XAMEiHX1kL9t3mx71aOKyS8ERi+CrMvWaaUZaU4pI9jXiw8ndqZWsA8TPtjMlqPnnVKHUpWZM48RvAU0A04CvwCPGWPy7X1MRKaISIyIxCQlJTmyRtcW2gyGfgCnd8FnD0CuczqHqxHgzbxJnQnx92LczI38fFCbiZQqS84Mgj7AdqA20BZ4S0QC85vRGDPdGBNljImqUaOG4ypU0Ph26PNv2Ps1LH8esjOdUkatoCosfrALdav6MuGDzfygB5CVKjPO7GJiAvCisYZIOygiR4BIYJMTa1L56fwAnNkP696wblWqWccPAmpaP/1rQkDYr/evTPMOtA4+l5HQAB8WPRDNvR9s5sGPtvDq8DYMbBteZutXylU5MwiOAb2ANSJSE2gKHHZiPaogItDvv1CvC5w/AqkJkHbauh1db/3MybhxOY8q4B9qC4lQCKoH0b+D4LolLiXY14t5kzozac5mHl+0nbSMbMZ0vqkUT04pZbcxi0VkAdbZQCHAaeB5wBPAGDNNRGpjnVlUCxCsvYOPfmu9OmZxOWQMpCdD6ulfAyJvWKSdtqadjwPPKjDwLWh2V6k2mZ6Vw0PztrJibyJ/6BfJA7c2LJOnolRlVdiYxTp4vXKcs4dgyf1wajt0nAS9X7CCoYSycnL5/eIdfLXjJA/3bMSTvZtoF9ZKFUAHr1flQ/WGMPEH6PIwbH4fZvSCpH0lXp2nuxuvjWjLyI51eWvlQf721W4d6UypEtAgUI7l4QV9/gmjP4a0BHjvVtgyp8Sd3Lm7Cf8e3IrJ3eoze10cz3yyU8dAVqqYNAiUczTpDQ/+DHU7wlePWk1G6SklWpWI8Mf+zfj9HU1YsiWeRxZsIyM7p4wLVqry0iBQzhNYC8Z9Drf9BXZ/AdO6QXzJjv+ICI/2asxzA5rzXWwCk+du4XKmhoFSRaFBoJzLzR26PwUTvrOah2b1gbX/K/FVzPffUp//DmnN2gNJjJ+1kQvpWWVcsFKVjwaBKh/qdYYH10DknbD8r/DRYOuU0xIY3rEub45qz/bjyYyesYFzF51zNbRSFYUGgSo/qgTDsDkw4DU4th6mdYWDP5ZoVXe2rsX08VEcOJ3GiPfWk5Di+OE3laooNAhU+SICURNg8krwDbH2DJb9pUR9HPVsGsqc+ztxKiWdYe+t49jZS3YoWKmKT4NAlU81m8PkFdBhgtW/0Qd94dyRYq8mukF15k3qTGp6NsPeW6c9lyqVDw0CVX55+cJdr1nNRWcOwnvd4fjmYq+mTd1gFj/QBW8Pd8a8v5FJczZzOMk54ysoVR5pFxOqYkg+BrMHQE4WPLAa/IvfHXl6Vg6z18Xx1oqDpGflML5LBI/1akyQr6cdClaqfNEuJlTFF1wPRnwEl8/BkgmQU/zB7Hw83Xnw1oasfKoHw6LqMnvdEW59eSVz1sWRpVcjKxdWpCAQkcdEJFAsM0Vkq4j0tndxSl2jVmu481WIWwMr/lHi1dQI8Obfg1vx9SPdaF4rkOe/3EW/19ewal9iGRarVMVR1D2C+40xF4DeQFVgHPCi3apSqiDtxlgHkH9+DfZ8VapVNa8dyLxJnZkxPorsnFzu+2Az987axIHTqWVTq1IVRFGD4Erfvv2BD40xu/I8ppRj9fsP1G4Pn/3OOohcCiLCHc1rsuyJW/nznc3Yeuw8fV9fw/NfxHJeL0RTLqKoQbBFRJZhBcFSEQkAtFFVOYeHNwyfC+6esHgcZF4s9Sq9PNyY1K0BPz3dk9Gd6vHRxmPc+tJKZq49Qma2vtVV5Vaks4ZExA1rgPnDxphkEakG1DHG7LRzfTfQs4bUVYdWwIeDodVQGDyjTMdH3n86lRe+2cPq/UnUD/HjT/2b0atZqA58oyqssjhrqAuwzxYCY4E/AyXrM1ipstLwNrjtT/DLx7BpepmuuknNAObe34kPJnTETWDS3BjGzdzE6QvaVYWqfIoaBO8Cl0SkDfAkcAiYa7eqlCqqW56EJv1g6R/h2MYyX33PpqF8/3h3/nZ3C7YeO8+kOTHavbWqdIoaBNnGakMaCLxljHkbCLBfWUoVkZsbDJoGQXXh43shrexPAfV0d+PemyN4c1Q7Yk+m8NTHO3RITFWpFDUIUkXkD1injX5jO2agl2Oq8qFKMIz4EC4nWyOdleBis6Lo1awmf+gXyTe/nOL1Hw/YZRtKOUNRg2AEkIF1PUECUAd4yW5VKVVcYa1gwP+si81+/JvdNjO5WwOGdajD6z8e4MsdJ+22HaUcqUhBYPvwnwcEicgAIN0Yo8cIVPnSdhRETbR6K939hV02ISK8MKglHSOq8vTHO9h+PNku21HKkYraxcRwYBMwDBgObBSRofYsTKkS6ftvCO8An0+FM/ZpvvH2cGfa2A7UCPBmytwYTqVctst2lHKUojYN/QnoaIy51xgzHugE/MV+ZSlVQlcuNvPwgkVjIcM+3U1X9/dm5r0duZSZw+S5MVzKtM9xCaUcoahB4GaMyXs6xtliLKuUYwXVgaGz4Mx++PIRsFNX603DAnhzVDt2n7zAk4v1TCJVcRX1w/x7EVkqIveJyH3AN8C39itLqVJq0ANu+zPs+hQ2TrPbZnpGhvLH/s34LjaB15bvt9t2lLInj6LMZIx5WkSGAF1tD003xnxmv7KUKgNdn4D4GFj2Z6jdDupF22UzE2+pz4HTabyx4iANQ/0Z2DbcLttRyl50hDJVuV1Ohhk9IfOSNbJZQE27bCYzO5exMzey/Xgyi6ZE065eVbtsR6mSKnFfQyKSKiIX8rmlisgF+5SrVBmqEgzDP4T0FGtkMzsdPPbycGPa2A6EBfow5cMtnEzWM4lUxVFoEBhjAowxgfncAowxgY4qUqlSCWsJd70OR3+GlxrBxxNg7zeQnVGmm6nm58XMe6NIz8xh0hw9k0hVHHrmj3INbUbAxB+g7Wg48hMsHA0vN4YvHobDqyC3bDqSa1wzgDdGt2NvwgWeWLRdzyRSFYIeI1CuJyfL+vD/ZQns/Roy08C/JrSwjW0Q3qHUYxvMWnuEv3+9m4d7NuKpPk3Lpm6lSqGwYwRFOmuohBudBQwAEo0xLQuYpwfwGlYHdmeMMbfaqx6lrnL3hMZ3WLfMS3BgqRUKMTNh47tQNQJaDoVWwyA0skSbmNA1ggOJqby18iCNQv25p52eSaTKL7vtEYhIdyANmJtfEIhIMLAO6GuMOSYiodddtJYv3SNQdnM52dpD+GWJ1XxkcqFmS2g5xLpVvalYq8vMzmX8rI1sPZbMgsnRdLhJzyRSzlPYHoFdm4ZEJAL4uoAgeAiobYz5c3HWqUGgHCL1NOz+3AqF+E3WY3U7Q4cJ0HKw1ZVFEZy/mMk97/zMxYxsvnj4FsKDq9ivZqUKURZDVdpDE6CqiKwSkS0iMr6gGUVkiojEiEhMUlKSA0tULiugJnR+ACb9AI/tgF7PweXz8PmD8L+WsOrFIg2CU9XPi5n3diQjO5dJc2JIy9AziVT548w9greAKKAXUAVYD9xpjCn0On3dI1BOYwwcWmF1WXFgGbh7WU1GnR+E2m0LXXT1/iQmzN5M05oBzLwvilpBumegHKu87hHEA0uNMReNMWeA1UAbJ9ajVOFEoFEvGPMxPLwFOtwHu7+E6bfCrH7WGAgFjI7WvUkNZt4bxbFzl7jn7Z+JPZHi2NqVKoQzg+AL4BYR8RARX6AzsMeJ9ShVdCGNoP9L8Pvd0PufcCEeFo+HN9rCz69bzUjX6dE0lE9+dzMebm4Mm7aeZbsSHF+3Uvmw51lDC4AeQAhwGnge2zjHxphptnmeBiYAucD7xpjXfmu92jSkyqXcHNj3HWx4F46uBU9faDPKajaq0eSaWRNT05k8dws745P5U/9mTLylPlLK6xaU+i1OO2vIHjQIVLl3aidsfA9++RhyMqBhL4j+nfXTzdoJv5yZw5Mfb+fbXxIY3bkef7u7BZ7ueqG/sh8NAqWcIS0JtnwAm9+HtNNQvbHV51GE1Zt7bq7h5WX7eGfVIbo1DuHtMe0J9PF0ctGqsiqvB4uVqtz8a8Ctz8DjsTB4BpgcmHs3bJkDgJub8EzfSP47tDXrD51lyDvrOH7ukpOLVq5Ig0Ape/PwgtbDYfJKqN8dvnoUvv/D1TOMhkfVZe7ETpy+kM6gd35m67EbDzQrZU8aBEo5SpVgGP2xdQB5wzswf7g1TgJwc8MQPpvaFT9vD0ZO38BXO046t1blUjQIlHIkdw/o9x8Y8JrVn9H7t8PZQwA0rOHPZw91pU2dIB5ZsI23Vhyg3BzDy82FE1utnltVpaNBoJQzRE2A8V/AxTMw4zY4/BNgDW7z0aTODGoXzsvL9vPkxzvIyC6bsRJKJCMNNs2AtztZQ37+8LzzalF2o0GglLNE3AKTV0BAGHw4yDq7CPD2cOfV4W144vYmfLr1BONmbuL8xUzH1nb+KCz9E7zaHL59CrwDoEFP2PQeJO1zbC3K7jQIlHKmavWtkdMa9YJvnoRvnoKcLESEx25vzOsj27L9WDKD3vmZw0n2GW/5KmMg7mdYNNa6QnrDu1ZdE3+wAmvwDPD0sw50l5cmK1UmNAiUcjafQBi1ELo8DJtnwEdDrnZRMbBtOPMnd+ZCejaD3lnH+2sOc66s9w6yM2D7fHivO8zuD3Froetj8PhOGPYB1O1k9bPkXwN6/B8c+hH2Ly3bGpRT6QVlSpUn2z6Crx6H4HowehGENAbg2NlL/H7xdmKOnsfL3Y3eLWoyqlM9ujSojptbCbunSD0NMbOskdkuJkGNSOsK6FbDwcs3/2WyM+Hdm61rIh7aaJ0aqyoEvbJYqYrk2AZYOMY6Q2fYB1bzjM3ehAss3HScz7adIOVyFjdV92V4VF2GdahDaKBP0dZ/chtsmAaxn0BuFjTpa53S2qBH0cZqPrAc5g2BO/5u7TmoCkGDQKmK5vxRWDAKkvZAn39bg+Tk+ZBOz8rh+9gEFm4+xobD53B3E3pFhjKyU11ubRKK+5W9BGOsZqaU45C41+ry4th68PKHtmOs9VZvWPz65g2Ho+vgkS3WID6q3NMgUKoiykiDT6fAvm+ssQ/6vXRjU0x2JsePHmR1zFb279tDYEYCjbyTaROYRricwTPtJGTl6bYi+Cbr23+7MeATVPLazhyEd6Kh9Qi45+2Sr0c5TGFB4OHoYpRSReTtDyM+ghX/gLWvQtJ+CG8PKfG/3tJOUxfDmCvLeEIyVTl6rip7TAgS3IabmjWhcZNmeFSrB2Gtwc299LWFNILoB2Hdm9DxfgjvUPp1KqfRPQKlKoKdi+Grx8DkQlCdPLd61/4eGA6ePsSfv8TimHg+jjnOqZR0Qvy9GNKhDgNa1aZF7cCSH2DOK/0CvNkBqkbAxGVFO77gitJT4NQOSPjF2iNr2BO8/BxehjYNKVUZ5GSBm0exPnBzcg2r9yexYNMxftybSE6uITTAm9siQ7ktMpRbGofg61WKhoFtH8EXU61rDFoPL/l6fkvCLzB3IPjVgMZ3QOPeUDe6/J21lHnJqvXkVuug/ImtcPbAtfO4e1udDzbtC036QVC4Q0rTIFBKce5iJqv2JfLj3kRW70siNSMbLw83ujSozu3NQukZGUqdqgWcNlqQ3Fx4/zZITYCHY6zmrDIv/DDM7GOFYI0m1kVvuVngFQANe1ih0OgOCKxV9tsuTHYmJO769QP/5DZI3GOdWgsQUAtqt4Pa7SG8HdRsZR383/c97P8OzsdZ84W1sgKhaV+o1e7q4EVlTYNAKXWNrJxcNh85x497E/lxz2nizloHlCPDArgtMpRezUJpW7fqr2cfFeb4Jph5B3R7Cnr9pWwLTU2Amb0hIxXu/x5qNLUOoh/5CQ4sgwM/wIUT1rxhraxQaNwbwqOsDv7KwtUzr+Kv/bafEGuNQAdQpartA7/9rx/+hQWTMXBmvzW86f7v4fhGq9nPvyY06WMFQ4MeBV/PUQIaBEqpQh1OSmPF3kR+3JPI5rhzZOcaqvp60rNpKLc1C6V7kxqFj572yWTY/QVM3Wh1m1EWLp+HD/pbp9Le+xXUyeeAtDGQuNsKhAM/WKfGmhzwCYaGt9n2Fm63roouSHoKpJyACyfhQrztvu125X7eM6+8AqB2W9vN9uEffFPpjpFcPAsHf7CC4eCPkJkKHj5Q/1ZbE1JfCKxd8vWjQaCUKoaUy1msOZDEij2JrNyXyPlLWXi4CZ3qV+Ph2xpxc8OQGxe6cNI6cNyol3WmU2llXrQ64ju5DUYvtg6wFsXlZDi8ygqFgz9YQ4Qi1rf0Rr2s5qWU+Dwf8ietD91riNURYGC41X4fGP7r/dDm1pCjdmq+Aawmp6M/W3sK+76D5KPW47XaQPRUaDOiRKvVIFBKlUhOrmH78fP8uCeRL3ecJP78Ze67OYJn+ja98SDz6pdgxQsw/ktocGvJN5qdCQtHwaEVMGw2NB9YsvXk5kLCTtvewjI4EfNr80tguPUN+8qZVnnvB4SBezkZO9oYSNr7axNSyyHWRYAloEGglCq1y5k5/HfpXj74OY6I6r68MrwNHW6q9usMWenWuAVefvDAmpK10efmwqeTIXYJ3PW6dSFdWclIA3ev8nemUXEYU+ImKB28XilValW83Hn+rhYsmBxNdq5h6LT1/PvbPaRn2c6S8fSBPv+02uy3fFD8DRgD3z1jhcDtfy3bEADrjKaKHAJgt2s1NAiUUsXSpWF1vn+8O6M61eO91Ye568217IxPtiZGDrDOkV/xAlw6V7wVr3rR6ob75keg6+NlXbYqhAaBUqrY/L09+NegVsy5vxOptrESXl22j8wcA33/AxkXYOW/ir7Cje/BTy9C27Fwxz/0KmUH0yBQSpXYrU1qsPSJ7tzTNpw3Vhzknrd/Zq+pA1ETrXEOTu/67ZXsXGw1CUUOsI4LaAg4nAaBUqpUgqp48srwNkwf14HE1HTuenMt73uOwvgEwffPFj6s5f5l8PnvIKIbDJlZdheBqWLRIFBKlYneLcJY9sSt9G4RxgsrEpjuPhKOrIa9X+e/wNH1sHgc1GwBI+dbB5uVU2gQKKXKTDU/L94e3Z43R7Vj+sVb2WfqcuGL/yM38/K1MybEwvwR1rn7Yz6xxm1WTqP7YUqpMndXm9p0blCNufMe46mEp1jwxrPcPOFf3FTdz+pE7qPBGC8/UoZ+TEqGL6kpKaSmZ5OWkU1qepbtZ7btsSxS07O5nJnD1J6NaFM32NlPr9LRC8qUUnZjjOHEe0OplrCGfjmvERLgw+uXnsXPXGJo5nMcMoV3wezpLgT4eOLv7cHZtAwahvrzxdSuiB5QLjYdoUwp5RQiQp0Rr2De6sTbgQuomh5PiNsFFkS+xaCQNvh7e1gf9D4eBOS97+OBv7cH3h5uVz/0F8cc55klO/k+NoF+rRzc5XQlp0GglLKvqhHIzY/Qcs3LVhcPYxZzX1E7kctjcLtwpq8+zMvL9nFH85p4uOshzrKir6RSyv66/R6a3Q3D5hS9J9HreLi78VTvJhxKusinW0+UcYGuzW5BICKzRCRRRGJ/Y76OIpItIkPtVYtSysm8/GDEhxDZv1Sr6dMijDZ1gnht+f5f+zhSpWbPPYLZQN/CZhARd+A/wDI71qGUqiREhGf6RnIyJZ15G485u5xKw25BYIxZDfxWr1OPAJ8AifaqQylVuXRtFELXRtV5e+VB0jKynV1OpeC0YwQiEg4MAt4twrxTRCRGRGKSkpLsX5xSqlx7pk8k5y5m8v6aw84upVJw5sHi14D/M8bk/taMxpjpxpgoY0xUjRqFjD2qlHIJbeoG07dFGDNWH+ZsWoazy6nwnBkEUcBCEYkDhgLviMg9TqxHKVWBPNWnCZezcnhn1SFnl1LhOS0IjDH1jTERxpgIYAnwkDHmc2fVo5SqWBqFBjCkfR0+XH+UE8mXf3sBVSB7nj66AFgPNBWReBGZKCIPisiD9tqmUsq1PH5HEwBeX77fyZVUbHa7stgYM6oY895nrzqUUpVXeHAVxkbfxOx1R5jSvSGNQv2dXVKFpFcWK6UqtKk9G1LF051Xlu1zdikVlgaBUqpCq+7vzaRuDfguNoEdx5OdXU6FpEGglKrwJnWrT1VfT15aqnsFJaFBoJSq8AJ8PJnasxFrD57h54NnnF1OhaNBoJSqFMZG30TtIB/+u3QfFW3ALWfTIFBKVQo+nu48fnsTdhxPZumu084up0LRIFBKVRqD24fTsIYfLy/bR06u7hUUlQaBUqrSsAavacrBxDQ+3Rrv7HIqDA0CpVSl0rdlGK3rBPHa8gNkZOvgNUWhQaCUqlREhKf7NOVE8mXmbdDBa4pCg0ApVenc0iiEmxvq4DVFpUGglKp0ruwVnL2Yycw1R5xdTrmnQaCUqpTa1atKnxY1mbHmMOcuZjq7nHJNg0ApVWk91bsplzKzeXfVQWeXUq5pECilKq3GNQMY3L4Oc9Yf5WQFH7zGnldLaxAopSq1x29vDAbe+PGAs0spsfSsHO6fvZlvdp6yy/o1CJRSlVqdqr6M7lyPxTHH+XrnSRJS0itUX0SXM3OYNCeGVfuTuGinM6DsNkKZUkqVFw/f1oivd57k4fnbAKjm50WzWgE0rxVI89qBNK8VRIMafni6l6/vxpczc5g0dzPrDp3lv0NaMyyqrl22o0GglKr0Qvy9+enpnuw+dYHdJ223UxeYs/4omdm5AHh5uNGkpr8VDrUCaV47iMhaAQT6eDql5kuZ2UycHcOGI2d5eWgbhnSoY7dtaRAopVyCn7cHHSOq0TGi2tXHsnNyOXzm4tVg2H3yAsv3JLI45td+iupV87XtPQQxpEM4dar62r3WS5nZ3D97M5uOnOPV4W0Y1M5+IQAgFamtDCAqKsrExMQ4uwylVCVljCExNeOacNh96gJxZy/i7+3Bvwa14q42te22/YsZ2UyYvZmYuHO8Orwt97QLL5P1isgWY0xUftN0j0AppfIQEWoG+lAz0IeekaFXHz929hKPLdrGIwu2seZAEn+9uwW+XmX7EXoxI5sJH2wm5ug5/jeiLQPblk0I/JbydWREKaXKqXrVfVn8QBce7tmIj7fEM+CNtcSeSCmz9adlZHPvrE1sOXae10e2c1gIgAaBUkoVmae7G0/1acq8SZ25mJnNoHd+5v01h8kt5SA4qelZ3DtrE9uOJ/PGyHZ2bXrKjwaBUkoV080NQ/j+se70aBrKC9/sYcLszSSlZpRoXRfSsxg/axM7jifz1qh23Nm6VhlX+9s0CJRSqgSq+nkxfVwH/jGwBesPn6Xf62v4aX9SsdZxIT2L8TM38Ut8Cm+Nbke/Vo4PAdAgUEqpEhMRxnWJ4MuHu1LNz5N7Z23in9/svnptQmFSLmcxbuYmdp1M4e0x7enb0jkhABoESilVapFhgXz58C2Mja7HjDVHGPzuzxxOSitw/pRLWYybuZHdJ1N4Z0wH+rQIc2C1N9IgUEqpMuDj6c4L97TivXEdiD9/mQFvruXjmOM39GuUcimLsTM3svdUKtPGduCO5jWdVPGvNAiUUqoM9WkRxnePdaNVeBBPL9nJowu3cyE9C4DkS5mMmbmBfQmpTBvXnl7NnB8CoEGglFJlrlZQFeZPjuap3k349pdT9H99DSv3JjJ6xkb2n07jvfEduC2yfIQAaBAopZRduLsJD9/WmMUPdAFgwuzNHExKY8b4KHo2Df2NpR1Lu5hQSik76nBTVb59rBtvrzhIj6ahdGlY3dkl3cBuewQiMktEEkUktoDpY0Rkp4j8IiLrRKSNvWpRSilnCvTx5A/9m5XLEAD7Ng3NBvoWMv0IcKsxphXwD2C6HWtRSilVALs1DRljVotIRCHT1+X5dQNg3w63lVJK5au8HCyeCHzn7CKUUsoVOf1gsYj0xAqCWwqZZwowBaBevXoOqkwppVyDU/cIRKQ18D4w0BhztqD5jDHTjTFRxpioGjVqOK5ApZRyAU4LAhGpB3wKjDPG7HdWHUop5ers1jQkIguAHkCIiMQDzwOeAMaYacBzQHXgHREByC5oPE2llFL2Y8+zhkb9xvRJwCR7bV8ppVTRyPU945V3IpIEHC3h4iHAmTIsp6yV9/qg/Neo9ZWO1lc65bm+m4wx+R5krXBBUBoiElOem5/Ke31Q/mvU+kpH6yud8l5fQcrLdQRKKaWcRINAKaVcnKsFQXnvz6i81wflv0atr3S0vtIp7/Xly6WOESillLqRq+0RKKWUuo4GgVJKubhKGQQi0ldE9onIQRF5Np/p3iKyyDZ9Y2HdZduhtroislJEdovILhF5LJ95eohIiohst92ec1R9tu3H2QYM2i4iMflMFxF5w/b67RSR9g6srWme12W7iFwQkcevm8fhr19+AzGJSDUR+UFEDth+Vi1g2Xtt8xwQkXsdWN9LIrLX9jf8TESCC1i20PeDHev7q4icyPN37F/AsoX+v9uxvkV5aosTke0FLGv316/UjDGV6ga4A4eABoAXsANoft08DwHTbPdHAoscWF8toL3tfgCwP5/6egBfO/E1jANCCpneH6vbcAGigY1O/FsnYF0o49TXD+gOtAdi8zz2X+BZ2/1ngf/ks1w14LDtZ1Xb/aoOqq834GG7/5/86ivK+8GO9f0VeKoI74FC/9/tVd91018BnnPW61faW2XcI+gEHDTGHDbGZAILgYHXzTMQmGO7vwToJbYOj+zNGHPKGLPVdj8V2AOEO2LbZWggMNdYNgDBIlLLCXX0Ag4ZY0p6pXmZMcasBs5d93De99kc4J58Fu0D/GCMOWeMOQ/8QOEj+5VZfcaYZcaYbNuvTh0cqoDXryiK8v9eaoXVZ/vsGA4sKOvtOkplDIJw4Hie3+O58YP26jy2f4QUrA7wHMrWJNUO2JjP5C4iskNEvhORFo6tDAMsE5EttrEgrleU19gRRlLwP58zX78rahpjTtnuJwA185mnvLyW91Pw4FC/9X6wp4dtTVezCmhaKw+vXzfgtDHmQAHTnfn6FUllDIIKQUT8gU+Ax40xF66bvBWruaMN8CbwuYPLu8UY0x7oB0wVke4O3v5vEhEv4G7g43wmO/v1u4Gx2gjK5bnaIvInIBuYV8Aszno/vAs0BNoCp7CaX8qjURS+N1Du/58qYxCcAOrm+b2O7bF85xERDyAIKHBgnLImIp5YITDPGPPp9dONMReMMWm2+98CniIS4qj6jDEnbD8Tgc+wdr/zKsprbG/9gK3GmNPXT3D265fH6StNZrafifnM49TXUkTuAwYAY2xhdYMivB/swhhz2hiTY4zJBWYUsF1nv34ewGBgUUHzOOv1K47KGASbgcYiUt/2rXEk8OV183wJXDk7YyiwoqB/grJma0+cCewxxrxawDxhV45ZiEgnrL+TQ4JKRPxEJODKfawDirHXzfYlMN529lA0kJKnCcRRCvwW5szX7zp532f3Al/kM89SoLeIVLU1ffS2PWZ3ItIXeAa42xhzqYB5ivJ+sFd9eY87DSpgu0X5f7en24G9xpj4/CY68/UrFmcfrbbHDeuslv1YZxP8yfbY37He8AA+WE0KB4FNQAMH1nYLVhPBTmC77dYfeBB40DbPw8AurDMgNgA3O7C+Brbt7rDVcOX1y1ufAG/bXt9fgCgH/339sD7Yg/I85tTXDyuUTgFZWO3UE7GOO/0IHACWA9Vs80YB7+dZ9n7be/EgMMGB9R3Eal+/8j68ciZdbeDbwt4PDqrvQ9v7ayfWh3ut6+uz/X7D/7sj6rM9PvvK+y7PvA5//Up70y4mlFLKxVXGpiGllFLFoEGglFIuToNAKaVcnAaBUkq5OA0CpZRycRoESjmQrWfUr51dh1J5aRAopZSL0yBQKh8iMlZENtn6kH9PRNxFJE1E/ifWOBI/ikgN27xtRWRDnn79q9oebyQiy22d320VkYa21fuLyBLbWADzHNXzrVIF0SBQ6joi0gwYAXQ1xrQFcoAxWFc0xxhjWgA/Ac/bFpkL/J8xpjXWlbBXHp8HvG2szu9uxroyFaweZx8HmmNdedrVzk9JqUJ5OLsApcqhXkAHYLPty3oVrA7jcvm1c7GPgE9FJAgINsb8ZHt8DvCxrX+ZcGPMZwDGmHQA2/o2GVvfNLZRrSKAtXZ/VkoVQINAqRsJMMcY84drHhT5y3XzlbR/low893PQ/0PlZNo0pNSNfgSGikgoXB17+Cas/5ehtnlGA2uNMSnAeRHpZnt8HPCTsUafixeRe2zr8BYRX0c+CaWKSr+JKHUdY8xuEfkz1qhSblg9Tk4FLgKdbNMSsY4jgNXF9DTbB/1hYILt8XHAeyLyd9s6hjnwaShVZNr7qFJFJCJpxhh/Z9ehVFnTpiGllHJxukeglFIuTvcIlFLKxWkQKKWUi9MgUEopF6dBoJRSLk6DQCmlXNz/A6Z4jNfEAV5eAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.plot(history.history['accuracy'])\n",
"plt.plot(history.history['val_accuracy'])\n",
"plt.title('model accuracy')\n",
"plt.ylabel('accuracy')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='lower right')\n",
"plt.show()\n",
"plt.plot(history.history['loss'])\n",
"plt.plot(history.history['val_loss'])\n",
"plt.title('model loss')\n",
"plt.ylabel('loss')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='upper right')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Tensorboard"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load the TensorBoard notebook extension on google colab\n",
"%load_ext tensorboard\n",
"\n",
"os.makedirs(logdir, exist_ok=True)\n",
"%tensorboard --logdir logs"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Y8oAT4oUGhJs"
},
"source": [
"# Part II : Transfer Learning\n",
"\n",
"\n",
"Having to train an image-classification model using very little data is a common situation,\n",
"which you’ll likely encounter in practice if you ever do computer vision in a\n",
"professional context. A “few” samples can mean anywhere from a few hundred to a\n",
"few tens of thousands of images. As a practical example, we’ll focus on classifying\n",
"560 images belongig to 8 actors. We’ll use 480 pictures for training, and 80 for validation.\n",
"\n",
"\n",
"## 2.1 Feature Extraction with a Pretrained Model\n",
"\n",
"Feature extraction consists of using the representations learned by a previously\n",
"trained model to extract interesting features from new samples. These features are\n",
"then run through a new classifier, which is trained from scratch.\n",
"\n",
"\n",
"As you saw previously, ConvNets used for image classification comprise two parts:\n",
"they start with a series of pooling and convolution layers, and they end with a densely\n",
"connected classifier. The first part is called the _convolutional base_ of the model. In the\n",
"case of convnets, feature extraction consists of taking the convolutional base of a previously\n",
"trained network, running the new data through it, and training a new classifier\n",
"on top of the output.\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {},
"outputs": [],
"source": [
"# General imports\n",
"import tensorflow as tf\n",
"tf.compat.v1.enable_eager_execution(\n",
" config=None, device_policy=None, execution_mode=None\n",
")\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import os, datetime\n",
"\n",
"# Shortcuts to keras if (however from tensorflow)\n",
"from tensorflow import keras\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"from tensorflow.keras.models import Sequential\n",
"from tensorflow.keras.layers import Conv2D, MaxPooling2D\n",
"from tensorflow.keras.layers import Activation, Dropout, Flatten, Dense\n",
"from tensorflow.keras.callbacks import TensorBoard \n",
"from tensorflow.keras import layers"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA/kAAAJjCAIAAADYrUbKAAAABGdBTUEAALGPC/xhBQAACjFpQ0NQSUNDIFByb2ZpbGUAAEiJnZZ3VFPZFofPvTe9UJIQipTQa2hSAkgNvUiRLioxCRBKwJAAIjZEVHBEUZGmCDIo4ICjQ5GxIoqFAVGx6wQZRNRxcBQblklkrRnfvHnvzZvfH/d+a5+9z91n733WugCQ/IMFwkxYCYAMoVgU4efFiI2LZ2AHAQzwAANsAOBws7NCFvhGApkCfNiMbJkT+Be9ug4g+fsq0z+MwQD/n5S5WSIxAFCYjOfy+NlcGRfJOD1XnCW3T8mYtjRNzjBKziJZgjJWk3PyLFt89pllDznzMoQ8GctzzuJl8OTcJ+ONORK+jJFgGRfnCPi5Mr4mY4N0SYZAxm/ksRl8TjYAKJLcLuZzU2RsLWOSKDKCLeN5AOBIyV/w0i9YzM8Tyw/FzsxaLhIkp4gZJlxTho2TE4vhz89N54vFzDAON40j4jHYmRlZHOFyAGbP/FkUeW0ZsiI72Dg5ODBtLW2+KNR/Xfybkvd2ll6Ef+4ZRB/4w/ZXfpkNALCmZbXZ+odtaRUAXesBULv9h81gLwCKsr51Dn1xHrp8XlLE4ixnK6vc3FxLAZ9rKS/o7/qfDn9DX3zPUr7d7+VhePOTOJJ0MUNeN25meqZExMjO4nD5DOafh/gfB/51HhYR/CS+iC+URUTLpkwgTJa1W8gTiAWZQoZA+J+a+A/D/qTZuZaJ2vgR0JZYAqUhGkB+HgAoKhEgCXtkK9DvfQvGRwP5zYvRmZid+8+C/n1XuEz+yBYkf45jR0QyuBJRzuya/FoCNCAARUAD6kAb6AMTwAS2wBG4AA/gAwJBKIgEcWAx4IIUkAFEIBcUgLWgGJSCrWAnqAZ1oBE0gzZwGHSBY+A0OAcugctgBNwBUjAOnoAp8ArMQBCEhcgQFVKHdCBDyByyhViQG+QDBUMRUByUCCVDQkgCFUDroFKoHKqG6qFm6FvoKHQaugANQ7egUWgS+hV6ByMwCabBWrARbAWzYE84CI6EF8HJ8DI4Hy6Ct8CVcAN8EO6ET8OX4BFYCj+BpxGAEBE6ooswERbCRkKReCQJESGrkBKkAmlA2pAepB+5ikiRp8hbFAZFRTFQTJQLyh8VheKilqFWoTajqlEHUJ2oPtRV1ChqCvURTUZros3RzugAdCw6GZ2LLkZXoJvQHeiz6BH0OPoVBoOhY4wxjhh/TBwmFbMCsxmzG9OOOYUZxoxhprFYrDrWHOuKDcVysGJsMbYKexB7EnsFO459gyPidHC2OF9cPE6IK8RV4FpwJ3BXcBO4GbwS3hDvjA/F8/DL8WX4RnwPfgg/jp8hKBOMCa6ESEIqYS2hktBGOEu4S3hBJBL1iE7EcKKAuIZYSTxEPE8cJb4lUUhmJDYpgSQhbSHtJ50i3SK9IJPJRmQPcjxZTN5CbiafId8nv1GgKlgqBCjwFFYr1Ch0KlxReKaIVzRU9FRcrJivWKF4RHFI8akSXslIia3EUVqlVKN0VOmG0rQyVdlGOVQ5Q3mzcovyBeVHFCzFiOJD4VGKKPsoZyhjVISqT2VTudR11EbqWeo4DUMzpgXQUmmltG9og7QpFYqKnUq0Sp5KjcpxFSkdoRvRA+jp9DL6Yfp1+jtVLVVPVb7qJtU21Suqr9XmqHmo8dVK1NrVRtTeqTPUfdTT1Lepd6nf00BpmGmEa+Rq7NE4q/F0Dm2OyxzunJI5h+fc1oQ1zTQjNFdo7tMc0JzW0tby08rSqtI6o/VUm67toZ2qvUP7hPakDlXHTUegs0PnpM5jhgrDk5HOqGT0MaZ0NXX9dSW69bqDujN6xnpReoV67Xr39An6LP0k/R36vfpTBjoGIQYFBq0Gtw3xhizDFMNdhv2Gr42MjWKMNhh1GT0yVjMOMM43bjW+a0I2cTdZZtJgcs0UY8oyTTPdbXrZDDazN0sxqzEbMofNHcwF5rvNhy3QFk4WQosGixtMEtOTmcNsZY5a0i2DLQstuyyfWRlYxVtts+q3+mhtb51u3Wh9x4ZiE2hTaNNj86utmS3Xtsb22lzyXN+5q+d2z31uZ27Ht9tjd9Oeah9iv8G+1/6Dg6ODyKHNYdLRwDHRsdbxBovGCmNtZp13Qjt5Oa12Oub01tnBWex82PkXF6ZLmkuLy6N5xvP48xrnjbnquXJc612lbgy3RLe9blJ3XXeOe4P7Aw99D55Hk8eEp6lnqudBz2de1l4irw6v12xn9kr2KW/E28+7xHvQh+IT5VPtc99XzzfZt9V3ys/eb4XfKX+0f5D/Nv8bAVoB3IDmgKlAx8CVgX1BpKAFQdVBD4LNgkXBPSFwSGDI9pC78w3nC+d3hYLQgNDtoffCjMOWhX0fjgkPC68JfxhhE1EQ0b+AumDJgpYFryK9Issi70SZREmieqMVoxOim6Nfx3jHlMdIY61iV8ZeitOIE8R1x2Pjo+Ob4qcX+izcuXA8wT6hOOH6IuNFeYsuLNZYnL74+BLFJZwlRxLRiTGJLYnvOaGcBs700oCltUunuGzuLu4TngdvB2+S78ov508kuSaVJz1Kdk3enjyZ4p5SkfJUwBZUC56n+qfWpb5OC03bn/YpPSa9PQOXkZhxVEgRpgn7MrUz8zKHs8yzirOky5yX7Vw2JQoSNWVD2Yuyu8U02c/UgMREsl4ymuOWU5PzJjc690iecp4wb2C52fJNyyfyffO/XoFawV3RW6BbsLZgdKXnyvpV0Kqlq3pX668uWj2+xm/NgbWEtWlrfyi0LiwvfLkuZl1PkVbRmqKx9X7rW4sVikXFNza4bKjbiNoo2Di4ae6mqk0fS3glF0utSytK32/mbr74lc1XlV992pK0ZbDMoWzPVsxW4dbr29y3HShXLs8vH9sesr1zB2NHyY6XO5fsvFBhV1G3i7BLsktaGVzZXWVQtbXqfXVK9UiNV017rWbtptrXu3m7r+zx2NNWp1VXWvdur2DvzXq/+s4Go4aKfZh9OfseNkY39n/N+rq5SaOptOnDfuF+6YGIA33Njs3NLZotZa1wq6R18mDCwcvfeH/T3cZsq2+nt5ceAockhx5/m/jt9cNBh3uPsI60fWf4XW0HtaOkE+pc3jnVldIl7Y7rHj4aeLS3x6Wn43vL7/cf0z1Wc1zleNkJwomiE59O5p+cPpV16unp5NNjvUt675yJPXOtL7xv8GzQ2fPnfM+d6ffsP3ne9fyxC84Xjl5kXey65HCpc8B+oOMH+x86Bh0GO4cch7ovO13uGZ43fOKK+5XTV72vnrsWcO3SyPyR4etR12/eSLghvcm7+ehW+q3nt3Nuz9xZcxd9t+Se0r2K+5r3G340/bFd6iA9Puo9OvBgwYM7Y9yxJz9l//R+vOgh+WHFhM5E8yPbR8cmfScvP174ePxJ1pOZp8U/K/9c+8zk2Xe/ePwyMBU7Nf5c9PzTr5tfqL/Y/9LuZe902PT9VxmvZl6XvFF/c+At623/u5h3EzO577HvKz+Yfuj5GPTx7qeMT59+A/eE8/vsbQFrAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAAJcEhZcwAAHsMAAB7DAbyXl9oAAB++SURBVHic7d1bsptKmgZQIDwEnxF0dM9/PFVRIzgeg9QPKssc7gKUwMda4YdtKblImz/1kUrY9fP5rAAAgDjN0TsAAAB8hawPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMP47eAbaq63ri2efz+b2Nvlfe+e/CNWxZHI6i4qAkFQcbyfoh2p3IdM94BuffQ5im4qAkFQeryfqB2mMJ7aGFAls8ZHE4loqDklQcfMR8fQAAyGRc/y7e8wXf3y12phK29Qchpr+RHJyMOLja9oOdXVqy+MTLGdtzOISKg5JUHIwxrn8v/c6l3S2O9YzvTqfdZuGGOqtt/3dibUv2qrNjY23gQCoOSlJx0CfrxxrrEdrdWX/8Y2y0o/34bFe4ZLWzi3d6zLHG6zYBu1NxUJKKg4XM4Qkx0evNLttp83w+d7ngqb/ajWvr75W+j6OoOChJxcFqsn6swT5i8MGY7wSfvamNUIyKg5JUHCwk64cofPavx+HmVByUpOJgNVkf3xJCUSoOSlJx3JxrcxkwO56xbsDjNRlx1R5BMhUHJak4bkXWZ/juY1s6rM4NENqrXT6+Mrg4BFBxUJKK4+Zk/Vvr37J37FZi7eGK5V3Su0sd7ATHhkCW7BVckYqDklQcVLI+7U5nrLvpt5ntj/p92Udd2JK9gitScVCSioOt95cFAADOybg+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1mcf/r43lKTioCQVx3XJ+uxAJwglqTgAFpL1AQBGObvm0mR9dqM3hJJUHJSk4rgoWR8AADLJ+mxlqANKUnFQkorj6mR9AIB5cj9XJOuzJ/0glKTiAJgm6wMAQCZZn00MK0JJKg5KUnEEkPUBLkwWgZJUHJcj67Mz/SAAwEnI+qwn1gMAnJmsD3ANzq6hJBVHBlmf/ekfoSQVByWpOK5F1gcAgEyyPisZ2ICSVBwAK8j6AAD/4OyaGPXz+Tx6H0hQ144lKEfFQUkqjusyrg8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQKYfR+8AEKWu66qqns/n0TsCm7yO5MrBDFycrA/s6fl81nUtJ3Fdjl4giawP7OydkIR+LuR9rFYOVyCIrA98Sz/0i1CcjYgPZKt1be2OHg4XXJJqjXMKLrpK3XEy2eV2Tsb1q6qqHo/H0bsAVVVVTZN8a6xXF99JHqqPwt5V9ng8mqa5Q/L4+9evo3cBqqqq/vr58+hduCNZHyjtne+bpmkHr+P2iHztE2kHG3Afsj5wmH7oF8LYl4gP3JysDxzvFcIM87MXER/gRdYHzsLcHrZz5AC0yfrA6Qj9fMpAPsAgWR84LxP6mSbiA0yT9YELMKGfNhEfYCFZH7gMc3vwewf4iKwPXI/QfzcG8gHWkfWBCzOhP5uID7CRrA8kMKE/iYgPsBdZH8hhbs+lifgAu5P1gUBC/7X4HQF8iawPJDOh/8wM5AN8m6wP3IIJ/ech4gMUI+sDN2Juz4FEfIDymvkmAHEev1W/c387ibKv9gSq99sOzPrr58+jd4HLM66fbGF22fK5W2wCtJnWfMmOI/0qbnAlL4r35l6Z9e9fv1Y8e4jyu9Tf4uuRdtw/1VvEVcj6yfofrhIzjNl+Fa+KexHxGfPXz5/S6kKd6G+An9VkfTbxQU6eM1/Fe5496RPxWULcHzP9tnjTWE3WBxjgKt7lvD8ApyXr82eaweAHdn8Kcv/Z9zhoZz399kvWOdYGDvGN0D9RdBeqOAP5fOrvX7/++vlz4dB+f9ZKZ0JLZyVj890ntjWxiRWNNzZ47227WefBjSv05cA9iVP81+An/eCtM6YzQXuRwfbtcDPdxi07OJX+rXs2rrC/hktUXPu2RYqUdWZnn7fjaX/Oej+zrpjOPr2JwcavloONZ9e2fHPtBu2f1+2/if7I+vwxmDDan+ILP9HfzSbGDmfbrNgulLHjzTo75w/ViStOxGcXH43otyPvRLMVm1u4iXbjdoOxxrNrW7K5JT7a/84pCncj6zNq8LN89tN9ycd/p0076MBV9If5Nx7Dp604EZ99vULnbFLvZNPZpToNVpwJjAXiwcdno/NsvN6evz99i7gn8/X5ryVf3K9b7dhEBcjQ/irsown9023OUHEuuuWrpifunzazjk28eV2K8P7vpw322pP+drdviEsTuZjStLweOeojX9Tg/LZP6D9DxXW2biCfb9geQNtj2AXOCl4xfTbN99t/1AC+wbg+oxbelmcF0YFsnUkyTdMsOeaPrbj+TXV8/8ZXzd6TZ8X5wPueM/v+1dvBfD82ut95tvMCZxt8xJg9S+jKmVEml8+mCrGDy+lE9oUj/QdWXP9CYShg+dj27GT9T1c4uImxxWeDdXvBwbn4sw2280UBfXpzDjB4vz/ZglTb5/ZsNFZxZ5iex51Nh91ObO1P19lyKergslu+DVh+F9Edzb5FUMn6zGrPQ2j/vG5tnRt6VkPzFvqRyGkAGQZva9tRrOKq8ZMQFUcxY/e9ef3Qv6nO8iC+sOU7Hy//w1vtsf9+2p5e4WyDhXZ5i7gJHTqj+ncX2eUqvSU3LRnc9Mbtwgm1B9cLV9zgDfVVHCfRzrIfpdgVJwOz6+/vTH8GzuAkorFb8u+Sy1e/RdxN/Xw+j96Hg9V17VONk2iaJr4kVVxf/7rY8685w00qTv7jJP76+TO+4k7IfXgADrb6Dv0TTMQHoJL1Ac5j+1waA/kAtMn6AKfTv152+R/ZFfEBeJP1AU5qdm6PiA/ANFkf4OwGQ3/nKQDoc89NgAtzL3wAJhjXBzi7ibk67ogPwARZH+CklkzH//QqXgBuRdYHOJ1Pg/s37tAPQABZH+Astt9XR+gHoM1FXRzDBYXw1vxWVdXjt43rbK+nf/ceGPPXz59H7wLX8NfPn46WS9D730v5j/zBLb4eaVpK7hIUM314fyPiT9yUU8Ux69joViw7btzQisUHF2k/8uk6O+3LvHWdrfz969e3t8gu9PiUtm+ygSvaPeJPb6u/IYmfMQJcGTEj4jEvJJj5+nzXbIgR97mPAn/mdnnFmdBPh9C20IrTodlFNp5iHXWG9vevX6/BfqeIZ2ZcB+C7vjFXZzsT+umT2Dqc/xDAuH6g/mf2dLCYbb+lQfuv/LSbvR8f/DNA01vsLNtfHAobPGLHRvHPVnHvB6d3WMVlG5tN/hq4fT3yPhPoN26fJPQX7LcZW8+SHVu+rSWTy/uvrrPmsUU678b0620v0pln3162v8jYCpfs0qCFv8TZ9bSXckZ0cgZy0rQ/jDsf/xPt+3N5l69w+RbbDSaGNheu0DAkJzF7xA7m5vNXXGedKu6eJs4BXv+WNBtsM7iesa2v2NbC+LvXVxmzr7e9xXebsde+8E0e815w7O1d8sYu+QX1V8gJ6buj9Efdlozod0b4BlvOrnDhFmd99BLOMx2C22ofsZ183D8yL1dx/Vek4m6onfb6w88TKbCdaDtPLVlPPz1/tK33gsUmJi3ZzyU+epOXr/C9nv5XGet+QVyFrB+o80k88dk8+NTsB/nsh/32NNB/CZVhRc6tM2A/WALXqrhd1swlTMxX6Rgc4h1c8EvRsD8NZvlSy9e5/A1ZuImPLH+TP13t7H8Nz0cyX58/xqYBTN+vY7bBXnvS3+72DcFq22+qc62Kq4a+BCDedMT8aFbJ93LkCYecd3y9G9czdt4i1t+HrH93s9cUdh4Z/LCfbQAxOhH/06+brltxg1P5lfkNDV7TuUtwPNVVnseeP+z1Jq/4gmLCqX5BLCfr39rgB/bYWGPn2aZpBqcdjzX4iADBCW0PuBkVJ/Tf2eBdawLyX/s+8Ye/nL3e5Omgf8IvQ/gSE6BvYfrm2bMf0v2L8z5tsJ3J+hyl+a365PCeKLqYimtfmTDdyXByn8a+L8XEw0P2qex+ge/CRbY04Jx0zVEGr2Ht3/rjI7Of39/4gB98CZIEJfUj/tjlttWuRXfFiuskfqV6UedMcoM3gvyGpHvOrPjDBa//nvMYYCM9cqb3R/WSzNGeA9CfBNx5amydsw0WGpyZ0HkKvmdhxB9csFp2/KdWXH+YX+hP9U6E79vYVx/GxPddXyYW79/3fezvXu2ocNhtvwNje/Lpm9y5oVDnX7XsjV3yC+IqdMRp+p/cEx/Y/Xm3/WQzOLF4cN7wwo3OMhuYQ6yL+NUnRXeTihP6gw3+sap1yXvi715NtBlsFmbHN3n5JqreG7vkF9RvzAnVz+fz6H04WF3XciQn0TRNfEmereLaMfRUO5bktG/yTSpuYQ57XZn67f05s33vWnMTH71pf/38GV9xJ2SsBbij1XN1WMFVvFdhngYrODs6OR0ucCMi/rHM7Tmzmyc2g/orODm8BPfXB27BtR/n4ZqcM7vhTB6BdYu7HS1XJOsDyU47U5xK6D+fe+a2e77q7W54WnhRsj4QSMS/lsE7FB26R8AMQf8qZH0gh4h/df3rd/0eAbaQ9YEEomESc3sA9iLrAxdmID+b0A+wkawPXI+Ifzcm9AOsI+sDlyHiY0I/wEdkfeACBDvazO0BWEjWB87LQD7ThH6AabI+cDoiPp8yoR9gkKwPnIWIz3Ym9AO0yfrAwUR8dmduD8CLrA8cRgjj24R+4OZkfaA0A/mUZ0I/cE/18/k8eh8OVtf10bsAfwSXpFrjnIKLrlJ3nEx2uZ2TrM8+6tqxxLB21HCQ7EXF7cXxyRIqjusyh4cdGDdi0PvA8Bm5LxW3o/fBWde1I5ZBKo5Lk/WBnRko5YqEfqYZ2ueimvkmsIyRD6rfh8Hzt6N3J5mK+5L20etNBq7OuD6wJ/meGA5mIIBxfbYy7gUlqTgoScVxdbI+wFVJIVCSiuOKZH32pB8EADgPWZ9NhHsAgNOS9QEuw9k1lKTiCCDrszM9I5Sk4qAkFcflyPoAAJBJ1mc9wxtQkooD4FOyPgBAl7NrMsj67E//CCWpOChJxXEtsj4AAGSS9VnJwAaUpOKgJBVHDFkfAAAy1c/n8+h9IEFdO5agHBUHJak4rsu4PgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAECm+vl8Hr0PB6vr+uhdgD/iS1LFcSoqDkqKr7gT+nH0DpzC4/E4ehcur2kab+N2TXOLr9ocKtupuF2oOBZScbu4ScWdjTedfegEoSQVByWpOK5L1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkOnH0TvAFzXNonO5x+OxcRNb1nC2DcFqKg5KUnGwhKyfrN9r6E3ge1QclKTiYAlZn010qVCSioOSVBwBzNcHAIBMxvWpqtb3nu/pj+/BjP6EyPY4R/sL0/5K+u2XrHOsDcRQcVDSLhU3uJ7+IrPrHGsDX+JQ449+1/Pu117Gmo0tMti+3dVOt+lsF8KoOChpl4qr5opOxXE2sj7/0O56+hc5LeyS3s0mRjJm26zYLlyOioOSdqm4arygVBwnJOszanC8YbZLWtJnddq0u124LRUHJa2ruCVtVBynYr4+f0z3X+v6qUdvXuPqVUEYFQclfaPiqqGiU3GciqzPlMELjA7pxY7aLpSk4qAkFccdyPqM6s87rHYarjA3EfpUHJSk4rgJJ5HMKNNnLbzvAcRTcVCSiiOeY4sDDE5t1NPBl6g4KEnFcSqOPGa8u6emado/r1tb/2bDY3co22VzcDkqDkpSccRzeDGq3Wf1/3TIjqutet+iDm5643bh5FQclKTiuIn6+XwevQ8Hq+tajXESTdPEl6SK4zxUHJR0h4o7IeP6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvVZr2mapllzCL2XWr2G7Q7cNKyzveK2rGQjFcflqDgyOA4oTe8DJak4KEnFcTY/jt4Bbu3xeBy9C3Avig5KUnEcztknAABkMq5/Uv0vAdtjA0uefTwenWbvNu8GgxtdvqHZxTsPdqYwjq1z46tbt/Pc3OzRMtHgQhX33skd+5N1O8/Nqbixjao4dmdc/4zeBfx4PDqd1+yznZUMtun3Sit2Y4X2eto/r9joxKvrN9hl5wk2e9QtOSxVXKXiWEbFzW5UxbEjh8XpdHqr6p9n6v36nziPX9JmbJGPNrSXvV5dfxylwM5zUdMVV31yWKq4fmPoUHEqjsJk/ZPq1O3YCEG78ez3fdOLLBkPWDhYsq91r67fQFfIhI8qrho6LFXc4HpgkIqb3qiKY0ey/l1k9wLTr675rdj+gIpTcZSk4lQcY1ybS1WNX8l0XYMXLekKOQkVByWpOO5M1r+vV7/QNE1S9/fSnw9aFf9aFjpUHJSk4uBF1r8MZfypvP6dkvIGAr/Ne8UWKu5T3isWEh9PZ/AynU4nOPjsxm0N9horNrT9nGSvVwdLLKm4sQZbtqXiuCcVp+IozBF2Xq/6f19w8+qn+nF/8Lu87VZsqLPI9KVCg8/u++raK+mvEDoGK64qVXQqjrtRcUs2OkHFsZBj4oz63cHgbXTb5b2xBxxcfMWGOns+exu1XTa6ZCUr9oT7mK646gtFp+K4MxX30UaXrETFMaF+Pp9H78PB6rpWIZxE0zTxJaniOA8VByXdoeJOyLg+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk61NVVdU0TdM4GKAQFQclqTjuzKEPAACZZH0AAMgk6wMAQKYfR+8A59KZ0fh4PKYb9NvMNljYBu5AxUFJKo4bMq7PH6/u6fF4vHulTofVbjDY5v3z7Eqm28AdqDgoScVxTw4+/uHdMY0NVLQfHxuomF3JdBu4DxUHJak4bkjW549Ol/T67+AQxdgig+ucXaqzIbgJFQclqTjuyXx9Vhrsth6PR/s2xmO9pC4PPqXioCQVRwxZnw8MXm/Uv9Tp/chshwhMUHFQkoojkqzPUoOd2tjIR+fZpmmWTIIE3lQclKTiSOVrJj4z24W1e8bBiYyzSwFvKg5KUnHkcfDxx9jdx1av4aMN6Qq5GxUHJak47smRxz/0pyFONBhr/H7q/UP/7mPmO0Kl4qAsFccN1c/n8+h9OFhd1yrw3Vv1L0LqN+s82+npBq9tGtzcdJt7apomviRVXKXiTkPF3YSKO4k7VNwJyfr6QU7kDv2giuM8VByUdIeKOyFzeAAAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQDg2ppmOM+MPX4qTdNcYj+5KMcWAHBhg0H5HaDPH6Mfj8fRu0CyH0fvAADAJp24/Mr318rQTdNca4e5irOf7AIAjDn/sP0Sr5Sf8Vo4G0cVAHBhhsNhgjk8AMAl9QfC24+8Z/J0fqhapwf9NbTPHCYG2j9aQ3vT/TbvR4zr8w2yPgAQoj0ZZnAS/+Aj7aXa8+b7ibyzktk1tJtNt3m39DUF+3IGCQDcwuO3qpW/J5J9x2Cyby84sYblW4F9yfoAwPV842Y7S66R7Wxx8L+zM3agGHN4AIB8GwP32KnFikn2puZTkkMNAGDKFW/YDy/G9QEARk0HfScAnJxxfQDgesqE7BUj+ubncCoORwDgqjYG67F7aC5f+eAi4j7n4VgEAG6n/8ewxm6p+fq5Y8ka4AxkfQDgjtphfV1M376GsRXCXlybCwBc0tjNK6dvgb/9qeXNll/Ra9oPX+LAAgAuLCYlG9TnG0LKAwC4oYx8HHO6wgk5tgCAawvIyhknLZyQ+foAwIVdPSU3TXP1l8CZXf48GADgugR9vkrWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEw/jt4B4ML+73//5/3zv/79nwP3BOIpN2AFWR/YhyACxSg3YCFZH9ifIALFKDdggqwPfJcgAsW8y02tAS+yPpxLOxnnyX51XFHqMZn6uoBP1c/n8+h9OFhd14/H4+i9gKqqqqZprlWSn+aJf/37PyqO87hWxa0ot8pnHGdyrYqLYVwf+DrTCaAMtQZ0yPrAV8gcUIxyA8bI+sBuBA4oRrkBS8j6wCYCBxSj3IBPuTbXdUucyB2uW1JxnIeKg5LuUHEn1By9AwAAwFfI+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyPTj6B04haZxzgPlqDgoScXBndXP5/PofQAAAPbnXJ991HV99C7Ajag4KEnFcV2yPjvQCUJJKg5KUnFcmqwPADBD4ueiZH12ox+EklQcALNkfQAAyCTrs5XBRShJxUFJKo6rk/UBrkoKgZJUHFck67Mn/SAAwHnI+mwi3AMAnJasD3AZzq6hJBVHAFmfnekZoSQVByWpOC5H1gcAgEyyPusZ3oCSVBwAn5L1AQC6nF2TQdZnf/pHKEnFQUkqjmuR9QEAIJOsz0oGNqAkFQclqThiyPoAAJCpfj6fR+8DCerasQTlqDgoScVxXcb1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATPXz+Tx6HwAAgP0Z1wcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdanqqqqruu6ro/dgQO3DuUdW3QqjrtRcdyWrM/xdIJQkoqDklQcx5L1AQAgk6wPAACZfhy9A5zR6wvH5/PZ+ebx+XyuazO48uqf32wONoY7UHFQ2GxBqThiGNdn1Ltj6vR9n7aZ0F6w/TPckIqDwmYLSsURQNZnSruTWt7GdUiwjoqDwmaLTsVxdbI+o5aMQHTaGLSA1VQcFDZbQSqOALI+S+njoCQVB4UpOiLJ+gAAkEnWBwCATLI+AABkkvXZZOz2ZBNtgNVUHJSk4ggg67PVu5vr93edO5TVdT3RJ04/C7yoOChJxXF1sj6bvP804LsLG7tDWf8vkgCfUnFQkoojQO2gZB1/7htKUnFQkoojhnF9AADIJOsDAEAmWR8AADKZrw8AAJmM6wMAQCZZHwAAMsn6AACQSdYHAIBMsj4AAGSS9QEAIJOsDwAAmWR9AADIJOsDAEAmWR8AADLJ+gAAkEnWBwCATLI+AABkkvUBACCTrA8AAJlkfQAAyCTrAwBAJlkfAAAyyfoAAJBJ1gcAgEyyPgAAZJL1AQAgk6wPAACZZH0AAMgk6wMAQCZZHwAAMsn6AACQSdYHAIBM/w9QVOE6+pp8PAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<IPython.core.display.Image object>"
]
},
"execution_count": 61,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import Image\n",
"Image(\"./Images/feature_extraction.png\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Why only reuse the convolutional base? Could we reuse the densely connected\n",
"classifier as well? In general, doing so should be avoided. The reason is that the representations\n",
"learned by the convolutional base are likely to be more generic and, therefore,\n",
"more reusable: the feature maps of a ConvNet are presence maps of generic\n",
"concepts over a picture, which are likely to be useful regardless of the computer vision\n",
"problem at hand. But the representations learned by the classifier will necessarily be\n",
"specific to the set of classes on which the model was trained—they will only contain\n",
"information about the presence probability of this or that class in the entire picture.\n",
"Additionally, representations found in densely connected layers no longer contain any information about where objects are located in the input image; these layers get rid of\n",
"the notion of space, whereas the object location is still described by convolutional feature\n",
"maps. For problems where object location matters, densely connected features\n",
"are largely useless.\n",
"\n",
"\n",
"Note that the level of generality (and therefore reusability) of the representations\n",
"extracted by specific convolution layers depends on the depth of the layer in the\n",
"model. Layers that come earlier in the model extract local, highly generic feature\n",
"maps (such as visual edges, colors, and textures), whereas layers that are higher up\n",
"extract more-abstract concepts (such as “cat ear” or “dog eye”). So if your new dataset\n",
"differs a lot from the dataset on which the original model was trained, you may be better\n",
"off using only the first few layers of the model to do feature extraction, rather than\n",
"using the entire convolutional base.\n",
"\n",
"\n",
"\n",
"In this case, because the ImageNet class set does not contain images of actors, we’ll \n",
"choose not to use the densely connected layers, in order to cover\n",
"the more general case where the class set of the new problem doesn’t overlap the\n",
"class set of the original model. Let’s put this into practice by using the convolutional\n",
"base of the VGG16 network, trained on ImageNet, to extract interesting features\n",
"from actors, and then train a classifier for the 8 actors on top of\n",
"these features.\n",
"\n",
"The VGG16 model, among others, comes prepackaged with Keras. You can import\n",
"it from the `keras.applications` module. Many other image-classification models (all\n",
"pretrained on the ImageNet dataset) are available as part of `keras.applications`:\n",
"\n",
"\n",
"- Xception\n",
"- ResNet\n",
"- MobileNet\n",
"- EfficientNet\n",
"- DenseNet\n",
"- etc.\n",
"\n",
"Let's instantiate the VGG16 model."
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "4Luec7pbGhJv",
"scrolled": true
},
"outputs": [],
"source": [
"# The target image size can be fixed here (quadratic)\n",
"# The ImageDataGenerator() automatically scales the images accordingly (aspect ratio is changed)\n",
"image_size = 150"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "eRes_n9BGhJ0"
},
"outputs": [],
"source": [
"conv_base = keras.applications.vgg16.VGG16(weights=\"imagenet\",\n",
" include_top=False,\n",
" input_shape=(image_size, image_size, 3))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "vEIWLeqSGhJ5"
},
"source": [
"You pass three arguments to the constructor:\n",
"\n",
"- `weights` specifies the weight checkpoint from which to initialize the model.\n",
"\n",
"- `include_top` refers to including (or not) the densely connected classifier on\n",
"top of the network. By default, this densely connected classifier corresponds to\n",
"the 1'000 classes from ImageNet. Because we intend to use our own densely\n",
"connected classifier (with 8 classes of actors), we don’t need to\n",
"include it.\n",
"\n",
"- `input_shape` is the shape of the image tensors that we’ll feed to the network.\n",
"This argument is purely optional: if we don’t pass it, the network will be able to\n",
"process inputs of any size. Here we pass it so that we can visualize (in the following\n",
"summary) how the size of the feature maps shrinks with each new convolution\n",
"and pooling layer."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here’s the detail of the architecture of the VGG16 convolutional base. It’s similar to\n",
"the simple convnets you’re already familiar with:"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "M7Bk7t1MGhJ6"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"vgg16\"\n",
"_________________________________________________________________\n",
" Layer (type) Output Shape Param # \n",
"=================================================================\n",
" input_9 (InputLayer) [(None, 150, 150, 3)] 0 \n",
" \n",
" block1_conv1 (Conv2D) (None, 150, 150, 64) 1792 \n",
" \n",
" block1_conv2 (Conv2D) (None, 150, 150, 64) 36928 \n",
" \n",
" block1_pool (MaxPooling2D) (None, 75, 75, 64) 0 \n",
" \n",
" block2_conv1 (Conv2D) (None, 75, 75, 128) 73856 \n",
" \n",
" block2_conv2 (Conv2D) (None, 75, 75, 128) 147584 \n",
" \n",
" block2_pool (MaxPooling2D) (None, 37, 37, 128) 0 \n",
" \n",
" block3_conv1 (Conv2D) (None, 37, 37, 256) 295168 \n",
" \n",
" block3_conv2 (Conv2D) (None, 37, 37, 256) 590080 \n",
" \n",
" block3_conv3 (Conv2D) (None, 37, 37, 256) 590080 \n",
" \n",
" block3_pool (MaxPooling2D) (None, 18, 18, 256) 0 \n",
" \n",
" block4_conv1 (Conv2D) (None, 18, 18, 512) 1180160 \n",
" \n",
" block4_conv2 (Conv2D) (None, 18, 18, 512) 2359808 \n",
" \n",
" block4_conv3 (Conv2D) (None, 18, 18, 512) 2359808 \n",
" \n",
" block4_pool (MaxPooling2D) (None, 9, 9, 512) 0 \n",
" \n",
" block5_conv1 (Conv2D) (None, 9, 9, 512) 2359808 \n",
" \n",
" block5_conv2 (Conv2D) (None, 9, 9, 512) 2359808 \n",
" \n",
" block5_conv3 (Conv2D) (None, 9, 9, 512) 2359808 \n",
" \n",
" block5_pool (MaxPooling2D) (None, 4, 4, 512) 0 \n",
" \n",
"=================================================================\n",
"Total params: 14,714,688\n",
"Trainable params: 14,714,688\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"conv_base.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "DBSrhVORGhKH"
},
"source": [
"\n",
"The final feature map (output volume) has shape $(5, 5, 512)$. That's the feature on top of which we will stick a densely connected classifier.\n",
"\n",
"At this point, there are two ways how we could proceed:\n",
"\n",
"- __Approach 1__: Run the convolutional base over our dataset, record its output to a NumPy array\n",
"on disk, and then use this data as input to a standalone, densely connected classifier\n",
"similar to those you saw in Block 4 of this course. This solution is fast and\n",
"cheap to run, because it only requires running the convolutional base once for\n",
"every input image, and the convolutional base is by far the most expensive part\n",
"of the pipeline. But for the same reason, this technique won’t allow us to use\n",
"data augmentation.\n",
"\n",
"- __Approach 2__: Extend the model we have (`conv_base`) by adding `Dense` layers on top, and run\n",
"the whole thing from end to end on the input data. This will allow us to use\n",
"data augmentation, because every input image goes through the convolutional\n",
"base every time it’s seen by the model. But for the same reason, this technique is\n",
"far more expensive than the first.\n",
"\n",
"We’ll cover both techniques. Let’s walk through the code required to set up the first\n",
"one: recording the output of `conv_base` on our data and using these outputs as inputs\n",
"to a new model."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "mlpIDmSCGhKI"
},
"source": [
"### 1. Approach : Fast feature extraction without data augmentation\n",
"\n",
"\n",
"We’ll start by extracting features as NumPy arrays by calling the `predict()` method of\n",
"the `conv_base` model on our training, and validation datasets.\n",
"Let’s iterate over our datasets to extract the VGG16 features."
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 480 files belonging to 8 classes.\n",
"Found 83 files belonging to 8 classes.\n"
]
}
],
"source": [
"from tensorflow.keras.utils import image_dataset_from_directory\n",
"\n",
"train_dataset = image_dataset_from_directory(\n",
" './train',\n",
" image_size=(150, 150),\n",
" batch_size=32,\n",
" shuffle=False,\n",
" label_mode=\"categorical\")\n",
"\n",
"validation_dataset = image_dataset_from_directory(\n",
" './validation',\n",
" image_size=(150, 150),\n",
" batch_size=32,\n",
" shuffle=False,\n",
" label_mode=\"categorical\")"
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"def get_features_and_labels(dataset):\n",
" all_features = []\n",
" all_labels = []\n",
" for images, labels in dataset:\n",
" preprocessed_images = keras.applications.vgg16.preprocess_input(images)\n",
" features = conv_base.predict(preprocessed_images)\n",
" all_features.append(features)\n",
" all_labels.append(labels)\n",
" return np.concatenate(all_features), np.concatenate(all_labels)\n",
"\n",
"train_features, train_labels = get_features_and_labels(train_dataset)\n",
"val_features, val_labels = get_features_and_labels(validation_dataset)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Importantly, `predict()` only expects images, not labels, but our current dataset yields\n",
"batches that contain both images and their labels. Moreover, the VGG16 model expects\n",
"inputs that are preprocessed with the function `keras.applications.vgg16.preprocess_input`, which scales pixel values to an appropriate range.\n",
"The extracted features are currently of shape `(samples, 5, 5, 512)`:"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(480, 4, 4, 512)"
]
},
"execution_count": 67,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_features.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And the labels are now referring to the order of the folders"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(480, 8)"
]
},
"execution_count": 68,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_labels.shape"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(83, 4, 4, 512)\n",
"(83, 8)\n"
]
}
],
"source": [
"print(val_features.shape)\n",
"print(val_labels.shape)"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {},
"outputs": [],
"source": [
"inputs = keras.Input(shape=(4, 4, 512))\n",
"# Note the use of the Flatten\n",
"# layer before passing the\n",
"# features to a Dense layer\n",
"x = layers.Flatten()(inputs)\n",
"x = layers.Dense(256)(x)\n",
"x = layers.Dropout(0.7)(x)\n",
"outputs = layers.Dense(8, activation=\"softmax\")(x)\n",
"model = keras.Model(inputs, outputs)"
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"model_4\"\n",
"_________________________________________________________________\n",
" Layer (type) Output Shape Param # \n",
"=================================================================\n",
" input_10 (InputLayer) [(None, 4, 4, 512)] 0 \n",
" \n",
" flatten_5 (Flatten) (None, 8192) 0 \n",
" \n",
" dense_10 (Dense) (None, 256) 2097408 \n",
" \n",
" dropout_5 (Dropout) (None, 256) 0 \n",
" \n",
" dense_11 (Dense) (None, 8) 2056 \n",
" \n",
"=================================================================\n",
"Total params: 2,099,464\n",
"Trainable params: 2,099,464\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": 72,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/30\n",
"15/15 [==============================] - 2s 120ms/step - loss: 47.5901 - accuracy: 0.3042 - val_loss: 21.6888 - val_accuracy: 0.4217\n",
"Epoch 2/30\n",
"15/15 [==============================] - 1s 104ms/step - loss: 18.2980 - accuracy: 0.5708 - val_loss: 18.4650 - val_accuracy: 0.4337\n",
"Epoch 3/30\n",
"15/15 [==============================] - 1s 82ms/step - loss: 7.1367 - accuracy: 0.7500 - val_loss: 20.4785 - val_accuracy: 0.4458\n",
"Epoch 4/30\n",
"15/15 [==============================] - 1s 82ms/step - loss: 6.9484 - accuracy: 0.7771 - val_loss: 18.7156 - val_accuracy: 0.5060\n",
"Epoch 5/30\n",
"15/15 [==============================] - 1s 100ms/step - loss: 5.1403 - accuracy: 0.8208 - val_loss: 25.2956 - val_accuracy: 0.5181\n",
"Epoch 6/30\n",
"15/15 [==============================] - 1s 87ms/step - loss: 5.6793 - accuracy: 0.8167 - val_loss: 22.8389 - val_accuracy: 0.4940\n",
"Epoch 7/30\n",
"15/15 [==============================] - 1s 84ms/step - loss: 4.9009 - accuracy: 0.8500 - val_loss: 25.7880 - val_accuracy: 0.5663\n",
"Epoch 8/30\n",
"15/15 [==============================] - 1s 104ms/step - loss: 3.1247 - accuracy: 0.8854 - val_loss: 21.7678 - val_accuracy: 0.4819\n",
"Epoch 9/30\n",
"15/15 [==============================] - 1s 82ms/step - loss: 4.9890 - accuracy: 0.8729 - val_loss: 28.8869 - val_accuracy: 0.4940\n",
"Epoch 10/30\n",
"15/15 [==============================] - 1s 90ms/step - loss: 2.8775 - accuracy: 0.8958 - val_loss: 33.2878 - val_accuracy: 0.5060\n",
"Epoch 11/30\n",
"15/15 [==============================] - 1s 100ms/step - loss: 1.6595 - accuracy: 0.9354 - val_loss: 35.3880 - val_accuracy: 0.3976\n",
"Epoch 12/30\n",
"15/15 [==============================] - 1s 85ms/step - loss: 2.5381 - accuracy: 0.9125 - val_loss: 28.2466 - val_accuracy: 0.5181\n",
"Epoch 13/30\n",
"15/15 [==============================] - 1s 80ms/step - loss: 3.0611 - accuracy: 0.9250 - val_loss: 29.1160 - val_accuracy: 0.4819\n",
"Epoch 14/30\n",
"15/15 [==============================] - 1s 91ms/step - loss: 1.9787 - accuracy: 0.9104 - val_loss: 27.5122 - val_accuracy: 0.4699\n",
"Epoch 15/30\n",
"15/15 [==============================] - 1s 88ms/step - loss: 1.8686 - accuracy: 0.9417 - val_loss: 34.6468 - val_accuracy: 0.4337\n",
"Epoch 16/30\n",
"15/15 [==============================] - 1s 85ms/step - loss: 2.2055 - accuracy: 0.9187 - val_loss: 33.5793 - val_accuracy: 0.5181\n",
"Epoch 17/30\n",
"15/15 [==============================] - 1s 87ms/step - loss: 1.7554 - accuracy: 0.9417 - val_loss: 29.8020 - val_accuracy: 0.5301\n",
"Epoch 18/30\n",
"15/15 [==============================] - 1s 102ms/step - loss: 1.1292 - accuracy: 0.9646 - val_loss: 29.8960 - val_accuracy: 0.4578\n",
"Epoch 19/30\n",
"15/15 [==============================] - 1s 86ms/step - loss: 2.0701 - accuracy: 0.9375 - val_loss: 36.3813 - val_accuracy: 0.4578\n",
"Epoch 20/30\n",
"15/15 [==============================] - 1s 81ms/step - loss: 1.9079 - accuracy: 0.9375 - val_loss: 33.0938 - val_accuracy: 0.5181\n",
"Epoch 21/30\n",
"15/15 [==============================] - 1s 97ms/step - loss: 2.1188 - accuracy: 0.9563 - val_loss: 33.5967 - val_accuracy: 0.5301\n",
"Epoch 22/30\n",
"15/15 [==============================] - 1s 79ms/step - loss: 1.8755 - accuracy: 0.9479 - val_loss: 33.4067 - val_accuracy: 0.5301\n",
"Epoch 23/30\n",
"15/15 [==============================] - 1s 85ms/step - loss: 1.4338 - accuracy: 0.9521 - val_loss: 37.3879 - val_accuracy: 0.4819\n",
"Epoch 24/30\n",
"15/15 [==============================] - 2s 112ms/step - loss: 1.5937 - accuracy: 0.9604 - val_loss: 36.4907 - val_accuracy: 0.4578\n",
"Epoch 25/30\n",
"15/15 [==============================] - 1s 83ms/step - loss: 0.7331 - accuracy: 0.9542 - val_loss: 36.2223 - val_accuracy: 0.5301\n",
"Epoch 26/30\n",
"15/15 [==============================] - 1s 86ms/step - loss: 1.1858 - accuracy: 0.9729 - val_loss: 34.4240 - val_accuracy: 0.5301\n",
"Epoch 27/30\n",
"15/15 [==============================] - 1s 90ms/step - loss: 0.8428 - accuracy: 0.9667 - val_loss: 34.2543 - val_accuracy: 0.5542\n",
"Epoch 28/30\n",
"15/15 [==============================] - 1s 94ms/step - loss: 1.0834 - accuracy: 0.9563 - val_loss: 38.1041 - val_accuracy: 0.4940\n",
"Epoch 29/30\n",
"15/15 [==============================] - 1s 89ms/step - loss: 1.4204 - accuracy: 0.9479 - val_loss: 34.2720 - val_accuracy: 0.5663\n",
"Epoch 30/30\n",
"15/15 [==============================] - 1s 85ms/step - loss: 1.4186 - accuracy: 0.9646 - val_loss: 38.5918 - val_accuracy: 0.4699\n"
]
}
],
"source": [
"model.compile(loss=\"categorical_crossentropy\",\n",
" optimizer=\"rmsprop\",\n",
" metrics=[\"accuracy\"])\n",
"\n",
"\n",
"logdir = os.path.join(\"logs_feature_extraction\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n",
"\n",
"\n",
"callbacks = [\n",
" keras.callbacks.ModelCheckpoint(filepath=\"feature_extraction.h5\", save_best_only=True, monitor=\"val_loss\"),\n",
" tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)\n",
"]\n",
"\n",
"history = model.fit(\n",
"train_features, train_labels,\n",
"epochs=30,\n",
"validation_data=(val_features, val_labels),\n",
"callbacks=callbacks\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that we’ll also use a `ModelCheckpoint` callback to save the model after each\n",
"epoch. We’ll configure it with the path specifying where to save the file, as well as the\n",
"arguments `save_best_only=True` and `monitor=\"val_loss\"`: they tell the callback to\n",
"only save a new file (overwriting any previous one) when the current value of the\n",
"`val_loss` metric is lower than at any previous time during training. This guarantees\n",
"that your saved file will always contain the state of the model corresponding to its bestperforming\n",
"training epoch, in terms of its performance on the validation data. As a\n",
"result, we won’t have to retrain a new model for a lower number of epochs if we start\n",
"overfitting: we can just reload our saved file."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let’s look at the loss and accuracy curves during training:"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEWCAYAAACEz/viAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABCpElEQVR4nO3dd3yV9fXA8c9JSAiBMAJhj7AhbAlD3BsnbsU9sW5brdX+bGvVtq7aauveVhEpolBFcSEOVAiIEDZIIIMREjIgCVnn98f3BgNk3CT35ib3nvfrlVdyn3XPk5s853m+U1QVY4wxoS0s0AEYY4wJPEsGxhhjLBkYY4yxZGCMMQZLBsYYY7BkYIwxBksGJsSIyGsi8pCX26aIyIn+jsmYpsCSgTHGGEsGxjRHItIi0DGY4GLJwDQ5nuKZ34rIChHZKyIvi0gXEflIRPJF5DMR6VBp+7NEZJWI5IjIlyIytNK6MSKyzLPfO0DUQe91hogs9+y7SERGehnj6SLyo4jkiUiqiNx/0PojPcfL8ay/yrO8lYj8XUS2iEiuiHzjWXasiKRV8Xs40fPz/SIyS0TeFJE84CoRGS8i33neY5uI/FtEIivtP0xEPhWRbBHZISK/F5GuIlIgIh0rbXeYiGSKSIQ3526CkyUD01SdB5wEDALOBD4Cfg/E4f5ubwMQkUHA28AdnnXzgP+JSKTnwvg+8B8gFviv57h49h0DvALcAHQEngfmikhLL+LbC1wBtAdOB24UkbM9x+3jifdfnphGA8s9+z0OjAUmeWK6Gyj38ncyBZjlec+3gDLg10An4HDgBOAmTwwxwGfAx0B3YADwuapuB74ELqx03MuBGapa4mUcJghZMjBN1b9UdYeqpgNfAz+o6o+qWgS8B4zxbHcR8KGqfuq5mD0OtMJdbCcCEcA/VbVEVWcBSyq9xzTgeVX9QVXLVPV1YJ9nvxqp6pequlJVy1V1BS4hHeNZfQnwmaq+7XnfLFVdLiJhwDXA7aqa7nnPRaq6z8vfyXeq+r7nPQtVdamqfq+qpaqagktmFTGcAWxX1b+rapGq5qvqD551rwOXAYhIODAVlzBNCLNkYJqqHZV+LqzidRvPz92BLRUrVLUcSAV6eNal64GjMW6p9HMf4E5PMUuOiOQAvTz71UhEJojIAk/xSi7wK9wdOp5jbKpit064Yqqq1nkj9aAYBonIByKy3VN09FcvYgCYAySISF/c01euqi6uZ0wmSFgyMM1dBu6iDoCICO5CmA5sA3p4llXoXennVOAvqtq+0le0qr7txftOB+YCvVS1HfAcUPE+qUD/KvbZBRRVs24vEF3pPMJxRUyVHTzE8LPAWmCgqrbFFaNVjqFfVYF7nq5m4p4OLseeCgyWDEzzNxM4XURO8FSA3okr6lkEfAeUAreJSISInAuMr7Tvi8CvPHf5IiKtPRXDMV68bwyQrapFIjIeVzRU4S3gRBG5UERaiEhHERnteWp5BXhCRLqLSLiIHO6po1gPRHnePwK4D6it7iIGyAP2iMgQ4MZK6z4AuonIHSLSUkRiRGRCpfVvAFcBZ2HJwGDJwDRzqroOd4f7L9yd95nAmaparKrFwLm4i142rn5hdqV9k4DrgX8Du4GNnm29cRPwgIjkA3/EJaWK424FTsMlpmxc5fEoz+q7gJW4uots4BEgTFVzPcd8CfdUsxc4oHVRFe7CJaF8XGJ7p1IM+bgioDOB7cAG4LhK67/FVVwvU9XKRWcmRIlNbmNMaBKRL4DpqvpSoGMxgWfJwJgQJCLjgE9xdR75gY7HBJ4VExkTYkTkdVwfhDssEZgK9mRgjDHGf08GIvKKiOwUkeRq1ouIPCUiG8UNO3CYv2IxxhhTM38OdvUarpXGG9WsPxUY6PmagGszPaGabffr1KmTxsfH+yZCY4wJEUuXLt2lqgf3XdnPb8lAVb8SkfgaNpkCvOHpHfq9iLQXkW6quq2m48bHx5OUlOTLUI0xJuiJSI1NiANZgdyDA7vXp3mWHUJEpolIkogkZWZmNkpwxhgTSppFayJVfUFVE1U1MS6u2qccY4wx9RTIZJCOG0OmQk/PMmOMMY0skMlgLnCFp1XRRNzIiTXWFxhjjPEPv1Ugi8jbwLFAJ88MTn/CjS2Pqj6Hm4TkNNx4MAXA1f6KxRhjTM382Zpoai3rFbjZX+9vjDHGe82iAtkYY4x/+bPTmTHGNGl5RSW8tyydVpHhDOkaw8DOMbSKDA90WAFhycAYE3IKi8t4/bsUnv1yE7mFJfuXhwnEd2zN4K4xDO4aw5CuMQzu2pbesdGEh0kNR2z+LBkY08yUlythQX5h8peSsnJmJqXy1Ocb2JG3j2MHx3HnSYOJbhnOuu35rN2ez7rteazZlsfHq7ZTMY5nVEQYg7rEMKJHO44d3JkjBnQkOrJxL5+79uwjJqoFLVv458ml2Y1ampiYqDYchQlF5eXK9MVbefTjtZw4tAsPnTO80S9IzVV5ufK/FRn849P1pGQVkNinA3dPHsL4vrHV7lNQXMqGHXt+SRI78li+NYe9xWVEtghjYr+OHD84juOHdKF3x+hqj9NQqdkFvPDVz8xMSuWBKcO4aFzv2neqgogsVdXEatdbMjCm6fs5cw/3zF7J4s3ZJHRry5rteQyIa8Ozlx3GgM7eTNncfO3ML2Jpym6WpOxmRVoOsa0j9xffDO4aQ3zHaFqEV90WRlX5cl0mj85fx5pteQzpGsPdkwdz3ODOiNT96WpfaRlLNu/mi7U7WbBuJ5t37QVgQOc2HD+kM8cN7kxifAciqomnLtZuz+PZLzfxwYpthAmcd1hPbjimP307ta7X8SwZGNOMlZSV8+LXP/PPzzYQ1SKM+85I4IKxPfl2Yxa3z/iRwpIy/nbuCKaMrnJYL59SVT5dvYPk9FxG9WrP2D4daB8d6fP32LxrL0kpu1mckk1SSjYpWQUAtGwRxvAe7dhdUEzKrr2Uey5dkS3CGNi5DUO6tvUkCVfWvyW7gEc/XsuSlN30jo3mzpMHcebI7j4tYtu8a69LDGt38sPmLErKlJioFhw9MI7D+3dkXHwsAzu3qdN7LknJ5tkvN/HF2p20jgznkgm9ufbIfnRtF9WgWC0ZGNNMJafncvesFazelsdpI7py/1nD6BzzywVhe24Rt769jCUpu7lsYm/+cEaC38qTF23cxaPz17E8NeeA5YO6tCExPpZx8R1I7BNLzw6tvL7jLiguJSOnkPScIjbsyGdJSjZJKbvJ2lsMQIfoiF+OHR/L8O7tiGzh7riLSsrYuHPP/jJ+9z2fnfn7DniPzjEtue2EgVw0rpdP7tZrsmdfKd9s2MWCtTv5cv1OduS5WNpGtSAxPpbE+A6Mi49lZM92h3xOqsoXa3fy7JebSNqym9jWkVw9KZ7LD+/js4RrycCYZqaopIx/fLael77eTGzrSB6cMpzJw7tWuW1JWTmPz1/H81/9zIge7Xjm0sPoFeu78uufUnN4bP46vtm4i27torjjxIGcPrI7yem5JKVksyRlN8u27CZ/XykAXdtG7b/oHda7AyXl5e6Cv7tw/4U/I6eQjNxCcgpKDniv3rHR+/cdF9+Bfp3qdkcNsHtv8f4EER4mnD+2V0CaiqoqqdmFLEnJ3v+1KdMVKUW2CGNUz3b7E11uYQnPffkz63bk06N9K6Yd3Y8LE30ftyUDY5qR73/O4t7ZK9m8ay8Xj+vFvacNpV2riFr3+2TVdu78708I8PcLR3NSQpcGxbFxZz5//2Q9HyVvp0N0BDcfN4DLJvYhKuLQC1RZubJuez5JW1xyWLI5m+15RYds16ZlC3q0b0X39lF0b9+K7u1beV63Ir5jNJ3bNqwYpKnL2rOPpVt2k7RlN0tSslmZlkupp6xrUJc2/OqY/pw5qrvfnmAsGRjjYwXFpWzLLaJ/XBufHTOvqIS/zVvL24u30js2mofPHcGkAZ3qdIytWQXcNH0pyel53HBMP3578uBqK1ark55TyD8/Xc+7y9JoFRHOdUf147qj+hITVXtCqqCqpOcUsjw1h+jI8P0X/rZ1OEYoKCwuY3lqDuWqHN6vo9+bC1syMMaHMnIKufrVJazbkc+lE3pzz6lD6nShrMqnq3dw3/sryczfx3VH9ePXJw6qdxFBUUkZD36wmrd+2Mr4+FieuGgUHVu3rHW/vKISnlu4ibe+3woCl0/sw03H9qdjm9r3Nc2DJQPTbOzML+Kn1FxOGNK5SXaqWrMtj6tfXcLefaVMHt6Vd5el0TkmiofOHs6J9SiWyczfx/3/W8WHK7YxpGsMj5w3klG92vsk1vd/TOfe2SspLCnzep8wgQvG9uL2EwfSvX0rn8Rhmg5LBqZZyN5bzPnPLeLnzL1M6BvL384dQT8fFsM01DcbdvGrN5fSpmULXrtmHEO6tmV5ag73vLuCtdvzOWNkN+4/axidvLiTVlVmL0vngQ9WU1hcxu0nDmTa0f18Xla8KXMPn63egTf/4WECxw/pwoDOTed3bnzLkoFp8gqLy7jkpe9ZlZHHr47ux2uLUigqLefXJw7i+qP61rnc29feXZrG795dwYDObXj16nF0a/fLXXNxaTnPL9zEv77YSHTLcP54RgLnjOlRbfPK1OwCfv/eSr7esIvEPh14+LyRdgE2jcKSgWnSSsvKueE/S1mwbifPXDqWycO7sjOviD/OWcXHq7YzrHtbHjlvJMN7tKvX8bdmFZCRW8i4+Ng6DzSmqjy9YCOPf7KeSf078tzlY6utBN24M5/fvbuSpVt2c/SgOP56znB6dviliWdZufL6ohQe/2QdAtxz6hAundCnSRaHmeBkycA0WarKPe+u5J2kVB46eziXTexzwPqPk7fxhzmryN5bzPVH9eOOEwdW2bTxYJn5+/hwRQbvL8/Y30mqR/tWXDKhNxcm9iIupvainNKycv4wJ5m3F6dyzpgePHLeyP0dnqpTXq785/stPPrxWhS4+5TBXH54PJsy93D3rBUsT83huMFxPHTOCHpYmbxpZJYMTJP1xCfreOqLjdx6/ADuPHlwldvkFpTw13lreCcplb6dWvPwuSOY0K/jIdvlF5Uwf9UO5ixP59uNuyhXGNqtLVNGd6dbuyjeWZLKok1ZRIQLpwzrymUT+zChb2yVxTl795Vyy/RlLFiXyc3H9eeukwfXaRyb9JxC/u+9lXy5LpPBXWL4edceYqIi+NOZCZw1qnu9xsQxpqEsGZgm6c3vt3Df+8lcmNiTR84bWesF8tuNu7h39kq2ZhdwiadJZ8sWYSxYm8ncn9L5fM1O9pWW0yu2FVNG9eCs0d0Z1OXAAdw2Ze7hre+3MmtpKnlFpQzs3IbLJvbhnMN67C/+2ZlfxLWvJbEqI5eHzh7BJRPqN0KkqjJneQZ/nbeGSf078oczEqyZpgkoSwamyfk4eTs3vbWU4wZ35vnLx3pdQVxYXMYTn67j5W82E9u6JftKy8gvKqVTm0jOGNmds0Z3Z0yv9rUmlsLiMv73UwZv/rCFFWm5REeGM2V0d04Y0oX7/7eKrD3FPH3pGI4f0rBevOCSgj0JmKbAkoFpUpakZHPpSz8wrHtbpl83sV6dq35KzeHxT9YRF9OSKaN7cET/jvVucbQiLYc3v9/C3J8yKCopp1ObSF65ahwje7av1/GMaaoCmgxEZDLwJBAOvKSqDx+0vg/wChAHZAOXqWpaTce0ZNB8rd+Rz/nPLqJTm5bMunESsa19O/xxQ+QWlDB/1XYmDeh4QCsgY4JFbcnAbw24RSQceBo4FUgApopIwkGbPQ68oaojgQeAv/krHhNY23ILufKVxbSMCOf1a8Y3qUQA0C46ggvH9bJEYEKWP3vzjAc2qurPqloMzACmHLRNAvCF5+cFVaw3QSC3oIQrX1nMnqJSXr96vE+HWDbG+IY/k0EPILXS6zTPssp+As71/HwOECMih7QbFJFpIpIkIkmZmZl+Cdb4R0ZOIVe/tpjNu/by/OVjSejeNtAhGWOqENh+/nAXcIyI/AgcA6QDh4yspaovqGqiqibGxcU1doymHsrLlTe+S+GkJxayZls+T148ps5DMhtjGk8LPx47HehV6XVPz7L9VDUDz5OBiLQBzlPVHD/GZBrBxp17uOfdFSRt2c1RAzvx13NGWNGQMU2cP5PBEmCgiPTFJYGLgUsqbyAinYBsVS0H7sW1LDIBUlJWzlOfbyC2dSSnj+x2wHy73u7//MJNPPX5RlpFhvP4BaM477DqB20zxjQdfksGqloqIrcA83FNS19R1VUi8gCQpKpzgWOBv4mIAl8BN/srHlOz4tJybn17GfNX7QDgwQ9Wc8SATpw1qjunDO9a6yxVK9JyuHuWG8759BHd+NNZCXVOJsaYwLFOZ+aARPCnMxM4YkAn5i7PYM5P6aRmFxLZIowTh3bmrFE9OHZw3AGDxVXuFdypTUsePHs4pwyrevJ2Y0zgWA9kU6Pi0nJumb6MT1a7RHD1EX33r1NVfkzNYe7yDD5YkcGuPcXERLXg1OFdmTK6B6rw+/fceEFTx/finlO9m7zdGNP4LBmYahWXlnPz9GV8unoH95+ZwFWVEsHBSsvKWbQpiznLM5i/ajt79pUCEN8xmr+eO4JJ/a2lkDFNWW3JwJ8VyKYJq5wI/nzWMK6cFF/j9i3Cwzh6UBxHD4rjLyXD+WLtTjLz93FhYq96T95ujGk6LBmEoOLScm56axmfrdnBA1OGccXh8XXaPyoinNNGdPNPcMaYgAh0pzPTyBqaCIwxwcmeDELIvtIybn5rGZ+t2cmDU4ZxuSUCY4yHJYMQsa+0jJveXMbna3fy4NnDufyg+YaNMaHNkkEIqJwIqpp43hhjLBkEucLiMm54cylfrc+0RGCMqZYlgyCWV1TCta8tYemW3Txy3gguGle/yd2NMcHPkkGQ2rVnH1e+spj1O/L519TDOH2kNQU1xlTPkkEQ2pZbyKUv/UBGTiEvXpHIsYM7BzokY0wTZ8kgyGzetZfLXvqBvMIS3rhmAuP7xgY6JGNMM2DJIIis3Z7HZS8tplyVt6dNZHiPdoEOyRjTTFgyCBLLtu7m6leX0CoinDevm8CAzjGBDskY04xYMggCizbu4ro3koiLacmb106wKSaNMXVmyaCJ2ZK1l9nL0unUJpLu7Vvt/2ob1aLK6SM/WbWdW97+kb4dW/Ofa8fTua3NLmaMqTtLBk3I7r3FXPbyD6RmFx6yrk3LFnRvH7U/OfRo34rycuWfn29geI92vH71ONpHRwYgamNMMLBk0ESUlrn5BXbk7WP2TZPo2aEVGTlFZOQUkpFTSLrne0ZOESvTcsnaWwzA4f068uKVibRpaR+lMab+7ArSRPxl3hoWbcri8QtGcVjvDgB0jolidK/2VW5fVFJGZv4+erRvRVjYocVHxhhTF36dz0BEJovIOhHZKCL3VLG+t4gsEJEfRWSFiJzmz3iaqplJqbz6bQrXHNGX88f29GqfqIhwesVGWyIwxviE35KBiIQDTwOnAgnAVBFJOGiz+4CZqjoGuBh4xl/xNFU/bt3Nfe8lc8SAjvz+tCGBDscYE6L8+WQwHtioqj+rajEwA5hy0DYKtPX83A7I8GM8Tc6OvCJu+M9SuraL4t9TD6NFuE08Z4wJDH9efXoAqZVep3mWVXY/cJmIpAHzgFurOpCITBORJBFJyszM9Eesja6opIwb/rOUPftKefGKRDq0tpZAxpjACfSt6FTgNVXtCZwG/EdEDolJVV9Q1URVTYyLi2v0IH1NVfm/95JZnprDExeOYnBX6y1sjAksfyaDdKBXpdc9PcsquxaYCaCq3wFRQCc/xtQkvPptCu8uS+P2EwYyebgNLW2MCTx/JoMlwEAR6SsikbgK4rkHbbMVOAFARIbikkFwlANV49uNu/jLvDWcnNCF208YGOhwjDEG8GMyUNVS4BZgPrAG12polYg8ICJneTa7E7heRH4C3gauUlX1V0yBtjWrgJunL6N/XGueuGi0NQs1xjQZfu10pqrzcBXDlZf9sdLPq4Ej/BlDU7F3XynXv5GEKrx4hfUYNsY0LYGuQA4JRSVl3D5jORt25vPvS8bQp2PrQIdkjDEHsNtTP0vZtZeb3lrG6m153H9mAkcNbP6toYwxwceSgR99nLyN3/53BWFhwitXJXL8kC6BDskYY6pkycAPSsrKefijtbz8zWZG9WrP05eMoWcHm3DGGNN0WTLwsYycQm6ZvoxlW3O4alI8vz9tKJEtrGrGGNO0WTLwoYXrM7ljxo8Ul5bzr6ljOHNU90CHZIwxXrFk4ANl5cqTn2/gX19sYFDnGJ657DD6x7UJdFjGGOM1SwYNtGvPPu6YsZxvNu7ivMN68tDZw2kVGR7osIwxpk4sGTTAxp35XPrSD+QUlPDIeSO4MLFXlZPWG2NMU2fJoAGm/5BKTkEJs2+axLDu7QIdjjHG1Js1c2mA5PRcErq3tURgjGn2LBnUU3m5siojlxE9LBEYY5o/Swb1tDlrL3uLyxhuTwXGmCBgyaCektNzARhuTwbGmCBgyaCektNziWwRxsAu1p/AGNP8WTKop5XpuQzt1paIcPsVGmOaP7uS1UN5ubIqPY/h3dsGOhRjjPEJSwb1sCW7gPx9pdaSyBgTNCwZ1INVHhtjgo0lg3pITs8lMjyMQV1iAh2KMcb4hCWDeliZnsvgrjE2T4ExJmj49WomIpNFZJ2IbBSRe6pY/w8RWe75Wi8iOf6MxxdUleT0XCsiMsYEFa8GqhOR2cDLwEeqWu7lPuHA08BJQBqwRETmqurqim1U9deVtr8VGFOH2AMiNbuQvCKrPDbGBBdvnwyeAS4BNojIwyIy2It9xgMbVfVnVS0GZgBTath+KvC2l/EEzEpP5bElA2NMMPEqGajqZ6p6KXAYkAJ8JiKLRORqEYmoZrceQGql12meZYcQkT5AX+CLatZPE5EkEUnKzMz0JmS/WZmeS0S4MKir9Tw2xgQPr+sMRKQjcBVwHfAj8CQuOXzqgzguBmapallVK1X1BVVNVNXEuLg4H7xd/SWn5zKoSwwtW9hsZsaY4OFVMhCR94CvgWjgTFU9S1XfUdVbgepukdOBXpVe9/Qsq8rFNIMiIlUl2YatNsYEIW9nOntKVRdUtUJVE6vZZwkwUET64pLAxbh6hwOIyBCgA/Cdl7EETNruQnIKSqwlkTEm6HhbTJQgIu0rXohIBxG5qaYdVLUUuAWYD6wBZqrqKhF5QETOqrTpxcAMVdW6hd74rOexMSZYeftkcL2qPl3xQlV3i8j1uFZG1VLVecC8g5b98aDX93sZQ8AlZ+TSIkwY0tV6Hhtjgou3TwbhIiIVLzx9CCL9E1LTtTI9j4FdYoiKsMpjY0xw8TYZfAy8IyIniMgJuMrej/0XVtNT0fN4RA8bttoYE3y8LSb6HXADcKPn9afAS36JqInKyC0ie2+x1RcYY4KSV8nAMwTFs56vkGSVx8aYYObt2EQDgb8BCUBUxXJV7eenuJqc5PRcwsOEhG5WTGSMCT7e1hm8insqKAWOA94A3vRXUE3RyvRcBnZuY5XHxpig5G0yaKWqnwOiqls8zUFP919YTUtF5fGw7lZEZIwJTt5WIO8TkTDcqKW34HoUh8xIbTvy9rFrT7G1JDLGBC1vnwxux41LdBswFrgMuNJfQTU1+4et7mlPBsaY4FTrk4Gng9lFqnoXsAe42u9RNTEr03MJExhqlcfGmCBV65OBZ1jpIxshliYrOT2X/nFtiI70tlTNGGOaF2+vbj+KyFzgv8DeioWqOtsvUTUxyem5HDmgU6DDMMYYv/E2GUQBWcDxlZYpEPTJYGdeETvz91lnM2NMUPO2B3LI1RNUWGk9j40xIcDbHsiv4p4EDqCq1/g8oiYmOT0PERjW3SqPjTHBy9tiog8q/RwFnANk+D6cpmdlei79OrWmdUurPDbGBC9vi4nerfxaRN4GvvFLRE1McnouE/vFBjoMY4zxK287nR1sINDZl4E0RZn5+9ieV2T1BcaYoOdtnUE+B9YZbMfNcRDUkjOs8tgYExq8LSYKyUl/k9NcMrDKY2NMsPOqmEhEzhGRdpVetxeRs73Yb7KIrBORjSJyTzXbXCgiq0VklYhM9zryRlBReRwTFRHoUIwxxq+8rTP4k6rmVrxQ1RzgTzXt4BnT6GngVNykOFNFJOGgbQYC9wJHqOow4A6vI28Eyem5DLMiImNMCPA2GVS1XW1FTOOBjar6s6oWAzOAKQdtcz3wtKruBlDVnV7G43dZe/aRkVtkw1YbY0KCt8kgSUSeEJH+nq8ngKW17NMDSK30Os2zrLJBwCAR+VZEvheRyVUdSESmiUiSiCRlZmZ6GXLDJGfkAVZ5bIwJDd4mg1uBYuAd3B1+EXCzD96/Ba6Z6rHAVOBFEWl/8Eaq+oKqJqpqYlxcnA/etnbJ6RWVx5YMjDHBz9vWRHuBKiuAa5AO9Kr0uqdnWWVpwA+qWgJsFpH1uOSwpI7v5XPJ6bn06RhNu1ZWeWyMCX7etib6tPIdu4h0EJH5tey2BBgoIn1FJBK4GJh70Dbv454KEJFOuGKjn72K3M9WpudaEZExJmR4W0zUydOCCABPhW+NPZBVtRS4BZgPrAFmquoqEXlARM7ybDYfyBKR1cAC4LeqmlXHc/C53XuLSdtdyAhLBsaYEOHt6GvlItJbVbcCiEg8VYxiejBVnQfMO2jZHyv9rMBvPF9NRkXPY0sGxphQ4W0y+D/gGxFZCAhwFDDNb1EFWHK6a0lkPY+NMaHC2wrkj0UkEZcAfsSV9Rf6Ma6ASk7PpVdsK9pHRwY6FGOMaRTeDlR3HXA7rkXQcmAi8B0HToMZNFam51oRkTEmpHhbgXw7MA7YoqrHAWOAHH8FFUhFJWVszS5gcJcgKiIqLoAdqwMdhTHmYIW7IWtToKMAvE8GRapaBCAiLVV1LTDYf2EFTtruAgD6dIwOcCQ+NP9eeP4oyEmtfVtjTON571fwwnHuhi3AvE0GaZ5+Bu8Dn4rIHGCLv4IKpK3Z7kPpFRskySBnK/z4FpSXQtIrgY7GGFMh40dY/zHsy4W1HwY6Gu+Sgaqeo6o5qno/8AfgZeBsP8YVMFuzXDLoHSzJ4Jt/uu89x8Oy16GkKKDhGGM8Fj4GUe2gbQ/46e1AR1P3aS9VdaGqzvWMRBp0tmYXEh0ZTqc2QdCSKC8DfvwPjLkUjvs9FGTB6vcDHZUxzU95GXz/LHzyB9Bau1jVbvtKWPchTLwJRk2FnxdA3raGH7cB6jsHctDaml1A79hoRCTQoTTct0+ClsORv4F+x0LHgbD4hUBHZUzzkrUJXj0NPr4HFj3lmxuqrx6Dlm1hwg0w6mL3f7pyZsOP2wCWDA6Sml0QHPUF+dth6WvuD61DHxCB8ddD+lJIq230cWMM5eWw+EV47kjYuQbOfha6jID590Hx3vofd8dqWD3HJYJWHaDTQOg5Dpa/7ZunjnqyZFCJqu5/Mmj2Fv0LyordU0GFUVMhsg0seTFwcRnTHOSkwpvnwLy7oPfhcNN3MPoSOO1RyEuDb/5R/2N//bj7P5x40y/LRl0MmWtg+4qGx15Plgwq2bWnmMKSsuafDPZkupZDIy6Ejv1/WR7V1v3RJc+GvbsCF58xTZWqa3337CRIXQJn/BMuexfaeebl6jMJRlwA3z4F2ZvrfvzM9e7/b9x1EB37y/Jh50J4pHs6CBBLBpVUNCtt9sngu39DSSEcfdeh68ZdD2X7YNkbjR+XMU1Z/nZ4+2KYcxN0HQE3fguJV7si1spOegDCWsD8/6v7e3z9d4hoBYffcuDy6FgYNBlW/hfKSup/Dg1gyaCS1GDoY1CQ7co5h5/nyiIP1nkI9D3aPTmUlTZ+fMY0RcnvwjMT4ecv4ZS/wZUfQGzfqrdt2x2O+a1rDbTxM+/fI2uTqyROvAbaVDFj46ipULALNn5er1NoKG9HLQ0JFU8GPTu0CnAkDfD9M1Cyt+qnggrjroeZl7sOL0PPaLzYTNXyd7gKxbJ9MOnWQEdTvbxtLk5vy7XDWrg74LhB/o2rKlt/8Dz9elEhm5fhmnb2SIRznqv6JupgE2+CZf+Bj34HN34HLbxoiv71E64oaNJtVa8fcCJEd3R9DgZXOR28X1kyqGRLVgFd20YRFREe6FDqp3A3/PA8JEyBzkOr327waa6jy5IXLRkEyt5d7sK66j1I+Yb9F60eY125dFOxJ9M1pVz1Pmz5FlBo0xXCvZgOtiDLndsNC6FljJ8DraS8HObeCrmp7uJam7BwOP4PcMQdEO7lJbFFSzj1EXjrfPjhWTji9pq3350CK2a4uoKYLtUcMxKGnw9LX3X/y606eBeLj1gyqCS1ubck+uF52JcHR/+25u3CW7hH1S8ehMx1EBeUw0w1PQXZsPYDV4G4+SvQMtf345i7YfCp8NYFsPBRuOL9wMe5Zq6LM+Vr1wa+02A49h4Ydo73fy8p38LrZ8CHd8I5zx9a9u4va+bArnVw3ssw4nz/vc/Ak2DQqe4zG3EhtO1W/bbf/AMkrPakMXoqLH7e3SQkXuPbeGthyaCSrdkFHDGgU2CDqGhnXNd/nKI8V0Q05AxX+VWbw66EhY/AkpfgtMfqHmegqTbexaUhijzjziTPdkUR5aXQoS8ceYdrQdJl2C/nMek2+PQPrhVLr3GNG2dhjotz1WxXbl5eCrH94ag7XQLonFD333f8EXDMPfDlX12nx9GX+CHwg5SXu2EeOg50cfvb5L/C0xPgsz/BudV06MxJdS2Uxl7p6htq0m00xA2Bn2ZYMgiUopIytucVBe7JYOda94+46j3YsxNOfgjGXOb9P+DiF9yFp6a6gsraxLmL0fLp7hE5qhkN2V1S5Cr7BpwApz4GYU20HURhDjw9HvbsgHa94fCb3e+826iqP9fEa9wd5FePwqX/bbw48zLg2SOgMBva93bl/MPPha4jG55wj77LPV18eKcrk/d3/cG6ebBzFZzzgiv+8bfYfq6e5+u/u8+v98RDt/n2n+77EXfUfjwR1/z7s/tdhXPlpuF+1kT/ixpf2m43cVvvjo1Yebxro7uLeeZweGaCe9xs08Xdhc29BaZf6N14JfvyXXPSgadA9zHev//4aVC8B1a8U/9zCISUb2D3ZvdUM++ugPbarNFPb7tEMPUduGOFa5LYfXT1F9iWbWDSLbDhE0hf1nhxfvIH16P26o/g9hVw0p+rT1h1FRYO577omlPOuto1efYXVfe0G9vPtaZrLEfd6erg5t3lxjCqLC/DVWSPvgTa9/LueCMvAsQ9HTQiSwYeqfv7GLT27xtlb3atCp47Ev49FhY85EYuPPUxuHMdXPUBXPUhTH4ENn/t7oBX/LfmC96Sl12F0zF31y2WnmNd8lj8YtO9oFZlw3yIiIaJN0PSy65FR1OLv2Iog14TXMsQby+s466HqPZu7JrGkPItJM9yxVZ9Jvmn6K1tNzj7OdiRDJ/c5/vjV1g/37V0Ouou7yuCfSGyNZz8oBt8bulrB6779imXII76TZW7Vqltd1estmKG+ztqJH5NBiIyWUTWichGEbmnivVXiUimiCz3fF3nz3hq4vcOZz++5SaxeGo0fP5nCG8Jp/wVfr0arvkYJkz7pZVBWBhM/BX86hvXzG32dTDziqp7DRcXuKEn+p8APRPrHtf4aa6ybfNXDTq9au3e4u7gc9N8czxV1yS27zFwyl9ckcbi510HoKaUEH7+ArI3ud9vXUS1dcVJ6+bBNj8PTVBWCh/dDe16eVeE0RCDTnaf1ZKXXCsqX6t4KmjfG0Ze6Pvj12bYuRB/lGuUUZDtluXvcC2DRk2FDvF1O96oqW4ukq2LfB5qdfyWDEQkHHgaOBVIAKaKSEIVm76jqqM9Xy/5K57abM0uoFWEn4auTvnG9WosK3ZFBbevgOs/d//0Fd3cq9JpAFwzH068310An54Aaz44cJulr7qOKsf8rn6xDTsXWsX6djTT3HT47ml48QR4cqQrL/7iL745duY6908y6GR3F3vyQzDhV/D90/DpH5tOQlj8IrTuDEPPqvu+46e5ES39/XSw9FV3t37KXyCyEerKTviTazo751Z3k+BLmz6HjGWuyMabZq++JuKamhblwQLP3/qip9z/fF2eCioMPcONX9SI8xz488lgPLBRVX/2zH0wA5jix/drEL8NXV1WCvM8d1/XfuqalnXo4/3+YeFw5K9h2kL3+PjOpTB7misWKil0w1T3PRp6T6hffBFRrpXDunkNmxYzf7tr2vryKfCPBJj/e/ePcOL9rvndug+h1AdTYKz/2H0feIr7LgKTH3bttxc9BZ8/EPiEsDvFFVmMvcq7zkgHa9XeJbg1c/03d/XeLPjiIfe3U5+EVR8tIl1zTxTevdZ3wy6owpePQNueMKoRWixVp8sw93eY9ApsWuAZH+yC+lUCR7Z2/YVWzWm0KTH9mQx6AJWvLmmeZQc7T0RWiMgsEamyhkVEpolIkogkZWZm+iNW/w1dnfSKa93Q0LuvLglw/Reuqd7KWa7S+cM7XQVlfZ8KKlQ0YVv6at3227vL1Ve8dgb8fYgrctiXB8fdB7cshV997RLZYVe4lk4pPiiK2vCJG0a48hOViKtzGXsVfPMEfPm3hr9PQyx52bUpT7y6/seYeKO7M/TX08EXD7qGB6c+2rhNdGP7wplPQtoSl4x8YfNCSFsMR/26fsnXl46713UWm36hu1k7ysvWfVUZdTEU57sbtUYQ6Ark/wHxqjoS+BR4vaqNVPUFVU1U1cS4uCrG9Gggvw1dvXeXqyDue4xv7r7CI9wf2/Wfu0rn5W9BnyMg/siGHbd9b3f3vvS12qfFLMh2rSPeOBseHwQf/sY9FRxzN9z0gxvq95jfuiKuCv2Pdxe2hpYVF+6Grd/DoFMOXRcWBqf/wzXHXfiIa5kVCMUF7vcz9Mza25TXJDrWzT+x6j030qUvZSx3n/WEG2ruqe4vw891ifvbf9ZtbJ/qLHwUYrrB6MsafqyGatXBFYeVFbvzbEhT2j5HuhKF5dN9F18N/Fnlng5UvtPv6Vm2n6pmVXr5EhCQ/+Bde4opKC6jd6yPm5V+/oBrsufru6/uY1yx0dLX3HgmvjD+eleUs/p9d0dSWZUdp+Jdkdfwc6HL8JrPLyLKjci49kN3wa5vS4+Nn7teu1UlA3AJ4UxP640Ff3Fj49SnvLYhkt+Fohz3+2yow29xRW9fP159h6a6UnVPcK07uR7FgTL5YUhdDLNvcKODxnSt33FSvnHDZEx+xP2dNQVjLnd/p0MaONRLWJhrZvrNE66JeU09nH3An08GS4CBItJXRCKBi4G5lTcQkcpndxawxo/xVGt/S6KOPnwyyPjR3SGOv8GNFOprEVGuxVHlO/CGOHhazH35rknr21PhsQHw/o2u8nbiTTDtS7htOZz4J9fb2ZtElzDFjVWz5dv6x7h+vhtrpsfY6rcJC4cpT7uy2s//7Jr2NRZV9/vrnOCe2BqqdScYd60b1jhrU8OPB65PSeoPri4nqp1vjlkfEa3g/FfdzdLsaYe2z/fWwkddRf3YK30bX0OEhXlGJu3c8GPtnxLT/50Q/ZYMVLUUuAWYj7vIz1TVVSLygIhUlJncJiKrROQn4DbgKn/FU5NUXzcrLS93lcatO8GxDSzPbywirhVL+lL4z7kuAcy+zhUpjLserv3MdZw6+UH3ZFLXJ50BJ7q+AfUtKiovg42fwoCTau9ZGhbu2rUPO8cN7/DV4w2bptBbaUtcO/fx1/vuSXDSbW6ky6//3vBjFeW5Flc9EgNb0Vqh8xA3c9jmhe4zqqut37t9j7jdJZdg1Gmg+7x+8v+UmH6tM1DVeao6SFX7q+pfPMv+qKpzPT/fq6rDVHWUqh6nqmv9GU91fhm62kfJYMU7rkIr0HdfdTXqYnfnvSPZjV109cfw61Vu/JVe4xp2gYuMdgN7rflf/e4C05a4OoPqiogOFt7C9XxNmOIqSx8bAP+92r1/bfUi9bX4BWjZzg1a5ittOsPYq11v1N0pDTvWV4+6BgenPdp0hvAYc7l7ivvyrzDr2l/a6Htj4aPu77UhFfXNweipsHO136fEbCJ/EYG1NbuALm1b+mbo6qZ291UXUW3hjpXwmzXugtHncN9eNBKmwN6d7o6urtbPBwl3ldHeCo+A81+DK//nyl43L4R3LvM89UyDdR/7prkruA5Gq953ww60bOObY1Y44nZX//H1E/U/RuZ6+P5ZV8FeUzFbYxNxE80f93+uvuqZw91nXZu0pa5vwaRbXTPMYDbsXAiL8PvwFJYMwLctiRY+Anszm9bdV11EtvbfAF8DT4YWUa79fF2tn++GS2jVvm77hYW5tvRn/hPuXA+XzYZhZ7vjvX0RPD4A3r8JNnzWsHbvy16H8hLXztzX2nZzzXOXT69fXxBV+PgeiGgNJ9zv8/AaLDzCtUa7/gt3pz/9Qphzi7uxqs5Xj7qWO/74fTc10bFuSBM/T4nZDK9WvufmMfDB3UXmOvjhuaZ399VUtIxxdQer59ZtzJWcVNdXY+DJDXv/8BZupNMp/4a7NsAl/3UT/az5H7x1nmsqu6IeFXVlJa4/Sf8TfFehf7Aj73DfK0bArIt189xd9HH3Vj3dYlPRbRRMWwBH/sY1m352Evy88NDtMpa7zoeH39y4k+YE0qip7ibTj1Nihnwy8NnQ1apuwLSI1q6dsalawhTIz4D0JO/32fCJ++5tfYE3WkS6IS3Oec4lhounu8q696bVPSGs/RDyt9V9HKK6aNcTxlzqWqjlZXi/X0khfHwvxA1tHnfRLVq6VmrXfOJ+fuMsmPfbAxsAfPWYq4vz5++7qRlwkisiDfNfb4CQTwbpOYWo+mDo6rUfuDb4x/2+ad99BdqgU1zrmLq0Klo/3/Vr6OSnsfAjomDI6XD5+65J6HvTXJ8Kby1+0XXcG3iSf+KrcORvXDPDb5/0fp9F/4KcLa7YMhBj9tRXr3Fww9cw4UZXMf/ckW5e4+3J7n9two3Nq3FGQ7WIhMvfg4E+6ldU1Vv47cjNhE9GKy0pdGPxdE5oHndfgRTVDvod54qKTn6o9hZKxQWu4vewK/0/bEJkNEyd4aaffPc6V3eSUMtwWjtWwZZv3ACE/p5MpUMf1+Jr6WuuSKq2TlbFBa7SOeFsV2/S3ERGw6kPu0Q95yZ4dbJLupExro+N8amQTwYVfQwaNC7Rt0+6kTSv/KBxx1FvrhKmuDkJMn6EHofVvG3K11Ba5Nsiopq0bAOXzoQ3z4NZ18CFb7iLUXUWv+gqxcdc3jjxHXWna1Uy/QLvto9o7ZJuc9b3KLhxkZsLYelrcPTdjT5ZfCgI+SvX1qwCoiLCiGvTsn4H2L3FTVU47Bz3R2tqN/hUV/a5ek7tyWD9fHdBa+j4S3XRMgYunQX/ORtmXgkXvelacxysMMf1KRlxvmvx0Rhi+8GN37kmut5u35AxkpqKljFugLsjbof2dRj113gt5JPBloYOXf3JfW6EyuZ+99WYomPd4H2r57iOedX97lVdMuh/nKtMbExRbV0z1DemwMzL4eK3Dy2v/eltKClwPbQbU9wg/88l3FTF9gt0BEEr5CuQU+vbxyB7Myz4q2szf9RvXGsP472EKW4e4+0rq99m52rIS2t4k9L6atXeVdrFDYYZl7gx6itUTGvZc7yb19iYZi6kk0HF0NVe1xfkbHX1Ay8c66avXPiIq8g7/FZ/hhmchpzhehTX1Kpo/0Q2AUoG4J5irpjrmp2+ffEv04PWd1pLY5qokC4mytpbMXR1DckgL8MNM7BqthsfB9xAbSc96OoJ2lc5H4+pTeuOrh5g9Rw4/r6qi4rWf+I6Ivl56N5aRcfCFXPcJD7TL3L1CRXTWtbW2siYZiKkk0G1zUrzd7iL1KrZsPU7t6zrCNeZbNjZVm7pKwlnudnaMtceOslKQbYb7O/o3wYmtoO17gRXzoXXTndNT0sK4Oi7Aj+zljE+EtLJ4IChq/dmwZo5rrPRlm9d5564oW4ArWHnuGIC41tDzoQP73KJ9+BksPEz9xkMbKQmpd5o09kNevfa6W4E0bFBPlqmCSkhnQx2bN/OBeFf0m/+i64sWMug4wA3b+mwc9y8w8Z/Yrq4wedWzzl01q31H0PrOFck15TEdIVrP4XctAPnYTammQu9ZFCUC2vnwarZXLPxC1pElMLueO+ncDS+lTDFTcO4a8MvT19lpe7JYMgZTXPk1+jYxutXYEwjCZ1ksPlrN577xs+gbB+068W86Cl8G3U0j9zSCEMdmKoNPdMlg9VzXBk8uGkZi3ID24rImBDTBG+7/CT7Z8hY5uaUvfZTuGMlD5deSkmX0ZYIAqltd+g14cAmphvmux7KdZnIxhjTIKHzZDBqqhs/xlPssK+0jG2+GLraNNzQs+CT/3MJO7afa1LaZ5LrBWyMaRSh82TQIvKA8ue03Z6hqy0ZBF7CWe776rlurKfMNTCoirGAjDF+49dkICKTRWSdiGwUkXtq2O48EVERSfRnPJX5ZOhq4xvte0P3w1xRUcVENk2pSakxIcBvyUBEwoGngVOBBGCqiBzSVlNEYoDbgR/8FUtVUi0ZNC0JU1ydztLXILa//6aPNMZUyZ9PBuOBjar6s6oWAzOAqvruPwg8AhT5MZZDbM0qoGWLMOJiGnk0TFO1iqKiHcmNN3eBMWY/fyaDHkBqpddpnmX7ichhQC9V/dCPcVRpa0OHrja+FdsPuo50P1syMKbRBaw1kYiEAU8AV3mx7TRgGkDv3r198v5b6zt0tfGfw66ARU9B70mBjsQEmZKSEtLS0igqatQCiICIioqiZ8+eRETUbc5rfyaDdKDykJ49PcsqxADDgS89d+ddgbkicpaqJlU+kKq+ALwAkJiYqA0NTFVJzS5gYr+ODT2U8aXx17svY3wsLS2NmJgY4uPjg7o0QFXJysoiLS2Nvn371mlffxYTLQEGikhfEYkELgbmVqxU1VxV7aSq8aoaD3wPHJII/CF7bzF7i8vo09GeDIwJBUVFRXTs2DGoEwGAiNCxY8d6PQH5LRmoailwCzAfWAPMVNVVIvKAiJzlr/f1hjUrNSb0BHsiqFDf8/RrnYGqzgPmHbTsj9Vse6w/Y6nMkoExxhwodHogV1LRx6BnB0sGxhj/y8nJ4Zlnnqnzfqeddho5OTm+D6gKIZkMtmYX0DmmJa0iwwMdijEmBFSXDEpLS2vcb968ebRv395PUR0odAaqq2RLljUrNSZU/fl/q1idkefTYyZ0b8ufzhxW7fp77rmHTZs2MXr0aCIiIoiKiqJDhw6sXbuW9evXc/bZZ5OamkpRURG3334706ZNAyA+Pp6kpCT27NnDqaeeypFHHsmiRYvo0aMHc+bMoVWrVj47h5B8Mki1PgbGmEb08MMP079/f5YvX85jjz3GsmXLePLJJ1m/fj0Ar7zyCkuXLiUpKYmnnnqKrKysQ46xYcMGbr75ZlatWkX79u159913fRpjyD0ZVAxd3cuSgTEhqaY7+MYyfvz4A/oBPPXUU7z33nsApKamsmHDBjp2PLAfVN++fRk9ejQAY8eOJSUlxacxhVwySLehq40xAda6dev9P3/55Zd89tlnfPfdd0RHR3PsscdW2U+gZctfxlELDw+nsLDQpzGFXDHR/mal1uHMGNNIYmJiyM/Pr3Jdbm4uHTp0IDo6mrVr1/L99983cnROyD0Z2NDVxpjG1rFjR4444giGDx9Oq1at6NKly/51kydP5rnnnmPo0KEMHjyYiRMnBiTGkEsGW7M9Q1e3saGrjTGNZ/r06VUub9myJR999FGV6yrqBTp16kRycvL+5XfddZfP4wvJYqLesdGEhYVG13RjjPFGCCaDQisiMsaYg4RUMqgYutqalRpjzIFCKhnsLihhz75SezIwxpiDhFQy2JK1F7CWRMYYc7CQSgbWx8AYY6oWUsmgoo9BLxu62hjThLVp0waAjIwMzj///Cq3OfbYY0lK8t3EkCGVDLZmFxBnQ1cbY5qJ7t27M2vWrEZ5r5DqdLbVRis1xnx0D2xf6dtjdh0Bpz5c7ep77rmHXr16cfPNNwNw//3306JFCxYsWMDu3bspKSnhoYceYsqUKQfsl5KSwhlnnEFycjKFhYVcffXV/PTTTwwZMsTGJmqIVOtjYIwJgIsuuoiZM2fufz1z5kyuvPJK3nvvPZYtW8aCBQu48847UdVqj/Hss88SHR3NmjVr+POf/8zSpUt9GmPIPBkUl5aTkVtofQyMCXU13MH7y5gxY9i5cycZGRlkZmbSoUMHunbtyq9//Wu++uorwsLCSE9PZ8eOHXTt2rXKY3z11VfcdtttAIwcOZKRI0f6NMaQSQbpOTZ0tTEmcC644AJmzZrF9u3bueiii3jrrbfIzMxk6dKlREREEB8fX+XQ1Y3Fr8VEIjJZRNaJyEYRuaeK9b8SkZUislxEvhGRBH/FUtGstI81KzXGBMBFF13EjBkzmDVrFhdccAG5ubl07tyZiIgIFixYwJYtW2rc/+ijj94/2F1ycjIrVqzwaXx+SwYiEg48DZwKJABTq7jYT1fVEao6GngUeMJf8Wy1oauNMQE0bNgw8vPz6dGjB926dePSSy8lKSmJESNG8MYbbzBkyJAa97/xxhvZs2cPQ4cO5Y9//CNjx471aXz+LCYaD2xU1Z8BRGQGMAVYXbGBqlaelbo1UH3tSQN1iWnJSQldbOhqY0zArFz5SyumTp068d1331W53Z49ewCIj4/fP3R1q1atmDFjht9i82cy6AGkVnqdBkw4eCMRuRn4DRAJHF/VgURkGjANoHfv3vUK5uRhXTl5WNUVM8YYE+oC3rRUVZ9W1f7A74D7qtnmBVVNVNXEuLi4xg3QGGNCgD+TQTrQq9Lrnp5l1ZkBnO3HeIwxIaymNvzBpL7n6c9ksAQYKCJ9RSQSuBiYW3kDERlY6eXpwAY/xmOMCVFRUVFkZWUFfUJQVbKysoiKiqrzvn6rM1DVUhG5BZgPhAOvqOoqEXkASFLVucAtInIiUALsBq70VzzGmNDVs2dP0tLSyMzMDHQofhcVFUXPnj3rvJ80t0yZmJiovhypzxhjQoGILFXVxOrWB7wC2RhjTOBZMjDGGGPJwBhjTDOsMxCRTKDmQTyq1wnY5cNwmoJgO6dgOx8IvnMKtvOB4Dunqs6nj6pW21Gr2SWDhhCRpJoqUJqjYDunYDsfCL5zCrbzgeA7p/qcjxUTGWOMsWRgjDEm9JLBC4EOwA+C7ZyC7Xwg+M4p2M4Hgu+c6nw+IVVnYIwxpmqh9mRgjDGmCpYMjDHGhE4yqG0+5uZGRFIqzR/dLAdrEpFXRGSniCRXWhYrIp+KyAbP9w6BjLEuqjmf+0Uk3fM5LReR0wIZY12JSC8RWSAiq0VklYjc7lneLD+nGs6n2X5OIhIlIotF5CfPOf3Zs7yviPzguea94xk9uvrjhEKdgWc+5vXASbgZ15YAU1V1dY07NmEikgIkqmqz7SgjIkcDe4A3VHW4Z9mjQLaqPuxJ2h1U9XeBjNNb1ZzP/cAeVX08kLHVl4h0A7qp6jIRiQGW4uYduYpm+DnVcD4X0kw/JxERoLWq7hGRCOAb4HbcDJKzVXWGiDwH/KSqz1Z3nFB5Mtg/H7OqFuMm0pkS4JhCnqp+BWQftHgK8Lrn59dpRhMeVXM+zZqqblPVZZ6f84E1uCltm+XnVMP5NFvq7PG8jPB8KW4a4Vme5bV+RqGSDKqaj7lZ/wHgPuxPRGSpZ47oYNFFVbd5ft4OdAlkMD5yi4is8BQjNYvilKqISDwwBviBIPicDjofaMafk4iEi8hyYCfwKbAJyFHVUs8mtV7zQiUZBKMjVfUw4FTgZk8RRVBRV4bZ3MsxnwX6A6OBbcDfAxpNPYlIG+Bd4A5Vzau8rjl+TlWcT7P+nFS1TFVH46YXHg8MqesxQiUZ1HU+5iZPVdM933cC7+H+AILBDk+5bkX57s4Ax9MgqrrD849aDrxIM/ycPOXQ7wJvqepsz+Jm+zlVdT7B8DkBqGoOsAA4HGgvIhWzWdZ6zQuVZFDrfMzNiYi09lR+ISKtgZOB5Jr3ajbm8sv0p1cCcwIYS4NVXDA9zqGZfU6eysmXgTWq+kSlVc3yc6rufJrz5yQicSLS3vNzK1xDmTW4pHC+Z7NaP6OQaE0E4Gkq9k9+mY/5L4GNqP5EpB/uaQDcPNbTm+P5iMjbwLG44XZ3AH8C3gdmAr1xQ5VfqKrNolK2mvM5Flf0oEAKcEOlsvYmT0SOBL4GVgLlnsW/x5WzN7vPqYbzmUoz/ZxEZCSugjgcd4M/U1Uf8FwnZgCxwI/AZaq6r9rjhEoyMMYYU71QKSYyxhhTA0sGxhhjLBkYY4yxZGCMMQZLBsYYY7BkYEyjEpFjReSDQMdhzMEsGRhjjLFkYExVROQyzxjxy0Xkec9AYHtE5B+eMeM/F5E4z7ajReR7zyBn71UMciYiA0TkM88488tEpL/n8G1EZJaIrBWRtzy9Yo0JKEsGxhxERIYCFwFHeAb/KgMuBVoDSao6DFiI62EM8AbwO1UdievZWrH8LeBpVR0FTMINgAZupMw7gASgH3CEn0/JmFq1qH0TY0LOCcBYYInnpr0VbiC2cuAdzzZvArNFpB3QXlUXepa/DvzXM3ZUD1V9D0BViwA8x1usqmme18uBeNyEJMYEjCUDYw4lwOuqeu8BC0X+cNB29R3LpfL4MGXY/6FpAqyYyJhDfQ6cLyKdYf98v31w/y8Vo0BeAnyjqrnAbhE5yrP8cmChZxatNBE523OMliIS3ZgnYUxd2B2JMQdR1dUich9uJrkwoAS4GdgLjPes24mrVwA3PPBznov9z8DVnuWXA8+LyAOeY1zQiKdhTJ3YqKXGeElE9qhqm0DHYYw/WDGRMcYYezIwxhhjTwbGGGOwZGCMMQZLBsYYY7BkYIwxBksGxhhjgP8HpAjexf8wpJgAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEWCAYAAABv+EDhAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA6oUlEQVR4nO3dd3xV9f348dc74ya52QsIgTBlyApTFAeKA1EcFQRX1Vqx1dbV1mq/HXbbX7fWPSouEBDqXgg4cbDDFEEgYQWy9/z8/vgcRiAJGXck976fj0ceufes+zlcct7nfMb7I8YYlFJKBZ8QfxdAKaWUf2gAUEqpIKUBQCmlgpQGAKWUClIaAJRSKkhpAFBKqSClAUCpZojIsyLyhxZuu0NEzm3vcZTyFQ0ASikVpDQAKKVUkNIAoDo9p+rlZyKyTkTKRORpEekqIm+LSImILBaRxKO2v0RENohIoYgsE5HBR60bKSKrnP1eBiKP+ayLRWSNs+9nIjK8jWW+WUS+EZF8EXlNRLo7y0VE/ikiuSJSLCJZIjLUWTdFRDY6ZdstIj9t0z+YUg4NACpQXAGcBwwApgJvA78AUrH/z28HEJEBwBzgTmfdW8DrIuISERfwP+B5IAmY7xwXZ9+RwDPALUAy8DjwmohEtKagInIO8GfgSiAN2AnMdVafD5zpnEe8s02es+5p4BZjTCwwFFjSms9V6lgaAFSgeMgYs98Ysxv4GPjCGLPaGFMJLAJGOtvNAN40xrxvjKkB/gZEAacB44Fw4F/GmBpjzALgq6M+YxbwuDHmC2NMnTFmNlDl7Nca1wDPGGNWGWOqgPuAU0WkN1ADxAKDADHGbDLG7HX2qwFOFpE4Y0yBMWZVKz9XqQY0AKhAsf+o1xWNvI9xXnfH3nEDYIypB7KBdGfdbtMwQ+LOo173An7iVP8Uikgh0NPZrzWOLUMp9i4/3RizBPgP8DCQKyJPiEics+kVwBRgp4h8KCKntvJzlWpAA4AKNnuwF3LA1rljL+K7gb1AurPskIyjXmcDfzTGJBz14zbGzGlnGaKxVUq7AYwxDxpjRgMnY6uCfuYs/8oYcynQBVtVNa+Vn6tUAxoAVLCZB1wkIpNEJBz4CbYa5zNgOVAL3C4i4SLyHWDcUfs+CfxARE5xGmujReQiEYltZRnmADeKSKbTfvAnbJXVDhEZ6xw/HCgDKoF6p43iGhGJd6quioH6dvw7KKUBQAUXY8wW4FrgIeAgtsF4qjGm2hhTDXwHuAHIx7YXLDxq3xXAzdgqmgLgG2fb1pZhMfAr4BXsU0c/YKazOg4baAqw1UR5wF+dddcBO0SkGPgBti1BqTYTnRBGKaWCkz4BKKVUkNIAoJRSQUoDgFJKBSkNAEopFaTC/F2AlkhJSTG9e/f2dzGUUqpTWbly5UFjTGpT6ztFAOjduzcrVqzwdzGUUqpTEZGdza3XKiCllApSGgCUUipIaQBQSqkg1SnaAJRSqrVqamrIycmhsrLS30XxusjISHr06EF4eHir9tMAoJQKSDk5OcTGxtK7d28aJngNLMYY8vLyyMnJoU+fPq3aV6uAlFIBqbKykuTk5IC++AOICMnJyW160tEAoJQKWIF+8T+krecZ0AFg0eocXvi82W6wSikVtAI6ALy5bp8GAKWUXxQWFvLII4+0er8pU6ZQWFjo+QI1IqADQEqMi7yyan8XQykVhJoKALW1tc3u99Zbb5GQkOClUjUU0L2AkqJdFJRVU19vCAkJjrpApVTHcO+997Jt2zYyMzMJDw8nMjKSxMRENm/ezNdff81ll11GdnY2lZWV3HHHHcyaNQs4kvqmtLSUCy+8kNNPP53PPvuM9PR0Xn31VaKiojxWxoAOAMkxEdTWG4ora0hwu/xdHKWUn/z29Q1s3FPs0WOe3D2O30wd0uT6Bx54gPXr17NmzRqWLVvGRRddxPr16w931XzmmWdISkqioqKCsWPHcsUVV5CcnNzgGFu3bmXOnDk8+eSTXHnllbzyyitce+21HjuHgK8CAjhYqtVASin/GjduXIN++g8++CAjRoxg/PjxZGdns3Xr1uP26dOnD5mZmQCMHj2aHTt2eLRMgf0EEB0BQF5pFf27xPi5NEopf2nuTt1XoqOjD79etmwZixcvZvny5bjdbiZOnNhoP/6IiIjDr0NDQ6moqPBomQL6CSAp2j4B5GtDsFLKx2JjYykpKWl0XVFREYmJibjdbjZv3sznn3/u49JZAf0EcLgKSAOAUsrHkpOTmTBhAkOHDiUqKoquXbseXjd58mQee+wxBg8ezMCBAxk/frxfyhjQASDReQLIK63yc0mUUsHopZdeanR5REQEb7/9dqPrDtXzp6SksH79+sPLf/rTn3q8fAFdBRQeGkKCO5w8bQRWSqnjBHQAANsOoG0ASil1vIAPACnRERzUKiCllDpOwAeAZE0HoZRSjQqKAKBVQEopdbyADwBJ0REUlFdTW1fv76IopVSHEvABICXGhTFQUF7j76IopVSTYmJstoI9e/Ywbdq0RreZOHEiK1as8NhnBnwAOJwOokwbgpVSHV/37t1ZsGCBTz4r8AOAMxo4X8cCKKV86N577+Xhhx8+/P7+++/nD3/4A5MmTWLUqFEMGzaMV1999bj9duzYwdChQwGoqKhg5syZDB48mMsvv9zjuYACeiQwQHK0poNQKui9fS/sy/LsMbsNgwsfaHL1jBkzuPPOO7ntttsAmDdvHu+++y633347cXFxHDx4kPHjx3PJJZc0Oafvo48+itvtZtOmTaxbt45Ro0Z59BQCPwDEHMkIqpRSvjJy5Ehyc3PZs2cPBw4cIDExkW7dunHXXXfx0UcfERISwu7du9m/fz/dunVr9BgfffQRt99+OwDDhw9n+PDhHi1jwAeAhKhwQgRNB6FUMGvmTt2bpk+fzoIFC9i3bx8zZszgxRdf5MCBA6xcuZLw8HB69+7daBpoXwn4NoCQECEpOkIHgymlfG7GjBnMnTuXBQsWMH36dIqKiujSpQvh4eEsXbqUnTt3Nrv/mWeeeTih3Pr161m3bp1HyxfwTwBg2wG0Ckgp5WtDhgyhpKSE9PR00tLSuOaaa5g6dSrDhg1jzJgxDBo0qNn9f/jDH3LjjTcyePBgBg8ezOjRoz1avuAIAJoOQinlJ1lZRxqfU1JSWL58eaPblZaWAnZS+ENpoKOiopg7d67XyhbwVUBgG4L1CUAppRoKjgAQrU8ASil1rKAJACWVtVTV1vm7KEopHzLG+LsIPtHW8wyOAOCMBdCsoEoFj8jISPLy8gI+CBhjyMvLIzIystX7Bk0jMNixAGnxUX4ujVLKF3r06EFOTg4HDhzwd1G8LjIykh49erR6P68HABEJBVYAu40xF4tIH2AukAysBK4zxnj11jzlUADQJwClgkZ4eDh9+vTxdzE6NF9UAd0BbDrq/V+Afxpj+gMFwE3eLkBStKaDUEqpY3k1AIhID+Ai4CnnvQDnAIdync4GLvNmGaBhFZBSSinL208A/wLuAQ5Nx5UMFBpjap33OUB6YzuKyCwRWSEiK9pbhxcbEYYrNISDOieAUkod5rUAICIXA7nGmJVt2d8Y84QxZowxZkxqamp7y2LnBtYnAKWUOsybjcATgEtEZAoQCcQB/wYSRCTMeQroAez2YhkOS9LBYEop1YDXngCMMfcZY3oYY3oDM4ElxphrgKXAoQkvrweOnxLHCzQdhFJKNeSPgWA/B+4WkW+wbQJP++JDU6JdHNQqIKWUOswnAcAYs8wYc7HzersxZpwxpr8xZroxxie35ckxLh0JrJTqPIr3wht3Qan3BrIFRSoIsGMBKmrqKK+uPfHGSinlbx/8Fla/ANWlXvuIoAkAOhZAKeU1lcWwbj7UeegGM2cFrJ0Dp/4Ikrw3mjloAsChdBAHtSFYKeVpb/0MFn4flj/U/mPV18PbP4eYrnDG3e0/XjOCJgAkR2tGUKWUF2xdDOvmgjsZlv4ZDm5t3/Gy5sPuFXDu/RAR65EiNiVoAkBStFYBKdVh7PgUNr8JtZ38ibyqBN64E1IGwqxlEB4Jr/3Y3sW36XilsPg30H0UDJ/pyZI2KijSQcORNgBNB6GUnxXshBeugNoKiEyAIZfDiJnQ8xQQ8XfpWueD30FRDnzvXUjIgAv+BK/eBiuehnE3t/54n/4LSvbC9NkQ4v3786B5AnC7wnC7QvUJQCl/Mgbe+ilICFzxNJx0HqydC89cAP8eAUv+0P4qFF/Z9Tl8+SSMmwUZp9hlmddA37Nh8f1QmN264xXugs8egmHTjxzPy4ImAICOBVDK7zb+D7a+B+f8EoZNgyuegp9thcsfh6S+8PHf4T9j4Imz4fPHvNoHvl1qKm1VT3wPmPTrI8tFYOq/baB74y77u6Xe/zUgtu7fR4KmCgjsWADtBaQCWmURhEbYuuiOprLI9m5JG2Hvmg+JiLVVQCNm2sFP61+xjarv/Bze/QX0ORNSToK47hDb3f6O6w6xaeBy++dcPv4bHPwarn0FImIarkvsZYPCOz+HdfNgxIwTH2/Hp7BhEUz8hQ0qPhJUASAl2sXeokp/F0Mp7zAGnr0Y6mrg5g/AFe3vEjX0we+h7ABc/TKENnHpiUuD035kf/ZvhKx58PV7sGeVDSDHikyAuHQnKKTBoIthwAVePQ32rYdP/gkjroL+5za+zbibbSB75+fQ72yI6dL08err4J17Ia4HnPZj75S5CUEVAJJjXGzYU+zvYijlHTkrYN86+/qNu+HyxzpOo2rOCvjqKTjlB9B9ZMv26XoydL3/SJVIValtIC3e4/zsPur9btt1ctVzMOZ7cP4fvfN0UFcLr/3IBp4L/tT0diGhcMlD8PgZ8PY9MP3Zprdd/YL93q542udPNEEWACLIK6vCGIN0lD8MpTxl9fMQ7oaxN9nGxN4TYNR3/V0q+0Ty+h22yuac/2v7cSJiIOIkWx3UmNpqWPJ7+OxB2PmZvaB2G9r2z2vMF4/CntUw7RlwJzW/bZdBcOY9sPQPMHQaDL74+G0qi2yZe46HoVd4tqwtEFyNwNEuauoMxZWaD6hDaU1DmWpcdRmsX2i7VJ77W9sT5a2f2eoKf/v8Udi/Hqb81bsDm8JccP7v4bpFUFEAT54DXzzuuf9f+dthyR9h4BQY8p2W7XP6ndB1KLz5E6goPH79R3+FsoNw4QN+eVoLrgBwOB+QNgR3CMV74OnzYdEP/F2Szm/jq1BdAiOvtdUP33kSohJh3ndtnhp/KdgJy/4MAy9q/A7YG/qdAz/4FPqeZatf5sy0F9n2MAZeux1Cw+Giv7f8Yh0aDpf+B8py4f1fNVyXt832dBp5TcurxTwsuAKApoPoOPastndo2V/Yoe/t/QMNdqtfgKR+kHGqfR+TaqtACnbY6hd/PGUd6vOPwJT/59vPjkmFq+fB5Adg2xJ4dAJsX9b24616DnZ8DOf9zjY4t0b3kbZxd9VzDcvw3i8hLBLO+XWTu3pbcAWAwwnhNAD41cbX4JkLISQcvvMUmDrbP1y1Td422Pmpvfs/+s609wSY9CvYsNA2wPraxleP9Pn3YdfGw0Rg/A/h+x9AZBw8dxm8/xvbJtEaxXvhvV9Br9Nh1PVtK8vE++w4h9dut9V125bAlrfgzJ9CbNe2HdMDgisAOE8AeZoOwj+MgY//AfOus41zN39gBwOlDoKsV/xduua98wuYfYkNXp5K+espa160I2tHXHX8utPugJPOt/3p96z2XZma6vPvD2nDbZ6e0dfbVAtPn2/r81vi0FNMXRVc8mDb0zOER9leQYU7YfFv7f+nxD42QPlRUPUC0oRwflRbBa/fCWtfsj0iLn34yGClodNsT4miHP/cKZ5IeT58+YS9o/z2Q4jvCWO/b3vYnKgniLfV1cKal6D/ebYf/LFCQuwo28fOgHnXwy0fQVSC98v1we9tvffVc5vu8+9Lrmg7QrffOXYE73/GQnSqbSeJTLD/JodfJ9r3kQm2e+nmN2zDenK/9pWh9+m2i+qXj9v3M16EsIj2HbOdOsA34zuusBDiIsO0DcDXyvLg5Wth12d2pONZ9zSsqhh2hQ0A6xfChNv9V86mbHoN6mvg5iW2OuCLx2zGxmUPwPAr4ZRboOsQ/5Rt2xLbF/7CZurY3Um2H/p/J9tEZTNe8G6Pk8N9/m/xW+Nmk06+1GbaXPGMHZRWUWCfVgp3wd519n1NWcN90jLtxCyecO5v4ZvFkDIABl3kmWO2Q1AFAICUGE0H4VMHtsBLV9oL5xVP2yqfYyX1hfTRtjG4IwaArAWQ3N9eONLF9mbZv8F2MVw3D1bNht5n2AvewCm2F46vrH4e3CkwYHLz2/Ucaxsw3/0FfP4InHqbd8pzdJ//s9vR59+bEnrCub9pen1ttQ0Kh4JDt6Gee4qJjINbP7fpOjrAWKSgCwBJ0S6tAvKVbUtg3g22f/YNb9qLUFOGToN377OZIJsa6OMPxXtgxycw8d6Gf7Bdh9g64XPvt707vnrKPuXEZ8Aps2D8rd4PBGUHYcvbNvCEuU68/fhb7QCp938NPcY1/32AvRDmboC9a6G63E544k6yP1FJ9n1EbMN/l0N9/me8YC92nVGYy/Yiikn1zvE7UIqOoAsAyTEuvj1YduINVft89bQdiJQ6yNYDJ2Q0v/2Qy+3dadYCOPs+35SxJTYsAowNUI1xJ9nBPqf+yPbq+OJx272vrsbr0/mxbp6tmsq8pmXbi9i2l8fPhPk3wA8+PtKGUVcDuZtsQ/HeNfb3/g1Qd4KbpZBwJygk25+cFfYpaJCP+vyrdgnCABDByp0F/i5GYNv0Brx5t+19csXTLbsTjEuzjWTrFxx/t+1PWfNtT5aU/s1vFxoGJ19if16+zrYPDLoIUgd6p1zG2Oqf9NE2Z05LRSXAlbNtT5iXr7PpCvasgX1ZtqcLQEQ8dB9he6h0H2nrwCPjbZVIeZ5tFC/Psz8V+Ucty7c9bqb8teN8f6pZQRcAUqLtnAB19YbQEP1P6nG1VfYOOHUwzJzTurrTYdNs/fHetdA902tFbLG8bfZO+Lzft26/i/5uBw29epudKcobVUF7VkHuRrj4n63ft/tIO0DqzbuP/FuPu9ku7z7Sdk9srLujO6n9PWFUhxJ0ASAp2kW9gcLyapJj/NsFKyB9+QQUfAvXLmx9w9ngS+DNn9q77o4QANa/Akjrk3TFdLG9chbebOvET/NQD5KjrX4BwqLankBs7E22miY61SdTD6qOKei++UMX/TztCup5ZQfhw7/aPun9J7V+f3eSza++YVHbJ9X2FGNsIOp1GsSnt37/YdNhwIU202PeNs+WrbrctpWcfKmtmmmr2K568Q9yQfftH0kIpwHA45b9GapL4YI/tv0Yw6bZwTe7lnuuXG2xL8vO+NTWO2wRuPgftrvfaz/2bEDb/AZUFdvUD0q1Q9AFgJQYTQfhFbmbYcV/7UjH9jR8DrzQ5rRfv8BzZWuL9QsgJAxOvqztx4jrDpP/ZPP0eDIXz6rnILE39JrguWOqoBR0AUDTQXjJe78EV4xNetUermgbBDb8r/VJuzylvt6OSu53DkQnt+9YmddAv0mw+H6bmbO98r+1DcyZ12r1jWq3oPsflOh2IaJzAnjUN4vhm/fhrJ+1/4IJtv68Ir996XvbI/sLKMpuuu9/a4jYHDQSYjNBtjct85qXAIHMRhK/KdVKQRcAQkOEJLdLG4E9pa4W3v2l7TroqayP/SbZRFxZ8z1zvNZav8DmaR80xTPHS+gJ5//OJpJb+Wzbj1Nf5yR+m9Qxk+apTifoAgDYhmCtAvKQVbPhwCabZ8ZTmQ3DXHZA1eY3bY8XX6qrsdVPAyZ7dvrCUTfYfEHv/cpmPW2L7cugOEcbf5XHBGUASIp2aSOwJ1QWwdI/2YkyBk/17LGHTrM9ira+69njnsj2D6H8oK2G8qSQEJsP3tS1fYau1c/bVMUDPfRkooKe1wKAiESKyJcislZENojIb53lfUTkCxH5RkReFpEWZLHyrOSYCH0C8ISP/mbTAFzwR88P/e99OsR0s/3dfWn9ApsK4aTzPH/spD4w6Te2zWTNS63btzzfPhENn+H3HPIqcHjzCaAKOMcYMwLIBCaLyHjgL8A/jTH9gQLgJi+WoVEp0doG0G7539q8+JlXe2fUbkioTRC39X2oKPT88RtTU2HzGA2e6r2L7LhZ0HO8zXxavLfl+2XNt4nZtPpHeZDXAoCxSp234c6PAc4BDt3WzQYu81YZmpIcE0FRRQ3VtX4ebdqZLf6N7Sd/zq+89xnDptsEZZvf8N5nHG3re1Bd0vicBZ4SEmIzctZW2Vw8La0KWv28TcrWbZj3yqaCjldzAYlIKLAS6A88DGwDCo0xhyZVzQEaHWcvIrOAWQAZGSdIJdxKh8YCFJRX0zUu0qPHDgo7P7MTfk/8RePTEHpK+ijbuyhrgW/ufLPmQ3QX6HOmdz8npb+dLOX9X9kgEBkPNZVQU26fQmrKobbyyOuaCjiwGab8zbvlUkHHqwHAGFMHZIpIArAIGNSKfZ8AngAYM2ZMOztPN5TipIM4WFqlAaC16uvhnfsgtjuc9mPvfpY4idg++QeU5toka95SWQRfvwejb/DNjF6n3gbbl9qpCUPC7ejn8Eg7eXi42/4Oi7KzfYVHQcapMGKm98ulgopPsoEaYwpFZClwKpAgImHOU0APYLcvynC0QwnhdG7gNsiaZycMufxxcLm9/3nDpsHHf7NdM0/x0DiDxmx+01Y3ebP652ghoTZjan1dx5g0XQUlb/YCSnXu/BGRKOA8YBOwFDj0V3Y98Kq3ytCUZE0H0TbVZbD4tzZn/LArffOZXQZDlyEtzw1UtNumYM5a0Lqullnz7axlPU4wTaIniejFX/mVN//3pQGznXaAEGCeMeYNEdkIzBWRPwCrgae9WIZGJUfbJwCdHL6VPvsPlOyBac/4Ng/NsGnwwW+hYCck9jp+fXm+bZPIWmATr+Fc+Fc9Z9MwJPVp/vilB2z//wl36ExWKqh4LQAYY9YBIxtZvh0Y563PbYm4qDDCQkS7grZGXa3NaDlgMvQ61befPfQKGwDWv3Jknt2qUjshetZ82PYB1NdC8kk2Gd3Q79iEae//Bh45Fc75Pzjlh03fbW/8nx2g5avqH6U6iKB8/hQRkmNc5GsVUMvt+BjKcm2/f19L7AU9xtk7/NRBtjpoy9u2h0xcOoy/1V68uw0/cgefcpINVm/+xGYqXf8KXPIf6Db0+ONnLbBTWHYd4tvzUsrPgjIAgK0G0nQQrbB+Abhi7UTv/jBsGrx9D8y9CqKSYMRVdlnP8U1XR8V1h5kv2RnG3r4HnjjLVvOceY/tcQNQuAuyP/fueAalOqjgDQAxLg7qE0DL1FbBxtdh8MW2S6I/ZF5tu4L2PAX6nQ2h4S3bT8RWCfWdCO/+Aj7+O2x8zebl6XWqM+8vbZ/5S6lOLHgDQLSLnXk+zjTZWW19H6qK/FtHHhELk9pxl+5Ogssfs+fw+l3w38kw9vt2UFv6mBM3FCsVgIIyGygcSginVUAtkjXfDkjqM9HfJWm//ufCrctto/BXT0PuRm38VUEreJ8AYlyUVddRUV1HlMsHIz87q6oS+PodGHld4PRZj4iBCx+wF/41L9n2BKWCUID8Rbfe4cFgZVX08MWI1s5q85s2L00g3iX3GGN/lApSwVsFFK3pIFokawHEZ9humEqpgBK8ASBG00GcUNlB2LbE9qLx5chfpZRPBO1fdUqMpoM4ocMjZD08PaJSqkMI2gCQdLgNQJ8AmnRo5K2OkFUqIAVtAHC7QokMDwncNoAdn8C3H7V9/8Js2LXcNv5qgjSlAlLQ9gISEZKjIwKzCmjnZ/DcZXa07G1fQkLP1h9jw0L7W0fIKhWwWvQEICJ3iEicWE+LyCoR8VNSGM9JiXF1jEbg3E2we6VnjpW/HeZeA/E97Pu3f96242TNd0bI9vVMuZRSHU5Lq4C+Z4wpBs4HEoHrgAe8ViofSYp2+T8hnDH2gv30+TZHTXtUFMJLM8HUw7WvwMR7Ycubti9/axzYAvuyArPvv1LqsJYGgEOVwFOA540xG45a1mklx0T4PyX0rs8hfxtEJcL8G2zmyraoq7X752+DGS9Acj+bJrnLEHjrHps/v6WyFoCEwJDL21YWpVSn0NIAsFJE3sMGgHdFJBao916xfCM5xsXBsmpMa6YO9LQ1L4ArBn7wqZ2OcMFNRzJUtpQxNt3x9qVw8T+hzxl2eWi4fV+cAx+28IHNGJv6ufcZENutdeVQSnUqLQ0ANwH3AmONMeVAOHCj10rlIynREVTX1lNaVeufAlSX2cnOh1wGsV3h2gU23fEr34d181t+nC+fgBVPw2m3w6jvNlyXcQqMuh6WP2KrdU5kzyrbjqDVP0oFvJYGgFOBLcaYQhG5FvglUOS9YvlGkr8nh9/4GlSXQua19n1ELFwzHzJOg0WzYO3cEx9j6/vwzr0w8CI49/7Gtzn3flvF9MZdUH+CB7esVyDUBYOntuZMlFKdUEsDwKNAuYiMAH4CbAOe81qpfORwOgh/jQVY86LtZZMx/siyiBi4Zh70Ph0W/QBWv9j0/vs3wvwb7UCt7zwBIU1kNXUnwQV/hJyvYNWzTR+vvs5WP/U/zwYMpVRAa2kAqDW2ovxS4D/GmIeBWO8VyzcOpYPwy7wA+d/aeXYzrz5+oJUrGq56GfqeBa/eBqueP37/0gPw0owj20bENP95w2fYev3F99uZtRqz81Mo3afVP0oFiZYGgBIRuQ/b/fNNEQnBtgN0an59Alg7B5Cmc9G73HDVXDv94Ws/gpXPHllXUwlzr4ayA3DVHIhPP/HnidgG4ZoKO0l6Y7Lm2wbpAZNbezZKqU6opQFgBlCFHQ+wD+gB/NVrpfKRI20APn4CqK+HNXPsxf3QgK3GhEfBzDm2Sub1O+wMVsbYgJDzpZ3iMH1Uyz835SSYcCesexm2L2u4rrbatkkMusgGH6VUwGtRAHAu+i8C8SJyMVBpjOn0bQARYaHERoT5/glgx8dQtAsyrznxtuGRMPNFOOkCePNueOE79k79nF/a3kOtdcbdkNgH3rjbPkkcsu0DqCyEoVr9o1SwaGkqiCuBL4HpwJXAFyISEFeKZH+kg1jzIkTEw6CLW7Z9WATMeB4GTrH5+YfPgDN+2rbPDo+Ci/5uB4x9+q8jy7PmQ1SSfSpRSgWFliaD+z/sGIBcABFJBRYDC7xVMF9JjonwbTqIyiJb1ZJ5lb27b6mwCJg+G75ZDP0ntS9DZ/9JNsnbx3+3uf5jusKWt2HETDt4TCkVFFraBhBy6OLvyGvFvh1aUrSPnwA2LILaiiN9/1sjzAWDpthg0F4X/AnCIm210pa3oaZcq3+UCjItfQJ4R0TeBeY472cAb3mnSL6VEuNiTXah7z5w9Yt2kpXWNN56Q2w3mPRreOunNhtpXDpknOrfMimlfKqljcA/A54Ahjs/Txhj2phn2IeK98Ce1c1ukhwdQX5ZNfX1PsgHdHCr7b2TeU3HmGRlzPeg+ygo3a/z/ioVhFo8IYwx5hWglVnK/MgYmx2zcBfc8hHEdGl0s+QYF3X1hqKKGhKdbqFes+ZFkFDbiNsRhITCJQ/a3EMjv3vi7ZVSAaXZWz4RKRGR4kZ+SkSk2FeFbBMR29ulogAWfM+mS27EkbmBvdwQXF9nc/ucdJ5N/NZRdBsGt30BqQP8XRKllI81GwCMMbHGmLhGfmKNMXG+KmSbdRtmR7/u+BiW/rHRTY6kg/ByQ/C2JVCyt2V9/5VSygcCv9I382oYfQN88g/YfHy7tc/SQax+AdzJmmZBKdVheC0AiEhPEVkqIhtFZIOI3OEsTxKR90Vkq/Pb+2knJ/8F0kbY7Jr53zZYlRzdyoRwtdUNR9C2RHk+bHkLhl1pu3IqpVQH4M0ngFrgJ8aYk4HxwG0icjJ2YpkPjDEnAR84770rPBKufM62C8y7ziZEcyS67cCngy2pAsrdDI+Mh3+PgG8+aPnnr38F6qrt04hSSnUQXgsAxpi9xphVzusSYBOQjk0pPdvZbDZwmbfK0EBib5szf18WvPWzw4vDQkNIdIeTf6IqoM1vwlOToKoEIuNsTp537mvZ08DqF2x7RNrw9p2DUkp5kE/aAESkNzAS+ALoaozZ66zaB/iuS8yAC+DMn8Hq5xvk2G82HUR9PSz7i02/nHISzFoGsz6EsTfD54/Ak+fYiVmasn8D7F3TtpG/SinlRV4PACISgx0/cKcxpkHXUWeSmUZHYInILBFZISIrDhw44LkCTbwP+k60I2D3rgUgOdrVeBVQVQnM/y4s+xMMnwk3vm1z77vccNHf4Op5UJYLT0yEzx9tfLrFNS9BSLjNuaOUUh2IVwOAiIRjL/4vGmMWOov3i0iasz4NaHR6KmPME8aYMcaYMampqZ4rVEgoXPG07ZEz77tQUUByjOv4KqD87fDUebbq54I/29z74VENtxlwAfzwMxtQ3rkXXpwGJfuOrK+rsbn3B06G6GTPnYNSSnmAN3sBCfA0sMkY84+jVr0GXO+8vh541VtlaFJ0Ckx/FopyYNEPSXGHN+wFtG0pPHG27bd/7UI49damUzfEdIGrX7aDznZ+Co+eZoMGwNb37KxdWv2jlOqAvPkEMAE7heQ5IrLG+ZkCPACcJyJbgXOd977Xc5zNiPn125xfOJeC8hpqa+tg+cO2gTeuO8xa2rL8+CIw9vs25URcd9te8PodsOIZm2q5/7nePx+llGqlFucCai1jzCdAUxnPJnnrc1tl3CzI/oLT1j/KxBA3NQsXEbZxPgyeCpc9duKJ1o+VOhC+/wEs+QN89hBg4LTbIdRr/8xKKdVmwX1lEoGpD1K2cw3PlvwVNgJn/5+dbautmTHDIuD839u7/s8fgXE3e7TISinlKcEdAAAiYth+9qNU/u92YibewZCzPDRYq+9Z9kcppTqowM8F1ALpAzKZUf1rPgk9xd9FUUopn9EAgM0I2ivZzcqdBf4uilJK+YwGAMeojERW7SrEjk1TSqnApwHAMSojgYOlVeQUVJx4Y6WUCgAaABwjM2xW6lW7tBpIKRUcNAA4BnWLxe0KZZW2AyilgoQGAEdYaAjDe8Szalehv4uilFI+oQHgKKMyEtm0t5iK6jp/F0UppbxOA8BRRmUkUltvWJdT6O+iKKWU12kAOMrIjAQArQZSSgUFDQBHSY6JoHeyW3sCKaWCggaAY4zKSGT1rgIdEKaUCngaAI4xslciB0uryc7XAWFKqcCmAeAYo3VAmFIqSGgAOMbAbrFEu0I1ACilAp4GgGOEhggjeiZoAFBKBTwNAI2wA8JKKK+u9XdRlFLKazQANGJUrwTq6g1rs4v8XRSllPIaDQCNGNlTG4KVUoFPA0AjEqNd9E2JZrUGAKVUANMA0ISROkOYUirAaQBowqheCeSXVbMzr9zfRVFKKa/QANCEUTogTCkV4DQANGFA11hiIsI0ACilApYGgCbYAWHxrNpZ6O+iKKWUV2gAaMaojEQ27yumrEoHhCmlAo8GgGaMykik3sBanSFMKRWANAA049AMYat1hjClVADSANCMBLeLfqnRrNqpDcFKqcCjAeAERmUksjpbB4QppQKPBoATGNUrkfyyanbogDClVIDRAHAChweEaTWQUirAeC0AiMgzIpIrIuuPWpYkIu+LyFbnd6K3Pt9TTuoSQ6wOCFNKBSBvPgE8C0w+Ztm9wAfGmJOAD5z3HVpIiJCZkcBKfQJQSgUYrwUAY8xHQP4xiy8FZjuvZwOXeevzPWlkRiJf7y+hVAeEKaUCiK/bALoaY/Y6r/cBXZvaUERmicgKEVlx4MAB35SuCaMyEuyAsOxCv5ZDKaU8yW+NwMb2q2yyb6Ux5gljzBhjzJjU1FQflux4h2cI02ogpVQA8XUA2C8iaQDO71wff36bxLvD6d8lRhuClVIBxdcB4DXgeuf19cCrPv78NhuVkaADwpRSAcWb3UDnAMuBgSKSIyI3AQ8A54nIVuBc532nMCojkcLyGrYfLPN3UZRSyiPCvHVgY8xVTaya5K3P9KZRvY60A/RLjfFzaZRSqv10JHAL9U+NITYyjFWaGVQpFSA0ALRQSIiQ2TOB1doQrJQKEBoAWmF0r0S27C+hpLLG30VRSql20wDQCqMyEjEG1mYX+bsoSinVbhoAWiEzIwERdDyAUiogeK0XUCCKiwznpC4x/PfTb1m+LY/E6HAS3S4S3S4S3OEkRdvXidEuEt3hJMdEEBOh/8RKqY5Jr06tdNe5A1i4ejeF5dVs2VdCYXkNBeXV1DcyPixE4Jaz+nH3eQMID9WHLaVUx6IBoJUuHJbGhcPSGiyrrzeUVNaSX15NQXk1BWXVFJTXsHxbHo8u28bybXk8OHMkGcluP5VaKaWOJ50htcGYMWPMihUr/F2MNnlz3V7uXbgOY+CPlw/l0sx0fxdJKRUkRGSlMWZMU+u1XsLLLhqextt3nMHAbrHcMXcNP5u/ljKdV0Ap1QFoAPCBHoluXp41nh+f058Fq3KY+tAnrN+tXUmVUv6lAcBHwkJD+Mn5A3np++Mpq67l8kc+5amPt7cou2hheTVLt+Tyz/e/5sdzVvPfT78lp6DcB6VWSgUybQPwg/yyau5ZsI7Fm/Zz9sBU/jp9BCkxEQDU1NWzeW8Jq7MLWLOrkNXZhXzrZCANEegSG8m+4koAhnSP4/yTu3H+kK4M6haLiPjtnJRSHc+J2gA0APiJMYbnP9/JH97cRHxUOBcNS2P97iKydhdRVVsPQEpMBCMzEhiZkUBmzwSG90ggJiKMbw+W8f7Gfby3YT8rdxVgDGQkuTn/5K6cP6Qbo3slEhqiwUCpYKcBoIPbtLeYO+eu4du8MoZ2j2NkRiKZPe1FPz0h6oR39bkllXywKZf3Nuzj02/yqK6rJynaxaRBXZg5riejeyV5pJyVNXX89d0thAjcM3mQjmtQqhPQANAJGGOoqzeEtfOiWlpVy4dbDvDexn0s2ZRLaXUtt5xpB6K5wtp+7G0HSrntxVVs3lcCwIT+yTxy9Wji3eHtKq9Syrs0AASp8upafvf6RuZ+lc2w9Hj+PTOTvm2YyObVNbu5b2EWEWEh/GNGJnml1dy3cB09k9z894ax9EqO9kLplVKeoOMAgpTbFcYDVwznsWtHsSu/nIse/ISXv9rV4jmNK2vquG/hOu6Yu4Yh3eN4644zOHtgF6aN7sELN51Cflk1lz38KV9+m+/lM1FKeYsGgAA3eWga79x5BiMzEvj5K1nc+uIqCsurm91n24FSLnv4U+Z8mc2tE/sx5+bxpMVHHV5/St9kFt06gUS3i2uf+oJFq3O8fRpKKS/QABAE0uKjeOGmU7jvwkEs3rSfyf/6mM+2HWx02/+t3s3Uhz4ht6SKZ28cyz2TBzXaNtEnJZqFt57G6F6J3PXyWv7+3hbqG8uIp5TqsDQABImQEOGWs/qx8IcTcLtCueapL3jg7c1UO11OK2vquPeVddz58hqGdo/nrdvPYOLALs0eM8HtYvb3xjFjTE8eWvINP567msqaOl+cjlLKA7QROAiVV9fy+zc2MudL20B893kD+Ms7m9m8r4Tbzu7HXecOaFWPJGMMj3+0nb+8s5kRPRJ48rtjSI2N8OIZKKVaQnsBqSa9s34f9y5cR2F5DUnRLv5x5YgT3vWf6Hh3vrya5OgInrlhLAO7xXqwtM2rrzfsyCtjbU4ha7OLSIp2MXNcT7rERvqsDEp1NBoAVLP2FVUy96tdzBybQbf49l8ss3KKuGn2V5RX1/HDif347qm9iI30/HiB/cWVrMkuZJ1zwV+bU0hJpc2yGhUeSkVNHeGhwtTh3blxQh+G9Yj3eBmU6ug0ACif21tUwf8tWs+SzbnERYZx44Q+fG9Cn3YNHNt+oJR3N+xnTXYBa7OLDudDCg0RBnWLZUTPBDJ7JDCiZwL9u8SwK7+c2Z/tYP6KbMqq6xjbO5HvTejDeSd3bfeAO6U6Cw0Aym+ycop4aMlW3tu4n5iIMK4/rRc3nd6XpGhXi/bfX1zJ62v38NraPazLsemzeye7GdEzgRHOxX5I9zgiw0ObPEZxZQ3zvspm9vIdZOdXkJ4QxXdP7cXMsRleG8lcU1evqTJUh6ABQPndpr3F/GfJN7y1fi9R4aFcN74X3z+jb6MNxcWVNbyTtY9X1+7ms215GAND0+O4dEQ6U0d0b3M1VV29YfGm/fz302/5fHs+UeGhXDE6nWvH96Jnopuo8FBCWphAr77esL+kkp155ezKK2dnfhm78ivYlVfGzvxyCstrSE+IYnBaLIPT4hjULY5BabH0To72apK+iuo63tu4jwMlVcRFhRMfFU5cpP0d7w4nLjKMmIiw4/JLGWMoraolr7SavLJq8kqryC+zrw86r0NFSI2LoEtsJF1iI+xPnH0dHaEzy3ZUGgBUh7F1fwn/WfoNr6/dgysshKvGZfCDs/oRHxXO0s25vLpmD0u25FJdW0+vZDeXZqZzyYju9O/S+hQWzdm4p5j/fvotr67dc7gbLNi2A7crlChXKNGuMKJc9r3bFYbbFUppVS0788rILqhosF9oiJCeEEVGkpuMZDcp0S6+zStn895ith8so84ZHxEZHsLAroeCQiyD0uIYmh5PTDsuoMYYVu0qZMHKbN5Yu5eSE8w2FxoixEWGERcVTlR4KEUVNeSVVTc4n6PFRISRFO2irt5woKSK6rrjt4t2hdIlLpLU2Ai6xUUyoGsMA7vZc+yReOKEhsp7NACoDmf7gVIeWbaNRat3EypCRFgIJVW1pMREMHVEGpdmpjOiR7zXLxx5pVW8v3E/RRU1lFfXUVFTR1lVLRXVdZRX11FWfeR1eXUtkeGh9Ep20ys5mowkN72S3WQkuemeENVklU9lTR3f5JayaW8xm/eVsGlvMZv2FlNQXgPYC/LQ9HjG903i1L7JjOmd1KKAsL+4koWrdrNgZTbbDpQRFR7KhcO6MX10T05Oi6O4soaiihqKK5zfh9/XUuQsK6+uI8EdTnK0i+QYF8nRESTFuEiJjiA5xkVStKtB9ZoxhsLyGnJLqsgtqSS3uOrI65IqDhRXsaeogpyCisP7xESEMbBbLAO7xTK4WywDu8UxsFss8VENq9+qa+uPKeehctdSV1fPxIFd6J2ieadaSwOA6rB25ZXz1CfbqaypY+qI7pzaNzkoGmiNsXfTG/cWs2pnAcu357Emu5CaOkNoiDC8Rzzj+yYzvm8yY3olHq5iqaqtY/HGXOavzOajrw9Qb2Bs70Smj+7JlOFp7XqS8KTSqlq27Cthy74SNu+zgW/z3mKKK488nXSPjyQmMuxwUKpowQDCYenxXDKiOxePSGuQmqQjq66tZ3dhBTvzytiVb6sMd+WXU1bd+JOacPxNz1+mDSc9oW3nqwFAqU6gorqOlTsL+Hx7Hsu357E2u5DaekOYExB6p0SzZHMuheU1pMVH8p1R6Uwb3ZM+neSu2BjDvuJKNu8tsQFhXzHVtfW2jcJpnzi63cK+tsuqaup5d8O+Bp0BxvVOYmpmd6YM7UZyzIkHHdqgVMymvfYpLKeggrT4SDKS3fRKinae7Nyt7rJcWVNHbnEVB0or2VdUxc78MrLzy9mZZ3/2FlVwdIaUyPAQMpLcxDXyOU1dif89M5Meie5WlesQDQBKdULl1bWs3FnA8m15fL49j625pUwc2IXpo3swoX9K0M749u3BMt5weoZtzS0lNESY0D+FqcPTuGBoN2JcYeQUVLBxbzGb9xU7VW4l7Mo/Mod2bGQYGUlu9hdXcrC0YWLEpGiXDQZJbjKSo+md7EYEe5EvOabKq6Tq8NiTo6XEuOiZdOQYh6oLeyW5SY2N8GmbiAYApVTAMcawZX8Jr63Zw+vr9pCdX4ErLARXaAilTkO4iE1aOLhbHIPTYhnULY7B3ePoHh95+CJ8qGHf9uYqZ2deWZN371HhoXSJsz2gUmNtj6jU2CPvu8ZF0jPJ3WGq4qCDBgARmQz8GwgFnjLGPNDc9hoAlFJNMcawJruQt7L2Ul1bz+C0OAanxTGgayxRrqbHiJxIVW0dOQUVCJAaG9FoF9qO7kQBwOehSkRCgYeB84Ac4CsRec0Ys9HXZVFKdX4iwsiMREZmJHr0uBFhofRrwyx6nYk/ulyMA74xxmw3xlQDc4FL/VAOpZQKav4IAOlA9lHvc5xlDYjILBFZISIrDhw44LPCKaVUsOiwna6NMU8YY8YYY8akpqb6uzhKKRVw/BEAdgM9j3rfw1mmlFLKh/wRAL4CThKRPiLiAmYCr/mhHEopFdR83gvIGFMrIj8C3sV2A33GGLPB1+VQSqlg55cRC8aYt4C3/PHZSimlrA7bCKyUUsq7OkUqCBE5AOxs4+4pwEEPFqcjCLRz0vPp+ALtnALtfKDxc+pljGmyG2WnCADtISIrmhsK3RkF2jnp+XR8gXZOgXY+0LZz0iogpZQKUhoAlFIqSAVDAHjC3wXwgkA7Jz2fji/QzinQzgfacE4B3waglFKqccHwBKCUUqoRGgCUUipIBXQAEJHJIrJFRL4RkXv9XZ72EpEdIpIlImtEpFNOkSYiz4hIroisP2pZkoi8LyJbnd+endnDi5o4n/tFZLfzPa0RkSn+LGNriEhPEVkqIhtFZIOI3OEs78zfUVPn1Cm/JxGJFJEvRWStcz6/dZb3EZEvnOvdy06uteaPFahtAM7MY19z1MxjwFWdeeYxEdkBjDHGdNoBLCJyJlAKPGeMGeos+39AvjHmASdQJxpjfu7PcrZUE+dzP1BqjPmbP8vWFiKSBqQZY1aJSCywErgMuIHO+x01dU5X0gm/J7HzUkYbY0pFJBz4BLgDuBtYaIyZKyKPAWuNMY82d6xAfgLQmcc6IGPMR0D+MYsvBWY7r2dj/zg7hSbOp9Myxuw1xqxyXpcAm7ATNnXm76ipc+qUjFXqvA13fgxwDrDAWd6i7yiQA0CLZh7rZAzwnoisFJFZ/i6MB3U1xux1Xu8DuvqzMB7yIxFZ51QRdZrqkqOJSG9gJPAFAfIdHXNO0Em/JxEJFZE1QC7wPrANKDTG1DqbtOh6F8gBIBCdbowZBVwI3OZUPwQUY+skO3u95KNAPyAT2Av83a+laQMRiQFeAe40xhQfva6zfkeNnFOn/Z6MMXXGmEzshFrjgEFtOU4gB4CAm3nMGLPb+Z0LLMJ+8YFgv1NPe6i+NtfP5WkXY8x+5w+0HniSTvY9OfXKrwAvGmMWOos79XfU2Dl19u8JwBhTCCwFTgUSRORQiv8WXe8COQAE1MxjIhLtNGAhItHA+cD65vfqNF4DrndeXw+86seytNuhC6XjcjrR9+Q0MD4NbDLG/OOoVZ32O2rqnDrr9yQiqSKS4LyOwnZ02YQNBNOczVr0HQVsLyAAp1vXvzgy89gf/VuithORvti7frAT+bzUGc9HROYAE7Gpa/cDvwH+B8wDMrBpv680xnSKhtUmzmcitlrBADuAW46qP+/QROR04GMgC6h3Fv8CW2feWb+jps7pKjrh9yQiw7GNvKHYm/h5xpjfOdeIuUASsBq41hhT1eyxAjkAKKWUalogVwEppZRqhgYApZQKUhoAlFIqSGkAUEqpIKUBQCmlgpQGAKW8TEQmisgb/i6HUsfSAKCUUkFKA4BSDhG51smzvkZEHncSbpWKyD+dvOsfiEiqs22miHzuJBJbdCiRmIj0F5HFTq72VSLSzzl8jIgsEJHNIvKiMzpVKb/SAKAUICKDgRnABCfJVh1wDRANrDDGDAE+xI70BXgO+LkxZjh2hOmh5S8CDxtjRgCnYZOMgc1AeSdwMtAXmODlU1LqhMJOvIlSQWESMBr4yrk5j8ImPKsHXna2eQFYKCLxQIIx5kNn+WxgvpOrKd0YswjAGFMJ4BzvS2NMjvN+DdAbO5GHUn6jAUApS4DZxpj7GiwU+dUx27U1d8rROVnq0L891QFoFZBS1gfANBHpAofnwO2F/Rs5lGHxauATY0wRUCAiZzjLrwM+dGabyhGRy5xjRIiI25cnoVRr6F2IUoAxZqOI/BI741oIUAPcBpQB45x1udh2ArDpdh9zLvDbgRud5dcBj4vI75xjTPfhaSjVKpoNVKlmiEipMSbG3+VQyhu0CkgppYKUPgEopVSQ0icApZQKUhoAlFIqSGkAUEqpIKUBQCmlgpQGAKWUClL/H3fRvV7Gd/0IAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.plot(history.history['accuracy'])\n",
"plt.plot(history.history['val_accuracy'])\n",
"plt.title('model accuracy')\n",
"plt.ylabel('accuracy')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='lower right')\n",
"plt.show()\n",
"plt.plot(history.history['loss'])\n",
"plt.plot(history.history['val_loss'])\n",
"plt.title('model loss')\n",
"plt.ylabel('loss')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='upper right')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3/3 [==============================] - 0s 5ms/step - loss: 38.5918 - accuracy: 0.4699\n"
]
},
{
"data": {
"text/plain": [
"[38.59177780151367, 0.46987950801849365]"
]
},
"execution_count": 74,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.evaluate(val_features, val_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We reach a validation accuracy of about 49% — much worse than we achieved in the\n",
"previous section with the small model trained from scratch. \n",
"\n",
"The learning curves indicate that we’re overfitting almost from the start—\n",
"despite using dropout with a fairly large rate. That’s because this technique doesn’t\n",
"use data augmentation, which is essential for preventing overfitting with small image\n",
"datasets."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Tensorboard"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load the TensorBoard notebook extension on google colab\n",
"%load_ext tensorboard\n",
"\n",
"%tensorboard --logdir logs"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "DJT-DgHvGhKu"
},
"source": [
"### 2. Approach : Feature Extraction with Data Augmentation\n",
"\n",
"\n",
"Now let’s review the second technique we mentioned for doing feature extraction,\n",
"which is much slower and more expensive, but which allows us to use data augmentation\n",
"during training: creating a model that chains the `conv_base` with a new dense\n",
"classifier, and training it end to end on the inputs.\n",
"\n",
"\n",
"In order to do this, we will first freeze the convolutional base. Freezing a layer or set of\n",
"layers means preventing their weights from being updated during training. If we don’t\n",
"do this, the representations that were previously learned by the convolutional base will\n",
"be modified during training. Because the Dense layers on top are randomly initialized,\n",
"very large weight updates would be propagated through the network, effectively\n",
"destroying the representations previously learned.\n",
"\n",
"In Keras, we freeze a layer or model by setting its trainable attribute to `False`. "
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "50DF9pH1GhKw"
},
"source": [
"#### Instantiating and freezing the VGG16 convolutional base"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {},
"outputs": [],
"source": [
"conv_base = keras.applications.vgg16.VGG16(weights=\"imagenet\", include_top=False)\n",
"conv_base.trainable = False"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Setting trainable to `False` empties the list of trainable weights of the layer or model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Printing the list of trainable weights before and after freezing"
]
},
{
"cell_type": "code",
"execution_count": 76,
"metadata": {},
"outputs": [],
"source": [
"conv_base.trainable = True"
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"This is the number of trainable weights before freezing the conv base: 26\n"
]
}
],
"source": [
"print(\"This is the number of trainable weights before freezing the conv base:\", len(conv_base.trainable_weights))"
]
},
{
"cell_type": "code",
"execution_count": 78,
"metadata": {},
"outputs": [],
"source": [
"conv_base.trainable = False"
]
},
{
"cell_type": "code",
"execution_count": 79,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"This is the number of trainable weights after freezing the conv base: 0\n"
]
}
],
"source": [
"print(\"This is the number of trainable weights after freezing the conv base:\", len(conv_base.trainable_weights))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can create a new model that chains together\n",
"\n",
"1. A data augmentation stage\n",
"\n",
"2. Our frozen convolutional base \n",
"\n",
"3. A dense classifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Adding a data augmentation stage and a classifier to the convolutional base"
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {},
"outputs": [],
"source": [
"data_augmentation = keras.Sequential(\n",
"[\n",
"layers.RandomFlip(\"horizontal\"),\n",
"layers.RandomRotation(0.1),\n",
"layers.RandomZoom(0.2),\n",
"]\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {},
"outputs": [],
"source": [
"inputs = keras.Input(shape=(150, 150, 3))\n",
"# Apply data augmentation\n",
"x = data_augmentation(inputs)\n",
"# Apply input value scaling\n",
"x = keras.applications.vgg16.preprocess_input(x)\n",
"x = conv_base(x)\n",
"x = layers.Flatten()(x)\n",
"x = layers.Dense(256)(x)\n",
"x = layers.Dropout(0.5)(x)\n",
"outputs = layers.Dense(8, activation=\"softmax\")(x)\n",
"model = keras.Model(inputs, outputs)\n",
"model.compile(loss=\"categorical_crossentropy\",\n",
" optimizer=\"rmsprop\",\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With this setup, only the weights from the two Dense layers that we added will be\n",
"trained. That’s a total of four weight tensors: two per layer (the main weight matrix\n",
"and the bias vector). \n",
"\n",
"Note that in order for these changes to take effect, you must first\n",
"compile the model. If you ever modify weight trainability after compilation, you\n",
"should then recompile the model, or these changes will be ignored.\n",
"\n",
"Let’s train our model. Thanks to data augmentation, it will take much longer for\n",
"the model to start overfitting, so we can train for more epochs — let’s do 50.\n",
"\n",
"__NOTE__ This technique is expensive enough that you should only attempt it if\n",
"you have access to a GPU (such as the free GPU available in Colab) — it’s\n",
"intractable on CPU. If you can’t run your code on GPU, then the previous\n",
"technique is the way to go."
]
},
{
"cell_type": "code",
"execution_count": 82,
"metadata": {},
"outputs": [],
"source": [
"logdir = os.path.join(\"logs_feature_extraction_with_augmentation\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n",
"\n",
"\n",
"callbacks = [\n",
" keras.callbacks.ModelCheckpoint(filepath=\"feature_extraction_with_augmentation.h5\", save_best_only=True, monitor=\"val_loss\"),\n",
" tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)\n",
"]\n"
]
},
{
"cell_type": "code",
"execution_count": 92,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/50\n",
"15/15 [==============================] - 213s 14s/step - loss: 5.9087 - accuracy: 0.8000 - val_loss: 38.9280 - val_accuracy: 0.5060\n",
"Epoch 2/50\n",
"15/15 [==============================] - 204s 14s/step - loss: 4.1850 - accuracy: 0.8521 - val_loss: 38.2262 - val_accuracy: 0.5422\n",
"Epoch 3/50\n",
"15/15 [==============================] - 199s 13s/step - loss: 3.1716 - accuracy: 0.8521 - val_loss: 35.7508 - val_accuracy: 0.5181\n",
"Epoch 4/50\n",
"15/15 [==============================] - 201s 13s/step - loss: 2.8206 - accuracy: 0.8854 - val_loss: 34.2308 - val_accuracy: 0.5301\n",
"Epoch 5/50\n",
"15/15 [==============================] - 201s 14s/step - loss: 2.8074 - accuracy: 0.9042 - val_loss: 35.6657 - val_accuracy: 0.5301\n",
"Epoch 6/50\n",
"15/15 [==============================] - 204s 14s/step - loss: 2.5794 - accuracy: 0.9000 - val_loss: 36.6189 - val_accuracy: 0.5181\n",
"Epoch 7/50\n",
"15/15 [==============================] - 203s 14s/step - loss: 2.5663 - accuracy: 0.9000 - val_loss: 35.3423 - val_accuracy: 0.5301\n",
"Epoch 8/50\n",
"15/15 [==============================] - 206s 14s/step - loss: 2.8004 - accuracy: 0.8938 - val_loss: 35.5800 - val_accuracy: 0.5422\n",
"Epoch 9/50\n",
"15/15 [==============================] - 202s 14s/step - loss: 2.2963 - accuracy: 0.9208 - val_loss: 34.6054 - val_accuracy: 0.5422\n",
"Epoch 10/50\n",
"15/15 [==============================] - 204s 14s/step - loss: 1.4943 - accuracy: 0.9146 - val_loss: 34.1561 - val_accuracy: 0.5542\n",
"Epoch 11/50\n",
"15/15 [==============================] - 205s 14s/step - loss: 1.2628 - accuracy: 0.9271 - val_loss: 33.4512 - val_accuracy: 0.5301\n",
"Epoch 12/50\n",
"15/15 [==============================] - 195s 13s/step - loss: 1.7920 - accuracy: 0.9125 - val_loss: 30.7742 - val_accuracy: 0.5542\n",
"Epoch 13/50\n",
"15/15 [==============================] - 210s 14s/step - loss: 0.7292 - accuracy: 0.9583 - val_loss: 33.0453 - val_accuracy: 0.5542\n",
"15/15 [==============================] - 199s 13s/step - loss: 0.8244 - accuracy: 0.9438 - val_loss: 31.8265 - val_accuracy: 0.5663\n",
"Epoch 29/50\n",
"15/15 [==============================] - 199s 13s/step - loss: 1.3684 - accuracy: 0.9396 - val_loss: 30.7292 - val_accuracy: 0.5542\n",
"Epoch 30/50\n",
"15/15 [==============================] - 199s 13s/step - loss: 1.4007 - accuracy: 0.9458 - val_loss: 30.8636 - val_accuracy: 0.5301\n",
"Epoch 31/50\n",
"15/15 [==============================] - 198s 13s/step - loss: 0.8514 - accuracy: 0.9563 - val_loss: 33.0234 - val_accuracy: 0.5301\n",
"Epoch 32/50\n",
"15/15 [==============================] - 212s 14s/step - loss: 1.1778 - accuracy: 0.9479 - val_loss: 33.0162 - val_accuracy: 0.5542\n",
"Epoch 34/50\n",
" 2/15 [===>..........................] - ETA: 2:32 - loss: 0.8565 - accuracy: 0.9375"
]
},
{
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-92-e759ae1d18a5>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0mepochs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m50\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0mvalidation_data\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidation_dataset\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m callbacks=callbacks)\n\u001b[0m",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/keras/utils/traceback_utils.py\u001b[0m in \u001b[0;36merror_handler\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 62\u001b[0m \u001b[0mfiltered_tb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 63\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 64\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 65\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# pylint: disable=broad-except\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 66\u001b[0m \u001b[0mfiltered_tb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_process_traceback_frames\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__traceback__\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)\u001b[0m\n\u001b[1;32m 1214\u001b[0m _r=1):\n\u001b[1;32m 1215\u001b[0m \u001b[0mcallbacks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_train_batch_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1216\u001b[0;31m \u001b[0mtmp_logs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1217\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mdata_handler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshould_sync\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1218\u001b[0m \u001b[0mcontext\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0masync_wait\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/traceback_utils.py\u001b[0m in \u001b[0;36merror_handler\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 148\u001b[0m \u001b[0mfiltered_tb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 149\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 150\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 151\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 152\u001b[0m \u001b[0mfiltered_tb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_process_traceback_frames\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__traceback__\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m 908\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 909\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mOptionalXlaContext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_jit_compile\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 910\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 911\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 912\u001b[0m \u001b[0mnew_tracing_count\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexperimental_get_tracing_count\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m 940\u001b[0m \u001b[0;31m# In this case we have created variables on the first call, so we run the\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 941\u001b[0m \u001b[0;31m# defunned version which is guaranteed to never create variables.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 942\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateless_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# pylint: disable=not-callable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 943\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateful_fn\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 944\u001b[0m \u001b[0;31m# Release the lock early so that multiple threads can perform the call\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 3129\u001b[0m filtered_flat_args) = self._maybe_define_function(args, kwargs)\n\u001b[1;32m 3130\u001b[0m return graph_function._call_flat(\n\u001b[0;32m-> 3131\u001b[0;31m filtered_flat_args, captured_inputs=graph_function.captured_inputs) # pylint: disable=protected-access\n\u001b[0m\u001b[1;32m 3132\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3133\u001b[0m \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_call_flat\u001b[0;34m(self, args, captured_inputs, cancellation_manager)\u001b[0m\n\u001b[1;32m 1958\u001b[0m \u001b[0;31m# No tape is watching; skip to running the function.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1959\u001b[0m return self._build_call_outputs(self._inference_function.call(\n\u001b[0;32m-> 1960\u001b[0;31m ctx, args, cancellation_manager=cancellation_manager))\n\u001b[0m\u001b[1;32m 1961\u001b[0m forward_backward = self._select_forward_and_backward_functions(\n\u001b[1;32m 1962\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36mcall\u001b[0;34m(self, ctx, args, cancellation_manager)\u001b[0m\n\u001b[1;32m 601\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 602\u001b[0m \u001b[0mattrs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mattrs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 603\u001b[0;31m ctx=ctx)\n\u001b[0m\u001b[1;32m 604\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 605\u001b[0m outputs = execute.execute_with_cancellation(\n",
"\u001b[0;32m/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/execute.py\u001b[0m in \u001b[0;36mquick_execute\u001b[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[0m\n\u001b[1;32m 57\u001b[0m \u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mensure_initialized\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 58\u001b[0m tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,\n\u001b[0;32m---> 59\u001b[0;31m inputs, attrs, num_outputs)\n\u001b[0m\u001b[1;32m 60\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 61\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mKeyboardInterrupt\u001b[0m: "
]
}
],
"source": [
"history = model.fit(\n",
"train_dataset,\n",
"epochs=50,\n",
"validation_data=validation_dataset,\n",
"callbacks=callbacks)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let’s plot the results again. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plt.plot(history.history['accuracy'])\n",
"plt.plot(history.history['val_accuracy'])\n",
"plt.title('model accuracy')\n",
"plt.ylabel('accuracy')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='lower right')\n",
"plt.show()\n",
"plt.plot(history.history['loss'])\n",
"plt.plot(history.history['val_loss'])\n",
"plt.title('model loss')\n",
"plt.ylabel('loss')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='upper right')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model.evaluate(validation_dataset)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you can see, we reach a validation accuracy of over 43%. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Tensorboard"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load the TensorBoard notebook extension on google colab\n",
"%load_ext tensorboard\n",
"%tensorboard --logdir logs"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "FZYRLtbkGhLV"
},
"source": [
"## 2.2 Fine Tuning\n",
"\n",
"Another widely used technique for model reuse, complementary to feature extraction, is _fine-tuning_. \n",
"Fine-tuning consists of unfreezing a few of the top layers of a frozen model base used\n",
"for feature extraction, and jointly training both the newly added part of the model (in this case, the\n",
"fully connected classifier) and these top layers. This is called _fine-tuning_ because it slightly \n",
"adjusts the more abstract representations of the model being reused in order to make them more relevant for the problem at hand.\n",
"\n",
"I stated earlier that it’s necessary to freeze the convolution base of VGG16 in order to be able to\n",
"train a randomly initialized classifier on top. For the same reason, it’s only possible to fine-tune the top\n",
"layers of the convolutional base once the classifier on top has already been trained. If the classifier isn’t\n",
"already trained, the error signal propagating through the network during training will be too\n",
"large, and the representations previously learned by the layers being fine-tuned will be destroyed. Thus\n",
"the steps for fine-tuning a network are as follows:\n",
"\n",
"The steps for fine-tuning are as follows:\n",
"\n",
"1. Add our custom network on top of an already-trained base network.\n",
"2. Freeze the base network.\n",
"3. Train the part we added.\n",
"4. Unfreeze some layers in the base network. (Note that you should not unfreeze “batch normalization” layers, which are not relevant here since there are no such layers in VGG16. )\n",
"5. Jointly train both these layers and the part we added.\n",
"\n",
"We already completed the first three steps when doing feature extraction. Let’s proceed with step 4:\n",
"we’ll unfreeze our `conv_base` and then freeze individual layers inside it.\n",
"\n",
"As a reminder, this is what our convolutional base looks like:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "cnObzTupGhLV",
"outputId": "3754b2b3-8885-44b3-cb87-82612d223ec3"
},
"outputs": [],
"source": [
"conv_base.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "aDtcl5X2GhLa"
},
"source": [
"We will fine-tune the last three convolutional layers, which means all layers up to `block4_pool` should be frozen, and the layers `block5_conv1`, `block5_conv2`, and `block5_conv3` should be trainable.\n",
"\n",
"Why not fine-tune more layers? Why not fine-tune the entire convolutional base?\n",
"You could. But you need to consider the following:\n",
"\n",
"- Earlier layers in the convolutional base encode more generic, reusable features, whereas layers higher up encode more specialized features. It’s more useful to fine-tune the more specialized features, because these are the ones that need to be repurposed on your new problem. There would be fast-decreasing returns in fine-tuning lower layers.\n",
"\n",
"- The more parameters you’re training, the more you’re at risk of overfitting. The convolutional base has 15 million parameters, so it would be risky to attempt to train it on your small dataset. \n",
"\n",
"Thus, in this situation, it’s a good strategy to fine-tune only the top two or three layers in the convolutional base. Let’s set this up, starting from where we left off in the previous example."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Freezing all layers until the fourth from the last"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "tBXYN1t2GhLc",
"outputId": "b33ae8d1-925b-4e8a-f15d-a62356070896"
},
"outputs": [],
"source": [
"conv_base.trainable = True\n",
"for layer in conv_base.layers[:-4]:\n",
" layer.trainable = False\n",
" \n",
"for layer in conv_base.layers[0:]:\n",
" print('layer name = ' + layer.name + ', shape = ' + repr(layer.output_shape)\n",
" + ', trainable = ' + repr(layer.trainable)) \n",
" \n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "XWw1mYfUGhLg"
},
"source": [
"Now we can begin fine-tuning the model. We’ll do this with the `RMSprop` optimizer, using a very low learning rate. The reason for using a low learning rate is that we want to limit the magnitude of the modifications we make to the representations of the three\n",
"layers we’re fine-tuning. Updates that are too large may harm these representations."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Fine-tuning the model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "4YBjFhSVGhLh",
"outputId": "c688820a-0f28-4aa0-b247-15a9684fa08f"
},
"outputs": [],
"source": [
"model.compile(loss=\"categorical_crossentropy\",\n",
" optimizer=keras.optimizers.RMSprop(learning_rate=1e-5),\n",
" metrics=[\"accuracy\"])\n",
"\n",
"for layer in model.layers[0:]:\n",
" print('layer name = ' + layer.name + ', shape = ' + repr(layer.output_shape)\n",
" + ', trainable = ' + repr(layer.trainable)) \n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"logdir = os.path.join(\"logs_fine_tuning\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n",
"\n",
"\n",
"callbacks = [\n",
" keras.callbacks.ModelCheckpoint(filepath=\"fine_tuning.keras\", save_best_only=True, monitor=\"val_loss\"),\n",
" tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)\n",
"]\n",
"\n",
"history = conv_base.fit(train_dataset,\n",
" epochs=30,\n",
" validation_data=validation_dataset,\n",
" callbacks=callbacks)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "9rwSMMQaGhLx",
"outputId": "0a58db5a-0f22-45e8-d1fb-0a664fceaf4d"
},
"outputs": [],
"source": [
"plt.plot(history.history['accuracy'])\n",
"plt.plot(history.history['val_accuracy'])\n",
"plt.title('model accuracy')\n",
"plt.ylabel('accuracy')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='lower right')\n",
"plt.show()\n",
"plt.plot(history.history['loss'])\n",
"plt.plot(history.history['val_loss'])\n",
"plt.title('model loss')\n",
"plt.ylabel('loss')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='upper right')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Tensorboard"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load the TensorBoard notebook extension on google colab\n",
"%load_ext tensorboard\n",
"%tensorboard --logdir logs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Confusion Matrix and Missclassified Images"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"prediction = model.predict(validation_dataset)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "WoDOi_F8GhL5",
"outputId": "17c21c92-2a5d-4e21-c367-57e818046762"
},
"outputs": [],
"source": [
"from sklearn.metrics import confusion_matrix\n",
"import sys\n",
"class_names = [\"angelina jolie\", \"brad pitt\",\"catherine deneuve\" , \"johnny depp\",\"leonardo dicaprio\", \"marion cotillard\", \"robert de niro\",\"sandra bullock\"]\n",
"\n",
"\n",
"Y_valid = np.zeros((num_valid_images,1),dtype=int)\n",
"\n",
"step = num_valid_images // num_classes\n",
"for ind in range(num_classes):\n",
" Y_valid[ind*step:(ind+1)*step] = ind\n",
" \n",
"confmat = confusion_matrix(val_labels.argmax(axis=1),np.argmax(prediction,axis=1)) \n",
"\n",
"for i0 in range(num_classes):\n",
" sys.stdout.write('[')\n",
" for i1 in range(num_classes):\n",
" sys.stdout.write('{:3d} '.format(confmat[i0,i1]))\n",
" \n",
" sys.stdout.write('], {}\\n'.format(class_names[i0]))\n",
" \n",
"sys.stdout.flush()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "nNp0qChLGhL-",
"outputId": "f22e9bfe-e5da-4d57-fbdc-2ea55d6681e7"
},
"outputs": [],
"source": [
"# Choose the class label you want to check\n",
"clbl = 7\n",
"step = num_valid_images // num_classes\n",
"pred_labels = np.argmax(prediction[clbl*step:(clbl+1)*step],axis=1)\n",
"wrong_labels = np.transpose(np.nonzero(pred_labels != clbl))\n",
"\n",
"\n",
"# Get the validation images as numpy arrays\n",
"\n",
"import numpy as np\n",
"def get_images_and_labels(dataset):\n",
" all_images = []\n",
" all_labels = []\n",
" for images, labels in dataset:\n",
" all_images.append(images)\n",
" all_labels.append(labels)\n",
" return np.concatenate(all_images), np.concatenate(all_labels)\n",
"\n",
"val_images, val_labels = get_images_and_labels(validation_dataset)\n",
"\n",
"\n",
"print('wrong classification for: {}'.format(class_names[clbl]))\n",
"\n",
"for i, i0 in enumerate(wrong_labels):\n",
" img = val_images[clbl*step + i0]\n",
" img = np.squeeze(img, axis=0)\n",
" plt.figure(figsize=(4, 4))\n",
" plt.imshow(img.astype(\"uint8\"))\n",
" plt.show()\n",
" plt.axis(\"off\")\n",
" print('matched to: {}'.format(class_names[pred_labels[i0][0]]))\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part III : Semantic Segmentation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this section, we’ll focus on semantic segmentation: we’ll be looking once again at\n",
"images of cats and dogs, and this time we’ll learn how to tell apart the main subject\n",
"and its background.\n",
"\n",
"We’ll work with the Oxford-IIIT Pets dataset (www.robots.ox.ac.uk/~vgg/data/pets/), which contains 7'390 pictures of various breeds of cats and dogs, together with foreground-background segmentation masks for \n",
"each picture. A segmentation mask is the image-segmentation equivalent of a label: it’s an image the same size as the input image, with a single color channel where each integer value corresponds to the class of the corresponding pixel in the input image. In our case, the pixels of our segmentation\n",
"masks can take one of three integer values:\n",
"1. (foreground)\n",
"2. (background)\n",
"3. (contour)\n",
"\n",
"Let’s start by downloading and uncompressing our dataset, using the `wget` and `tar`\n",
"shell utilities:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!wget http://www.robots.ox.ac.uk/~vgg/data/pets/data/images.tar.gz\n",
"!wget http://www.robots.ox.ac.uk/~vgg/data/pets/data/annotations.tar.gz\n",
"!tar -xf images.tar.gz\n",
"!tar -xf annotations.tar.gz"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The input pictures are stored as JPG files in the `images/` folder \n",
"(such as images/Abyssinian_1.jpg), and the corresponding segmentation mask is stored as a PNG file with\n",
"the same name in the `annotations/trimaps/` folder (such as annotations/trimaps/Abyssinian_1.png).\n",
"Let’s prepare the list of input file paths, as well as the list of the corresponding\n",
"mask file paths:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"input_dir = \"images/\"\n",
"target_dir = \"annotations/trimaps/\"\n",
"input_img_paths = sorted(\n",
" [os.path.join(input_dir, fname)\n",
" for fname in os.listdir(input_dir)\n",
" if fname.endswith(\".jpg\")])\n",
" \n",
"target_paths = sorted([os.path.join(target_dir, fname)\n",
" for fname in os.listdir(target_dir)\n",
" if fname.endswith(\".png\") and not fname.startswith(\".\")])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, what does one of these inputs and its mask look like? Let’s take a quick look.\n",
"Here’s a sample image:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"from tensorflow.keras.utils import load_img, img_to_array\n",
"plt.axis(\"off\")\n",
"# Display input image number 9\n",
"plt.imshow(load_img(input_img_paths[9]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And here is its corresponding target:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def display_target(target_array):\n",
" # The original labels are 1, 2, and 3. We subtract 1 so that the\n",
" # labels range from 0 to 2, and then we multiply by 127 so that\n",
" # the labels become 0 (black), 127 (gray), 254 (near-white).\n",
" normalized_array = (target_array.astype(\"uint8\") - 1) * 127\n",
" plt.axis(\"off\")\n",
" plt.imshow(normalized_array[:, :, 0])\n",
"\n",
"# We use color_mode=\"grayscale\" so that the image we load is treated as\n",
"# having a single color channel. \n",
"img = img_to_array(load_img(target_paths[9], color_mode=\"grayscale\"))\n",
"display_target(img)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, let’s load our inputs and targets into two NumPy arrays, and let’s split the arrays\n",
"into a training and a validation set. Since the dataset is very small, we can just load\n",
"everything into memory:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import random\n",
"\n",
"# We resize everything to 200x200\n",
"img_size = (200, 200)\n",
"# Total number of samples in the data\n",
"num_imgs = len(input_img_paths)\n",
"\n",
"# Shuffle the file paths (they were originally sorted by breed). We \n",
"# use the same seed (1337) in both statements to ensure that the input \n",
"# paths and target paths stay in the same order\n",
"\n",
"random.Random(1337).shuffle(input_img_paths)\n",
"random.Random(1337).shuffle(target_paths)\n",
"\n",
"def path_to_input_image(path):\n",
" return img_to_array(load_img(path, target_size=img_size))\n",
"\n",
"def path_to_target(path):\n",
" img = img_to_array(\n",
" load_img(path, target_size=img_size, color_mode=\"grayscale\"))\n",
" # Subtract 1 so that our labels become 0, 1, and 2\n",
" img = img.astype(\"uint8\") - 1\n",
" return img\n",
"\n",
"# Load all images in the input_imgs float32 array and their masks in the\n",
"# targets uint8 array (same order). The inputs have three channels (RBG values)\n",
"# and the targets have a single channel (which contains integer labels)\n",
"input_imgs = np.zeros((num_imgs,) + img_size + (3,), dtype=\"float32\")\n",
"targets = np.zeros((num_imgs,) + img_size + (1,), dtype=\"uint8\")\n",
"for i in range(num_imgs):\n",
" input_imgs[i] = path_to_input_image(input_img_paths[i])\n",
" targets[i] = path_to_target(target_paths[i])\n",
" \n",
"# Reserve 1000 samples for validation\n",
"num_val_samples = 1000\n",
"\n",
"# Split the data into a training and a\n",
"# validation set\n",
"train_input_imgs = input_imgs[:-num_val_samples]\n",
"train_targets = targets[:-num_val_samples]\n",
"val_input_imgs = input_imgs[-num_val_samples:]\n",
"val_targets = targets[-num_val_samples:]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now it’s time to define our model:"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"from tensorflow import keras\n",
"from keras import layers\n",
"\n",
"def get_model(img_size, num_classes):\n",
" inputs = keras.Input(shape=img_size + (3,))\n",
" \n",
" # Don’t forget to rescale input images to the [0-1] range\n",
" # x = layers.Rescaling(1./255)(inputs)\n",
" ### [First half of the network: downsampling inputs] ###\n",
"\n",
" # Entry block\n",
" x = layers.Conv2D(32, 3, strides=2, padding=\"same\")(inputs)\n",
" x = layers.BatchNormalization()(x)\n",
" x = layers.Activation(\"relu\")(x)\n",
"\n",
" previous_block_activation = x # Set aside residual\n",
"\n",
" # Blocks 1, 2, 3 are identical apart from the feature depth.\n",
" for filters in [64, 128, 256]:\n",
" x = layers.Activation(\"relu\")(x)\n",
" x = layers.SeparableConv2D(filters, 3, padding=\"same\")(x)\n",
" x = layers.BatchNormalization()(x)\n",
"\n",
" x = layers.Activation(\"relu\")(x)\n",
" x = layers.SeparableConv2D(filters, 3, padding=\"same\")(x)\n",
" x = layers.BatchNormalization()(x)\n",
"\n",
" x = layers.MaxPooling2D(3, strides=2, padding=\"same\")(x)\n",
"\n",
" # Project residual\n",
" residual = layers.Conv2D(filters, 1, strides=2, padding=\"same\")(\n",
" previous_block_activation\n",
" )\n",
" x = layers.add([x, residual]) # Add back residual\n",
" previous_block_activation = x # Set aside next residual\n",
"\n",
" ### [Second half of the network: upsampling inputs] ###\n",
"\n",
" for filters in [256, 128, 64, 32]:\n",
" x = layers.Activation(\"relu\")(x)\n",
" x = layers.Conv2DTranspose(filters, 3, padding=\"same\")(x)\n",
" x = layers.BatchNormalization()(x)\n",
"\n",
" x = layers.Activation(\"relu\")(x)\n",
" x = layers.Conv2DTranspose(filters, 3, padding=\"same\")(x)\n",
" x = layers.BatchNormalization()(x)\n",
"\n",
" x = layers.UpSampling2D(2)(x)\n",
"\n",
" # Project residual\n",
" residual = layers.UpSampling2D(2)(previous_block_activation)\n",
" residual = layers.Conv2D(filters, 1, padding=\"same\")(residual)\n",
" x = layers.add([x, residual]) # Add back residual\n",
" previous_block_activation = x # Set aside next residual\n",
"\n",
" # We end the model with a per-pixel three-way\n",
" # softmax to classify each output pixel into one of\n",
" # our three categories\n",
" outputs = layers.Conv2D(num_classes, 3, activation=\"softmax\", padding=\"same\")(x)\n",
"\n",
" # Define the model\n",
" model = keras.Model(inputs, outputs)\n",
" return model\n",
"\n",
"\n",
"# Free up RAM in case the model definition cells were run multiple times\n",
"keras.backend.clear_session()\n",
"\n",
"# Build model\n",
"model = get_model(img_size, num_classes=3)\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from tensorflow import keras\n",
"from keras import layers\n",
"\n",
"def get_model(img_size, num_classes):\n",
" inputs = keras.Input(shape=img_size + (3,))\n",
" x = layers.Rescaling(1./255)(inputs) \n",
" x = layers.Conv2D(64, 3, strides=2, activation=\"relu\", padding=\"same\")(x)\n",
" x = layers.Conv2D(64, 3, activation=\"relu\", padding=\"same\")(x)\n",
" x = layers.Conv2D(128, 3, strides=2, activation=\"relu\", padding=\"same\")(x)\n",
" x = layers.Conv2D(128, 3, activation=\"relu\", padding=\"same\")(x)\n",
" x = layers.Conv2D(256, 3, strides=2, padding=\"same\", activation=\"relu\")(x)\n",
" x = layers.Conv2D(256, 3, activation=\"relu\", padding=\"same\")(x)\n",
" x = layers.Conv2DTranspose(256, 3, activation=\"relu\", padding=\"same\")(x)\n",
" x = layers.Conv2DTranspose(256, 3, activation=\"relu\", padding=\"same\", strides=2)(x)\n",
" x = layers.Conv2DTranspose(128, 3, activation=\"relu\", padding=\"same\")(x)\n",
" x = layers.Conv2DTranspose(128, 3, activation=\"relu\", padding=\"same\", strides=2)(x)\n",
" x = layers.Conv2DTranspose(64, 3, activation=\"relu\", padding=\"same\")(x)\n",
" x = layers.Conv2DTranspose(64, 3, activation=\"relu\", padding=\"same\", strides=2)(x)\n",
" outputs = layers.Conv2D(num_classes, 3, activation=\"softmax\", padding=\"same\")(x)\n",
" model = keras.Model(inputs, outputs)\n",
" return model\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"# Free up RAM in case the model definition cells were run multiple times\n",
"keras.backend.clear_session()\n",
"\n",
"# Build model\n",
"model = get_model(img_size, num_classes=3)\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The first half of the model closely resembles the kind of convnet you’d use for image\n",
"classification: a stack of `Conv2D` layers, with gradually increasing filter sizes. We downsample\n",
"our images three times by a factor of two each, ending up with activations of size\n",
"$(25, 25, 256)$. The purpose of this first half is to encode the images into smaller feature\n",
"maps, where each spatial location (or pixel) contains information about a large spatial\n",
"chunk of the original image. You can understand it as a kind of compression.\n",
"\n",
"\n",
"One important difference between the first half of this model and the classification\n",
"models you’ve seen before is the way we do downsampling: in the classification\n",
"ConvNets from the last chapter, we used `MaxPooling2D` layers to downsample feature\n",
"maps. Here, we downsample by adding _strides_ to every other convolution layer. We do \n",
"this because, in the case of image segmentation, we care a lot about the _spatial location_ of information in the image, since we need to produce per-pixel target masks as output of the \n",
"model. When you do $2\\times 2$ max pooling, you are completely destroying location information within each pooling window: you return one scalar value per window, with zero knowledge of which of the four locations in the windows the value came from. So while max pooling layers perform\n",
"well for classification tasks, they would hurt us quite a bit for a segmentation\n",
"task. Meanwhile, strided convolutions do a better job at downsampling feature maps\n",
"while retaining location information. Throughout this book, you’ll notice that we\n",
"tend to use strides instead of max pooling in any model that cares about feature location,\n",
"such as generative models.\n",
"\n",
"The second half of the model is a stack of `Conv2DTranspose` layers. What are those?\n",
"Well, the output of the first half of the model is a feature map of shape $(25, 25, 256)$, \n",
"but we want our final output to have the same shape as the target masks, $(200, 200,3)$. Therefore, we need to apply a kind of _inverse_ of the transformations we’ve applied\n",
"so far — something that will _upsample_ the feature maps instead of downsampling them.\n",
"That’s the purpose of the `Conv2DTranspose` layer: you can think of it as a kind of convolution\n",
"layer that _learns to upsample_. If you have an input of shape $(100, 100, 64)$, and you\n",
"run it through the layer `Conv2D(128, 3, strides=2, padding=\"same\")`, you get an\n",
"output of shape $(50, 50, 128)$. If you run this output through the layer \n",
"`Conv2DTranspose(64, 3, strides=2, padding=\"same\")`, you get back an output of shape $(100,\n",
"100, 64)$, the same as the original. So after compressing our inputs into feature maps of\n",
"shape $(25, 25, 256)$ via a stack of `Conv2D` layers, we can simply apply the corresponding\n",
"sequence of `Conv2DTranspose` layers to get back to images of shape $(200, 200, 3)$.\n",
"\n",
"We can now compile and fit our model:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os, datetime\n",
"import tensorflow as tf\n",
"\n",
"model.compile(optimizer=\"rmsprop\", loss=\"sparse_categorical_crossentropy\",\n",
" metrics=[\"accuracy\"])\n",
"\n",
"logdir = os.path.join(\"logs\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n",
"callbacks = [\n",
" keras.callbacks.ModelCheckpoint(filepath=\"fine_tuning.keras\", save_best_only=True, monitor=\"val_loss\"),\n",
" tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)\n",
"] \n",
"\n",
"history = model.fit(train_input_imgs, train_targets,\n",
" epochs=50,\n",
" callbacks=callbacks,\n",
" batch_size=64,\n",
" validation_data=(val_input_imgs, val_targets))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let’s display our training and validation loss:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plt.plot(history.history['accuracy'])\n",
"plt.plot(history.history['val_accuracy'])\n",
"plt.title('model accuracy')\n",
"plt.ylabel('accuracy')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='lower right')\n",
"plt.show()\n",
"plt.plot(history.history['loss'])\n",
"plt.plot(history.history['val_loss'])\n",
"plt.title('model loss')\n",
"plt.ylabel('loss')\n",
"plt.xlabel('epoch')\n",
"plt.legend(['train', 'valid'], loc='upper right')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can see that we start overfitting midway, around epoch 25. Let’s reload our best\n",
"performing model according to the validation loss, and demonstrate how to use it to\n",
"predict a segmentation mask"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from tensorflow.keras.utils import array_to_img\n",
"model = keras.models.load_model(\"oxford_segmentation.keras\")\n",
"i = 4\n",
"test_image = val_input_imgs[i]\n",
"plt.axis(\"off\")\n",
"plt.imshow(array_to_img(test_image))\n",
"mask = model.predict(np.expand_dims(test_image, 0))[0]\n",
"\n",
"# Utility to display a model’s prediction\n",
"def display_mask(pred):\n",
" mask = np.argmax(pred, axis=-1)\n",
" mask *= 127\n",
" plt.axis(\"off\")\n",
" plt.imshow(mask)\n",
" \n",
"display_mask(mask)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are a couple of small artifacts in our predicted mask, caused by geometric shapes\n",
"in the foreground and background. Nevertheless, our model appears to work nicely."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part IV : Object Detection with Yolo and Retinanet"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Please consult the following sources:\n",
"\n",
"1. [Retinanet](https://keras.io/examples/vision/retinanet/)\n",
"2. [Yolo](https://machinelearningmastery.com/how-to-perform-object-detection-with-yolov3-in-keras/)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "R0dfpdDOGhM2"
},
"source": [
"# Part V : Instance Segmentation with Mask R-CNN\n",
"\n",
"### Please run this section on Colab !"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "vOAEQt-pGhM3"
},
"source": [
"Object detection is a task in computer vision that involves identifying the presence, location, and type of one or more objects in a given photograph.\n",
"\n",
"It is a challenging problem that involves building upon methods for object recognition (e.g. where are they), object localization (e.g. what are their extent), and object classification (e.g. what are they).\n",
"\n",
"In recent years, deep learning techniques have achieved state-of-the-art results for object detection, such as on standard benchmark datasets and in computer vision competitions. Most notably is the R-CNN, or Region-Based Convolutional Neural Networks, and the most recent technique called Mask R-CNN that is capable of achieving state-of-the-art results on a range of object detection tasks.\n",
"\n",
"In this section, we will discover how to use the __Mask R-CNN__ model to detect objects in new photographs.\n",
"\n",
"After completing this tutorial, you will know:\n",
"\n",
"- The region-based Convolutional Neural Network family of models for object detection and the most recent variation called Mask R-CNN.\n",
"\n",
"- The best-of-breed open source library implementation of the Mask R-CNN for the Keras deep learning library.\n",
" \n",
"- How to use a pre-trained Mask R-CNN to perform object localization and detection on new photographs.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "ra-bXlWXGhM4"
},
"source": [
"## Mask R-CNN for Object Detection\n",
"\n",
"Object detection is a computer vision task that involves both localizing one or more objects within an image and classifying each object in the image.\n",
"\n",
"It is a challenging computer vision task that requires both successful object localization in order to locate and draw a bounding box around each object in an image, and object classification to predict the correct class of object that was localized.\n",
"\n",
"An extension of object detection involves marking the specific pixels in the image that belong to each detected object instead of using coarse bounding boxes during object localization. This harder version of the problem is generally referred to as object segmentation or semantic segmentation.\n",
"\n",
"The __Region-Based__ Convolutional Neural Network, or R-CNN, is a family of convolutional neural network models designed for object detection, developed by Ross Girshick, et al.\n",
"\n",
"There are perhaps four main variations of the approach, resulting in the current pinnacle called Mask R-CNN. The salient aspects of each variation can be summarized as follows:\n",
"\n",
"- __R-CNN__: Bounding boxes are proposed by the “selective search” algorithm, each of which is stretched and features are extracted via a deep convolutional neural network, such as AlexNet, before a final set of object classifications are made with linear SVMs.\n",
"\n",
"- __Fast R-CNN__: Simplified design with a single model, bounding boxes are still specified as input, but a region-of-interest pooling layer is used after the deep CNN to consolidate regions and the model predicts both class labels and regions of interest directly.\n",
" \n",
"- __Faster R-CNN__: Addition of a Region Proposal Network that interprets features extracted from the deep CNN and learns to propose regions-of-interest directly.\n",
" \n",
"- __Mask R-CNN__: Extension of Faster R-CNN that adds an output model for predicting a mask for each detected object.\n",
"\n",
"The Mask R-CNN model introduced in the 2018 paper titled [Mask R-CNN](https://arxiv.org/abs/1703.06870) is the most recent variation of the family models and supports both object detection and object segmentation. The paper provides a nice summary of the model linage to that point:\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "GlXwuVoOGhM7"
},
"source": [
"### Matterport Mask R-CNN Project\n",
"\n",
"Mask R-CNN is a sophisticated model to implement, especially as compared to a simple or even state-of-the-art deep convolutional neural network model.\n",
"\n",
"Source code is available for each version of the R-CNN model, provided in separate GitHub repositories with prototype models based on the Caffe deep learning framework. For example:\n",
"\n",
"- R-CNN: [Regions with Convolutional Neural Network Features, GitHub](https://github.com/rbgirshick/rcnn)\n",
"\n",
"- Fast R-CNN, [GitHub](https://github.com/rbgirshick/fast-rcnn)\n",
"\n",
"- Faster R-CNN Python Code, [GitHub](https://github.com/rbgirshick/py-faster-rcnn)\n",
"\n",
"- Detectron, Facebook AI, [GitHub](https://github.com/facebookresearch/Detectron)\n",
"\n",
"Instead of developing an implementation of the R-CNN or Mask R-CNN model from scratch, we can use a reliable third-party implementation built on top of the Keras deep learning framework.\n",
"\n",
"The best of breed third-party implementations of Mask R-CNN is the [Mask R-CNN](https://github.com/matterport/Mask_RCNN) Project developed by Matterport. The project is open source released under a permissive license (i.e. MIT license) and the code has been widely used on a variety of projects and Kaggle competitions.\n",
"\n",
"Nevertheless, it is an open source project, subject to the whims of the project developers. As such, I have a fork of the project available, just in case there are major changes to the API in the future.\n",
"\n",
"The project is light on API documentation, although it does provide a number of examples in the form of Python Notebooks that you can use to understand how to use the library by example. Two notebooks that may be helpful to review are:\n",
"\n",
"- Mask R-CNN Demo, [Notebook](https://github.com/matterport/Mask_RCNN/blob/master/samples/demo.ipynb)\n",
"\n",
"- Mask R-CNN – Inspect Trained Model, [Notebook](https://github.com/matterport/Mask_RCNN/blob/master/samples/coco/inspect_model.ipynb)\n",
"\n",
"There are perhaps three main use cases for using the Mask R-CNN model with the Matterport library; they are:\n",
"\n",
"- __Object Detection Application__: Use a pre-trained model for object detection on new images.\n",
"\n",
"- __New Model via Transfer Learning__: Use a pre-trained model as a starting point in developing a model for a new object detection dataset.\n",
" \n",
"- __New Model from Scratch__: Develop a new model from scratch for an object detection dataset.\n",
"\n",
"In order to get familiar with the model and the library, we will look at the first example in the next section.\n",
"\n",
"#### Object Detection With Mask R-CNN\n",
"\n",
"In this section, we will use the Matterport Mask R-CNN library to perform object detection on arbitrary photographs.\n",
"\n",
"Much like using a pre-trained deep CNN for image classification, e.g. such as VGG-16 trained on an ImageNet dataset, we can use a pre-trained Mask R-CNN model to detect objects in new photographs. In this case, we will use a Mask R-CNN trained on the [MS COCO object detection problem](http://cocodataset.org/#home).\n",
"\n",
"#### Mask R-CNN Installation\n",
"\n",
"The first step is to install the library.\n",
"\n",
"At the time of writing, there is no distributed version of the library, so we have to install it manually. The good news is that this is very easy.\n",
"\n",
"Installation involves cloning the GitHub repository and running the installation script on your workstation. If you are having trouble, see the [installation instructions](https://github.com/matterport/Mask_RCNN#installation) buried in the library’s readme file.\n",
"\n",
"#### Step 0. Open Colab and Upload this Notebook\n",
"\n",
"#### Step 1. Clone the Mask R-CNN GitHub Repository\n",
"\n",
"This is as simple as running the following command from your command line:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 104
},
"colab_type": "code",
"id": "HGiDmuejGhM8",
"outputId": "ce5ca013-96e5-4766-d2ed-b4cde9b3ca94"
},
"outputs": [],
"source": [
"!git clone https://github.com/matterport/Mask_RCNN.git"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "S7uXyFVPGhNA"
},
"source": [
"This will create a new local directory with the name Mask_RCNN that looks as follows:"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "raw",
"id": "DhKn5ytcGhNA"
},
"source": [
"Mask_RCNN\n",
"├── assets\n",
"├── build\n",
"│ ├── bdist.macosx-10.13-x86_64\n",
"│ └── lib\n",
"│ └── mrcnn\n",
"├── dist\n",
"├── images\n",
"├── mask_rcnn.egg-info\n",
"├── mrcnn\n",
"└── samples\n",
" ├── balloon\n",
" ├── coco\n",
" ├── nucleus\n",
" └── shapes"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "WvFlDgvJGhNB"
},
"source": [
"#### Step 2. Install the Mask R-CNN Library\n",
"\n",
"The library can be installed directly via pip.\n",
"\n",
"Change directory into the _Mask_RCNN_ directory and run the installation script.\n",
"\n",
"From the command line, type the following:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"colab_type": "code",
"id": "aEUeZhX5GhNB",
"outputId": "be5de5a1-e821-477c-ce28-91bb9f8c3194"
},
"outputs": [],
"source": [
"import os\n",
"os.chdir('./Mask_RCNN')\n",
"!pip3 install -r requirements.txt\n",
"!python3 setup.py install "
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "DlySPeHPGhNE"
},
"source": [
"The library will then install directly and you will see a lot of successful installation messages ending with the following:"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "raw",
"id": "nAww1LboGhNF"
},
"source": [
"...\n",
"Finished processing dependencies for mask-rcnn==2.1"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "55X0zSm7GhNG"
},
"source": [
"#### Step 3: Confirm the Library Was Installed\n",
"\n",
"It is always a good idea to confirm that the library was installed correctly.\n",
"\n",
"You can confirm that the library was installed correctly by querying it via the pip command; for example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 191
},
"colab_type": "code",
"id": "kKXRZ1vTGhNG",
"outputId": "9f0df55c-755f-4e11-a6c3-e8b7418eefcb"
},
"outputs": [],
"source": [
"!pip3 show mask-rcnn"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "f0vwUrMcGhNJ"
},
"source": [
"### Example of Object Localization\n",
"\n",
"We are going to use a pre-trained Mask R-CNN model to detect objects on a new photograph.\n",
"\n",
"#### Step 1. Download Model Weights\n",
"\n",
"First, download the weights for the pre-trained model, specifically a Mask R-CNN trained on the MS Coco dataset.\n",
"\n",
"The weights are available from the project GitHub project and the file is about 250 megabytes. Download the model weights to a file with the name ‘mask_rcnn_coco.h5‘ in your current working directory.\n",
"\n",
"[Download Weights (mask_rcnn_coco.h5)](https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5) (246 megabytes)\n",
"\n",
"#### Step 2. Download Sample Photograph\n",
"\n",
"We also need a photograph in which to detect objects.\n",
"\n",
"Download from Ilias the photograph to your current working directory with the filename ‘african-elephant.jpg‘\n",
"\n",
"\n",
"african-elephant.jpg"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "n8ccmDSvGhNK"
},
"source": [
"#### Step 3. Load Model and Make Prediction\n",
"\n",
"First, the model must be defined via an instance MaskRCNN class.\n",
"\n",
"This class requires a configuration object as a parameter. The configuration object defines how the model might be used during training or inference.\n",
"\n",
"In this case, the configuration will only specify the number of images per batch, which will be one, and the number of classes to predict.\n",
"\n",
"You can see the full extent of the configuration object and the properties that you can override in the [config.py](https://github.com/matterport/Mask_RCNN/blob/master/mrcnn/config.py) file."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "qAfMaOOzGhNL"
},
"outputs": [],
"source": [
"%tensorflow_version 1.x\n",
"from mrcnn.config import Config\n",
"from mrcnn.model import MaskRCNN\n",
"# define the test configuration\n",
"class TestConfig(Config):\n",
" NAME = \"test\"\n",
" GPU_COUNT = 1\n",
" IMAGES_PER_GPU = 1\n",
" NUM_CLASSES = 1 + 80"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "1CmHYT4RGhNN"
},
"source": [
"We can now define the MaskRCNN instance.\n",
"\n",
"We will define the model as type “inference” indicating that we are interested in making predictions and not training. We must also specify a directory where any log messages could be written, which in this case will be the current working directory."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "Sg482-mcGhNO"
},
"outputs": [],
"source": [
"# define the model\n",
"rcnn = MaskRCNN(mode='inference', model_dir='./', config=TestConfig())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install 'h5py==2.10.0' --force-reinstall"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "9BtI50MlGhNR"
},
"source": [
"The next step is to load the weights that we downloaded. You should save it on google drive and then load it."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "_TWgehzsNOSV",
"outputId": "73225d99-e9df-4d1c-c733-a092c97e336c"
},
"outputs": [],
"source": [
"from google.colab import drive\n",
"drive.mount('/content/drive')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 245
},
"colab_type": "code",
"id": "46t9gwLdGhNR",
"outputId": "842b58f4-2678-4ad9-bbcf-aac4656392b7"
},
"outputs": [],
"source": [
"# load coco model weights\n",
"rcnn.load_weights('/content/drive/My Drive/mask_rcnn_coco.h5', by_name=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "bTBwZPvBGhNU"
},
"source": [
"Now we can make a prediction for our image. First, we can load the image and convert it to a NumPy array."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "6k8CgLmCGhNW"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing import image\n",
"# load photograph\n",
"img = image.load_img('/content/drive/My Drive/african-elephant.jpg')\n",
"img = image.img_to_array(img)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "h2hsqN-5GhNZ"
},
"source": [
"We can then make a prediction with the model. Instead of calling `predict()` as we would on a normal Keras model, will call the `detect()` function and pass it the single image."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "ubUzpG2lGhNZ"
},
"outputs": [],
"source": [
"# make prediction\n",
"results = rcnn.detect([img], verbose=0)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "UfgnKPgSGhNc"
},
"source": [
"The result contains a dictionary for each image that we passed into the `detect()` function, in this case, a list of a single dictionary for the one image.\n",
"\n",
"The dictionary has keys for the bounding boxes, masks, and so on, and each key points to a list for multiple possible objects detected in the image.\n",
"\n",
"The keys of the dictionary of note are as follows:\n",
"\n",
"- __‘rois‘__: The bound boxes or regions-of-interest (ROI) for detected objects.\n",
"- __‘masks‘__: The masks for the detected objects.\n",
"- __‘class_ids‘__: The class integers for the detected objects.\n",
"- __‘scores‘__: The probability or confidence for each predicted class.\n",
"\n",
"We can draw each box detected in the image by first getting the dictionary for the first image (e.g. results[0]), and then retrieving the list of bounding boxes (e.g. [‘rois’])."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "Gb2Q5QgLGhNc"
},
"outputs": [],
"source": [
"boxes = results[0]['rois']"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "qUxs3u4qGhNf"
},
"source": [
"Each bounding box is defined in terms of the bottom left and top right coordinates of the bounding box in the image"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "wKPg5GodGhNg"
},
"outputs": [],
"source": [
"y1, x1, y2, x2 = boxes[0]"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Mp9EfU8vGhNj"
},
"source": [
"We can use these coordinates to create a `Rectangle()` from the matplotlib API and draw each rectangle over the top of our image."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 286
},
"colab_type": "code",
"id": "VbLvAtkvGhNk",
"outputId": "1db15efd-d2a8-4a0c-fcac-e00ab09e24c7"
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"from matplotlib import pyplot\n",
"from matplotlib.patches import Rectangle\n",
"ax = pyplot.gca()\n",
"# get coordinates\n",
"y1, x1, y2, x2 = boxes[0]\n",
"# calculate width and height of the box\n",
"width, height = x2 - x1, y2 - y1\n",
"# create the shape\n",
"rect = Rectangle((x1, y1), width, height, fill=False, color='red')\n",
"# draw the box\n",
"ax.add_patch(rect)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "6pekthaaGhNm"
},
"source": [
"To keep things neat, we can create a function to do this that will take the filename of the photograph and the list of bounding boxes to draw and will show the photo with the boxes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "MPA85WZZGhNn"
},
"outputs": [],
"source": [
"# draw an image with detected objects\n",
"def draw_image_with_boxes(filename, boxes_list):\n",
" # load the image\n",
" data = pyplot.imread(filename)\n",
" # plot the image\n",
" pyplot.imshow(data)\n",
" # get the context for drawing boxes\n",
" ax = pyplot.gca()\n",
" # plot each box\n",
" for box in boxes_list:\n",
" # get coordinates\n",
" y1, x1, y2, x2 = box\n",
" # calculate width and height of the box\n",
" width, height = x2 - x1, y2 - y1\n",
" # create the shape\n",
" rect = Rectangle((x1, y1), width, height, fill=False, color='red')\n",
" # draw the box\n",
" ax.add_patch(rect)\n",
" # show the plot\n",
" pyplot.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "TKjNOnR5GhNq"
},
"source": [
"We can now tie all of this together and load the pre-trained model and use it to detect objects in our photograph of an elephant, then draw the photograph with all detected objects.\n",
"\n",
"The complete example is listed below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 269
},
"colab_type": "code",
"id": "XscAeWiLGhNq",
"outputId": "8c0f20a6-1ff0-4162-f7a0-d2ed64370872"
},
"outputs": [],
"source": [
"from keras.preprocessing.image import load_img\n",
"from keras.preprocessing.image import img_to_array\n",
"from mrcnn.config import Config\n",
"from mrcnn.model import MaskRCNN\n",
"from matplotlib import pyplot\n",
"from matplotlib.patches import Rectangle\n",
" \n",
"# draw an image with detected objects\n",
"def draw_image_with_boxes(filename, boxes_list):\n",
" # load the image\n",
" data = pyplot.imread(filename)\n",
" # plot the image\n",
" pyplot.imshow(data)\n",
" # get the context for drawing boxes\n",
" ax = pyplot.gca()\n",
" # plot each box\n",
" for box in boxes_list:\n",
" # get coordinates\n",
" y1, x1, y2, x2 = box\n",
" # calculate width and height of the box\n",
" width, height = x2 - x1, y2 - y1\n",
" # create the shape\n",
" rect = Rectangle((x1, y1), width, height, fill=False, color='red')\n",
" # draw the box\n",
" ax.add_patch(rect)\n",
" # show the plot\n",
" pyplot.show()\n",
" \n",
"# define the test configuration\n",
"class TestConfig(Config):\n",
" NAME = \"test\"\n",
" GPU_COUNT = 1\n",
" IMAGES_PER_GPU = 1\n",
" NUM_CLASSES = 1 + 80\n",
" \n",
"# define the model\n",
"rcnn = MaskRCNN(mode='inference', model_dir='./', config=TestConfig())\n",
"# load coco model weights\n",
"rcnn.load_weights('/content/drive/My Drive/mask_rcnn_coco.h5', by_name=True)\n",
"# load photograph\n",
"img = load_img('/content/drive/My Drive/african-elephant.jpg')\n",
"img = img_to_array(img)\n",
"# make prediction\n",
"results = rcnn.detect([img], verbose=0)\n",
"# visualize the results\n",
"draw_image_with_boxes('/content/drive/My Drive/african-elephant.jpg', results[0]['rois'])\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Gl69hYXeGhNt"
},
"source": [
"Running the example loads the model and performs object detection. More accurately, we have performed object localization, only drawing bounding boxes around detected objects.\n",
"\n",
"In this case, we can see that the model has correctly located the single object in the photo, the elephant, and drawn a red box around it."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "2JHZGM-gGhNt"
},
"source": [
"## Example of Object Detection\n",
"\n",
"Now that we know how to load the model and use it to make a prediction, let’s update the example to perform real object detection.\n",
"\n",
"That is, in addition to localizing objects, we want to know what they are.\n",
"\n",
"The `Mask_RCNN API` provides a function called `display_instances()` that will take the array of pixel values for the loaded image and the aspects of the prediction dictionary, such as the bounding boxes, scores, and class labels, and will plot the photo with all of these annotations.\n",
"\n",
"One of the arguments is the list of predicted class identifiers available in the `class_id` key of the dictionary. The function also needs a mapping of ids to class labels. The pre-trained model was fit with a dataset that had 80 (81 including background) class labels, helpfully provided as a list in the [Mask R-CNN Demo, Notebook Tutorial](https://github.com/matterport/Mask_RCNN/blob/master/samples/demo.ipynb), listed below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "TLdQQg8gGhNv"
},
"outputs": [],
"source": [
"# define 81 classes that the coco model knowns about\n",
"class_names = ['BG', 'person', 'bicycle', 'car', 'motorcycle', 'airplane',\n",
" 'bus', 'train', 'truck', 'boat', 'traffic light',\n",
" 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird',\n",
" 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear',\n",
" 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie',\n",
" 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball',\n",
" 'kite', 'baseball bat', 'baseball glove', 'skateboard',\n",
" 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup',\n",
" 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',\n",
" 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',\n",
" 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',\n",
" 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote',\n",
" 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster',\n",
" 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors',\n",
" 'teddy bear', 'hair drier', 'toothbrush']"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "llndXml9GhNz"
},
"source": [
"We can then provide the details of the prediction for the elephant photo to the display_instances() function; for example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 632
},
"colab_type": "code",
"id": "mIlhDj57GhNz",
"outputId": "9e57f9b3-97af-4cb5-c389-6d6f2435ddc7"
},
"outputs": [],
"source": [
"from mrcnn.visualize import display_instances\n",
"# get dictionary for first prediction\n",
"r = results[0]\n",
"# show photo with bounding boxes, masks, class labels and scores\n",
"display_instances(img, r['rois'], r['masks'], r['class_ids'], class_names, r['scores'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "S8daLDB4GhN2"
},
"source": [
"The `display_instances()` function is flexible, allowing you to only draw the mask or only the bounding boxes. You can learn more about this function in the `visualize.py` source file.\n",
"\n",
"The complete example with this change using the `display_instances()` function is listed below."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "vorB1IyBGhN3"
},
"source": [
"Running the example shows the photograph of the two elephants with the annotations predicted by the Mask R-CNN model, specifically:\n",
"\n",
"- __Bounding Box__: Dotted bounding box around each detected object.\n",
"- __Class Label__: Class label assigned each detected object written in the top left corner of the bounding box.\n",
"- __Prediction Confidence__: Confidence of class label prediction for each detected object written in the top left corner of the bounding box.\n",
"- __Object Mask Outline__: Polygon outline for the mask of each detected object.\n",
"- __Object Mask__: Polygon fill for the mask of each detected object.\n",
"\n",
"The result is very impressive and sparks many ideas for how such a powerful pre-trained model could be used in practice."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "9zKyjQ7nGhN4"
},
"source": [
"# Suggestions for Your Project in DLV\n",
"\n",
"1. Get familiar with different ConvNet architectures such as _EfficientNets_, _MobileNet_, etc. and apply transfer learning to your own dataset. Discuss the resulting confusion matrices and record test set accuracies, F1-scores, etc.\n",
"\n",
"2. Scrape your own image dataset and label objects in your images by means of e.g. [labelImg](https://github.com/tzutalin/labelImg). Use [YOLO](https://github.com/Ma-Dan/keras-yolo4) and [SSD](https://github.com/pierluigiferrari/ssd_keras) to detect objects in your dataset. Compare your results with respect to speed and Intersection of Union (IoU) or Mean Average Precision (MAP) (see lecture notes).\n",
"\n",
"3. Get acquainted with the [Coconut Annotator](https://github.com/jsbroks/coco-annotator) to annotate and segment objects in your images. Use Transfer Learning for object detection and classification. See [Mask RCNN for Object Detection and Segmentation](https://github.com/matterport/Mask_RCNN)\n",
"\n",
"4. Discover which parts of an image are relevant for image classification. Apply GradCam and get familiar with [Layer-Wise Relevance Propagation](https://towardsdatascience.com/indepth-layer-wise-relevance-propagation-340f95deb1ea). Use LRP with Keras (https://pypi.org/project/keras-explain/) to your image classification task.\n",
"\n",
"5. Label joints of animals in your images by means of [DeepLabCut](http://www.mackenziemathislab.org/deeplabcut). Classify animals or poses of animals by means of (relative) joint coordinates. See as well [Real Time Pose Estimation](https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation)"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [
"mlpIDmSCGhKI",
"vOpS1YRXGhKb",
"DJT-DgHvGhKu",
"50DF9pH1GhKw",
"FZYRLtbkGhLV",
"KaLGByZgGhMD",
"3kccXpvRGhMK",
"-XPyCOH1GhMp",
"1cHtuvpBGhMw",
"2JHZGM-gGhNt",
"9zKyjQ7nGhN4"
],
"name": "Jupyter Notebook Block 5 - Object Detection.ipynb",
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}