{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 1 : Toy Neural Network Example" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this section we will walk through a complete implementation of a toy \n", "Neural Network in 2 dimensions. We will first implement a simple linear \n", "classifier and then extend the code to a $2$-layer Neural Network. As we \n", "will see, this extension is surprisingly simple and very few changes are \n", "necessary." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Generating Spiral Dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us generate a classification dataset that is not easily linearly separable. Our favorite example is the spiral dataset, which can be generated as follows:\n" ] }, { "cell_type": "code", "execution_count": 96, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABzuUlEQVR4nO2dd3hUVdrAf+dOS4MUSgih914DSFEQpCkqduy9rH6WVdfF1dVde13L6q66rmvvFWyoYAOkSu+d0EIgvU093x9nJplk7qROCuT8nidPZm6b90457zlvFVJKNBqNRtN8MRpbAI1Go9E0LloRaDQaTTNHKwKNRqNp5mhFoNFoNM0crQg0Go2mmWNtbAFqQ+vWrWWXLl0aWwyNRqM5pli5cuURKWWbituPSUXQpUsXVqxY0dhiaDQazTGFEGKP2XZtGtJoNJpmTkQUgRDiNSHEYSHE+jD7hRDieSHEdiHEWiHEsKB9lwshtvn/Lo+EPBqNRqOpPpFaEbwOTKtk/3Sgp//vOuDfAEKIJOB+YBQwErhfCJEYIZk0Go1GUw0iogiklL8AWZUccibwplQsARKEECnAVOB7KWWWlDIb+J7KFYpGo9FoIkxDOYtTgfSg5/v828JtD0EIcR1qNUGnTp3qR0pNk+FoZiHrVx/AarMwJK0DsXH2xhZJozluOWaihqSUrwCvAKSlpelKeccxH7+9im+/2IgwBEII/udbwrW3jGHUuC6NLZpGc1zSUFFD+4GOQc87+LeF265ppqxZuZ/v5m7C7fbhcnpxlnhwu7z85/nFHM0sbGzxNJrjkoZSBHOAy/zRQycAuVLKg8A8YIoQItHvJJ7i36Zppnz/5WacTm/IdumTLPppRyNIpNEc/0TENCSEeA+YALQWQuxDRQLZAKSULwFfA6cC24Ei4Er/viwhxIPAcv+lHpBSVuZ01hzn5OeWmG73eHzk5TobWBqNpnkQEUUgpbywiv0SuCnMvteA1yIhh+bYZ+Cw9uxLz8Hj9pXb7oiy0m9Qu0aSSqM5vtGZxZomxZTT+xIdY8MwROk2m81CSmpLhgw3DSjTaDR15JiJGtI0D1rGR/HAP2bw2bur+X3ZPqxWg3GTunPGuQMwLHreotHUB1oRaJocSa1iuPrmMVzd2IJoNM0EPcXSaDSaZo5eEWhK2bX9KAu+3UpOVhEDhrbnpEndiY7RGb0azfGOVgQaAOZ/s4X3/7cSt8eH9Ek2r8/gm8838venTyU+IbqxxdNoNPWINg1pKMhz8t5rK3C5vEifqt7hcnnJyynmk3dWN65wGo2m3tGKQMO6VQewWEO/Cl6vZPnivY0gkUajaUi0ItCAqGRXJfs0Gs3xgVYEGgYNa4/XG1rQ1WI1GDm2cyNIpNFoGhKtCDTExjm47NoR2O2W0oxeh8NKYlI051w8pHGF02g09Y6OGtIAcNLknnTr1Zofv9tG9tEiBg5tz5jxXXFE2RpbNI1GU89oRaAppUPnRC69dmRji6HRaBoYbRrSaDSaZo5WBBqNRtPM0YpAo9FomjlaEWg0Gk0zRysCjUajaeZERBEIIaYJIbYIIbYLIWab7H9GCLHa/7dVCJETtM8btG9OJOTRaDQaTfWpc/ioEMICvAhMBvYBy4UQc6SUGwPHSCn/GHT8zcDQoEsUSymH1FWOpoSzxI3T6aVFSwdC12jQaDRNnEjkEYwEtkspdwIIId4HzgQ2hjn+QuD+CLxuk6Mgz8l/X/iNtb/vByA+IZrLrh/JkBEdGlkyjUajCU8kTEOpQHrQ833+bSEIIToDXYEFQZujhBArhBBLhBAzw72IEOI6/3ErMjMzIyB2ZPH5JI/cM481K/fj8fjweHwcPVLIi0/+wpYNGY0tnkaj0YSloZ3Fs4CPpZTeoG2dpZRpwEXAs0KI7mYnSilfkVKmSSnT2rRp0xCy1ohN6w5xNLMQr9dXbrvL5eWz99aUPne7vWxad4gtGzLweHwVL6PRaDQNTiRMQ/uBjkHPO/i3mTELuCl4g5Ryv///TiHETyj/wY4IyNWg7N+bg8drPrDvS88FYNniPfz3n78BIH0SwyK46U8nMXBo+waTU6PRaCoSiRXBcqCnEKKrEMKOGuxDon+EEH2AROC3oG2JQgiH/3FrYCzhfQtNmjbJcVhNmrsAtGkby769Ofzn2UWUFLspKXbjdHooLnLz9APz2bMrq4Gl1Wg0mjLqrAiklB7g/4B5wCbgQynlBiHEA0KIM4IOnQW8L6UMLnzfF1ghhFgD/Ag8FhxtdCwxaHgq0TF2hFE+SsjusHDm+YP4/svNuNzekPOkhH88uCBku6axkMBR4ACQ18iyaDQNQ0Sqj0opvwa+rrDtvgrP/2Zy3mJgYCRkaGwsFoN7HpnC84/9zKH9eRgWA6TkvMuGMWREB+bN3aTGGBNyc4rZtzeHDp0SGlRmTUWKgNWAh7IPqwUwGLA0kkwaTf2jy1BHkDbJLXjwmRlkHMynqNBFaqcE7HY1gPTs24aNaw+ZnmexGBw+lK8VQaMigbWAs8L2PGAb0KfBJao7TmAnkInqR9oG6A7oHhOa8ugSE/VAckoLuvZoVaoEAE6Z3rvS/r+pHeMbQDJNeAoJVQKgFEQGYZdzTRY3sAI4BHhRq5xD/m2hJkpN80YrggaiZUI05146NEQZWK2CvgPbkZzSsnEE0/hxoWbNZvj8f8cSB1CDfzASdZ+HG14cTZNGm4YakBlnDyA62s7Hb/+Oy+VFIBg1rjOXXa+7gjU+LQg/64+m4X0EPqAE9RO11+L8bMyVlw/lDE+pvWia4w6tCBqYSdN7cfKUHuTklBAbZ8fh0B9B08CGSodJp/wAagC9KhybD+wFioF4/3lRYa6bg0qLKQh6jQ6EX32AMuFs88shgZZAf8BR3ZuhcuURTlZNc0WPQo2AYTFIahXT2GJoQuiKmv3vRfkL4oBuqEF7HWrgtwO5lCmLAuAgMMx/fDDZKAd04NiA87YI6B1GhixgC+WVUS7wO3AC5grEhVIeRaiVTTJK2WQSuioQfnkPAEko53F0GFk0zQWtCDSaUgTKZBJsNtmPmp0HzEaFFc6RKOfrFmB4hX3bCR2IfahBuwvmM/xdJueAcv5mAa2CtnlQUU3r/XL4UI7tXX5ZuqNWIwHl4fUfF/AdZKKU1cgwsmiaC1oRVMKu7UfZsfUICYnRDE5LxWbTseTNixJgazWPzUMNtMHfkYIwxwqUecls8C0Oc44M2leEyt00S3gLOLY3o6q1JKMUSD5KqVVUMl6UOaxHmNctRtWRLESteDpQPdNSkf+6+UAM0InQFZOmqaAVgQkul5dnH17Ats2ZSAkWi8BiMfjzA5Pp3C2p0nN9Xh8L5m1j/tdbKC5yMXBoe2bOGkyrNrENJH1zpwQ4ghrwWqMGodqyvQbHCkLNNhbCh2qGi+WPRs3+za4fjZrN/x7mmGBy/cfaUMogH/OVhkQpCjNygDX+Y6T/+QFgCMpvYUYBasWzj7JVVD5q9dEf9ZlomhpaEZjw2Xtr2LoxE7e/JETgJ/fUA/N57r/n8PvyfXz92QZysorp1bcNZ14wiJRUlQfwr6d/Zc3K/bic6tyFP+5kxZJ0Hnp2hlYG9U46ygYfGIB2oSqid6dy52w4smtwbCKh0dipqAGx4gBsI/xA2hXlj6h4jh1l099P7fIA7Kj3wCwyyu7ffhjYg/JlxKBWA8FyBMxgm4BRFa7hQ5moKotW2gSMo3afhaY+0XkEJvz03bZSJRCMy+nh1Rd+45VnFrFjyxGOZhaydOEe7r/ja/bszGLPzqxySgBUn4KSYjdffLi2IW+hGVKIUgKBSJuAzXw/4We8lWEWh2+GgRpIzZy/XSlTEBb/nwNVsiLcYJjkv5bNf54AElBmnoBJqTo5DbGUn+clh3lNA2Xu2YMyJxVS5nsIt+ooJjT5bjfhlUAAD2qFoRPamhp6RWCCsyTMD0DCb7/swuctm1X5fBJniYd3X1vB0BEdyu0LPmbt7wfqS1wNoCJhws1ED1DeyVoVBSgHcWWk+q8dD7TFPM/AAAahBtd8lMJIpOoZcTuUQihEmYOCbfIx/uuGG3CFf39FxeRAmWY2BL2+RIWzJvi31yRpruI9HKjm+TmoAsPHRYmx4watCEzo1qs12zaFdkFzu73YbBac3tCZ4taNhxkzvisWi2HacCY6Rtd3qV8qm71XZU+viJlTNZhUlLLIQ9m+i1Cz/3AL7Fj/Xwlq1ZJPeMerGzUzP0qZ36Gb/1hQEU17KnmdBNTgbhYS2hplmjmKur9ElILIpmbmmhjK5ykUUL3VE5T5JJzoSKWmgzYNmXDhlWnYHZZyvw27w8KQER3C1guyOyykje6ENLHB2h0WTjk1XNy4JjK0IvysvKYd7cxqDgWIQ60+cikLxdyHsutXRh6wDOXHyPafsxQ1Qw5mLWqgDpi2vKgQ0EDBQjvKtOSgzORkQ608RqKS3yrLC7CgVjDtKBuIbVReS8kI+m8B+vmfe4FVwMoqzq+IQCnF6lCA8vXsJHwUlqauaEVgQvderbn30WkMHpZKXAsH7TvEc+m1I7nxzhOxWkMHG6vNYOyEbsTGObjpTydht1twOKxYrYZSIGkdOHlKz0a4k+ZEK8rMJgEEauCsaTmFysIcvZjnBuSgZvrh2ERZHD+UDfSbgrYV+P8qDqo+lA0+QDwwGkhD+Q7GUjPTV0ViMQ8JDVQs7eT/3xmV1BZ4f7ajFFxN6zB5qZ6fYDtKyexGrYJWUj6nQxMpmo1pyOn0sHHNQbw+Sb+B7YiJrbx+S+duSdz+14mlz3Oyivjq0w106ZHElvUZCMPA4/Zgt1tJ6dCS8y8bBsCQtA48+9o5rFySTlGhi74D21UZcqqJBAZqUNyHmrFL1My3EzX/mrsq2VfZTDYPldlbEWcl57n8+6JRPoFwJpqK5wvUAB4JBMpmv4oyZRW4fh/M3z+JWqWEUwIOlMI4Gmb/TpQfJBw5hJroAv6e1iizliZSNAtFsHzxHv7z/GIMIVQAnNfHxVencfLUijVkzNm66TBP/X0+Pq8Pt9uHzW4gBEw5vS+DhqXSb1A7RJDNKDbOwUmnhEvQ0dQPPtRAHIeaKdflq13ZzN7AfDYbWH2Eozqz2JhKjqtve3oMapWRjVI6cagQ19pUZLWgmg8KlD/A7J4KCE3AC6Yy5/9BtCKILMe9Isg4mM8rzy7C5Sr/4333vyvo0r0VXXtUvqT2+SQvPvkLzpIyZ5jb5UMIH7u2Z3HhlWn1IrcmQKCOfjZqoE0l1HSTg7LRB5tdulPmYK0pMVSeFWwWtSMIb55x+K9ZsTxFYF/ALNMCNQuvaB4yUCUp6huD6puYLCi5zVY6EvUZlRBecULlDurKTEfVdUxrqktEfARCiGlCiC1CiO1CiNkm+68QQmQKIVb7/64J2ne5EGKb/+/ySMgTzM/fb8PrDZ1ZuN1evv9ys+k5Pp/k0P48MjMKSN+dTXFRaNSJlLB982FKimsakaKpPi6Ug3UHKlv4AMpOvL/CMWtRg0PA9uzzn1OThLBgOhJ+kPKiInOCcwNsqGzbyn5Off3HBq4b7HgNfq3BlIWYBq7fjaZZNronofccyEuwoZRBuBm/WQJeMG0q2V9VhJP2IdSUOq8IhBAW4EVgMspAu1wIMcekCf0HUsr/q3BuEnA/ai0vgZX+c2v7Cw4h60ghXpPYfinh6JHQGdq6VQd49fnFFBW5kBISEqORMvwXq7J9mrqyEzXQB7/HPpQTsS1qsDmM+Q/fh6oiWhsTgoH6aZgpeel/3VEoU5SV6uUGtEA5Wg9QFj7anlCTjw2lDFwo5RZF043paI3yLexErXbsKIdyQGkJYABlZSp8lCm3qlp/tkF9fmYrs6OoiUFwuQrpPz4d9blFo1aFNY0Ya55EwjQ0EtgupdwJIIR4HzgTlTVSFVOB76WUWf5zvwemAe9FQC4A+g5sx+/L9pUz7QDY7Bb6Dy4/y9q3J5vnH/upXGZwZkb4kLWOXZKIjqlN0xBN9cgk/CC/EDW4RhHeVl1ZGKgZPtTXNhBnHw7D/7o1retvp/omHju1a0jT0CRRudM3EOF0CGUqaoEanKsq4GigsqHNfn8SNeAHK4KtlHdeF6M+yz7+62gqIxJTjVTUpxJgn39bRc4RQqwVQnwshOhYw3NrzQkndSWuhQOLpWzGJgRERVmZWMFZ/PXnG/G4QwcAq83AsAgMQ13DajWIirZy1U0nRFJUTY3JR80Mw32Na9oHeh/VUwJN0UzTlAk05OmJyl+obhXfynwBwUo+0I/BLKw3uPaUJhwN5SyeC7wnpXQKIa4H3gAmVnFOOYQQ1wHXAXTq1Kna5zkcVv725HTef+N3Vvy2F59PMiQtlQuvTCOuZfll+f69Ofh8oV8aj9vHqHGdiY6xcXB/Ht16tWbyqX10Ebl6pw3qB17ZDzmwr2JBNQsqdLQmVJVRbKDMOTVVMJraEY95BVdBeZNfVWG3AZOUJhyRUAT7Ueo+QAfKe/OQUgYHE78KPBF07oQK5/5k9iJSyleAVwDS0tJqpOJbJkRz3a1jue7WsZUe16lLInt3Z4fUC3I4rPQZkMzEab3JySpiwbytvPWfZXTqksjJ03qRmKS7jUUOJ8rRG/jKGJTZl8MhUcv/QEeuBNTss6adtyqbgSainLbhqoZqIk8SZfkVFaOoOgc9txN+smCh6fpYmg6RUATLgZ5CiK6ogX0WcFHwAUKIFCnlQf/TM1DplADzgEeEEAH1PgW4OwIy1YrpZ/VnycLduLxlMxAhwGY3GD2+G7u2H+Wxv36Hx+PD4/axbtUB5s3ZxF0PTKZ7L11nve64UF+nik7aQASKWSMWUF/jQNmDQDJUbUhAmZoqEuhbrBV+wyJQSYK7KDP9JKKa6AT7ZwK1nCrmfwRWcMHfh3zURCPg6E9FzWObt7Ko891LKT3A/6EG9U3Ah1LKDUKIB4QQZ/gPu0UIsUEIsQa4BbjCf24W8CDq178ceCDgOG4M2neI5/Z7J9ImOQ6b3YLVZtCleyv++th0oqNtvPzMQkqKPaV+BI/bR0mJh5efWaijhyLCfsxn5T4q74wVbLOvS637boSaEAK1irQSaBysqNXdicB4VE0ls89iIEoZBMJyDZTS8AKLKQtD/h0VVuxFrT53U724ltpQjHKBphO+81zTQByLA1haWppcsWJFvV3fWeLm7f+uYOmvu3A6vXTplsSMswfw8nOLcLtCE11sdguP/vMM2iTrVnx1YyXhZ/0pqAgSs6zfZMpWBHWlEDUDzUENQh1Qs0bdTKXpIymr15RP+OzkihioCPZI+vx2ocJZA3IJlDmrSwRfo+YIIVZKKUOyYI/7zOLa8NyjP7F1U2bpoL97RxYvP7fItNcA6CEicoQLlwwkV5ll5oLyDVRWrqAmxKJi3zXHHgKlBLZS80J4uUROEeSglEDF7m57UH6Ppudnat6GMRP27Mxi2+bMkJm/2+UtF4IaTEJSDK3b6giiutMB869koEtXOJUbaKGoad6UUDslUFWdqJoSrklPoGhe00Mrggrs3HbENBMZQBgCR5QVq029bVargSPKyvV/HFuu6JymtiSils/B5RsMVFJQIuEjQ6yEbwavaT6EyzKvCkHlSXE1pbLos6ZZJ0mbhoBd24+yad0homNsFBe58Zp0GAOIjbPz18ens+CbLezdlU3HLolMmt6LpNZ6NRA5uqD8AVmUFUGzomb8dkKLnBkoJ69WxBoP1VMEgflvoAPcECI7J26Nef9mg/LZ0E2HZq0IvF4fLzzxC+tXH8Dr8WGxmreZDBCfEE1SqxjOvWRoA0rZHHEQmr27C/OSETrTVxMgCRWhY/YbDgz6fVGhyLmoVWQSkTeMxBE6MTFQORFtI/xakaFZK4L5X29h/eoDpbWFvN7K7cyOKJ2dWDUFqJC8PFS4Z2fq1j0rwAHMZ3vS/1o621cTjzIhVpyNW1ArzXaU+QLqKxw4C1USvaIyaoPKRWma1vjmrQi+2VquwFxl2O2q5aSmMnKB1ZT9CJyoH4WVsnr9Xah5sbaqnMG6FLgGyqqdHqSsXEhrVKmRhijgJ1GpVGYrkhyacpmLZq0ISkrCDyCGIUrrDlksBnEtHYyfrPsOV45ZxIakbKA+iAr1HEHNlIFALbfDVaJseuF4msbCQOV9RLR2ZTUpIbwz2E1ZS9KmR9NcpzQQg4a2L60oGozNbmHMhK7EJ0bTIt7B+Ck9+PvTp1XZ57h54yN8V69gPJRvxF5dwjVBac+xUa5Zc/xTWcCCpCkPt816RTBz1mBWLk2nuNhdmizmcFgZNqoD195SeYE6TUUCzrjqRG3UpopIAiq6Yycqa9SOqhHTvhbX0mhqgqSsQ54XZe9vT6ipJ4qyInkViaX++07XnmatCFq1ieXBZ2Yw56N1rFt1gOgYG5NP681Jp2gTUM0RqIiI6sRyGyhlIFAOPrOZkgdVnyWKshyBeFQRMo2mIdkCZFBm9sxHKYXhhA6h/VH1jHz+PwP1Pe/eIJLWlmatCEApgytv1A1m6oYXFd55lOqtCIqB9UHP+1MWWRRoRXmQshVGa1RSWdN1tmmOV/IprwTwPy4GNlPmE0hBrRRiUYP+1qBjQfXVboEqjtf0TJlN12ilOUaQqEihcJVDw+EN+ltPWaLYTspS9AON6DNRPzqNpqE5gnkUkER9L7P9f5tREXI5wDZCJ0SBMOc19SVondCKQFNHslE20ZrWdwlGUjb478P8R3SY8D2MNZr6ImDaqQofSglsp/LfQhHmFXQbF60INHUkh7oXfJOoFYGLygf6DcAKdN6ApuFoQ/XLl/hQA31lCELLpDQ+WhFUgcvlJTOjAJezaRaLanzshP8axaIcvdVpF9iCqhWKRK0+ttVEQI2mDsSgkiCrO1RWVfxQonJimhbN3lkcDp/Xx0fvrOaHrzYjEEgpOXlaLy64fBgWi9afZbRFdX6qiAF0RTmBc1E5BjsIP+NvS/VmXgEzUR/0PEbTMATKpBxC+cEchPYbCJDs32f2PReoEhhNL6lMK4IwfPT2Kn74eku5EhQ/ztuK1yu59NoRjShZU8OOioQIjgIKtJZsTdmXPx6VSGa2sgqOsY5HKY7KkGhfgaZhiUP1Sgb1/d4T5rj0MNsFKrKoR5j9jYueUpngcnpClIDa7uXn77dRXKxt1OVJAsaiKjv2AkajQuiCZ/iGf39F55vFvz1Af1TuQGVfzRh0KKmmYXGiQpoPoHxV4SYivjD7OgC9aarf24goAiHENCHEFiHEdiHEbJP9twshNgoh1goh5gshOgft8wohVvv/5kRCnrqSnVUcttGMxSLIOhKuZWJzxoJyrLUjfAZla1Rv2HaomX9HYCTKPxDAAZyAKh7WCvNyvr0iJrVGUzV7gSWo3IAtqJDSmpIdUYkiTZ1NQ0IIC/AiMBkV+7dcCDFHSrkx6LBVQJqUskgI8QfgCeAC/75iKeWQusoRSRISo0oLzlXE6/GRmFRfJWybA7Eo+35lBLKK4/x/WagZWRzK76CLzGkaihxUsmRdwqOhqXfQi8SKYCSwXUq5U0rpAt4Hzgw+QEr5o5QyEFe1BLVOapK4nB4sVgsnndIduz10GeeT8Oo/F3Nwf1V27OMVNyoxxqxJTCTIAhajHMt7UHMLgFHAYLQS0DQs+6i+EgiXcxCoiNp0iYSzOJXyHpJ9qF9tOK4Gvgl6HiWEWIGaBj4mpfzc7CQhxHXAdQCdOnWqi7ymbN10mDdfXsb+vTkIQzB0RAdGjO3MskV78Hp8pSsEr8fH70vT2bDmIH9/+jTatW8uA5MPFbZ5iLLSD0ko+36kYg4CWca+CtsKULOy46sGVNHBo3gKS2jRLQVhaHdd08RVzeNsqPLqOags4+ACjO1pqi0qAzRo1JAQ4hKUkXh80ObOUsr9QohuwAIhxDopZUg8opTyFeAVgLS0tDqFjOzZmcXShbvx+SQjxnTCZrPw5N9+KHMO+ySrlu2jbbs4/njPyfzjoQXlTEVSgtPp5bP31vCHO06siyjHEDtRSiB4kD4KbAQGReg1wtlRpf+1jw9FkLfjAD9f9BDZa3ciLBZs8TGMeemPdDp9TGOLpgmhFSoTuLJVgYH6bjpQ4aNJlJWmSKIphotWJBKKYD/K6xegg39bOYQQpwD3AOOllKV2BSnlfv//nUKIn1DlJc0C0yPCe6+tYMG3W3G71aA//5sttIyPCokQ8np9ZGcVsXThbtPrSJ9k0/pD9SVmE8NHWcenYCRKGaxE/QBSMI+KyECZeYLt/Akmx1WWUFZXG23TwF1UwtxRN+LKLlAzCtx4ikr46cKHmP7jP2gzoir/iaZhaY8ycrgJjQayUBYqHdyL2Max1kc7EuvR5UBPIURXIYQdmAWUi/4RQgwFXgbOkFIeDtqeKIRw+B+3RsUgBjuZI8q2zYdZMG8rLpcXKdXv0OX0ciTTPArIWeKhsMAVNoEsNq7p1hePLB4qj9vPQ+nuFaj0+XzKfAi7UUvlQv91clCxA2a6PqGS10msmchNkNwt6XzU+UJcWfl+JVCGt9jF2sfeayTJNOGxURbpZvU/T0XlzvQFxhAaKn3sUecVgZTSI4T4P2AeSkW+JqXcIIR4AFghpZwDPImaCn7kD8vcK6U8A/VOviyECBTufqxCtFFEWbhgB26XyawzzNhjd1gZMCSFdasOmOyzMGVGc5m9BcpEVFZmI1Ca9zf/sRIVFpqH+Ru8F5UvEOxEc6AWl+mENh9vmok41cXn9fLtpDtwHs0zP0BKcjbsblCZNNXFgYp0O35/7xHxEUgpvwa+rrDtvqDHp4Q5bzFKtTYILqe34kSsUqxWgzETupGSGs8zDy8AwOeTSAmjxnZpRj2MBSrNvqowusCbG1C2VUVW7UAtvYNnU91QkUF7UY66RP9r17ThfdPi4PxVuPOLwx8gBAn9uzSYPBpNMM2qxMSIMZ1ZsnB3aVvKYIQhMIRaBfh8krgWDm77ywQcDit9BiTzz9fPY83K/RQWuOgzILkZRQsFCLiBdhM+e7Km+FAmpIqDfGuaepSFGc6cAjKXbMTWMpa2J/RFGAbFGVlsfvlL9n6+CG9J+AgUS7SdwXdf1IDSajRlNCtFMCQtFbvNQok31MQhfZJxk3swZkI3oqNtdOqaWC672O6wMmJM55Dzmg8C6IRyjOWj7PyRUAbhvoISZYqqTuXSxmfNI++w5qG3MexWpE9iaxHDyH/8gcXX/wOfy1OpEgAY9vcraJ3Wu4Gk1WjK0/R/YRHEsBikje6EWfUIh8NKl+5J9OmfTOduSWFLTGgMVHmIrtT962PFXBEcQiWVLQJ+RTmb69rzoP7Y+cGPrH7wLbwlLtx5RXgKiik+eJSfL34Ed15RlUoA4MAPvzeApBqNOc1qRQBw2tkDWLZ4T7lwUSHAajcoyHdy5/WfUVLspv/gFM6+aAjJKS0quVpzpjMqRvogauZuQ4WYBq8SBKo6abgsZDO7fyaqnkvAFyFR4adOVGZx0+Lgj6v45eJHkD4T34nZtjBkLttEwd4M4jolh+w7umobWWt2ENc5mXbjB9cq+cxTVELetv1EJScS0y6pxudrjm+ErIn3tImQlpYmV6xYUevzN607xH9f+I2c7GKklHTsnEhUtJUdW4+UKgjDEDiirDzwj9No204rg+pxAJV45kUN4G1RYXfrCHUyG6hIoIqp90tQ0UcVMVCZm02nzpMzp4APO16Ap7DuHaeExcCwWWk/JY0J796DNSYKd0Ex38/4C0dXbAWh/FhRbRKYtuBpU4VhhpSSNQ+/w7rH30MYBj6Xm+Txg5nw7j04kpqbn+tYwIOaXB2hLFQ1cqHTQoiVUsq0kO3NURGA+oFkHSnCajPIPlrEw3+ZF5JUZhgwenw3LrlmBL/9souD+3Lp1C2JUeO64HA0u8VUNZGoaB8rZWGkW1DNZALvr4GKJu6LStSJoawo149hrhsoV92mXqSuKSVHcll843Ps/Xwh0hPGbBVcZaCaWKLsdDn3JE56824WXv0kO96dj89ZVvZcWAxiOyVjj48hZ+MeotomMODOC+h381mm5szNL89l+R0v4SkqU1aG3UqroT2Z8dsL5Y7N23GAzf/6gtzNe2lzQj96Xz+D6LbHfv7GsYMblYvjomziZKACNbpF5BW0IqiEb+ds5KM3V+HxhC7lW8Q78Lh9eL0+XE4vjigrjigr9z0+nTbJTa/lXNOkCGXyyUWNjK1RiiGPstEyBZWmvxjz+i4GMIzyJasbhy3/+YrfbnoO6fVWPtAHbs0wwOdTjmQJ0l1521NLlI0ZS//FnGHXI71Vm5csMQ56XzeDUf+4sXSbz+0hc/kWFpx9HyWHc0zPmfHbCyQNVAPM/nnLWXDO/XjdHqTbiyXKjiXKzmmLniehb3MOkmhItqOymCt+qQxUbc+6l6oIpwialbM4HLGxDixW87eiIM9JcZG7dLXgLPGQn+fkP88takgRj1FcqPITy1ElJrJRs/8MlFLwoVYJPtRyeA8qMsnss4ihKfR6XffUhyy+/h9qFVDVHMq/XxiChAFd6HfrOdjiqpEPYRh8PeGP1VICAN4iJ1temkvJEZW3seeLRbyXfA7fT59tqgQADKuFvG2qEozX4+Gnix7GU+RE+kuveEtcuHILWXTt09WSQRMJDhP+S3UY5YPbjvr9RLbkilYEwPATOoZ9/80WTNIn2b71CIUF1a1M2FxZg5r1Bw/4ByhbGQTjQ2UUp6JCVA3KQkdbohzFjRvJlbF4PStmv1Lj86THi/T6GHLvJbjziqo83uf21NjvYDhsZK3eTvaG3fx88cO4cgpw54d/LZ/bS0K/zmQsWs9HnS/ClZ1vIrgkc9nmcmYlTX0S7vstUfk721G/kS0oX1rkSsFrRQDExNq55e7xOKKsREVZsZn0IaiIALzhbMMaVOlos4GosplMoKZRd1TZqcGoJfFwVPRRw+MuKCZ7w25+uvAhvh53q2pIUQucR3LJ2bK3ylm+NTaKqFbxSFfl5qOKSI+XqORE1jz8Nt7iyicohsNG2zH9sUY7+G7anyk+eLRGr6WpDUWoGX0G6ntegKq7Ffw5t8NcGUjU7ybw3fGilMCWiEmnPZ5+Bgxpz/P/O5dVy/axY1smv3y/A6cz/I+xdXIcLROafnnZxqOEms/g7ZTNTayofIXGwV1YzOIbnmHXRz8rc0ldfGmGIPnEgax56J1KD3O0bsngv1xC+tdLKD6UZXpMi+7tKdp/pFxugrAYxHZpx8bnP2XXBz+Fl9UQGFYLXc45iTEv/ZFfr3i8ypVHyx6ppH+1lPaTh+NIaHzT3LGHROXBHA7aFiitFnAidUM5hDuhooWKKQusEEHXqUhW0LXqhl4RBBEVbWP0+K4MSeuAYQk/iNnsBlffNLoBJTsWiSW8vdMsW9hAJak1jUS+H864l53vLVAz8zoGVFhjHAz9+5VkrdoW9phBd1/EhRmf0v+2c+h/y9lYY0N9CZZoO5O/fpSUScOwRNuxtYjBGhdNXNcUupx9Ijve/L5yWYXA5/aw7+ulLPvTS+z5fGGVshemH2bRNU/xQfvz2PLqV9W6X00wB1FKwEf5WX2wuXQnSgFYUKvfPqiy7h39jysbpiMT7KNXBCb06ts2bM/ixKRorrppNEsX7eaLD9fSo08bJk7rRbxeHVQgGpVwdpTQL2uKf9tB/38r0AVVgK7xyV6/i0M/rq71+e2njSBr1XY8BcUkjxtI2uPXkti/C3FdUyhMzww53hoXTeLArqXhnx1OO4G+N81k4/OfquQxQyC9Xk58fTbxPTswee7D5GzeS9bq7cR2bEts52Q+7npR1c5l/35XTgFbX/6y8mMNAT5ZbsWw9LYXaZ3Wm1ZDju1KsA1LdVpd+lCBEq1Rg35byvobeIGtYc6Lxbz/R81pduGj+Xkl/Dp/B/vTc+ncLYlxJ3cjJjbU/vzbL7t47YXf8PjbVNpsFuwOC+ddNpR3X12Bx+vD55XY7AY2m4W/Pjad9h0bz5TR9JCotpNHTPYFZv8dKMtKbhorAYCF1zzFtte+qfrACgiLQYfTTuCUzx803b9/3nLmn3M/3qIgJ58QOFq15Py972ONKv89LNibwf55K7BG2+kwY3RY08yPsx5k94c/1VjesAiBMISpYmk3YTBTvn0ci71pN2NvOiyieu0u7Si/GCj7vwsVKWdBTZi2UqZQhP9vCDU1n+o8AmDX9qM8/tfv8Xh9uF1e7A6LGsQfn0ZKaugbund3Nj98tZn0PTnk5RRz5LB5AxuEWkXc88jUGst0/JKOWvKGmw0ZqC9+uEWpF5V7kIMqRZGCqgtfv/i8Xt6MOw0ZlMRVGYbdis/lwRobhT0+lhlLXyQ2NXzS2+aX5rD8rpcRhoH0+ohp35pJnz9Qp1j9N2OnV+kgrjZC0LJHKnnb9pnvthi07NWBGYv/iT1e+wyqZiPKQVwVgf7fG1BRdQZqMtUFVc4lB1WavRgVRdeZ2mTZh1MEzcY0JKXkX0/9QnFx2Q/c5fTidnl55dlF3P/kqSHndOqSyBnnDeSeW7+kpLiSgUHC9s2ZuJwe7Drj2I9Za8tgBCpyIsH/3Ivqd3CQ8nWGJOpHsQfVuqJ+6+SsvPvVqpWAxSB1chojnrqBPZ/+Qv7Og7Q9oR/dLp6ELbZyE2GfG86gx+VTyVq9HVvLWBL6dW5SBQ4t0XZ63zCDVfe9bupIll4f+TsPsvqBtxj59B8aQcJjja6oVXFlEYYGasBfjYoukkHH70atFlIwb+8aGZrNqJVxIJ+c7NAaNlLC3l3ZFOQ5iWsZOuP85vMNuKsTyifAtKxps6Wq9yww449DlbVeR/gfS0AxrAfGoX44BcA2ymZP7VDRF7X/SntKXKz/x0eVHtNmTH8mfnQ/MSmtAEjsd2mNX8ca7aDt6P61ktGMTmeMZffHP1c7Aa0yEvt3pfe1M9j43Kd4ip2m4bI+p5sd7/ygFUG1iEa1utyJSqi0oL6jgQE/BujlP7YE8/yaPdR3D+Rmowi8Pl/4mZdQ+83YsvEwXpNGNuVOF9C7X1vs1cg/aD4kUj5kzoz9qBVAdfMxfKjlcWvg96DzvKhEtTxU1EUgLC+wmqgee79YVDbwxTqQ00bgjY3BvnEnvhXbATjpjdnEpLSiIM8JAuJaNH7f6pFP38ChX9bgyi1U/oda1DgClFmoVyq2uGhOX/YvFpz7Nw4vXG96aNj6ShoTYoABFbYFmjsFxoxDlZwfucSxcEREEQghpgHPoe7qVSnlYxX2O4A3Ub/So8AFUsrd/n13A1ejfs23SCnnRUKmiqSkxuOIsuIsCZ2ptkmOCxv107pNHOm7c8Je125XTuQrbzwhUqIeJ3RFfdSVDRiyiv1mx+/2/1Uc6SRqlpWNWmHsRa1K7H5ZwkUkSdQPzcLhhWvocFoPOKEPn6xvg9crceUC7dsjzhxLfOZBWs/fzpLnV3Jofx4SZT689tYxpHZMqMF9RJaY9q05e9PrbH99HgcXrCJj8QacR8zbhLYZ1YeWPTuw68Of8bnKm8As0Xb6/d9ZAES3TeTkD+/n424X4S0pf5ywWuh89on1czPNhooTlMrCreu/TWudncVCCAvKpT0ZFSu1HLgwuAm9EOJGYJCU8gYhxCzgLCnlBUKIfsB7qPTR9sAPQC8pZaWjQ22dxetWHeD5x37C41aRQBaLwGqzcNffTqFHH3MH3+YNGTz9wPyQyqQWi2DA0BT69m/HSZN7EBvX+DPDpkcRyu6fTVnHsfomhtCMZgOVrdwhaFshqmdyNgHTk5QSl0ty65UHKS4y+10EtoWuLKNjbAxJ68B5lw6lVZvYut1CHfl+xl/Y9/XSkO1GlJ1hf7ucgXfNYteHP7HwqicQFjUj9bk9jHj6Bvr+4cxy56x9/D3VdKfYBVJiibJjT4zjjJUv674GEWclahIT/N0zgH5EqupuvUUNCSFGA3+TUk71P78bQEr5aNAx8/zH/CaEsKLWQW2A2cHHBh9X2WvWJXz0QHou387ZyP70XLp0T2Lq6X2r7Dfww9db+OD1lVisBlJKLBaDW2ZPoM+A6tWE14D6yIMbzjQGUahwu1iUggr97q9cUswrz2VRUlzz34Xwm4oeef70Rs063//9ChacdX9IjSBLtINzt71JTHvVD9pdUMyB71fi83hpf8owHInmv4NDv65l0wufU3woiw7TR9H7+hlhj9XUBQ8qC/kIarJhQU1gIucfqM+ooVRUrGCAfcCocMdIKT1CiFyglX/7kgrnVuxUAoAQ4jrgOoBOnTrVWtj2HeO5qoZZwaec2puxJ3dj68bD2O0WevVri8Wik7JrRqvGFgDljKu8pEJhoQ9ZS10lJeTnObn3ti8ZOa4LI8d2pmefNg0eFZQ6OY1B91zMmgffQtgsCATS52PCe/eWKgEAW1w0nc8aV+X12p04iHYnDqpPkTWAGo4HoBSCF2XWbJjvzjHjLJZSvgK8AmpF0NCvHx1tY/BwUx2lqRY2VL+BbTTuqqByevdzhM0qry65OSV8/+VmfvpuG/0HteOWuyc0+MRh8N0X0evq6RycvwpLlJ3UqWlYYyJva/Y6XWSt3oE1NoqE/l2aVCjssUu4Xt71+4p1ZT+qKEaADv5tZsfs85uG4lGexOqcqzluaI/66A8Q2t+4aZCcYuWEk2JY/HMR3jq6NNwuLxvWHuTHeVuZNL03ebklWK1Gg/mTotsm0u3CiRG/rjOngBV/foVtb3yHdLlVm027jZiUJCZ++neSBnWP+Gtq6pdI+AisKGfxJNSvezlwkZRyQ9AxNwEDg5zFZ0spzxdC9AfepcxZPB/oWV/OYk1TYjcqPrrprQ58PslP3xXw0Vt5FBXWXVlZLIL4xGjycktAQreerbjutrG0ST727Ow+j5cvhl5H7pZ00xBSW0IsJzz7f2St30Vcp2S6XzRR90ZuQtRriQkhxKnAsyjvxmtSyoeFEA8AK6SUc4QQUcBbwFBU7dRZUsqd/nPvAa5CGcZuk1JWWeRFK4LjAYkyEx2kLPC9pf9/IHIiYE7piQrx3E3DrSIsgJ2jR1oy58Nsli1Kp6iwemUnqkIYghYtHDz1ylnHXO/rPZ8v5JfLHsNTEJqcCYAQquyG040lxoGwGEz99vGIJtBpao+uNVTPSCnxeiXWMC0vNeFwo8I9HZTFS+ehMobtqOSxQNJNFirks8D/3CByKwqByvCMRSmkUFv3/r053H/H17jddU+mckRZueTaEZw06diq5Lli9iuse+KDGp0TlZzIBfs+wLDohMvGptnXGqoOGQfz+eit31m/+iB2u4WTTunBGecNrLR+kNfr44sP1vLdl5spLnLTum0s5182jFHjujSc4Mc0NkIrKLb0/1Ukyf8nUcrChQo0y6nk+g6UsgF/XzmTYwygB1WVwU7tlMAd903k5WcWUlTkBqkGdJfLQ0lxzRwKzhIP6buza3ROY+IpdlJyJJeo5EQs0Q68xdXPdvUWOclcupnkMXpV0FTRisDP0cxC/nbnVxQXe5A+SXGRm2/nbGLzhgzueWRq2GiI1/+9hCW/7i5NODtyuJBX/7kYKSUt46PYtPYQsS0cnHBiFxKSal4tUGOGoEx5xAErUAO8rHBMHKpUb+BrLlEJZIdRysGCSkBLRtWEqZq+A9vxzH/P4cC+XASClA4tWbNyP/966lfTrPVwOBxW2rVv+rZzr8vN0tteZPsb81R5amHUvLyEEDVSHJqKOFFVR6OoryxjrQj8zP1kHSUlSgkEcLu87N2VzaZ1h+g3KDSpIye7mMU/78LjLm+ecDm9/Of5xVgsBs4SDzabwSfvrOba28Yyckztyw1rzIhGxRqkU5bB3BLV2COR8iYeQdmqovYIIcqVlOjZJ3wjo3BYrILRJ3WtkxwNwaJrnmL3J7+WK3NtRNmwWOxgCNUy0ycRhoEtPgZXdkHINaTXS9vR/RpS7OMEH2VtLgNlqROA/kR66NaKwM/GNYfwmRSXc5Z42Lop01QRpO8y7ysL4HH7ShWE2///lWcX0W9guyZRqOz4woEy7TQOsXF2Zl4wiM8/WIPbVbXPQgi4efYE04ZITYniw9ns+uhnfBXKcvtK3NjiYxn/7j34nG6ShvYgum0iuVvSmTvqRtXjOYjO546vlxyG45/tqAq9wTW5slE9DiKb4Kc9m35atDT/otrtFlqEGbh/XbAjZDVQGYYh+H1ZetUHao4pcrKL2bIhA69HglCfc0Ji+IFPSnjq7/NZ/PPOBpSy5uRtP4AlylxZeUtctBnZh84zx9Giczus0Q6KDx7FsIbOLXd/+CP5Ow/Ut7jHGV7K9+YIEDBvRtbUphWBn6ln9MXhMIlqEJg6fnOyili5tGaDus8nQ4rXmSGlrHN2q6Zh8Hp9PHz3t6xfc1B9ZlJ9zgX5lXcM83p8vPbiEo4cDjWlNBVadEkOa9sXhhHSoWzTi1+YHi+9Pra/+V29yHj8Upm/SVC99pfVR5uG/IwY04kdWzL54ZstWCwGQqiZ281/Hm/asGbX9qPYbJYarQgABgwJX0AqL6eYt19dzool6fi8Pnr3T+bSa0fQoXNije9H0zCsWbmfvNySELOix1P190L6JIt/2skZ5zfNOj4x7VsT3S6Jwr2hfSWk14vP7cGwlk2eSjLNS1/7XB5KwpTF1oTDRvjwaEl1gxuqi1YEfoQQXHhVGlNO78vGtYeIirYyaHhq2ISflglRYWftVqthOhAMGtY+bKSI2+3l73d9Q/bRotJGOJvXZ/Dg7G955PkzGr20scac/XtyahQtFIzH46Mgv2lH0ziz8k23W6LsHJz/Ox1nlBVw7HDaKLLW7cBXoX+BNS6a9pNDQtc1lWKg+hLvorwyMFBhzpEdurVpqAKt2sRy4qTujBjTudKsz249WxOfEBXSndJmM8J2O8s6UrFOfhkrFu8lP88Z0g3N7fLyzRcbw5ylaWzatIurdXaw3WHB7fYx96N1bNt8mIZM7vR5vWQu20zmss34vOHNlbKSfd4KLVz7/d9MHEktEbayVYIlyk5C3050nKEbN9Wcjqj2qzbKylJ3oj4CI/SKoJYIIbjz/kk8ft8PFBY4QYLXJ+nUNZH9e3NME4x27TiKz+vDMKlEuWNrpunM0uuVbN2YUS/3oKk7w0Z14u1Xl+N0eqjJOG5YBG6Xl4ULduB2e7HbrfTq15bb7jm53rPT93+3gp8veQSfU9mZDYed8W//hdQpobP21KkjSJ/zG7LC5Mbn8pAycWi5bY6klpy58iVWP/gWez5diGG30vPKaQz80wU6q7hWCJQy6IByHluor7LUusREHfH5JFs2ZJCdVUTXHq0ozHfx5N9+oMRkULfZDP7z4UWmyWnffLGRT95ZjdtVYQYmYPiojtwye0I93YGmrhzcn8s/H/uZzIwCpJSl4cJmtGjpwGazkJtTHLL6s9stnH3RYKbPrL8M3PydB/h80DV4isqbpKwxDmaufZUW3dqHHD9n5I14CktUGKkQGA4b7cYNRPp8JPTrTN+bzyK+Zwc0TZ9wJSa0aaiOGIag78B2jBnfjZTUeLr1ak10jC3kOKvNYPRJXcNmKI+d0A3DCN1nt1uYdqZOxmkKSClZ/PNO7r/zK+647lP+968lHM0sJCU1nkf+eQYPPjuDUSeGTxKLibXz9CtnMe3MflhMZv0ul5cf522rz1tg07++wGtSK8nn9rLp33NCtrfo1p6z1r/GgNvPo/XIPqRMHIIwBId+WcvBBavY/NJcvhh6Hfu/axoTM03t0IogwhiG4I/3TiQ2zk5UlBXDInBEWWnfIZ4LrwrvMGsZH8Xt904kroWdqGgrjigrVpvBWbMG06tv2wa8A0043nx5Ka//aym7t2dx5HAhv8zfzr23zeXwIeVQTU5pwfrV4ePlo2Ks3HjJh7z/xsqwYcQlJZGpcBqO3K37kO7Q1arP7SFv6z7Tc2LaJTH84as5fcmLOLPy8RY5SxvfS48Xb5GTXy57LMR8pDl20IqgHujcLYln/nsOl/9hFOdePJRb757AA/84rcpM0j4DknngmRkkJEbj9fqw2Sx88u5qXnvxN3xe/SNrTDIO5vPr/J04nWWDqM+ralJ98s5qQOWW5GSFKc8M5GQV4/H4TDPYQU0iBg+vXxNL29H9TJPELNF22pzQt9Jzndn55KzfbbrPU1RC9rpdkRBR0whoZ3E94XBYGTO+W43Pe/ahHzl8qACfT5bmKPz2yy6SWscy84KmGW/eHNi07hDCZNokJaxbpVYBO7YerfQa4RQAqOY1UdG2ev+Me187g/VPfYjX6abUuy0Elig7va+dUem5wsR0GUB6vKx55B2ExaDTmWPpcvaJuPOL2PjC56TP/Q1HUgv63jSTjqeP1u0smyBaETQh9u7K4tDBvJD8BJfTy3dzN2lF0IhERVsxwgxgjij1M/L5JMIAWYPFmxDQpm0cA4e1Z8a5A0lqVb8VaqNaxzNj8QssuvZpDi9RYcltR/dj7Ct3ENW6Yjnw8tjj42g1vCeZSzdTMUTK63Sx+5NfwCdJn/sb6x5/j+KMbJzZ+aV5BYcXb6DXNacy6pmb6ufmNLVGK4ImRNaRIn+T81D7cWGBC59PmjqUNfXPkLQOpr3R7HYLJ0/tBUCvvm0whMBbgy5qKR3iefSfZ0RIyuoR37sjp/7yLJ6iEoAaFYQb99pdfDX2ZrwlbrzFToTdinR5VLKrXzl4CkvIWrdL9Z0LMml6CkvY8vKX9L1pJi17pEbyljR1RPsImhAduySG7X7Vtl2cVgKNSFS0jZv/PB67w4LdYcEwlPmvR582nDpTRXUlJMWQ1Dr8jL7igsLusDTqKs8aE1XjqqAJfTpx7ra3GPrAFXQ5bzwpJw3CsJnkCHh95ZRAAAns+3ppLSXW1Bd1WhEIIZKAD4AuqIay50spsyscMwT4N6pIvBd4WEr5gX/f68B4VF9CgCuklKvrItOxTKs2saSd0Infl6bjCsonsDssXHDF8EaUTAMwcGh7nnn1HJYv3kNBvpPe/ZPp2acNQggyDubzwesrycwoDHt+y/goiorcWCwG0ic568JBx2QnO0dSSwbecT4Aax97j4M/ran2ucIQYSuaahqPupqGZgPzpZSPCSFm+5//ucIxRcBlUsptQoj2wEohxDwpZY5//5+klB/XUY7jgsOH8hGGQAhRWvSuddtYLrhiOGkndGps8TRAXAtHqSkI1GeWmVHAPx//mZLiKkI/Bdzy5/G0TIiifYf4SlugHit0OnMMqx98E2+FrmXCYoAQod3MfJJOM8c2oISa6lDXb+KZwAT/4zeAn6igCKSUW4MeHxBCHAbaUHmj2eOWfXtz2Lsri1atY+nVr21pBMXhQ/ncd/tXOEvcBMKx7Q4L3Xq1jkhXs+IiF9u3HCEq2kr3Xm20mamOHEjP5YUnVDax1+sLyRI2Ize7hH8+8TN3PzTluFACAAl9O9PnxjPZ8tJcPIVlPofYzm2xtYwlZ8MuPAUlGHYrwmIw+sVbiW6rq+k2Ner6bUyWUh70Pz6Eav4aFiHESMAO7Aja/LAQ4j5gPjBbStm0yzHWEpfTw3OP/sTWjYcRFoGQ0DIxmtkPTKZVm1g+f39tOSWgzvGyetk+0ndn07FL7X8833y+gU/fXYPFaiClxBFl44/3nEzXHq0icGfND2eJm4f/8i0FBS5q4BcG1Gf67msruPfRaSH7vF4fG9YcJC+nhG69WtO+Q+VRPJEkUGqmNqGdI5+8gQ7TRrL11a9wZhfQ9dyT6HbxKVgcNvZ9u5wD3y3HntSSzmeNY89nC/mkz+UIi4WeV06l700zsUbrjn2h5KFaVErUvDme+qozBNWoNSSE+AFoZ7LrHuANKWVC0LHZUkrTEUsIkYJaMVwupVwStO0QSjm8AuyQUj4Q5vzrgOsAOnXqNHzPnj2Vyt3UeOOlpfw6f3u5OjSGIUjtlMBDz87glis/Ije7JOQ8q83ggsuHMWVG5ck+4VizYj8vPPlzSCZrdIyNZ/97DlHRoeUwNOXxen2sXr6PA+m5tE1pQVGRi/deW1nr8tMWi+C1Ty4pt23f3hwev+97XP7idT6fZNCwVG6888R6LUKXvWE3S299gUM/r0FYLXQ5bzyj/nFjlaGkNcVTVMKctD9QsPuQ6nMMWKIdJA7symm/PodhOz5WSJFhG3CAsvLTBkoZ9KWuyiBcraEq330p5SmVXDRDCJEipTzoH9RDO1io41oCXwH3BJSA/9qB1YRTCPE/4M5K5HgFpSxIS0s7pirl+bw+f5XJChUcfZKMg3ns25tDdLTdVBFYLEadett+9el603IGPp9k+eK9nDipe62vXR+Eq87aWGRnFfHQ7G8pyHdSUuzBajNASjye2n8FHVHlla/P6+PJv/1AXk75z3/d7/uZ+/E6zpo1uNavVRkFezL4auzNuPOLQUqk18fuD34i87eNnLXhNSz2yE0Str/5PYV7M0qVAIC32EnOxj3s/WIRXc4dH7HXOrbJobwSwP84E2gLtK6XV63rL24OcLn/8eXAFxUPEELYgc+ANys6hf3KA6HWozOB9XWUp8ngcXtZ8usuXn5mIe/8d0XYsFCLxSA/t4QTJ3UPCS8E1cVq+KiOtZbj6BHzKBaX00N2Vvj+CA2JlJJvPt/ATZd+yJXnvMNtV33MLz9sb2yxAHjl2UUczSwsLSvucfsqVQKGRRAdbcPusBATGzqQ2uwWxk8pX09+y8bDpo5ml8vL/K+31PEOwrP+6Q/xFDvLJYf53B6KM7LZ89nCGl9v5wc/8vnga3i39Uy+mXQHGYs3lO7b89mvIRVPATwFxez5YlHtbuC45BDmXcl8qB7G9UNd12OPAR8KIa4G9gDnAwgh0oAbpJTX+LedBLQSQlzhPy8QJvqOEKINar2zGrihjvI0CUqK3Tx097ccPlSAs8SDYYiwteo9bh+duiby2fvmIXhDRqQSHaNWBM4SN5+8u4aF83fgcnvpOzCZC69Io33H8Mv47r3bcPRIEbJCtrLDYaVL96Ta3WCE+eitVXz/1ebSlUt2VjFv/WcZbreXSdN7N5pchQUutm48XO0+A0LA2RcOJrVjAgOGpHBgXy5P3P+DciZ7fBiGQbderTn7wiHlzsvLDV0JBigqqr8idId+WYs0maB4CorJXLKRbhecXO1rrXn4bdY++m7pYH/ox9XMm/InTvn8QdqfMhx7vHmHPWExcCS2qN0NHJdUlpZef/XG6qQIpJRHgUkm21cA1/gfvw28Heb8iXV5/abK159t4ND+/NJVQLiWlnaHhVNO60NhgYud246aDjhrVh5QBeeE4NF7vyd9T3ZpDaK1vx9g68avefCZGbRtZ/5jOvP8gaxetq9csTSr1aB12zgGDGlvek5DUlzk4rsvN4f0YXA5vXzyzmpOntKz0UxFLpenRs5Tq9VC23YtGOZfwXXp3orn/ncuq5fvIyermO69W9OtZ+jSvkfvNnjD9DiuT2Ud1zmZ7LU7Q7Zboh3Edq407qMcrrxC1jz8TjmzD4C3yMmSW/7J2Rtfp/d1M9j3zbLSyKIAhsNGjyum1u4GjkvaAkcIrS5gUEUsTp1oOsbY44iFP+40NQVZbQZJrWOw2Sy0ahPLrCuGc/6lQ8k6UhTWIehxe3G6vGxYc5AD+3JLlQAAUg2Ycz8Ob1FL7ZjA7Icm071Xa4RQMpxwUhf+8sjUJhFCeuhAfth7d7m85OXVLIjM5fKy6KedvPnKMr75fEOls+2qSEiMJjqm+nMlt9vLf//5G3f94XO++HAtJcVubDYLI8Z0ZvKMPqZKAFQi4ejx3bA7ymfo2h0WLryy/hIJB9xxPpaY0IgdYQh6XFzmGsxev4vvps/mzdhTebft2az4y6t4ggb9Iyu34gtTgjpv2348xU5SJg2j9w2nY4m2Y9isGHYblig7g++9hNbDepme2zxphcq9Df5NGEAcSknUD9pVXw+EWwEYQjDj7AFMOrW8uaN9h5Z4wvgQYls4iIqysmPrEdMoFZ9Psnn9oZDtUkqWLdrDN19sJD+nhD4Dk7nu1jEkt2/ZpKo/JiRFh713JDVylOfmFPP3P31DQb4TZ4kHm93CZ++v5Y6/TqR3/5rPpoQQtE1pSV5uZrXPcTo9ZBzMZ+5H61i+eA/3P3kqNrMSDBW48sYT6NApnnlzN1OQ56Rrz1acd+lQevRuU2O5q0u7kwYx4onrWf6nl0vLRAirhYkf/42oNgkA5G5J58sxN6uZvJR4i51sfPYTMpdsZNr8pxFCcHjhelVvyAyLoXIIhGDkkzfQ66rppM/9DWEx6HzWuJCOaBoBDELF3RxEhY8mAynU57xdK4J6YNTYzvzw9RY8FZb7UsLgtNBiWy0TohkzoRu//bKrXISP3WHh3IuHIIQgPiEKu8NiGgEUnxgdsu3911fy47fbSk1Ci3/ayYrf0rn/iemV+hQamsSkGHr3T2bz+oxy75fNbmHM+K7Y7dXvdfvmK8vIzioqLfccMDf98/Gfef5/59bKxFRYULu0Frfbx+FDBSxbuIexJ1ddjtwwBFPP6MfUMxq2G13fG8+kx6WTyVi4DkuUneQTB2FYy97zVQ+8gbeovEPZW+LiyPItHP5tI8lj+rPro5/CXr/tqL7l+hUn9O1MQt+6J0ge3xioiH2zqP36e0VNhDn9vIEkJEWXDWJCDeqnndOf1m3jTM+5/IZRTD29ryp3bKiB/5JrRjB+ck8ARo7tbDqTdzisTK/QyjLraBHzv95SvomKT3W/eve1FfwyfzvfztnI3l1ZEbrjunHjnSfSrVdr7HYL0TE2bDYLA4e055JrRlT7Gj6fZNXSdNOa/263l+1bj9RYrq0bD+Nyms90haDK+H5niYeVS/fW+HUbGluLGDpMH0XKyUPLKQGAjJ/XmXYe87k8HPZHBTmP5JlfWECH6SMjLq8m8ugVQT0Q18LBQ8+dzi8/bGfVsnRatIxi4rRe9B0YXsNbLAbnXjKUsy8agtvlwe6wlhv4Y+Mc3H7vRJ575EeU5Uni9fiYekYfhleoQ7R5/SEsFiO0ibq/icrWTYfxen3+jlip3HjHidWeLTudHvbtySYm1k5KamRWFrFxDu55ZCoH9+eSmVFASmo8bZLNFWZYpAxrkhNChDijq2LOh2uZ+4l5DoYwBC1aOrjm5tGsWLyXzRsyOHyowOR11b0dy0S1jafoQKgSNRw2otsmAJB84sDSXgTBWGOiSJk4rCHE1NQRrQjqiehoG1NP78vU02uWEWwYIiThKECfAck8/8Z5bFxziJISN30HJNMyIdQsFBVtC615HESwr2HNyv0s+HYrp5zWJ+Q41SXNi81uQQjBd3M38fHbqzEsAq/HR5t2Lbj17gkkp0Qm/C8lNT6scjm0P4+P3l7FxrWHiIq2MnFaL6bP7F86KzcsBj37tmXrxtCcRp9X0qNP9W3tRzMLmfPRulBF6qdN2zjiE6NY8M1WTjylB+MmduepB+aHKA2b3cKECjkDxxoDbj+PxX94NiTaRwhB57NPBGDo368oiwjym5As0Xbaju5P6xGNF/6rqT5VlphoiqSlpckVK1Y0thhNFpfLyy1XfERxNWPQU1Jb8tiLZ5Y+93p9fPruan74egsup5f4xGhGjunEj99tKzfYCaEia57+z9n+hjr1Q8bBfO6//StKStylpmq73UKfAcnccV9Z9HL67mweuvtb3C6vKgIn1HGXXjOCk/wmtgB7dmbxw9dbOHq4gD4D2zFxai/iWqrZ+4Jvt/Le/1aYrgYsFoHVauD073NEWRk6ogNtU1rwzecbVb6GAIHg9PMGcOb5x3ZXOSkly+74N1v+PQdhtyIQCIvBKV8+QvKY/qXHZa/fxfK7Xibj13VY46Lpc/0MBt19ERaHLjndlAhXYkIrguMMn0+ydeNhNq07xNefb0AgcLs9GIaBx+szLZKWkBjNc/87t/T5f55bxLLFe0IGfbOvisUiuPiaEfWa+PXKs4tY/PPOkNd3OKzMfmhyubDMI4cL+OaLjWzbdJg2yS2Yfma/kNXAwvnbeePlZbg9PqRPYrEaWK0G19w8mhFjOvPTd9t49zVzRWCGw2HljvsmkpAUw6rl6QAMH9WRNslNL1EqZ+NuDsxfha1lDJ1njsUeX2aC83m97J+3gkM/rSYqOZHuF00iJkUVJizcl8mhX9Zij4+l/eThES0/oWk4tCJoBhxIz+WJv/1AcZGK8fZ4fHTt3opBw9oTnxjNW68sCzF3CAH9Bqdw532TMAxBTlYRd1z3aY1q6VgsgqtuPIFxk+rHDHLbVR+TnVUcukNA+9R4pp/VjxNOrF6EUUmxm5sv/6hc458AhiHo1DWRG24/kb/eNjfUNCQwrzYqYPKpvbnk2qbrGJU+HwuvepJdH/2MlFI5hX0+Jrz/VzrOGI2nqIRvTr6dnE178RQUYzhsCMNg/Lv30PlM3T/geCGcItBRQ8cJPq+Px+//nuysIkqKPZQUe/C4fezZlYXNYWX85J4MHdkxJGlJSti+KZPZN31BTlYRO7cdwaTDYKV4vZLXX15GQX5oqKWUkv3pOWzfnBk2AqcqYluEcbhKOLAvlzdeWsb9d3yJs8TcFLZjaybPPLSAO677lEfv/S5sAUefT7JvTw5zPlzLrCuGY7erlpSgTEDhFI2AJlUoz4ztb3zH7k9+wVvsxFfiwlNQjKfIyY+zHsSZlcfqh94ma+1OPAVK4fqcqifxzxc/jDu/ejWppJRkrdnB/u9WUJKZU493o4k02ll8nLBpfYYqXFZhxupyepk3ZxPTz+zHH24fx4J52/jgjZXlzB5Op4fMjAL+/Y+FtIyPCqlLVB0shmDNiv3lYuYP7c/j2Ud/5GhmIRaLgc8rOe+yoUw2cUxXxpQZfXj71eVhTTUet5cD6Xk89cAC/vLwlHLRVquWpfOvp39VKwAJRw6HbyUJahW1bNEerrllDL37J/PzD9vJzy1h0PBUvF4fb7+yrNQ/EMBmtzD6pK41uqdI4C4sZuurX7Pnk1+xtoim97Wn0enMsaZhxhuf/zTE4QvKl7Hro5/Z/vo8fM5QRWpYLKR/tYRusyqvBlOwJ4PvT7ubgj0ZCKsFn9NNr+tmMOoff0AYTVtJarQiOG7IyS4O2yOl0D9TNywGo8Z15r3XQs1qPp9k+5bKM2j7D05h49qDpr4CKcEbFG/ucXt5+J555OeW+I9Xg+eHb/5Om7ZxDBnRoTq3BcCJk3qwdeNhFv4YWhcnmG2bDvP9V5tLezf4fJLX/7202rb+AIFoqY5dEsvlMvi8vtJwUWeJByGUEpg4tVeDN/lx5xcxd+SNFKQfVglfQMYva+l6wcmMezW0mrsrNzS8FcDrcuPKKcDrdJnulz4f3mLzfaXHSMm3p9xJwa5D5XIOtr36NS26pdD/lrOre1uaRkKr6uOE7j1bmyZTAeUyiYsK3WEjfCwWo3wtowr7ppzeh5Mm98BiCZ1x+nw+Bg0ry5petXxfaZOVYFxOL3M+WlfV7ZTDMARde7ZCVPFtlRK++rSs9PHRzEKKiiofxMxo2y7ONITXsBjcds/J/N9dJ3HSpO6cPK0Xf35gMhdeFWJyrXc2vvAZBXsySpUAgKewhJ3vL+Doqm0hx7efMgJhDTVtWRw2UiYMocOpo1Sf4QpIr4/2kyuvd3R40XqKM7JDEs88RSWsf/KD6t6SphHRiuA4oV1qS4akpZrasfftyeGRe+aRl1tCm7ax2MLYuq1Wg45dEkz3GYaqknnORUNoER8VkjU9c9ZgEoJKXRw5XBg2Dv9IZuXmGTN+XbATWQ3fRXBzF0eUNWySmdVq0K1Xq5DCe3a7hUuvC+/0NQzBoGGpXH3zGC6/flS91gKqjJ3vLQip9gngLXGzd+5vIdsH33MxtpYx5QZ7S4yDdhMG03pkH4Y/dDX2hDgMe5mRwBobRb/bziG2Q+X3WJieGbZ+VcmR3OrekqYR0YrgOOKG20/k1LP7hziEPR4f2zdn8o8HF2BYDC66Oi1EYdgdFi66cjiXXTdKnR/0u3Y4rJx2zgDiWjiIT4jmkefPYOasQfTu35aRYzpz532TmHHOgHLX69glAVuYEgydatF/uaiaNX+Ck9taxkfRrUfoYC8MQccuidz/xKnceOeJdO6WSIuWDvoPbsefH5jcJMpzV0W48E1hEVhM2j7GdWzLmStfpvtlk4lul0TLnqkMe/AqJn36AEII4jonM3Ptq/S75WwSB3QlZdJQxr97D2mPXFOlLK2G9cTnMQ8E0HWFjg10+OhxyC1XfERuTqhj0O6w8NfHp9OpSyLrVh3gs/fWkHEwj7btWjBz1mAGD1emnd07jvLpu2vYtf0oiUnRnHbOAEaN61IjGXw+yb23zuXQwfxytfbtdou/LHbNZtLPPvwjq5bvq/QYu93CDbePK1dy42hmIQ/O/pbiIhfOEg+OKCuOKCt/fWxateP8d+84ypJfd+P1+hgxujM9+7Zp9Aqum/71BcvvermcaQjAEmXnzDX/Ib5n9X0wkeCHM+7hwPzfy/kTLNEOJn3+AKmTG950pjFH5xE0Iy4/6y3TePfoGBvX3Tq2tHFKfVOQ7+T1l5ayamk6Ukpat43jsutH1mrGvXHtQR6/74ew+1vGO7jo6hGm0TsFeSUs/20vudnFpHSIZ9iojtUqDQ3wwRsr+eGrLaX9Jex2K8NGdeT6P5pH5zQUPreH76bPJnPZJjwFJQiLgWG3MeS+Sxn05wtrd02PuseKheeqg9fpYuVf/suWV77EU+SkZc9URv7jRjqeOqpWsmjqh1o3r9cce7RpG0dmRmiUiMfjI7VTw5Wgjmvh4P/+dBJutxe3y0t0jK3Wg2f3XuGbdkdH27jh9nH0H1xewRQVunjtX0tYtTQdIQTRMTZmXTGs2kpg+5ZMVWbDVT7U9vdl6fy+ND2k2F9DYtisTP3uCfZ9u5z0uYuxxkXT49LJJA3qXuNr5e88wOIbn+Pg/N+RUmKLjcbn9hDdLomBd11A7+tPr/JzszjsjHz6D4x46gakx4thYp7SNF30p3Uccu4lQ/jvC7+VC5u02Qz6DWpHckrLBpfHZrNUe/ANh91hpUVLB/kmHcs8Hi/t2ocquKcfnM/uHVmlfQ7cuV5e//dSomPtDBtZ9apo0U87TauWOks8/PLD9kZVBADCMOh46qiQWXfx4WyQkujk8m0ufV4v+79ZxtFV24nt1JYu556Ez+Vh7gk34czKL60eGkggK9h9iOV3vkzezoOMfOL6sHJIKUn/8je2+XMRul04ka7nT9DK4BiiTp+UECIJ+ADoAuwGzpdSZpsc5wUCMYN7pZRn+Ld3Bd5H9WdbCVwqpax5vJ+mHCec2BWPy8sHb62iqMCFMARjJ3Tj4quPXVutEIIzzh/ER2+tKpehbLNbGDi0Pa3alG+OvnvHUfbuyg4Jh3W5VC/k6igCj8sbtnG9WYmKxiZr7Q5+vfxxcjbtBQEte6Ry4ut/pvXwXpQczeXrcbdSeOAInoJirLHRLPvjv+hxxVQ8hc6QEtIBPEUlbH7hcwb9eRZRrUKVrZSSXy59lL1fLCpNWDv08xo2vzyXaT88pWsSHSPUVWXPBuZLKR8TQsz2P/+zyXHFUsohJtsfB56RUr4vhHgJuBr4dx1l0gDjJvVgzMndKSxwEhVtq/OMvCkw+bTeuFwe5n60HumT+Hw+Ro3tzOU3hNqh9+/NxQhjzjh8ML9ar5c2ujNLF+0JaRFqdzROJnFllGTm8PX4P+LOLQvNzdmwm28n3s5ZG//HsjteIn/nQXxudS+BUhJbXp5bZcKY4bCRtWo77U8JzSfI+GVtOSUAKp8ha9V2dr4zn55XTovE7WnqmboqgjOBCf7HbwA/Ya4IQhDK6DgRuCjo/L+hFUHEMAxBi5ZRjS1GxBD+ns/TTu9LdlYxLVo6VO8FE9qmxIXNtE5sFVOt1xs4rD29/D0OAt3e7A4LqR3jGT2+aSmCLa9+ZVoiwuvysOmFz9n72cJSJRCMz+NFWC1IT/gVjjuvkNUPvYWtRQxtRpXvr7Hro5/wFJmY6wpL2P7291oRHCPUNY8gWUp50P/4EKrLshlRQogVQoglQoiZ/m2tgBwpZeDbuQ8IbejrRwhxnf8aKzIzq99MXHP8YbVZaJMcF1YJgEp+a902FqNCFrTdYeHMC6rXI8AwBH+892Quu2Ekvfsn06N3G2ZdMZy/PDKtya2wjq7cZppg5nO6ObJiCzJMJUGLw26aUVwOCRm/ruebiXeQ/tWScruEJfz7oGsMHTtU+UkJIX4QQqw3+Tsz+Dip4lDDTcI6+0OWLgKeFULUOLRBSvmKlDJNSpnWpk3jZHNqjh1cTg8nTuxOXAuHv+ubyh+YOWswYydU3Uw+gMViMO7k7vzl4Sn89fFpTJreu1rlrhuaxAFdMBwmZTFsVpIGdydxUJgVjM/HhPfuxZ4Yh7VFdHilICXeYieL//AswSHn3WadjDU6tDqsNTaKnldMrdW9aBqeKhWBlPIUKeUAk78vgAwhRAqA/39on0B1jf3+/ztR5qOhwFEgQQgRME91APbX+Y40zZ6jmYX86YbP+ez9teTllGBYBFJKbvvLBE47q3/VFzgG6X3dDNMoHcNupe//ncUJL9yKNSaq3CzdEuMg7fHr6DxzHBce+oQpXz3K6BdvxRob3pzoPJpL4b6yFXmbE/rR/fIp6hy/T8YaF0XyiQPpesHJEbxDTX1S17XbHOBy/+PLgS8qHiCESBRCOPyPWwNjgY3+FcSPwLmVnX8skrlsM1+Nu4XX7VN4J+lMlt/1Mh6TZbumfnjtxd/Iz3OWOnk9bh8up5eXn1kUtvbQsU5M+9ZMnfc4sZ2TscQ4sMQ4cLRqycA/z8KeEEd8z1RO+fJhupx3Ei26pdDu5CFM+uwB+t40E1ArhzYn9OP3v/7PtFx1AOmVWGPKVgBCCEa/cAuTv3qUnldOpdvFk5jw7r1M/vKRWiWmaRqHOmUWCyFaAR8CnYA9qPDRLCFEGnCDlPIaIcQY4GXAh1I8z0op/+s/vxsqfDQJWAVcIqWssqhMU84sPrp6O1+Pu6WcA80SZaftmP5M++GpRpSseeByebnhwvdUz+IKREVZmf3QlAYvGd2QSCnZ/PJclt/xEobNgvT68BQ5ERaBsFho0aUd4/53F21P6Bdy7oEFq1hw1n1hG9EIi0HbcQM59cd/1PdtaOqJesksllIeBSaZbF8BXON/vBgYGOb8nUDT7e9XC36/9zU8FcLxvCUuMpdu4siKLbROq7/evhpVPz/s3EaIcnWPjkcK9x5m+R0v4S124g3q7ik9EunxkbslnXlT/sTMNa/SomtKuXNdOQVhu7cBxLRvxfi37q4nyTWNiXbrR5jMpZtMu7xLr0/t09Qrjigbnbslme4TArp0N993vLD1tW+Q3sqT3bxODxuf+zRke/LY/vhc5u0+4/t24tztb1dZklpzbKIVQYSJSjYvsSxsFqLbHd+DUFPhyhtPICrKisVfBlsYArvDwlU3nYC1iYV9RpqifZn4XJX3hpZuD0d+3xqyPTo5iX63nF3eWSwE1tgoTnrzbl0y4jhGK4IIM+CO802jLiw2Kx1nnNAIEjU/OndL4qHnTmfitF5069mK0Sd24d5HpzFybJfGFq3eaTd+MNa4KpIILQYJ/buY7hr+6LWMefl2koZ0J7pdEp3PGsuM316g9fBekRdW02TQZagjjJSSZXf8m83/noPFYQMpscZEMfmbx2g1pEdji6c5zvGUuPh8wFUUpmeaZhIHSB43gGnzn9az/GaG7kfQwBQdPMrh3zbiSIwj+aRBGJVkYGo0kaTkSC4r/vwKuz76Ga/LhTQpkGeJcZD26DX0u1k3lm9OaEWg0TRDNr34OcvvegVvcWhUdkK/zpy1/rVGkErTWIRTBNpHoNEcx3iKnGELynlMlIOmeaIVgUZzHJM6NQ1hluEroPWIPg0vkKZJohWBRnOMI6Us7TdckaRB3el6/oTQYnIS0ucuJmPhOtPzNM0LrQg0mmOAA/N/58sxN/N24hl8Nuhqdn30M668QhZe/SRvxkznDcdUvhh+PRmL1oecO/jei01LQnuLXSy/6+WGEF/TxNGxYxpNE2f3Zwv55ZJHSh2+ObmF/HrF4ziSWlByJLe0IU3Wqu3Mm3oXp/36HK2G9iw9/8jyLRhRNtNw0qOrtjfMTWiaNHpFoNHUAZ/Xy945i1n6xxdZ88jbFOzNiOj1pZQsu+2FkKgfb7GTov1HQrqSeYtdrPrbG+W2RScnhi0h5EhsEUlxNccoekUQIQr2ZrDj7e8pycwl5eShdDhtlM4dOM5xFxbzzYTbyd2SjqegGMNhY83D73Di67Ppet74iLyGMyuP4oyc6p8gJUdWbCm3KfmkQdjiY3EXlJSrg2WJcdDvFp1HoNGKICLs+vAnfr3ycaRX4nO52frfb4jv04lTf/oH1pjjp2ewpjxrH3mXnA27S1tEBmbnv17xOKlThmOPj6vza6jvT81yfWJTyxeGMywWpn73JPOm/AmXv7m9dHvpPHMcA/50fp1l1Bz7aEVQR5w5Bfx65eN4g0pPewqKyV6/izWPvsfwB69sROk09cm21+eZ9gkWFoP0L5fQ/eJT6vwa1mgHnWaOY+/ni0IrgwpCdIQ1JoqBsy8MuU5Cn06cv/s9Dv28huKMbNqM7EOLbu3rLJ/m+ED7COpI+tzfTBt4+0pcbH/920aQSNNQhCvZjJSmCqK2jHnpjyQO6oolxlHaDhL8jeMFGA4btpYxWKLsDLrnYrqcfaLpdYRhkHLyULrNmqiVgKYcWhHUEW+JC8K0P/Q6QweD/N2H2P3JL2Qs3sCxWN4j0kifj+z1u8jesLtB3w93fhG/3fw8byeewRvR0/jutLvJ2by3RtfoeMZo02Qt6fWROnVEyPaMRev5atwtvBl7Kh90msX6Zz5G+qpulONIiOP0pf+i44zR5fIBpMcLUrWLPPHN2cw6+BGD776oRveg0YA2DdWZ1Klppj9mYbXQ6Ywxpc99Hi+/Xvk4ez75FcNuRfok0W0TmPrdE812dnbwx1X8fMkjqjWiBHtiHOPfuYd2Jw6q19f1eb18Pf42cjbtLbXr7/92OV8uWs+Zq/9Diy7tqnWdYQ9cyb4vl+LKKyy9jjU2in63nh3SwOXQL2v5bvrs0uifon2Z/P7X18jZuJtx/7mzytcSQnBwwe+m5SIMmxXp8kTEJ6FpntRpRSCESBJCfC+E2Ob/H9KVRQhxshBiddBfiRBipn/f60KIXUH7htRFnsYgrlMy/W47p1wPAsNhw5HYgqF/v6J029rH3mXPZwvxlrhw5xXhKSgmf/ch5k25KyIzYU+JK2x2aVMkf+cBfjj9XooPZuEpKMFTWELRviN8f+rdFKQfrtfX3v/tcvK2HygfeiklniIn6554v9rXiU1tw8x1rzLg9vNoNbQnqVNHMOGD+xj+0NUhxy6789+hIaBFTna+/UO17zdcwxkpJV5nGDOVRlMN6roimA3Ml1I+JoSY7X/+5+ADpJQ/AkNAKQ5gO/Bd0CF/klJ+XEc5GpW0R64hedxANj7/KSWZOXSYPor+t55NVJuE0mM2Pv8Z3qIKRb58kuLDOWQu2Ujb0f2r9VrO7Hz2fPILrtxC2p08BOn2svjGZ8lesxNhMeg0cyyjX7yVqNbxNboHT4mLPZ/+Ss6G3bTsmUqX88Zji42u0TVqwqZ/fWGa4ORze9ny8pcMf+iqCts9HPp1Hd4SF8njBmBvGVvr185cshFPQXHIdunxcujH1aHbpSRzyUb2fL4Ii8NG1wtOJtHf2CW6bSLDH76a4Q+HDv7BZK3eYbrdcNg4smwzcR3bVil3h1NHsevDn6HCClR6vLQ/ZViV52s04airIjgTmOB//AbwExUUQQXOBb6RUhbV8XWbHB1PHUXHU0eF3e/KzjfdLgxB0cGsar1G+ldL+PGCBxBC4HN7EIaBz+UpNU1Jn4/dn/7K0VXbOHvj6xhmxcZMKNibwVdjbsblX6lY46JYftfLnPrLcyT06VSta9SUnA17zBWBy03Oxj3lth36eQ3zz7m/1Czic3tJe/xa+v3fWbV67eh2SViiHaalmaPbtyr3XPp8/HLZY+z9YhGeohKEYbD+6Y8Y+OdZDL3vsmq/pj0+FufRvJDtUkqi2iZU6xppj17Lge9X4i4oLjNFxUQx6O4LiU7WbVA1taeuzuJkKeVB/+NDQHIVx88C3quw7WEhxFohxDNCCEe4E4UQ1wkhVgghVmRmZtZB5MYhcUBX0+0+l5vWaVW3ASw6nM38c+7HW+TEU1iCz+XBW+IK9U94feTvOsiezxdWW7ZfLnuM4ozs0lmyp6AE59F8fjz/79W+Rk04smILWet2me6zRNnLvR8lmTl8P+MvuLLycecV4c4rwlvsZMXs/3Dwp9W1ev2us05GGKG5ttaYKAbcUT6ufs+nvyolUFgCUjmCvcVO1j3+PkdXV788Q58/nIElusLXWwgcCS1IHjugWteI65zMWev/S/8/nkurYT3pcNooJn3+AIPvuaTacmg0ZlSpCIQQPwgh1pv8nRl8nFSG7rDGbiFECjAQmBe0+W6gDzACSKKS1YSU8hUpZZqUMq1NmzbhDmuyjHjy+pCBwBLjoMt5E4jrVLn+LDpwhM/7XYmsoil5KV7J1v98VeVh7sJi1j7+Hhm/rkV6KygUKcnfcZD8nQeq95rVJGfzXr45+XaKDxwx3W84bPS+9rTS5zvenY80icryFjlZ//SHtZIhqlU8p8x5CHt8LLaWMdhaqNDLgbMvDFnVbfnPV0oJVHx9p4sd7/xQ7dcc8tdL6TB9BJYoO9a4aGwtYohp34op8x43LQgXjujkJNIeuYYzVrzE5LmP0P6U4dU+V6MJR5WmISll2KwYIUSGECJFSnnQP9BX5vU6H/hMSlnq1QpaTTiFEP8Dqg6fOEZpf8pwTpnzEMv/9BLZ63fjSIyj7y1nM8gk+aciC69+Cmd2QY1eL3u9+Yw7gCuvkLkjb6Qw/XBY9S0sBp6Kfo0a4C4oRvp8HP19G7s/+QWL3UrOxr1hY+xb9khl4mcPlPOtFKYfNjXhABTurb1TOeXkocw69DEHF6zCU1hCuwlDTP0qYe/fJ8n4eQ1SSoQIXV0c+GElm178gpLMHDrOGE3vG05n4sd/J3dLOkeWbyY6pRXtJgzWZUg0TYK6+gjmAJcDj/n/f1HJsReiVgClBCkRAcwEQmvoHke0nzSMM39/pUbnuAuLObhgVbkaMdWh5HAOW175kl7XnmY6UG149hMK9x6uNPHJEmUjvm/NfQS5W9NZeNWTZC7fjPT4VAasT4IhlNIxuxcB3S+dTEK/zhxespGMX9cR1TqexIHdscZFhzh3hc1C8kl1CzO1OOx0mB7erwPQ9YIJHPl9K77i0Pcpa90udr47PySDeOV9r7HxmU9KVxJHV21j04ufc8bKl4jv3ZH43h0B5R84/NsGjizfQkz7VnQ8fTQWh71O96TR1Ia6KoLHgA+FEFcDe1CzfoQQacANUspr/M+7AB2Bnyuc/44Qog1qqFgN3FBHeY47fC5PrcJLpdfHsjv+TeGBIwz72xUh+3e+t6ByJRDjYPS/bqvxjNWZlceXo2/GlVNQNuAHxA+TeAfKPh+VnMh30/7M4cUb8LrcWOw2EGCPj8PndJc5l4XAGuVg4J0X1Ei22tDrqumsf/pDCveErj58TjdrH323nCIo2JPBhqc+KvfeeotdlBzOYfWDbzH6n7cA4Ckq4bvpszn6+zZ8Hi8WuxXDbmPaD0+RNLh7vd+XRhNMnZzFUsqjUspJUsqeUspTpJRZ/u0rAkrA/3y3lDJVSumrcP5EKeVAKeUAKeUlUsqa2T+Oc9z5RSy6/h9Ir3l+gGG30u3iU1TpARM8hSWsf/JDlbBV8Vxb+AE+oX9nps9/mq7n1ryC5tb/fqMGwRoqL2GxUJh+mIyF6/EUliDdXjyFJXgKSnDlFtLlvPFYouwIi0HKxCGctvifxHWuKjah7lhjouh/+3mIMO9XUQVfx75vlqmVTwV8bg97Pvm19PmKu1/lyPItyvHvdOPOL8Z5NI/vTru7WtnGGk0k0SUmmjDzz76P9Lm/mdrwLTEOxr56Jye9OZuxr9xhOvgoJAfmrwrZ2vPK6aFRLKionVPmPEybUX3DypW7JZ0Nz37C5n/PoehQ+dDXzKWbwtr0y7+Q4XfURhOdksTU759g+//mmZ4rhKDzWeO4rOgbLnd9x7TvnyqN428I2o7qp1YnJiQOKj97N2wWU1NcYF+Aba9/a7oic+cXmXYZ02jqE60Imii5W9M5vHhDSOMRUJE1U+c9QY9LJiOEoMs5J2KxmVv5vMUufrrwQRZe/SS+oJVF3xvPoNXQHljjVNKYsBhYoh0M+fsVtOiaYnotKSVLbn2BL4Zdx4q7/8OyO1/i424Xs+W/X5ceE9+nE0aYQTNY/pM/uo+TP7yfyd88zgXpH9BmRB/cJkleoGL53XlqVRNukK1PWo/oTdKQ7hiO8vdliXYwrEJ12U5njAmNwEIp2O6XTS19bhaJBOr+XDUMDNBo6opWBE2UvK37wg6oPqebtqP7lT63OOx0vXAilihzR6PP6WbnBz+y8blPy50z/adnOOmtu+lx5VT63nwWMxb/k0F/Cm93T5/7G9te+wZvsQuf04232Im3xMXSW14gb4cKM+19/YxKzU6gMmHT5y4hdUoayWP6l4ZPpkwaarqykV4f7SYMrvSa9YkQginfPE73i08pNU+17N2RSZ89EFIXKapNAie8eAuWaHupOckaF018n44Mmj2r9LjWI/qYvpa3wmer0TQE4lisgJmWliZXrFjR2GLUK7lb0/li6HXl+hwEiE5pxaz95WPo3YXFzJ/5VzJ+XR+2PHJMamsuSP+g1jLNm/ZnDnwX+r4bNiuD/nIRQ++/HICDP63m5wsfwl1YgtfpRppkEFtiHCG9dXO3pDN31I3KR+CfVVtjo+h+6WTG/Ou2WssdSXxeLz6XB6uJWS2Y3G372P76t5Rk5pI6JY1OZ47FCFq1HV6ykXmn3ImnuMyfYo2Nou//nUXao9eEu6xGUyeEECullGkVt+sVQRMlvldH2o7pH2KOsMZEMeS+S0OOt8VGM+37pxj69yvCrgycWeZlLqqLK8z5PrenXPmElAlDuGD/h5z68zMhVThLz3G6Sf9ySblt8b07csbKl+l20SRiUluTOLg7J7xwC6NfvLVOckcSw2KpUgkAxPfswPCHr2HsK3fQ5dzx5ZQAQNsT+nHqr8/R4bRRRLVNIHFQN8a89EeGP1J5zSKNpj7QZaibMJM+fYBfr3qC9C+XlA4kg++9hN7XzQh7Tuezx7H672+Y7ms1tEed5Ok0cyzZ63eFODmtcdGkThtZbpswDFoN7Yk93rw4nLAYWByhpq+W3dtz0huz6yTnsUKroT2ZPOfhxhZDo9GKoCljaxHDxI/+hjM7n5LMHOI6J1eZcBTfswMdZ5xA+ldLy0XgWKIdpD1+XZ3k6fOHM9j80hxKDueUlkS2RDtIGtydDtNCG7EA9LrmVJbf9XJI5VVhGHSJUIN3jUZTN7Rp6BjAkdiC+F4dq511Ov6dexh41wU4Wsdj2Ky0HtmHKd8+Vu3iZpXJcebvr9D3ppnEdmpLy56pDLn/MqZ+/2TYejm9rz2N5DEDsMapfg3CasESZWfYw1eFjU7SaDQNi3YWa+od6fNx4Iff2Tt3MfaWsXS/5BQS+nZubLE0mmZHOGexNg1p6h1hGKROSSN1Ssj3T6PRNAG0aUij0WiaOVoRaDQaTTNHKwKNRqNp5mhFoNFoNM0crQg0Go2mmXNMho8KITJRjXAamtaAebPdpoWWM3IcCzKCljOSHAsyQu3k7CylDKn7ckwqgsZCCLHCLAa3qaHljBzHgoyg5Ywkx4KMEFk5tWlIo9FomjlaEWg0Gk0zRyuCmvFKYwtQTbSckeNYkBG0nJHkWJARIiin9hFoNBpNM0evCDQajaaZoxWBRqPRNHO0IqiAECJJCPG9EGKb/3+iyTEnCyFWB/2VCCFm+ve9LoTYFbRvSGPJ6T/OGyTLnKDtXYUQS4UQ24UQHwghqtfsIMIyCiGGCCF+E0JsEEKsFUJcELSvXt9LIcQ0IcQW/3sQ0hZNCOHwvzfb/e9Vl6B9d/u3bxFCTI2kXDWU8XYhxEb/ezdfCNE5aJ/pZ99Icl4hhMgMkueaoH2X+78j24QQlzeynM8EybhVCJETtK9B3k8hxGtCiMNCiPVh9gshxPP+e1grhBgWtK9276WUUv8F/QFPALP9j2cDj1dxfBKQBcT4n78OnNtU5AQKwmz/EJjlf/wS8IfGkBHoBfT0P24PHAQS6vu9BCzADqAbYAfWAP0qHHMj8JL/8SzgA//jfv7jHUBX/3UsjSTjyUHfvT8EZKzss28kOa8AXjA5NwnY6f+f6H+c2FhyVjj+ZuC1Rng/TwKGAevD7D8V+AYQwAnA0rq+l3pFEMqZQKDp7xvAzCqOPxf4RkpZVJ9CmVBTOUsRQghgIvBxbc6vAVXKKKXcKqXc5n98ADgMmHe8jywjge1Syp1SShfwvl/eYILl/xiY5H/vzgTel1I6pZS7gO3+6zW4jFLKH4O+e0uADvUgR1VU570Mx1TgeylllpQyG/gemNZE5LwQeK+eZAmLlPIX1OQyHGcCb0rFEiBBCJFCHd5LrQhCSZZSHvQ/PgQkV3H8LEK/LA/7l2zPCCEcEZdQUV05o4QQK4QQSwLmK6AVkCOl9Pif7wNSG1FGAIQQI1EztR1Bm+vrvUwF0oOem70Hpcf436tc1HtXnXMbSsZgrkbNFAOYffb1QXXlPMf/WX4shOhYw3MjQbVfy29i6wosCNrcUO9nVYS7j1q/l82yQ5kQ4gegncmue4KfSCmlECJsfK1fCw8E5gVtvhs16NlRcb5/Bh5oRDk7Syn3CyG6AQuEEOtQA1pEiPB7+RZwuZTS598csffyeEcIcQmQBowP2hzy2Uspd5hfod6ZC7wnpXQKIa5HrbQmNpIs1WEW8LGU0hu0rSm9nxGlWSoCKeUp4fYJITKEEClSyoP+welwJZc6H/hMSukOunZgBuwUQvwPuLMx5ZRS7vf/3ymE+AkYCnyCWk5a/TPdDsD+xpJRCNES+Aq4x7/UDVw7Yu+lCfuBjkHPzd6DwDH7hBBWIB44Ws1zG0pGhBCnoBTveCmlM7A9zGdfHwNXlXJKKY8GPX0V5T8KnDuhwrk/RVzCsteq7uc2C7gpeEMDvp9VEe4+av1eatNQKHOAgLf9cuCLSo4NsSH6B7yAHX4mYOr5jwBVyimESAyYU4QQrYGxwEapPEs/ovwbYc9vIBntwGcom+fHFfbV53u5HOgpVPSUHfXDrxgJEiz/ucAC/3s3B5glVFRRV6AnsCyCslVbRiHEUOBl4Awp5eGg7aaffT3IWF05U4KengFs8j+eB0zxy5sITKH8CrtB5fTL2gflbP0taFtDvp9VMQe4zB89dAKQ65801f69bAgv+LH0h7IBzwe2AT8ASf7tacCrQcd1QWlgo8L5C4B1qEHrbSCuseQExvhlWeP/f3XQ+d1Qg9d24CPA0UgyXgK4gdVBf0Ma4r1ERV9sRc3q7vFvewA1qAJE+d+b7f73qlvQuff4z9sCTK/H72NVMv4AZAS9d3Oq+uwbSc5HgQ1+eX4E+gSde5X/Pd4OXNmYcvqf/w14rMJ5DfZ+oiaXB/2/i30o388NwA3+/QJ40X8P64C0ur6XusSERqPRNHO0aUij0WiaOVoRaDQaTTNHKwKNRqNp5mhFoNFoNM0crQg0Go2mmaMVgUaj0TRztCLQaDSaZs7/A5rbjJunZZ3xAAAAAElFTkSuQmCC\n", "text/plain": [ "<Figure size 432x288 with 1 Axes>" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pylab as plt\n", "%matplotlib inline \n", "\n", "N = 100 # number of points per class\n", "D = 2 # dimensionality\n", "K = 3 # number of classes\n", "X = np.zeros((N*K,D)) # data matrix (each row = single example)\n", "y = np.zeros(N*K, dtype='uint8') # class labels\n", "for j in range(K):\n", " ix = range(N*j,N*(j+1))\n", " r = np.linspace(0.0,1,N) # radius\n", " t = np.linspace(j*4,(j+1)*4,N) + np.random.randn(N)*0.2 # theta\n", " X[ix] = np.c_[r*np.sin(t), r*np.cos(t)]\n", " y[ix] = j\n", "# lets visualize the data:\n", "plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap=plt.cm.Spectral)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- `np.zeros` produces an ndarray of all 0's. The shape of the ndarray `np.zeros((N*K, D))` is $ (N\\cdot K, D)$, that is dimension $ 1 $ has size $N\\cdot K$ and dimension 2 has size $D$.\n", "- `dtype='uint8'` refers to the data type of the array. `uint8` refers to as \n", "_unsigned 8-bit integer type_\n", "- The `range` function produces a list of evenly-spaced integers, up to but not including the endpoint\n", "- `np.linspace(0.0,1,N)` produces $ N $ evenly spaced numbers over a specified closed interval, from $ 0.0 $ to $ 1 $ \n", "- `r*np.sin(t)` is a multiplication of two equally sized ndarrays. The multiplication proceeds elementwise\n", "- `np.c\\_` is stacking two arrays 'columnwise' together" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Normally we would want to preprocess the dataset so that each feature has zero \n", "mean and unit standard deviation, but in this case the features are already \n", "in a nice range from $-1$ to $1$, so we skip this step." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Training a Softmax Linear Classifier" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialize the parameters" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us first train a Softmax classifier on this classification dataset. As we saw in the previous sections, the Softmax classifier has a linear score function and uses the cross-entropy loss. The parameters of the linear classifier consist of a weight matrix \n", "$ W $ and a bias vector $ b $ for each class. Let us first initialize these \n", "parameters to be random numbers:" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[ 2.58507353e-02 -1.65804593e-06 -2.06983186e-02]\n", " [ 2.18356399e-02 6.19492831e-03 -1.69727720e-02]]\n", "[[0. 0. 0.]]\n" ] } ], "source": [ "# initialize parameters randomly\n", "W = 0.01 * np.random.randn(D,K)\n", "b = np.zeros((1,K))\n", "print(W)\n", "print(b)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Recall that $ D = 2 $ is the dimensionality and $ K = 3 $ is \n", "the number of classes\n", "- `np.random.randn(D,K)` produces an array of shape $ (D, K) $ \n", "consisting of standard normally distributed random numbers (with mean $ 0 $ and \n", "standard deviation $ 1 $.)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Compute the Class Scores" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since this is a linear classifier, we can compute all class scores very simply in parallel with a single matrix multiplication:" ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(300, 3)\n" ] } ], "source": [ "# compute class scores for a linear classifier\n", "scores = np.dot(X, W) + b\n", "print(scores.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- `np.dot(X, W)` is the matrix product of $ X $ (a $300$ by $ 2 $ ndarray) and \n", "$ W $ (a $ 2 $ by $ 3 $ ndarray)\n", "- In this example we have $ 300 $ $2$-D points, so after this multiplication \n", "the array `scores` will have shape $[300 \\times 3] $, where each row gives the \n", "class scores corresponding to the $ 3 $ classes (blue, red, yellow)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Compute the Loss" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The second key ingredient we need is a loss function, which is a differentiable \n", "objective that quantifies our unhappiness with the computed class scores. \n", "Intuitively, we want the correct class to have a higher score than the other \n", "classes. When this is the case, the loss should be low and otherwise the loss \n", "should be high. \n", "\n", "There are many ways to quantify this intuition, but in this \n", "example let us use the _cross-entropy loss_ that is associated with the Softmax \n", "classifier. Recall that if $ z $ is the array of class scores for a single \n", "example (e.g. array of $ 3 $ numbers here), then the Softmax classifier \n", "computes the loss for that example as:\n", "\n", "\n", "$$\n", "L_i=-\\log\\left(\\frac{e^{z_{y_i}}}{\\sum_{j}e^{z_{j}}}\\right)\n", "$$\n", "\n", "where $z_{y_i}$ denotes the element of the score array that represents to the \n", "correct class. We can see that the Softmax classifier interprets every element of $ z $ \n", "as holding the (unnormalized) log probabilities of the three classes. \n", "We exponentiate these to get (unnormalized) probabilities, and then \n", "normalize them to get probabilites. Therefore, the expression inside \n", "the $\\log$ is the normalized probability of the correct class. \n", "\n", "Note how \n", "this expression works: this quantity is always between $ 0 $ and $ 1 $. \n", "When the probability of the correct class is very small (near $ 0 $), \n", "the loss will go towards (positive) infinity. Conversely, when the \n", "correct class probability goes towards $ 1 $, the loss will go towards \n", "zero because $ \\log(1)=0 $. Hence, the expression for $L_i$ is low when \n", "the correct class probability is high, and it is very high when it is low.\n", "\n", "Recall also that the full Softmax classifier loss is then defined as the \n", "average cross-entropy loss over the training examples and the regularization:\n", "\n", "$$\n", "L=\\underbrace{\\frac{1}{N}\\sum L_i}_{\\text{data loss}} + \\underbrace{\\frac{1}{2}\\lambda \\sum_{k}\\sum_{l}W_{kl}^2}_{\\text{regularization loss}}\n", "$$\n", "\n", "\n", "Given the array of __scores__ we have computed above, we can compute the loss. First, the way to obtain the probabilities is straight forward:\n" ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(300, 3)\n", "[[0.33333333 0.33333333 0.33333333]\n", " [0.33341756 0.33333943 0.33324301]\n", " [0.33351434 0.33334343 0.33314223]]\n" ] } ], "source": [ "# Get unnormalized probabilities\n", "exp_scores = np.exp(scores)\n", "# Normalize them for each example\n", "probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)\n", "print(probs.shape)\n", "print(probs[0:3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Recall that `scores` has shape $[300 \\times 3] $\n", "- `np.exp()` is a _unary_ universal function, or _ufunc_. These functions perform elementwise operations on data in ndarrays. That is, `exp\\_scores` \n", "has shape $[300 \\times 3] $\n", "- The function `np.sum(exp_scores, axis=1, keepdims=True)` computes the sum over \n", "axis $ 1 $. If `keepdims` is set to `True`, the axes which are reduced are \n", "left in the result as dimensions with size one. With this option, the \n", "result will broadcast correctly against the input array." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We now have an array `probs` of size $ [300 \\times 3] $, where each row now \n", "contains the class probabilities. In particular, since we have normalized \n", "them every row now sums to one. We can now query for the log probabilities \n", "assigned to the correct classes in each example:" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(300,)\n" ] } ], "source": [ "num_examples = X.shape[0]\n", "correct_logprobs = -np.log(probs[range(num_examples),y])\n", "print(correct_logprobs.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- The array `correct_logprobs` is a $1$D array of just the probabilities \n", "assigned to the correct classes for each example.\n", "- Take a moment to understand, how the fancy indexing for the ndarray `probs` \n", "was carried out: the elements \n", "$ (0, y[0]), (1, y[1]), \\ldots, (N*K-1, y[N*K-1]) $ were selected\n", "- `np.log()` is a unary ufunc that evaluates the natural logarithm of the ndarray elementwise" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "The full loss is then the average of these log probabilities and the regularization loss:\n" ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.0977848504902463\n" ] } ], "source": [ "# compute the loss: average cross-entropy loss and regularization\n", "data_loss = np.sum(correct_logprobs)/(num_examples)\n", "reg = 0.5\n", "reg_loss = reg*np.sum(W*W)\n", "loss = data_loss + reg_loss\n", "print(loss)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Aggregations (often called _reductions_ ) like `sum` can either \n", "be used by calling the array instance method (`correct_logprobs.sum()`) or using the top level NumPy function `np.sum()`\n", "- In this code, the regularization strength $\\lambda $ is stored inside the `reg` .\n", "The convenience factor of $0.5$ multiplying the regularization will become clear in \n", "a second. Evaluating this in the beginning (with random parameters) might give us \n", "`loss = 1.1`, which is `np.log(1.0/3)` since with small initial random \n", "weights all probabilities assigned to all classes are about one third. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We now want to make the loss as low as possible, with `loss = 0` as the \n", "absolute lower bound. But the lower the loss is, the higher are the probabilities \n", "assigned to the correct classes for all examples." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Computing the Analytic Gradient with Backpropagation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have a way of evaluating the loss, and now we have to minimize it. \n", "We will do so with gradient descent. That is, we start with random \n", "parameters (as shown above), and evaluate the gradient of the loss \n", "function with respect to the parameters, so that we know how we \n", "should change the parameters to decrease the loss. Let us introduce \n", "the intermediate variable $ p $, which is a vector of the (normalized) \n", "probabilities. The loss for one example is:\n", "\n", "$$\n", "p_{k} = \\frac{e^{z_{k}}}{\\sum_{j}e^{z_{j}}} \\quad \\quad L_i=-\\log(p_{y_{i}})\n", "$$\n", "\n", "We now wish to understand how the computed scores inside $ z $ should change to \n", "decrease the loss $ L_i $ that this example contributes to the full objective. \n", "In other words, we want to derive the gradient $\\frac{\\partial{L_i}}{{\\partial z_{k}}} $. \n", "The loss $ L_i $ is computed from $ p $, which in turn depends on $ z $. \n", "It is a fun exercise to the reader to use the chain rule to derive the gradient, \n", "but it turns out to be extremely simple and interpretable in the end, after a \n", "lot of things cancel out:\n", "\n", "$$\n", "\\frac{\\partial L_{i}}{\\partial z_{k}}=p_{k} - \\mathbb{1}\\left(y_i=k\\right)\n", "$$\n", "\n", "Notice how elegant and simple this expression is. Suppose the probabilities we \n", "computed were `p = [0.2, 0.3, 0.5]`, and that the correct class was the middle \n", "one (with probability $ 0.3 $). According to this derivation the gradient on \n", "the scores would be `dz = [0.2, -0.7, 0.5]`. Recalling what the interpretation \n", "of the gradient is, we see that this result is highly intuitive: increasing the \n", "first or last element of the score vector `z` (the scores of the incorrect classes) leads to an increased loss (due to the positive signs $+0.2$ and $+0.5$) - and increasing the loss is bad, as expected. However, increasing the score of the correct class has negative influence on the loss. The gradient of $-0.7$ is telling us that increasing the correct class score would lead to a decrease of the loss $L_i$, which makes sense.\n", "\n", "All of this boils down to the following code. Recall that `probs` stores the \n", "probabilities of all classes (as rows) for each example. To get the gradient \n", "on the scores, which we call `dscores`, we proceed as follows:" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(300, 3)\n" ] } ], "source": [ "dscores = probs\n", "dscores[range(num_examples),y] -= 1\n", "dscores /= num_examples\n", "print(dscores.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- The fancy indexing for the ndarray `probs` was carried out as follows: the elements \n", "$ (0, y[0]), (1, y[1]), \\ldots, (num\\_examples-1, y[num\\_examples-1]) $ were selected. That is, we subract $ 1$ from the probability corresponding to the true class and leaving the other probabilities unchanged.\n", "- We divide `dscores` by `num_examples` because \n", "$$\n", "L=\\underbrace{\\frac{1}{N}\\sum L_i}_{\\text{data loss}} + \\underbrace{\\frac{1}{2}\\lambda \\sum_{k}\\sum_{l}W_{kl}^2}_{\\text{regularization loss}}\n", "$$\n", "so we need to divide by the number of data points." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lastly, we had that `scores = np.dot(X, W) + b`, so armed with the \n", "gradient on `scores` (stored in `dscores`), we can now backpropagate \n", "into $ W $ and $ b $:" ] }, { "cell_type": "code", "execution_count": 81, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(1, 3)\n", "(2, 3)\n" ] } ], "source": [ "dW = np.dot(X.T, dscores)\n", "db = np.sum(dscores, axis=0, keepdims=True)\n", "print(db.shape)\n", "dW += reg*W # don't forget the regularization gradient\n", "print(dW.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- `X.T` denotes the transpose of $X$ and has shape $(2, 300)$. \n", "`dscores` has shape $(300, 3)$. Thus, `dW` has shape $(2, 3)$ which is of course identical to \n", "the shape of `W`\n", "- We see that we have backpropped through the matrix multiply operation, \n", "and also added the contribution from the regularization. Note that the \n", "regularization gradient has the very simple form `reg*W` since we \n", "used the constant $ 0.5 $ for its loss \n", "contribution, i.e.\n", "\n", "$$\n", "\\frac{d}{dw}\\left(\\frac{1}{2}\\lambda w^2\\right)=\\lambda w\n", "$$\n", "\n", "This is a common convenience trick that simplifies the gradient expression.\n", "\n", "`dW` is actually a short-hand notation for the Jacobian\n", "\n", "$$\n", "\\frac{\\partial L}{\\partial W_{ij}}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Performing a Parameter Update" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we have evaluated the gradient we know how every parameter \n", "influences the loss function. We will now perform a parameter update \n", "in the negative gradient direction to decrease the loss:" ] }, { "cell_type": "raw", "metadata": {}, "source": [ "# perform a parameter update / illustration\n", "W += -step_size * dW\n", "b += -step_size * db" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Putting all of this together, here is the full code for training a \n", "Softmax classifier with Gradient descent:\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Putting it All Together: Training a Softmax Classifier" ] }, { "cell_type": "code", "execution_count": 82, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "iteration 0: loss 1.099738\n", "iteration 10: loss 0.929506\n", "iteration 20: loss 0.869086\n", "iteration 30: loss 0.843276\n", "iteration 40: loss 0.830723\n", "iteration 50: loss 0.824075\n", "iteration 60: loss 0.820346\n", "iteration 70: loss 0.818167\n", "iteration 80: loss 0.816855\n", "iteration 90: loss 0.816047\n", "iteration 100: loss 0.815541\n", "iteration 110: loss 0.815220\n", "iteration 120: loss 0.815014\n", "iteration 130: loss 0.814880\n", "iteration 140: loss 0.814793\n", "iteration 150: loss 0.814736\n", "iteration 160: loss 0.814699\n", "iteration 170: loss 0.814674\n", "iteration 180: loss 0.814657\n", "iteration 190: loss 0.814647\n" ] } ], "source": [ "#Train a Linear Classifier\n", "\n", "# initialize parameters randomly\n", "W = 0.01 * np.random.randn(D,K)\n", "b = np.zeros((1,K))\n", "\n", "# some hyperparameters\n", "step_size = 1e-0\n", "reg = 1e-3 # regularization strength\n", "\n", "# gradient descent loop\n", "num_examples = X.shape[0]\n", "for i in range(200):\n", " \n", " # evaluate class scores, [N x K]\n", " scores = np.dot(X, W) + b \n", " \n", " # compute the class probabilities\n", " exp_scores = np.exp(scores)\n", " # [N x K]\n", " probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True) \n", " \n", " # compute the loss: average cross-entropy loss and regularization\n", " correct_logprobs = -np.log(probs[range(num_examples),y])\n", " data_loss = np.sum(correct_logprobs)/num_examples\n", " reg_loss = 0.5*reg*np.sum(W*W)\n", " loss = data_loss + reg_loss\n", " if i % 10 == 0:\n", " print(\"iteration %d: loss %f\" % (i, loss))\n", " \n", " # compute the gradient on scores\n", " dscores = probs\n", " dscores[range(num_examples),y] -= 1\n", " dscores /= num_examples\n", " \n", " # backpropate the gradient to the parameters (W,b)\n", " dW = np.dot(X.T, dscores)\n", " db = np.sum(dscores, axis=0, keepdims=True)\n", " \n", " dW += reg*W # regularization gradient\n", " \n", " # perform a parameter update\n", " W += -step_size * dW\n", " b += -step_size * db" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We could verify that we have converged to something after about $ 190 $ iterations. We can evaluate the training set accuracy:" ] }, { "cell_type": "code", "execution_count": 83, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "training accuracy: 0.49\n" ] } ], "source": [ "# evaluate training set accuracy\n", "scores = np.dot(X, W) + b\n", "predicted_class = np.argmax(scores, axis=1)\n", "print('training accuracy: %.2f' % (np.mean(predicted_class == y)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- `np.argmax` returns the indices of the maximum values along an axis. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This prints $ 55\\% $. Not very good at all, but also not surprising \n", "given that the dataset is constructed so it is not linearly separable. We can also plot the \n", "learned decision boundaries:" ] }, { "attachments": { "spiral_linear.png": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtAAAAJACAYAAACkMVHfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdAVfX/x/HnZW9ExQGICrj3TnOPTHNb5qocpamZmvnVLGdWzswyZ2nLlWapuXKhufeeqCgi4gBBlgjc3x/8unZDQUqZr8df8rnnnPs515IXh/fn8zYYjUYjIiIiIiLyRCwyewIiIiIiItmJArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpIMCtIiIiIhIOihAi4iIiIikgwK0iIiIiEg6KECLiIiIiKSDArSIiIiISDooQIuIiIiIpINVZk8gtzAYDJk9BREREZEnZjQaM3sKWZYCdAbqN237U7nO9ehIkoyJ+DWMo7VPAtXzWeF03+GpXPvfGjtxMWNHdM3UOWR1+ozSps8odfp80qbPKG36jFKnzyeZhVubzJ5ClqYSjmzIw9EFLyc3AvztmL7AkYN3EoiyjcnsaYmIiIjkCgrQ2ZiXkxsWBiumL3BkzAGjQrSIiIhIBlCAzuY8HF2wMFgR4G/PmANG/KMiM2UeDetWyJT3zU70GaVNn1Hq9PmkTZ9R2vQZpU6fjzwJg1EV4hnCYDA8tRroR8mKddEiIiKSPVm4tdEiwlToCXQOobpoERERkYyhAJ3DeDm5AQbVRYuIiIg8IwrQOdBfiwsD/O0UokVERESeMgXoHCqrLC4UERERyWkUoHOw5BBtSYC/PWsuWeEfFamn0SIiIiL/kQJ0DqfFhSIiIiJPlwJ0LqGmKyIiIiJPhwJ0LqK6aBEREZH/TgE6lzGvi7ZUiBYRERFJJ6vMnoBkPA9HFwAC/GG6vxF6RapzoYiIiMgT0hPoXOzvTVe0uFBERETkyShA53JaXCgiIiKSPgrQosWFIiIiIumgAC2Amq6IiIiIPCkFaDFR0xURERGRtClASwp/1UWvuaRNWkRERET+SQFaRERERCQdFKAlVSrjEBERETGnAC2P5OHoolpoERERkUdQgJbH0h7RIiIiIikpQEuqHu4RbacQLSIiIoICtDwBNVoREREReUgBWp6IGq2IiIiIJFOAliemRisiIiIiCtDyL2hxoYiIiORmCtDyr6guWkRERHIrBWj518zroi1VFy0iIiK5ggK0/CcP66LtVRctIiIiuYICtDwVXk5ugEEhWkRERHI8BWh5arS4UERERHIDBWh5qrS4UERERHI6BWh56tR0RURERHIyBWh5JtR0RURERHIqBeg09OrVi4IFC1KhQoVHvu7v74+rqytVqlShSpUqTJgwIYNnmLWpLlpERERyGgXoNPTs2ZMNGzakekyDBg04cuQIR44c4aOPPsqgmWUfD+ui7VQXLSIiItmeAnQa6tWrh5ubW6rHGI3GDJpN9vX3xYV/NV0RERERyY4UoP8jg8HA7t27qVSpEi1btuT06dOZPaUsS4sLRUREJCdQgP6PqlatSlBQEMeOHWPgwIG0a9cus6eUpWlxoYiIiGR3Vpk9gezO2dnZ9OcWLVrQv39/wsLCyJs3b4pjD2xcaPqzh29lPP2qZMgcsyIvJzeuR0cyfYEjfg3jGFcjBqf7Dpk9LRERkVzJf+cJ/HeeyOxpZBsK0P9RaGgoBQoUwGAwsH//foxG4yPDM0CN5j0zeHZZm4ejC9ejI5ObrhBLa59IGjq5ZPa0REREcp2GdSvQsO7DHcfGT1qaibPJ+hSg09ClSxe2b9/O7du3KVKkCOPGjePBgwcA9O3blxUrVjB79mysrKxwcHBg6VL9B5cefw/Ra4gDn0iq57PS02gRERHJsgxGbSGRIQwGA/2mbc/saWRp16LCASNDekUrRIuIiGQiC7c22mUsFVpEKFmGmq6IiIhIdqAALVnKP5uuKESLiIhIVqMALVnO35uuqHOhiIiIZDUK0JIlqemKiIiIZFUK0JJlqemKiIiIZEUK0JLlaXGhiIiIZCUK0JItqC5aREREsgoFaMk2zOuiLVUXLSIiIplCAVqylYd10faqixYREZFMoQAt2ZKXkxtgUIgWERGRDKcALdmWFheKiIhIZlCAlmxNiwtFREQkoylAS7anpisiIiKSkRSgJUdQ0xURERHJKArQkqOoLlpERESeNQVoyXEe1kXbqS5aREREnjoFaMmR/r648K+mKyIiIiJPgwK05FhaXCgiIiLPggK05GhaXCgiIiJPmwK05ApaXCgiIiJPiwK05BpquiIiIiJPgwK05CqqixYREZH/SgFach3VRYuIiMh/oQAtuZbqokVEROTfUICWXO2fTVcUokVERCQtCtCS62lxoYiIiKSHArQIWlwoIiIiT04BWuT/aXGhiIiIPAkFaJF/0OJCERERSY0CtMgjqC5aREREHkcBWuQxzOuiLVUXLSIiIoACtEiqHtZF26suWkRERAAFaJEn4uXkBhgUokVEREQBWuRJ/RWiMVhm9lREREQkEylAi4iIiIikgwK0SDpN/9ZOZRwiIiK5mAK0SDqoFlpEREQUoEXSSY1WREREcjcFaJF/QY1WREREci8FaJF/ybzRipUarYiIiOQSCtAi/8HDRit2qosWERHJJRSgRZ4C1UWLiIjkHgrQIk/Jw7poO4VoERGRHEwBWuQp0uJCERGRnE8BWuQp0+JCERGRnE0BWuQZ0OJCERGRnEsBWuQZ0uJCERGRnEcBWuQZU120iIhIzqIALZIBzOuiLVUXLSIiko0pQItkkId10faqixYREcnGFKDT0KtXLwoWLEiFChUee8y7775LiRIlqFSpEkeOHMnA2Ul25OXkBhgUokVERLIpBeg09OzZkw0bNjz29XXr1hEQEMCFCxeYN28e/fr1y8DZSXalxYUiIiLZlwJ0GurVq4ebm9tjX1+9ejVvvPEGALVq1eLu3buEhoZm1PQkG9PiQhERkexJAfo/Cg4OpkiRIqavvby8uHbtWibOSLITNV0RERHJfhSgnwKj0Wj2tcFgyKSZSHakpisiIiLZi1VmTyC78/T0JCgoyPT1tWvX8PT0fOSxBzYuNP3Zw7cynn5Vnvn8JPvwcnLjenQk0xc44tcwjnE1YnC675DZ0xIRkVzAf+cJ/HeeyOxpZBsK0P9RmzZtmDlzJp07d2bv3r3kyZOHggULPvLYGs17ZvDsJLvxcHThenRkcl00sbT2iaShk0tmT0tERHK4hnUr0LDuwx3Hxk9amomzyfoUoNPQpUsXtm/fzu3btylSpAjjxo3jwYMHAPTt25eWLVuybt06/Pz8cHR0ZOHChWlcUSR1fw/Ra4gFhWgREZEsxWD8ZwGvPBMGg4F+07Zn9jQkG7keHUmSMREwMqRXNNXzWamkQ0REMoSFW5sUa7zkIS0iFMmi/lpcqKYrIiIiWYsCtEgWp6YrIiIiWYsCtEg2oKYrIiIiWYcCtEg2oaYrIiIiWYMCtEg2oqYrIiIimU8BWiQbUl20iIhI5lGAFsmmHtZF2+W4EB0XF8/J01e4eetuZk9FREQkBQVokWwspy0uNBqNTJ7xK4VL9qRZ28/wrdyP1p0/Iyz8XmZPTURExESdCEWyOfPOhXHgE5ltm64s+Gkzn8/cRKOao3FxKsiDhPscP7+Mtl0m8eeGCZk9PREREUBPoEVyhJyyuHDi579RudTruDgVBMDaypYqZbpz5tx1jp24nK5rHTxygbeHzOHl16Yy77uNxMTcfxZTFhGRXEgBWiQHye6LC69cCyG/m4/ZmIXBgvx5i3Hh0vUnvs6M2Wt4od0EDu534EZwKSZP30n1hsO4GxH1tKcsIiK5kAK0SA6T3eqijUYjPy3bRo2Gw7GytOXmnfNmryclJRASeoGSfh5PdL2QG2F8NGExjWuNonyJNvh51+P5yu9BojefTvvlWdyCiIjkMgrQIjmQedMVyyzddOWDcT/xv1G/4Gr3IuV8W7Pn6ALCIq4AcD8+it1HviUpycCrPT4n9GZ4mtf7feMBihSuhJNDftOYwWDAt0hTlv+655ndh4iI5B5aRCiSQ3k4ugAQ4A/T/Y0M6RVN9XwxWWpx4Y3QcL6ev46X6k/GztYZr0JVsLa2Z9PuSRiNYDQm4l24Oq0bTeT0xTX0HTyX3xaPyOxpi4hILqcn0CI5nJeTG2DIkosLd+8/g0eBktjZOpvGShZrRNvGk0hIvE/HF77g+apvYWPtQFnfNvyx9TBRUbGpXrNV8xoEhRwjKua2acxoNHIxaDOvtK/9zO5FRERyDwVokVwgqy4uzOPqSGxcRIrx+/H3sLV2xMba3jRmbWWLhYUFsXHxqV6zcKG8TPioK1v3fczJC6sJuPonu458DpZXGTm041O/BxERyX0UoEVyiay4uLB+nfIkEcWV6wdMY0lJiRw8tQTPgpXMjg0OPUYRzwLkz+eS5nUH9WvNplWjqF4zhsJe5/jfe3U56D+FPK5OT/0eREQk91ENtEguktWarlhZWbJqyQhadPyYoFB/HGwLcuPOCQq4O3I9JIgLV7aTL09xbt45y9nANfz83XsYDIYnuna1yn5Uq+z3jO9ARERyI4PRaDRm9iRyA4PBQL9p2zN7GiIm16LCgb8WF2Zu58LY2PusXr+fGzfDqVWtJLWql2Lbn8eZ/MVqLgWGUr6sNyOHtqd6lRKZNkcRkdzEwq0NioiPpwCdQRSgJSu6Hh1JkjEBv4ZxjKthyFI7dIiISOZRgE6daqBFcrH/UhcdFxfP6nX7+GnZNq4F3077BBERkRxCNdAiuZx5XXQs+ETS0Cn1hXr+O0/w8mtTcHH2wNbamX7vzaNKRR9Cb0aSkJBI25eq88F7HXHP75pBdyEiIpJxVMKRQVTCIVldcjlHImnVRUdERFOsYh9qlu9PYfdyAMTF32PDjo8p7lUbr0JVuBzsT/T9sxzeMRW3PNr5QkQku1EJR+pUwiEiQPKT6CdpurJi9S4K5CttCs8AdjbOVCn7CqF3zpEvTzGql+uBnXVx5izYkIF3ICIikjEUoEXETFpNV27djsTOOn+K85wc3ImLv2f62tO9Jus3HX/m8xUREcloCtAikkJqiwvr1CzNjTtHSUpKNDsn6MZhCuR9uM1c7P0I8ro5ZticRUREMooCtIg8UnKItkxeXHjJCv+oSKJsY6hXpxzlyxZiz7GvCY+4SkxsOCcv/M65y1so59cCSK6JDghaT58eTTL5LkRERJ4+LSLMIFpEKNnZP5uuWEVY8cm0FSz8aStR0bGU8PHk9LkrFPWojIWFLUEhh3i7d3Mmjn3tiTsHiohI1qFFhKlTgM4gCtCS3aXVdCX0Zjir1u3jfnwCLZtVw7d44UyaqYiI/FcK0KlTgM4gCtCSE6hzoYhI7qAAnTrVQIvIE3uSzoXx8Q9Yu/EAi37252rQrUyYpeQ2SUlJXA+5Q1RUbGZPRURyCXUiFJF0+atzIViBAaJsY0xPovfsP0u7rhNxsHPHztaV4NB59OjWmBkTe6sWOgOFhd/j1JmreHrkw6dYocyezjO16Gd/Rn+2mKjoWOLi4unQqjZffPYWrq7aAUZEnh09gRaRx4qLjiDw1C6uX0y5bd0/xcTcp/Wrn1LBrwcNqn9IrQrv0KrBFH757QTfLd6SQTPO3ZKSkhg+9jt8q/RhxMc/Uqf5/2jx8lhu3Y7I7Kk9E6vX7ePDT35i0eIPuXnzVwIDl2DlYEennpMye2oiksMpQIvIIx3a9COLPunCsfXL2LnsCxZ/0pWbQWdNrwf4W7Pm4sOnyqvW7cXNpShehSqbxmysHSnr04GZczdm6Nxzq8+/XsXOfWc5f/4Hdu+ZSVDQMspX9qNzrylAcsA+fOwiBw5fICEh9R+InrXTZ6+yduMBAq+G/utrTPnqV2bMeIfatZO7YubN68LcuUO4cCmEw8cuPq2pioikoBIOEUnh0vEdnNu9jjYNPsXB3g2AK9cPsG7+CLp9uMRUxhHgb8cYYmntE8nN2xHY27qnuJaTgztnr+TMJ6BZQXz8A6ysLLGwsGDm/N9ZtfoTChRI/juzsbFm4sS38PHpyneLNvPJ58uxsrLE2tqKu3ejmD2tHy81r5Gh870TFkmX3lM5c+EaFcoX5+Ch8zRvXIX5M97Bzs4mXdc6dfYq9epVMBuztLSkTp1ynDpzlaqVfJ/m1EVETPQEWkRSOPnnSiqVbG8KzwBFPWqQz7U4l07sAFI2WrEtX4Trt4+SmJRgdq1roYepW7tMhs4/N/DfeYLnm/8PJ89OuBXrwoBhs7kadIuKFX3MjrOysqRsmaIMHjmfGV8O5MzZ7zlxcgGLFn9Ir4Ffcvb8tQydd88BMyhXyZfAwCWs3zCJK1eWEBOfwMiPf0z3tXyLF+LQofNmY0ajkUOHzuNbPGfXfotI5lKAFpEUYiLDcHVKuY+zi0NBYiLvmL72cHTBy8mNAH871h+uSInKRdl99EvCIq4QGxfB2Ut/EBC0gVHDOmbk9HO8fQfP8WqvyQx5vxOxsRs4c+Y7ouIScM/vyrZtR82OjYmJY+++M7Ru/TwvvfScaTFn/fqV6Nu3NXO/25Bh874adIu9B88zceJbWFlZAuDgYMdXMwcy77sN/Lh0Kw8eJKRxlYcGv92GQe/O5MKF5B8C4uLiGTnyG/K4OFK7Zulncg8iIqAALSKP4O5dims3j5mNGY1JXL99EvciKYOJl5MbFgYr8tUdTqEmfhw+9xUbdn2Au8dFdqybQNnS3hk19Vxh8oyVjB3bg1deaYiVlSWenu4s/O5/WFpa0KvnZHbuPIHRaOTq1VC6dP4Yz8J5qV+/YorrVKniR+DVmxk27+s37lC8WKEUpRqenu5YWVkya+EGXuo0nvv3Hzzy/Pv3H3D42EUuXg4BoFunhrz1+gvUqfMOFcr3wsurE8cOnefXn0Zq1xcReaZUAy0iKVRp0o1VMwdia+2Ej1cd4uIjOXruV+zzuOHpV+WR5yTXRUP+4t15bmlHWvsk0tDJJYNnnjscPn6RKV/0NxuztLSkXft6RIffo1fPSYTcCMfa2pJe3ZpRv1YZtm09Qp8+rczO2br1KBXKFjUb+6txwrMIoGVKFuFCQDAhIXcoXDifaXzfvjPky+fKnztn0LLFByxctJm3e7UwO3fBT5sYOf5HChTIw+07kfgWK8R3swYxpH9b3u75ImcvXMM9nytenvlN56zfdIhvf/yDW3ciqVOzNAP7tMLjb+8rIvJv6Qm0iKSQt1AxWvWdSnDMSZasf5u1f47F0bsQL/WZhMFgIOZeOOcObuD8oU3cj7lnOs+8LtrykY1W5L/z8sjPqVOBKcZPnbxMhza1Obt/NkEnFxB6/kemfNyTXt2bsmvXSaZMWUZMTBz378czZ85qVizfzts9XwSSd8Vo3/0T7Ap2xNW7M28NmsntO0/378/V1ZH+vVvSvt0oDh06T1JSEtu2HaFbt08YPfo1rK2t6T+gLavW7TM774+tRxg/ZRmbt0zlxMkFBAUto2Onhrz48lgePEjA3t6WKhV9zcLzp9N+ZtDI+bzUri5jP+5JdEIStZq+z+UrN57qPYlI7qRW3hlErbwlpzi+4xcObPiWwgXKYzQmcePWaZ5vN5DSNR8+MUxu+Z0IGBnSK5rq+azU9vspiIuLZ9nKP3nvw29wzePMn3/OwMvLHaPRyPz5a/ns00WcOzAba+uUv1wMuHSdISO/YeuOExgMULtmaaZ93IuK5YsTdO0WNZu8z/ARXXjzzZZER8fxySeL2L7tCAe2TsPGxvqp3UNSUhIz5qxh4vQV3L4TSdmyRRk5shtduzYBYOXKHcybtZr1K8aaznmp0zi6vv4C3bs3M7tWg/qDGNSnFVUq+vD9kq2E3rpLrWolqV+nPNUaDuH06YVmT7rHjPmOqwHBLPh60FO7H5GcSq28U6cAnUEUoCUnuBF4ko0LRvFinY9wckh+2hdxL4SNez6h7TtfkbdQMbPjr0WFoxD93yUmJjLmsyV8/c1aEhISWbfuM3bvPsWkSUspX74Yly6F4OJkz8ofR1K6pFeq14qOjiMpKQln54d/F/8bvZBEK0s+//xhWYjRaKRxo/fo81ozOnes/9Tv6dDRANp1/4QTJxbg5uZsus9mTYfxats69OnxounYMjX7s/K38ZQtW8zsGkOGfE34zXB+33iQrl2bUKKEJ7+v2cPpU4HUeq4sK34ZZ3b81auh1Kjejxvnf3jq9yOS0yhAp0410CLyxM7uXUeZYi+YwjOAq3Nh/Io04NyB9dRu3c/seC8nN65HRzJ9gSN+DeMYVyNGIfpfGPXJIvYcOk/37k2xs7Ohfv1K1K9fibffbsPBg+dYvHgLrvY2aYZnAEdHuxRjB48G8OHo183GDAYDLV+qxYEjF55JgK5W2Y86NcpQvnwvBg/uiIuLI7NnryaPswM9/v9p9F/KlS7Cjh3HzQK00Whku/9RAgND2fjHZGrUSF7c+s477WnZcgS3H9F98d69GOztn2yv6fj4B9wJu0e+vM5P9Qm8iOQMqoEWkceKi47gTshFHtyPASA26i6O9ikXYTnZ5yMu6tHNUpLroq0I8LdnzAFjrq2LvhMWyeAP5uNdvhdeZXvS//3Z3AgNT/O86Og45n63gZ8WfUhkZAzlyhUzvZYnjxNNm1bjhReqE3z9zuMvkgbPwvk4c+ZqivHTp6/gUShvinGj0cifu0/x8ZSlfDXvd0JvPv4+4uMfsNn/KL9vOEBERLRp/Mjxi2zbeZx3321PQMB1du48QenS3gRfv80/H3r1eq0Zo0cvZMOG/RiNRiIionj//TlERERTubKvKTxDcvD//PN+7Nt/ln37zpjGk5KS+PSTRXTuUC/VzyIpKYnxk5fiVa4nVRsOwatcT8ZPXkpSUlKq54lI7qIn0CKSwoP7sexY8TmBJ3fi4JCX2Ni7lK/bnkJ+lbh69CDFPGuZjjUajVwJPUTpBi899noPOxfas4Y48InMVSUdwSF3qNdiBE2bVWPzlqlYWVkyc+Zv1H/pAw5smYarq2Oq5+Z1c8bLy52qVUuwadMhevY036Hijz8OUrmCz2OukLY+bzSnW99pNG9enZIliwCwZcth1qzZw8Q9M82OffAggc69pnDy7FXad6jHxZOXGTtxMd9+9S7tXnrO7NitO47xWt/peHsXwNnZgdf7TWfCh93p/2ZLZsxew/DhXXj//VfNzmnaZCi//r6Hzh3rExkZQ//3Z7Nu0yGcnR149dXxYIQko5GWzarx0dBO/LxmT4r7cXNzwcLCQMuWI2jfvi7Fixdm1W+7sLK0YPakPql+FmM+W4z/7lPs3fs1vr6eBAQE0+ONiSQkJDJ+ZLd/8/GKSA6kAC0iKWxbOgnj3Xg6NP0cG2sHomPvsOPwLLyrPE949DX2n/yJ0sWakJSUxOnLG4gnBr8qjVO9podj8pZ2Af7hTPf/qy4655d0LF6+nbcGzaRWrdLMmzfUtD3c9OkDCLl+hwWLNjOkf9vHnl+4oBt3wu5x40YYPXq8yPTpK/j000UMHNgeg8HA7Nmr+WPjASZu/yLNudyNiCIhIYn8+cy3F6xXpxwfDe3Ec88NoHIlP6KiYgkOvs3PC/9HAfc8ZsfOnL+WqLh4Tp5aYCptOHz4PE2bvk+D58vjlscJgFu3I3i11xSWLRtNkyZVAbh4MZhGjd6jXBlvTp8Pov/glA12nq9bgdPnggDoMeAL3NzzcPXqUpydHQgMDKFd21F06VCP/w3qSERENO99tICAgGD8/DxN15g7dw0dW9dh0rgeLFruz649pwkNDeN6SBg+VfrQ543mjBneOUVpRkzMfWZ9u45jx76hSJECAPj5ebJ4yUdUqdKHEYNfxsHBNs3PWURyPpVwiIiZqPCbXDt/kOcq9MDGOjncOtrno3aFXpzevYq273yJVUFHNu2fytbDX+DoXYi2A2ZgZf1kweKvpivTFzgy5oCRKNuYZ3k7mSrg0nUGj/yGFi1q0qVLkxR7K7fvUI9de8885uxkzs4OvN65Eb16TubBgwT8/aezd+9p8uZti5tbG3ZtP8rW1Z/gnt/1sde4eDmElq+Mw7t8b/yq9uG5pu+zZ/9Zs2P69HiRwGPf8H7/tnw2qjuXj82nUb2UzVcWLffno1HdzcJn1aoladK4Kr+t3Wsa++lnf1q3qm0KzwC+vp6MGNGVuQs34Fe8MPv/MQeAgwfOUsLHg4uXQ9i9/yyzZw82LXgsVqwwC78bzqxv12E0GnF1dWTimNdp2HAIU6cuY9WqXbz55hS+mb+WcSO7UqigG34+hTl4LIBvvh1GfPwf7N37NcfOXKHv4K9TvHdQ8C3y5XUxhee/eHsXxC2PE8Eh/75MRkRyFj2BFhEzkWEhuLp4YmVlHojzuHjy4H4s1rb21Os4mHodB//r93hY0mHHGHLu4sIfl/nz2mvNsLa24sqV0BSvX74cQuS9GHoP/Io8rg689mqjR5ZiTB7Xg/dHLaRkydfx9MhH8PU7vPXGC0wZ3xMHh5SLAv8uKiqWJm0/4p2BHVi5egI2NlYsX76dtt0msHvjZPx8PEyBT3KwAAAgAElEQVTHOjs70KJZtdSvFx1H3rwpG+S45XUmKjrO9HXIjTBKlS6S4rjSpYuwbPEWPhv9Oh3fmEj16iWpXbscCQmJzJmzmpMnL7P822H8uec0FSv4YGtrvuivcmU/gq/f4cGDBGxsrOnT40UqlivGNz9uwn/zYWpWK8Eh/+mmHygmTP2ZuXPfo1mz6kDyE+XlK8bi7d2ZwKuhFPMuaLp24YJ5uXU7glu37uL+tyfvN2+GcyfsHoUKmD+NF5HcS0+gRcSMq7sXdyOu8eBBrNl4WMRVbO2dsLaxfyrvkxsWF94Oi6Ro0YL06NGc+fPXcubMFdNrly+HMGXKMsIioqn5fDmc87nSstN4Zs77PcV1bGys+XJSHy4dncfieUO5dHQeX0/tl2Z4Bli8YjtVq5Zk2LBXsbe3xdLSks6dG/PWW62YOX9tuu+pWcPK/PDDH2ZjkZHRrF69mxcaVTaN1ahagnVr96XYBmvt2n3UqFKC2jVL89WkPrzaaTwl/Lrj5dWJxT9uYuMv47C3t6WUnydHjgYQG3vf7Pz9+89S1LuA2V7Xz9UozTdfDmTN0lGMGtbZ7Gn8kWOXTOH5Lw4OdjxXqwzHTwaajbu4ONCtUwPe7D2F8PDkBkHh4fd4682pdO/U0GzrPxHJ3RSgRcSMo0s+ipV7nl3H5hMbl7yzRsS9EHYf+4bKjbtgsHh6/2yYdy60wj8qMkeVdNR7riyLF2+hdGlvpk17mzp1BtKmzYe0aDGCsmV7ki+fCwcPzeHtt9swblwP9u6dyZiJix+7o0YeVycqlCtGHlenJ57DqbNB1G+QshSjQYNKplrj9Bg+qCNLl2zl3Xe/Ys+eU6xcuYNGDd+jY5valCrxcBu9di89x73IaAYMmEFw8C0iI6P5/PPlLPppMwP/v6V4xzZ1uHhkHit/+IDdGyexa+Nk0zWKehegWcPK9OwxiZv/v8vH6dOBvPbap7zYpBqv9f2cgiVew69qH8ZOXExcXPwj51vEK2XXxqSkJE6fvkIRr/wpjp/2cW8K5nPBx6cbVav0wcenG4XyuzJlfM90f1YiknMpQItICg06DcXF24tV20bwy+YhbNg9AStHewIOb2XrkoncDr7w1N7Lw9EFLyc3AvztmL7AkYN3EnJMiO7QujYBAcF07/4plSr5smnTZBITkzh8+DzdujWlceMqZnXRRYsWok3rOqxevy+Vqz5eUlISsbH3zZ76+hQryKGD51Mce/DgOXyKFUr3e3h55mfPH5OxMcLAATOY9dWvDO7biq8m9TU7ztraik2/jicxNp5Spd4gX752jB69EFtbKzZsOWSao5WVJeXLFqV40ZRz+ebLgTjZ2VCsWFfy529HvXqDKFQoLwsXbaJC1RIcPjKPX1aO5/jZIDq89ukjmz4MePMlBr7zJWFhyb/hSEhIZNy4HyhUIE+KcpkHDxJYs2E/drY29O3RnDH/e5WtqybQoE45duw+SUJCYro/LxHJmRSgn8CGDRsoXbo0JUqUYNKkSSle9/f3x9XVlSpVqlClShUmTJiQCbMUeTyj0ciNwJPsXjWL3au+JuTyiVQ7TFlZ21Lv5SG8PvYXnmv7NmDA06U8lbzb4hjrwprZQ7lyOuX2Yf9Fdl5cePnKDXoP/Ioi5XpSumY/xk9eSlxcPDY21tSqVpLIyBg6dBhDx45jKV68EEePzufChWt4erqnuJaFpQWJienbczg+/gEjxn5PgRKv4VasKxWff5cVq3YB8Nqrjdi8+TCLF28x7WW8Y8cxvvxyJf17tUjtso/lUTgfUz7uyYGtn7Pp14/p/mqjFAskAfLldaF40YKU8PNk9+6vuHdvLct+HsPMb9YxY87qNN/HwcEW32KFaNyoMgcOzObGjV8oX744Q4a8wogRXfHycqdyZT+WrxjL5as32bH7FElJSaxZv5/eA7/irUEzKVOyCNUr+eDr251GDYdQvHgXtm0+xPLvhpvN+d69GBq2GskXc9dQtIQn0QmJvNHvCxq3/YhfNxxg1GeLKVn9bY6duPyvPjMRyVm0iDANiYmJvPPOO2zevBlPT09q1KhBmzZtKFOmjNlxDRo0YPXqtL8hiGQ0o9HIzpVfEnh8J75edTEAW77/GO9yz1Hv5SGPDD5/sbKx49AfP/J8pTfxKpRc31oof2nyuRZn5y8z8C5d66mXdPy1X/QYYmntE0lDp5QL1rKSa8G3eb75cPr2bc2OcTMID49iwsc/0q7bJ6xfMZYh/dvy5qCZrF8/kbJli5GUlMTChRs4efIy3kUKYDQaTX8HN26EsXr1bkYPfjldc3jr3ZmER8Vy8OAcihUrxNatR+jRYyLW1pa0bfkca38eTa93vmTEiHnY29ly/3488794h4rliz+Lj8QkNvY+U2f+yv79s/D5/8WKzz1XluUrxtKg/mD6926ZZpe/Fat38cWXAylevDAAu3adZMGC/5kdY2VlSes2dfhzzynmLFjP2YBg3nzrJZKSjIwY/z01qpTgzL6vOfH/jWHKlvZO8T6fTV9BcV8Pflr0IQaDgZkzf6VMmaJs3DgJF5fkfbqXLNlCm64TCDg816wGW0RyH/0LkIb9+/fj5+dHsWLFAOjcuTOrVq1KEaDVL16yCqPRyPWLR7l0bDsYDDjnLcSVE7tpVf9jbKyTFwCW8WnOul3juXb+IEVK1XjstaLu3uR+dCSeBSuZjRfKX4akhATu3r6GW4GUYeS/yE5NV+Li4mnZaRyvdGrI2HE9TOPLV4ylcqU32brjOE0bVuajoZ1o0GAwRb0LcudOJHlcHVmzdBT93ptF2zYf0aVrY0JCwvjyy5W8178tRb0LPP5N/+HylRus33KYK1eWmBYVNmlSlblz32PMqIW0bfkcVSv5cmTHF5y7EEx8/APKlfHG0tLyaX8cj5hbKPnyupjC819Kl/bG3sGWa9fvpFlGkpiYZBayCxRwIzDwBtWqlTQ77krgDfI6O3D+Ugj79s8y7d7x5pstqV7tbY6euEzzv22p90/LV+3il5XjTT/MzJ+/lq++GmgKzwBdujRh1ter2LjlCK1efPj/TUzMfZau3MHpc0H4FitE15cbpNocR0SyP5VwpCE4OJgiRR5uxeTl5UVwcLDZMQaDgd27d1OpUiVatmzJ6dOnM3qaIkByeN6+bAr+iyZhc9cam3Arjm9Zhr21C9ZWD3dssLa2p6R3Iy4c2pzq9SytbEhMekCS0bz202hMIiHh/hPv/Zxe2aUuevjY7wmPiKZz50Zm48lPRJ+neYfR1H1xON5e7lw5/i1ffvYWqxd/xCH/6dSuUZpdGybzYsNK/LJ0G6ePXeTH2UP44L1X0jWHYycDqf1cmRQ7cjRtWo0jxy6avjYYDJQu6UXF8sUzJDwDFCyQh9Cb4URGRpuN37kTwd27UeR/xHZ4/9T6xZrMmb3a9JCid+8WjB37PXfuPGwdv23bEbZuO0r43Sj69W9jtvWdg4Mdb771Eit/Ny85MhqN7Np7mh+WbuXQ0QDi4xPMmqSEhobj62se/AF8fT0IvfWwdfmlwBuUr/MOv204QMEiBdi+7wxlnxvA8ZMq9RDJyfQEOg2p/Xr7L1WrViUoKAgHBwfWr19Pu3btOH8+5aIdkWft6tl9hJw/xkv1xmP9//s4lyrelN/9PyLk1kk8ClQwHWtlaUNSYkKq13NwdiO/RwnOXd5MWd8XTeMXrmzH1d0LZ7eCqZz933k5uXEtKpzpCxzxa5i19ouOi4vnh6VbqVatJEFBt1K8fvVqKFOn9sPbuwBv9P+CH2YPplmjKmbHODra0a93S/r1bvmv5+Ht5c6pU1dISkrC4m/lNCdOJJeIZKZ8eV1o1bwGgwZ9zZw5g7G1tSEuLp5B786kY5s6uLik/Xc5pH9bGrYaSccOY+jarQm3bt3l5s1wfHy60eLFmty+HcGJE5dZtmAYC37a/Mh/sy0szMduhIbTrtsnREbFUq1aScZOXIKVpQVz56xh6rR+ANSuXZbfftvFgAHtTOfFxcWz8Y+DDOvXxjT2zrC5vN2vDcOHdzGNLVy4nt7vfsX+LdOe6HuIiGQ/CtBp8PT0JCjo4VZPQUFBeHl5mR3j7Oxs+nOLFi3o378/YWFh5M2b1+y4AxsXmv7s4VsZTz/zb6Yi/9XFI9so6d3IFJ4BrK3sKFW8GYHB+0wBOikpkYBrf1K5Rdc0r9mw8zBWzxrCjbCzuLv6cjvyMmGRgbTuN+2Z3cffeTm5ZcmmK3cjorGxsaJfvzaMG/cDzZpVMzUY2bv3NOvW7eOLLwbg7p4Ho9HIhKk/mwJ0UlISX3+zjnnfbSQkNIxa1Ury4dBO1KlVJrW3fKQqFX0o6O7K2LHfM3r061hZWXL7dgTvDvySAW/++2D+tMya+jav95uOt3dnKlfy5cjRABrUKc+crwc90flueZzYuX4iCxdv4ceFG3B1duDHOUMoXcIL/50ncXa258UmVbGzsyEsPIpJX67k9ddfMJV9xMbe55v5a5k8tofpmr3emUHjZtX49NM3MRgMJCYm0r37Z8z/Zi2hN8Pp3LkxpUt7M3z4PGxsrOjQoR5Xr97kgxHzaVyvgqmGOvxuFLv2nWHlavOF42+80ZxRoxYScCmEEo94ii2SFfnvPIH/zhOZPY1sw2BU8W6qEhISKFWqFFu2bMHDw4OaNWuyZMkSsxro0NBQChQogMFgYP/+/XTq1InAwECz6xgMBvpN257Bs5fcZvOPE8hn9KJkMfOSgnOXt3Ai4HeqlekEBgPnr27FJo8LLXp/yo3AE4ReOY29kxu+lRpiY5eydvPB/VjOHlhP4MldWNnYUqZWK4qWrZ2hT9euR0eSZEzEr2EsrX0SM3VxYWJiIms2HOCtd79i0+ap/PyzP/PmraVFi5qEhUWybdtRfvzxA15+uQEAMTFxuLm1IS50JQCDRszn0PGLTJ7yNiVKeLJ27V6GD5/Hiu9GUK9OuXTPJ+RGGF3enMrFwBv4+Xpw9NhF3nz9BSaNfcPsqXRmCrh0nQsXQyhVwvNfbZ/3JBITE+nSeyqXg27yVp9WJCYmMXfOGiqXK8bCWYMwGAxcC75N1YZDuHZtmVmpR1hYJN7er9KgQWUSEhLJn9+VOnXKsX79frZvP4a1tRVDB7Rl2LsdTAsIQ2+GU/a5Ady+/VuKz7lkiddY8d1wKpQr9kzuVeRZs3Bro/VdqdAT6DRYWVkxc+ZMmjdvTmJiIr1796ZMmTLMnTsXgL59+7JixQpmz56NlZUVDg4OLF26NJNnLblV8Yp1Obz2B/y862Fhkfy/d2JSAuevbsOnSn2u3jkKRiNlG7WleIX6rJ33P6Lv3MTTvSIhsUfZu2YuL/aaQGEf88YbNwJPcmD9AjwKVMCJvOxZOYvj25fT8q2Jz6wO+p+yyuLC8LtRNO8wBgsrS56rXZb27Ufz008jefvt1ixcuIGFC48zcGB7U3gGOHcuCI/C+YDkXTt++nkbly4tJk+e5IYoPXu2wNrainGTl7D5t5TbYD54kJDqrg+FC+XF//dPOXMuiJDQMCqWK07+fFlr9xI/Hw+ztuHPgqWlJUu+fZ/V6/ezau0+DAaYMLIbLV+oZvph79adCAoXypuiRbibmzMWFhYkJSaxceNk0/iAAe0YNGgmDpYWjBzayeycAu55KF60IL/+upOOHeubxv/88zjx8Q8o+4hW5iKSM+gJdAbRE2jJCElJiWz49kOib92iVNEmYEzizKU/iIm7i3O+QrTqNxU7h/8vM/h9LuEXLlK3aj8sDMlPz4JDj7Pn5AK6j/oZS6vkX4E/iI/jp/Gv0KDqOxTMXzr5fYxJ/Hnoa/KXLk3Nlm9m+H1eiwoHjAzpFZ3hIfrt92aBtRVz5iRvAbhw4XrGjv2eoKBbeHnmw2g0MmjwKwwd+goGg4HIyGhat/qQpvXK89Gwzqxcs5sflm9n1epPzK4bFRVL/vztiL3xC5C8yG3uwg1MnfkrgVdu4l3EnaED2tH/zZaqq/0PYmPv412hN3v2zKTE3zon7t59kq5dJmBna03b9vUYPrwzdnY2fPPNOj79ZBEHtk7DyzNl58I/d5+i4xufMWBAe+rXr8DBg+eZNu1n5k0fQJuWtTLy1kSeKj2BTl3W+N2eiDwVFhaWNO/5MXGJ9zh7aRPXQo9Szq8lHZtNJ69dEXb9+pXp2HMHNlKxZDtTeAbwLFgRJ3t3ggMOm8aCzu3HzbWoKTwDWBgsqFCiLecPbcqYG/uHzGq6YjQaWbJiO2PGvG4KsT17tuDKlaVUrFCcn+YOZfvaz/jph42ULv0GTRoPpXDhl7kQcI3PZ61i/OSl5M/nQmBgaIpvTIGBNyjgnsf09fRZq5i9cAOLl4wiIWETy1eM5dtFm5k8Y2WG3GtOZW9vy4dDX6FN6w/ZuPEAd+5E8Ouvf9K588d88tFrbF39CSFXQvH07ISra2s2b9jPllUfPzI8A9SrU44daydyM/gW48d8x9kTl1m/fIzCs0gOpxIOkRzm7q0gSEiiZePRGP4WjiuVas/KzUNpmDAMSysbHsTHYmeT8tf8trbOxMc93Hbswf1YbK1T1kXb2jjx4H7ss7mJJ5AZTVeMRiMxMfdxc3NO8VrevC5Ex8RRvGghDm//grovDsfK2pLDh+dSqpQ3QUE36dB+NC7ODsTfj+fbb9fx5psvAcm7O/xv2Fx6dW8KwP37D5j0xS/8uXMGpUolL1irXr0UK34ZS61aA3i3byvs7TOmdCYnGtyvLe75XPnwg/lcCrxB2VLezJzUl9YtagLw/ZwhLJw1CKPR+ERb/pUu6cXXU95+1tMWkSxEAVokh4mPi8bOzsUsPAPYWDuC0UhiwgMsrWzw8qvK5Wu7KePb3HRM7P1IQm+dobHPSNOYp18Vdq6cQdz9SOxsHwbUS0E7U23CkhHM66Jj4RmHaAsLC5o1qsKPP26iT59WpvHLl0M4eiyAOjWTFxefDwgmMOgmV3Z+aapdLlKkAHPmvkfHDqNZ9/MY2nb7hHlzf6dECU+2bD1Ck/oV+WBIcgfCoOBbODramcLzX3x9Pcnr5syVoFuULmm+G5CkT7dODenWqeFjX3/Wiy/Dwu8xZ8F6tu86hVseR3p0bcKLTas90/cUkadHAVokh8nvWYJ7UTeJjArBxamwafxayGHyuHubdtmo0bI3q2cNIT4hBq+CVbgXfZPjF36jQr2OOLjkM53nlKcA5Z/vwB97JlLBrw1ODu5cvXGIS9d30W7gzAy/v3/6e4ie7m+EXs92ceEno7rz4stjCQ6+TYsWNTl79irjx/3A2BFdTfsaX7x8g4oVfFIs/KtatQRXg25R0s+DM/u+ZuuO44SEhvPhoI6UKfVwwZl7PlfCwu8RFhZp2hoP4O7dKG7euksBd9dncm+SMUJvhlO3xQjq1q3A4Pdf4fr1Owz64Bu6HDzH2BFpby0pIplPiwgziBYRSkY6sfNXjm5aTJVSHXFz9ebGrdMcD1hNs9fH4FXy4VOuu7eucXTrEkIvn8Te2Y1yddviU7FhikVqRqORSyd2cGbXGmKj7lLItwKVG76Kc95nsx3Zv/VvFxdeDbrFkl92cC8qliYNKtKwboVUF+qdDwhm+uxVHD52Cbc8jlQp70PTRpVpWLc8lpaWXL5yg1pNh3H16lKzUos//zzOW72ncmb/rDTn1HvgV8QnJfHNt8Ows0tuQPJ2388xxifw3ezBT3RfkjUN/mA+2FgzY8Y7prFbt+5SuvQbHNo2PV2t3EWeFS0iTJ0CdAZRgJaMFnh6Nye2/8K9sBvk8/ClSpMuFPBOf6OO7CZ5v+iE/+9caEgzRP+0bBuDR35Dp06NcHd3ZcXy7ZT0LczPC4enunVcQkIib783i9Xr99O4UWUuXrxOREQ0v/00krKlvenSewpGSwu+/GogBQvm5dixi7zaaRzDB3WkR9cmad5HdHQcPQfMYPvuk1Su5Mux45d4vmYZvps1CGfnzG8kI/9eyepv8+tvH1O+fHGz8R5vTOS5yn707fniY84UyTgK0KlTCYdIDlWsbB2Kla2T5nHGpCTCb17FYDCQp4B3tt8iLT2LC2+EhvPuiPns3v0VZcoUBeCjj7rTrOkwPp6yjPEjuz32fabN/I3LQbcIDFyMo6M9AAsWrKdd9085s+9rFsx8l/dHL6RUqTews7PBytKCD4d2eqLwDMltvn/+bjiXr9zgfMB1Svh6PLMGJJKxrKwsiImJSzEeExOHjY2+LYtkB3oCnUH0BFoyW2JCPBePb+fmlbM45XGnZPUXCA8NxH/ZVJIePMBoTMLazoFGnYdTqHj5zJ7uf/awc2EcrX0SHlnSMeubdew/fpHvf/jAbHzTpoO8+urHTBzzOm+90ZxHKVGtL0uXjaZ69VJm49Wq9mHy2DdoXL8SADEx97kbEUUB9zxYWaW9o4PkbHcjoihTsz81a5Xht98+Ni1WPH06kOeff5cLh+aQL2/WaoIjuZOeQKdOP+qK5AKxUXdZ9fUg7AxOeOQvz61rpzm86UeSjEnUr9oPjwLJnQeDQg6x/tuRdBq2AEfXR+97m114OCaHkAD/cKb7/1UXHWMWomNi7+Pi6pTiXFdXRwoXzsuIcd/TvHFVvIu4ExERbXoN4Oatu/j6puys5+vrQejNu6avHRxscXDQlnOS7Ov562jUqArXQ+5Qu/Y7vPxyfa5du823365jSL82Cs8i2YQaqYjkAnvXzKWQcyma1hpGOb+W1KnUm/rVBmAwQmH3chgMBgwGA94e1SlauAZn9q7N7Ck/Nak1XXmxSVV++WU79+6ZN2JZuHADrVvX5uWXGzBz/u80afsRXuV64lm2BxXqvMP87zfyXI3SrFq1y+y8qKhYtm47Sq1/PJUW+csf247Q+82WbNkyjQ8+6Mr163fIl8+FHj2ak5ikp30i2YUCtEgOZzQaCTi2lfJ+rczqmwu7l8PRPj8375w3Oz6vS1Eibl3L6Gk+Ux6OLlgYrAjwtzML0eXLFqV9q+eoUaMfS5duZevWw/TuPYXNmw/z3nuv4OzswJwF63mlc2PCwlYRFraKfgPaMvTDb7l4OYRhw+aycOF6wsIiOXDgLK1bjaR9q9qqVZbHcnSwIzz8HlZWlrRrV5fp0wcwevTrgAFH/aZCJNtQgBbJBZISE7CySvnN2dLSmsSkBLOxW3cDyOvhk1FTyzAPQ7Q9Yw4Y8Y+KBOCrSX0pW9KLUR8tYMyY7ylSxJ09e2bi4GDHokWbafZCdfr1a4ONjTW2tjYMGNCeVzo1pGixQpQp4cnPi7dSvHhXunWdQMsmVZk9TR3p5PG6vlyfyZOWmi0ivHgxmKVLt9K5Y/1MnJmIpIdqoEVyOIPBQJGSNbhwxZ+yvg+3x4qMCiE84irRMbdJSkrEiJGLV3cQfOs4dWu+l4kzfnbMOxfGgU9y05X5M96hXosRFPUuQO3a5di06SBTJi/DzdWRl19ukOI6jRpVZt26faxfv58Lh+aSP5/qVuXJdH2lAVv/PEHZsj3p1KkhkZExLF/uz6ejXtNvLkSyEe3CkUG0C4dkprAbgaz+ehDFPWrj4V6BiHvXOXV5HWVqt+LauUOEh17GCOT3KEH9V4aQt1DxNK+Z3f2z6UrCzSTmLNjAH9uO4ORoR7dXGnL0xCXiDQY+/7y/2blDhnyNk5M933+/kW2rP1HwkXQ7cPgC6zcfwsHellfaPq/mKZLlaBeO1ClAZxAFaMlsUeE3ObHzF24GnsUxT37K1W1H4eIVgORdOgwGA3aOuatFdFpNVy5fuUGtJu/z06IPeeGF6gBs2LCf11+fyLx5QxkyeCYBh+diaant6UQkZ1GATp0CdAZRgBbJmh7uFx1La5/EFE1Xtu44xmt9p2OwMGBraw1A165N+f77DUwa8wZdHlHiIfIsnDgVyLmAYEr6elCxfM7/LZFkLgXo1KkGWkRytcfVRV86FMq4SUvZtvMELs72lCvtzZ3we9wIDefg3tN8++VAmjWqktnTl1wgIiKazr2ncOrcVWpUL8XBQ+cp7efJ0gX/wy1Pyn3MReTZ0xPoDKIn0CJZ31910e3rnmd4508ZNep1OnduREjIHUZ9tAASk/j1p5HZvt25ZC+v9f0cB1dHZs0ajJWVJQkJiQwc+BXhN8NZ8u2wzJ6e5FB6Ap06BegMogAtkj1ciwrn9JoZ9O7oywcjuprGHzxIoFTJ11k8f6gapch/djciir0HzuHq4kjNaiVYvHwHCxdvJiw8irq1yzDsnQ4U9S5A+N0ofCq/xZUrS3D9W9fMe/diKFLkVQIOz1X3QnkmFKBTpxIOEZG/8XJyY8+1c7Rv94bZuLW1FS1a1mLPgbMK0PKfTPlyJZ9NX0HVKn6EhoZz+3YEhQrlZdz4nnh5ubN8+XbqNP8fO9Z9htFoxC2Pk1l4BnB2diB/Phdu34lUgBbJBArQIiL/4ODkwsWL1yld2tts/MyZKzxfxS+TZiU5wa+/7+HbnzZz/Pg3FClSgICAYGrW7M/2HV+YQnK1aiWxs7Phs+krmD2tH3Fx8Zw6dZly5R4uHDxz5gr37sVSzLtgZt2KSK6mToQiIsCD+7FER97BmJQElg4MHz6P8PB7ptd//30Pe/eeoVQJr0ycpWR3sxes5+MJvShSJHnf523bjtCmTZ0UT5i7d2/Kpm1Hsba24qP3X6VD+zFs2XKY2Nj7bNt2hA7tRzNy6CumnWFEJGPpCbSI5GrxcdHsXzuTgKP+2NraYGVjz/24GJp0qI2f32s0blyZkJAwAgNv0KhRZX7esIOqlXwze9qSTV0Lvk3ZskVNXzs7O3D7dkSK427fjsDFOXlf8v5vtsTF2Z6hQ77m7PlrlPTzZMTgjrzeuXGGzVtEzOkJtIjkKjH3wgkOOEzE7WsA+C8ZQ2U/I0FXFxEetpI1v32Iva0BH5/CHDs2nw4d6vHhh924fHkxlpYW/LIjkCjbmEy+C8muqlX2Y8OG/aavW7Wqzd69p9m166RpLB0kW2wAACAASURBVD7+AePH/UD3Tg1NY91fbcSRHTOIvfELx3Z+qfAsksm0C0cG0S4c8izEx0Vzdv96/o+9+wyMouoaOP6fLUk2vffeewIkoffeu6Bgo1pQQQQsqIiIIioWFAuoCEiTroCAmACht1ACISGF9N7rlnk/RIN5g6g8QCDM75N7MzN7ZonZs3fvPSfn6nlUJuYEtB+EtZNPc4d1z6muKCH5XAzJcb9RlJ1IULA3V5PSMbFypqo0m6yMtSgU17sJbt8ey9SpH5GWtg49vfqvyE+eTKBHj5nY+EYy9KOnbth0RSL5J+cupNBr+Ot8+OEzjB3bnby8EiY8+T6xhy8waGA7nF1s2b4tliB/F9Z/O7vh908iudukKhw3J81ASyT3qaqyQjZ+OJmMU8dx0PNHUSLn5y9f4tKxX/7na4uiiEZd2yL+eF459SsbF4/DSjxFjw52GBkq6dk9iKzMtQzu64NKX94oeQbo0iWM4uIKAgOfZNasLxk/fiF9+szGy9sFE6eQ+qYryXKiK8qa6a4k96vQYA+2/ziX71fsRKXqT1DQk/h42BF/9HN6dAjC2ljF95+/wOZVr0rJs0RyD5PWQEsk96kTu7/D2TKUiKCHG8bcHaPYuW0+nqFd0FeZ/OdriqJI/NGfObNvNZVlBRioTAnpOopW3R9GkN1/n7fLCrM48ctSTp5Y2lBRo6iojI4dn6dDhyA+/GAqq1ft4fLla40qbpw8mYCBSp9OncNQKuUEBrpiaGTItp/PMGT0IPI1apKiVSyJFmFCfedC41rD5rpNyX2mXaQ/v21bgFarRSaTNTTmmfJEv2aOTCKR/Fv33zuiRCIBIOX8QfzcezUaMzV2wNbKj/SEE7d0zQuxW4nbu5ZOIVMZN2gFPaNeIuV4NMd2fnM7Qr7rks7sZfz4Xo2SY0tLU2bPHsPKlXvQ01Pi6eXIG2+upKamDoCrVzN5+pmlBHV5nJOXNHz86XbeeXcDxy5q6T/lU5T6KhyNTHE2tgAElnxrxMlCjbQuWvKfyeVyqaulRHKfkmagJZL7lSAAN1piId7Sm7JOp+X03lX0iJiBpVl9wmlh6kyXNtPYHv0KrXqOQ19l/A9XubfU1VTg7GTZZNzBwYri4nJKSytISMiiuMoYe4cxWFubkZNTSFCn0YR2GY0gCHR56O+v72xsQVZlGUu+NcK7Ww1vRVY160x0ba0a4K6WNisqLmf1+t9JScmldStvRg/riIGB3l17folEImkO0gy0RHKf8gzpwqWUPY3GSsozySu8grNv5H++Xk1FCTqNuiF5/pOhgTnGRrYNVSvuJw6erVn9YzRarbbR+Lp1+wkL82LosLfwbtWDkG6PoRMF9A30aN8hhIuHNrLzmxkc++VLsq7GNVkLXltVTmlBBlqNGkcjU2SCgqRoFW+eEJtlXfSlhHT6DHgNM6eHMHN6iEHD5pGcmvOvz8/MKuSNt1czbMR8Zr/27b8+99jJBHxDp/DT/F1kfR3Px7PWEtzmGTKzCm/1ViQSieS+IFXhuEukKhyS262qvJhtnz2HiYEtLjatqKgu4Mq13+kw7Fn8Ivr+5+tp1LWsfHM4Q7ouxFBlcX1cU8umfTMY8/IPGJla3c5buON0Oi17v5uFj5sec18di7GximXLtrN69T5kciVBHYYT0H4km5Y8xuaf5tKrVxsAcnKK6NBhGm3bBnL46BVM7ALpPOoV6mqrOLZ9CanxRzAzN6Wysoaw7o8S1HEkWZVl6EQt3t1qGOypuWsVOnLzigmLmkavMge6iA7oENkvy+KweQEXTn+BudnNvzU4c+4qfQbOpbXaCq9aE64pKzmizOOnda/SvXPo356n0+nwCZ7MgGw72gi2DeNb5SnIe9iyaf1rt+0eJRLJ3SdV4bg5aQZaIrlPGZpYMGrm17i37UI+aQi2egyd9klD8qzVqEk6u58jO77k3MFN1FQ2bdbwVwqlPv6R/Tl24QfUmpr6a2jVnIj/ESef1vdd8gwgk8np+di7lMlDePixz+g/eAEnk4wYPv17Oo6cTUH6BbZ9NgFrSyOMjAwazrO3t2TBgomUlVVyOf4bZDVpJJ3dz4H184kM0iczYy2Z6WuIPfgB6We3cOXkr3/MRMsB+d8HdAd8tWIXwTXm9MIZPUGOgaBggOiKR7UR36/57R/Pf/qZpQytcOXhOm+iBDtGaTx5vMqbyVM/uemb55lzyWjK6miNTaPxPhoXdu0/1bCcRCKRSFoiaQ20RHIfU+obEtxxGMEdhzUar64oYfsX01GKBjhaBZOdfIpTv35Pvwnv4OD597OK7QZPJXr9B2ze9yIW5q6UlGVi7x5Ej7H372yiQqlPePdxhHcf1zB2MXYTKSd/4qMPJxMW5sVvv51h2LA3WLXqZfr0qV/+4uXlSG5uMSqVPi/PHsUbC3+hrOAa33y9oKHsXVCQByuWT2f8E5/jewuz/rfDyWOJ+NWawf9b9u5TbcKp41fgmb8/N7+glIuJ6UymQ6PxEKxYX5bCpYR0Av1db3huTU0dBjJFk/X2esgQRdBqdbd0PxKJRHI/kBJoiaQFOrLjS+yMfYkIeqQhwcnIjWPfqrcZ9/o6ZLIbz5LKFXr0HPcqFaX5lORew9TKAVMrx7sZ+h2nrq3m9L7vOX1yKT4+zgD4+Djj5GTFnDnf0Lt3BIIgsGPHYdq1CwDA2NgAdU0lbdr4N6kZ3b59EHnZ6Q2Pk6KV7EADnmV3ZRmHp7c9qbGXiGy8zJssvWqivB0oKCzjjfmr2bQ1Fo1Wy5D+bXl73qM4O1kjkwloNLobbkXViTffjBrRyocisYY0sRw34XrJxCPkEBnqjaGh/m26Q4lEIrn3SEs4JJIWQtTpSLt0lJgNH5B05jfsrQMbJUDOdmHoKYzISb1wk6vUMzazwdm3TYtLngEKs67i5u7QkDz/aeDAdiQmZpCcnMUHH6xn+fKdvPjiaERR5PPPt2Ns68/p0wloNI0z1aNH47GxdwJoWMZR32hFQXRF2R0vb/fUlIEcVOaQJF5fohMvFnFCkc/4Md3o0ms2iWvjmVkWzGuV4RRsTaFD95coKi4nPbMAhShwiOxG1zxNAShl+Ps6//+na6Cvr+STj55iqeoie4R04sUitihS2GaUzkcfTr5j9yuRSCT3AmkGWiJpAXQ6LXtXvkVxZipeTh0J8RnMifOryS283KjRilJhgFZd14yRNj89lTEFecXodDpkf2kOU1RUjlqtISDgSayt62tFx8enMW3apxw7Fo9WTMDcxpmnnv6Ujz6cgqmpEZcvX2PS5E8I6DC24TqORvWzzknRxSyJFpkxoZIIqztX3s7f15nvls9gyjOfYqJTohNFavRE1q94mdjjl1Hla3lE7dfwYWqk1pOyiit88/2v1NSqaS3YsE1MIVkswwcz0ijnGLm4WdrdcAZaFEUKCstQGejxyOiueHnY8+ln2zmckoO5tQW2yTV06TsHVztrXpo1iomP9pZqHUskkhZHSqAlkhYg6ex+yrIzGNDxDeTy+hrAfh69+Dn6ddwcIrGx9KasIpuikjTift/AwZ8+xtzWhfAeY3H0Cm/m6O8uCzs3lIYWfPbZVl54YQRQnxTOmfM1RiamPPF4L9q09mbduv3s2nWCgQPbUlhYRkSEH7v2p3L0XBWOTmOxsDCjrLyasO7j8Isa0OR5nI0tyKgovis1oocMaEv/xJWcOJ2ITCYQ2doHuVzOmHHvEVxp1iSBDak2Z/uWI/Ts2wpDmZK3dFHEkk0CJdiiYgw+JJg2XcP8/Y/7ePPN1RSVViACfbu3YtnSZ1mzchYrf/yNV15awdhqL3zxIS2jnIWvrCY/t4RXZt2kmLbklqRn5PPa26vY8stRAIYPbMfCNx7D2cm6mSOTSB4MUhm7u0QqYye5k3YtfxVnVTCeLh0bjcdd3kJpeRa21n6cT9yOVqshInAs1pZe5BclEXdlCx1HPo93ePdmirx5XDi8nbN7lxEe5kVYmBfR0XEYGxuQkVHI9OnDmTlzTKPjFyxYRXl5FUs+3szjb/2MVl1LdWUpppb2yBU3bxpSX95O80cSLdzVRivOXo8RXGTCWMGn0fhOMY1oRRY1Ch2CFuarIzAT6tcsa0Udn6ouMnX+CJ6ZNBAAtVrD0IfeJjr6HE8RRAhW1KDlZ8U10t20nDryMV6Bk3iiwAtP4fq67wKxmndUZ8lIXNmoyonkf1NSWkHrrjMY/2hvnn++/kPgxx9vYu2P+zgd/TFmZkbNHKGkJZDK2N2cNAMtkbQAoqhDuMHGQEEmJ78iBbmVIQgiPdvOwMayPpkyN3HCzNiBQ9uX4Rna5W83FrZEJZkXeHfhRJydbUlNzWHEiM506xbOxx9vYvv22CYJ9KVL12jVyhsBAZlMhtLIDAMjs3/1XI5GpmRVltU3WqGawXdwc6FOpyP60Hmyc4pJz8inrKiSw1TSQbTH9Y+NfnliFftIZ5omFJVGziLFWd7WP00XtT0qnYJoMimpqWXNqt/p3jmUAD8XFixaz8kDlxmJF6FC/QynCgWjNB4syj3H+s2HqKioaZQ8A1gLKqwUKhKSMmkd5nVH7vlB9N3qfUS1DeDttyc0jL3zzkSuJmWyct1+np86uBmjk0geDFICLZG0AB6hnbkc/TNujlHIhPp1vRpNLcmZsfQc9ypGZjbsWDq9IXn+k62VL6JGS3lRNmbWf79hrKURRR16ekqGDm08Y69QyIg7l0JychaenvUbKH/99QR79pzEzt4KR49Afl/9GmWFWVjYuePfYcxNywL+6a9J9A5qwLOMCCvFbZ2NTryaxYChb0KpmroaNdmaClwxoQ8uLOYMLqIxAgKplDOa6zPFw7TuXA7Usjf+Gm46I0bgiUwU2HQumVbtnsPP04n07AL0dOCHeaPnFAQBjxpjUq/loRF1lIl1mArXZ+TVopbCuipsrf/dhw3JP9PpdCxftZdX545v8rP+A9qyd+exZohKInnwSFU4JJIWwLdNH+QmBuw9+h7J6bEkpkWz6/ACHHzCcPQKR8/AkDp1FVpt4w2EGm0dak01Sv0H6ytfR79OfPb5L9TVXW/2UVNTxyefbMbMIZiQ0KlERT1NcPAExo9/B1c3R75b+RsVxem8OacX0b8tYOYzbTmw/k1SL8b+u+c0MsXZ2IKkaAOWfGvEyULNbavQIYoiQ0fOp0O2OQMqnCjV1BCEJTlU4YcF79MBI5QYoeB92tNNcGo411k0Jv1aHl0ER14SWiECq7nCENx5nw70TLagpLoaO1SkUd7kua8pKvD3dWbUkA5sUCajEevXTutEkW3KNNpG+Enrcm+jjVtjKSuvJiEhvcnPEi6n42BncYOzJBLJ7SYl0BJJCyBXKBk45X2Ceo4gq/YS+boU2g6bQvexcxAEAUMTS+xcArmYtLPReReTfsHONQhDkwfrTdcjpDMVahNatZrCihU7+eabn+nQYRqurrYUpF9g6LSvMPN9CMEsHOfAnph4DESn1bFvz3uMG9cLf39XJk0ayPq1r3Bmz1f/aZ2gs7EFMkHBkm+NePOE+D8l0Vt+PkJk+xcwsR9NwbViuukcOUg2rpjgjwU9cOZj4rhGOeFYU0Qthv/vi8fzQiEGSj281CaIoshmkplMIFGCHaaCHqGCNa4Y44EpW0gmXawA6hPkPWI65cY6BveLRC6XcV5bwExi+USMYyaxnNAv4LvlMzh5JpGHxy8iPOJZxox7j+Onrtz0vi7Ep/H4xA8Jbf0MQ4a/xf4Dcbf8GrU067cc5Lnnh/PVVz9z9mxSw/iZM4ks+3I7E8b3bsboJJIHh7SJ8C6RNhFKmltFSR47ls1EKRhgbeZJQWkyGrGGQU9/iLG5bXOHd9fFbv4Af+dK5DKQyWSMGNGJYcM6MXHSEi7nOdKqx/WvyAuykji5dT7JSd81uoYoilhYjWDoc99iZPbfZlnrNxdq8e5WzWBP7X9aF63RaJm/aC1ffLyD8RofqtAQSzazhFYsFE/hiQnF1PEUQcSQxX4yKKQGEeiAPUPxwAAFR8lhLYl4ezvimaagp9qJVzjKZ3RuVLnjvFjIV1ykLXacJA9jlJRRh6m5ETG/LSInt4Qxo95hblUriqghiyos0WejYQqDJ3Tl2xW/0q/WCS+dGSlCGbsMMlixfDpDB7Rrcm/frPyVGTO/pp/OhSDRknQq2GmYwYJ3HmfSE83T7fFe0n/UPJ5+bji1tWqeeeZjgoM9EEWRs2eT6Nw+kB3r3mjuECUthLSJ8OakNdASSQsn6nRkJJ4iPeEE3q16YGhmhbq2Gnfbrrj6RSGTP5h/Bmoq8nh0/BAGDWrfaLxNa0/ObrjWaEzPwIiS4jLUag1K5fXXq6KimrpaNUp91X9+/sbroqv/defC9ZsPMO2FZVRU1DCbVngIphSLtaziMlWiGh/MqEZDIiUcIIvOONAVR2LJ5keuUIOWWRxGi4gv5kwmkO8yrpChkOGmNkaHSAVqTLi+ljkAC+RKOcXuCmquajE0VfH0+D68O+9x5HI5n32xg/bVNqgEBU4YkyFWsooEMqoq+Hzpdp4lhCDBEgTwxBTnaiNemP4Vg/tFNarF/cPa33hxxteMF31oJ9iDAB6Y4lVlxpy53zFuTDdUqge7w2G/Xq35dsVOtm5bwKBB7YiJiUOt1jB71ldMf3poc4cnkTwwHsx3TonkAaHVqNn97WuU52XjZhdFpSabCxmbaTt4Cu6BHZo7vGZlauPNrt2nmiTQv+w8SVZKLpsWj6ayogKdDrzDu2Fs6cCSJZuYPbu+Qocoirw5bxXuAVHoGdzaGvLrTVdgSbQIE26+ufCzr3/mlZe/pYfoxO9k4vHHRkALQR9DUcFHxDEQN1aRQDjWxJDFJpIRAI0cXPTNmVQdyAQxAB0iCkGGKIpoNZdZvfIlpr/4FXoFctZqE5kkBiITBERRZJcinfBQD2L2vX/DuERRRKB+xvqAmMVO0ngUPyzQYzFnCaTxEiFfzKmrTCIpORtf7/r12JWV1TzzwheoRS2RNP5GxEkwwkZmyKmzV+nUPvCWXuuWYtKjffhxYwwjhr/B5CkDqaysYclHPxEW5E6PLv+8oVUikdwe0hpoiaQFu3h4G+qSKgZ2mkeo3xAig8bRv9PrHNn+JRWl+c0dXrMKaD+M1Wt+54tl26mpqaO8vIp5837g4IGzzJnRly2b5jJ71kMYqhQEudZSVV7Mog+2EhH5HNOeW0pI2NOs33yaqMHT/+dYnI0tAOGmmwsvJaTz6ivfMUz0oBw1GnRUiZqGn7tgTC0afuIq1Wi5QBEF1KBGC/oy4s8sI0dbRblYh0wQUPxRrSWRUuyszRncP4qk+OXs+fUdxBATXjc8yfeGiSwwPkuiWx1rVs7+2/hHDO/IUVU+1aKGbaTwNMEECZYYoYcaHRoafw2sRaRaq8bI8Hpt6HETPsBSrYcA1NC4XbooilTo6jA2lmpJGxkZ8Nu2BXSO8ufD99fx3Te/MOWx3qz5ZqbU8VEiuYukNdB3ibQGWtIcNi95ihCXwTjaBjcaPxy3ArvwUEI7j2qmyO4NhdlXOfPrMlITziIgYKDS46eNb9C7d0TDMatX7+Wbb37BwECfYycSkAkyNFqBwE6jMTazpaaqFAf3EGxc/P7neG7UdEWn07Hoo594b/FGDOoEJhPEGfK5SBGemDEeX2SCwD4xnZ9JwxVj2mNPBWp+IwNPTKnxMeD08c94cfY37F5zhLFVHjhjzGWKWa26yqIPJ/Lowz0axXLyTCLn49PwdLenS4egmyZnoigy8alP2LvjJKXV1XwqdG742QfiGYKwpL/g1jC2kSTSPLXs3/UutjbmXE3JJjRqGjYafRwxwhQ9xuDd8JyxYjYHXUq4FPellCT+g+ycIpZ9u4vTcVdxcrRi6hP9pBrcklsirYG+OWkJh0TSgmnUdSgVTdeMKuT6aNV1NzjjwWLl4EWvJz5Ao64lN/UiyYe/bJQ8A4wZ051Jkz5gyZJncXCw4Lvv5rB16yHGjV9IWLg/wUFu/LJhA2b2fnQZ8wZyhfKW47lR05Xtb6/n19VHcKgzQA85CRTTFjtiySaTCl7mCF6iGVcppQo17phwhgIMkDMOH/aosnhqUv3muw/em4idnTmffL6d3OJSfFzs+fCNKTw8qmuTWCJa+RDRyqfJuCiKXEpIp6S0krBgD4yMDBAEgQXzHkWuJ2fVmt+oENUYC/Wvw+P4s5gznBbz8cKU47J8NDod7rkW+IRMpn1bf46fvIKXxoRMKmmFDTtIZSGnCBAtSaWMFGUFh9YslpLnf5CQmEGPIXMZNrwTU58dysWLaQwcM58P357AI6Ob/htLJJJbJyXQEkkL5hbcgcSrBxo1UKlTV5OWfYKwEY80Y2T3FoVSHz2VMeVllfXref+SqFVW1iCXy0lOzsLe3hJBEBg+vDNjx3Rn8+aDjBrRnqWfrWTgoDc4d2AdrXo82nCuKIrotGpkcuW/Tv7+mkRvLCvgh5V7eKc2kl9Ioxo1e0jHASP64souruGPOVWoqRI09OoSxoHjlwkRrDBSK/hRmUzX7qFMndAfqK82MmfmaObMHI1Wq0Uu/2/dJxOvZjHmkXfJzijETG5ArqaSeXPHYWKsYsbsbwgRLDERlawigcliIApBhhUGtFLYkuctI76glK7FjgzGHVmlwCkxj5UH4hmIG6fIZzKBfMEFWmGFCBwjhxKhjvijy/DydPhPsT6IXn5rJTNfeoiZMx8CYMiQjgwc2JaePWcyfFC7B34DpkRyO0kJtETSgoV1Hc2Ws89y6MxXeDi2o6augvjkXXiFd6M4N43YLUupqSjB3jOEsO5jMLV8cJMUaycfajRyNmyIZsyY7kB9Arxw4Rq6dw/n669/Jjr644bjO3YMprS0kjVr9iEIAovem0Dvvq9ibO6As28EWVfPcD56JXmZ1zA0Niag3RDCez6B/F9UPflzc2HclngsMcVU0KOH6MRbnKAvLmwnpWF9cxyF+GJGWJgnO7fOp7CojJ+2Haa0rJK3u4TecBYZ+M/Js1qtoc/AuXTKt2SaLhKZIJArVvHu2+uo0qqZpg5iBfG4Y0IB1cwkFmfRmDy9WgJD3Pjw9Ud49JHFDNa6I/vjw8RBshmLN1HY8SvXqELD20RxmBwKqUGl1OOp54dIyfO/oNPp2LnnFGvWNy5jFxrqha+PM4ePX6Zn17Bmik4iaXmkTYQSSQtmYGTGiBnLsAkO4lLOb2RWXyBq6GSU+oYc374cD/MoIn3GIear2fzx05TkZzR3yM1GEAQ6j57L1Kc/p3//l5k//weiop7h2293ERt7nvBwb1as+KXh+IMHz9O+fSCrV7/K4sXrsbU1p6aqgpr0n9mwaCzHdnzEqm+fRa3ew6kTn2GsSeDo1g//U0weDp4UaitQizrsBEMG484+MiiiBhBxxZTXiaBO0PH44/UNNKwsTZn6ZD9mvzDyb5PnW7Fz70mMKwV6is4NCbCdYIh7jSGRGmtiySYCW54VQnldiGQOrQnHmmo07Nj8JmUV1TgpjBvOBcihCi/MyKUKNTpWEM8yLpBJJcfIxbu1K6/9UfVE8s/kchm1teom47W1apSK//aBSSKR3JyUQP+D3bt34+/vj4+PD4sWLbrhMc8//zw+Pj6EhYVx5syZuxyhRHJz+ioTWvccx9DnPqH/pIXYuvpz8fA2+rR7GQ+ndthYetEm8CH8XXty8tfvmzvcZmXj7Ev38fM5eiyB8vIqevQIZ/nyl8jI2MDatXNZtWov1dU1fPHFNvbsOcmTT/YjMNAdjUbL99/vJiDADU93G+LjV6CUC5w8mYAgCPj4OLNt65skn4tmx9JJ7F/16r9qAW5q6YCNsy/rSEQj6uiNMz6YYYIeI/GiH65sIIlrYjnjRne5o69N6rU8HNVNy+sZIMdM1OMMBfTEuWHcUTCit+CCv74l+2LOEhbsTmJtEbXi9Qob9hiSRCmrSGAEXnxMZ3rhghem9MWVvILSRnW3JX9PJpMxckgHPvhgQ6Px6OizZGcX0T7Kv5kik0haJukv001otVqmTZvGvn37cHJyIjIykiFDhhAQENBwzM6dO0lKSiIxMZFjx47x9NNPc/To0WaMWiK5ucykMzjaBmOgb9Jo3NOlI78cequZorp36DRqvLxdWbz4qUbjBgZ6VFbWYGs7kshIP/btW4yVlRmpqTnU1qr59NMtfP31i7zyynLc3Ox57bVxLFiwCj8/F4YN64SxsYo2bXzo3z8KR0crXn11Cb//+Bbm1vb4RA0joN0Q8jMS0Gk02Lr6o1DWr1dVC3BYlscB8hFFLQqFPjIR9msyMUcPCwyQ6ekT3P5FbK3NeG5qX8Y91O22b7gLDXLnE0UpYm3jNeJ6yDkqzwUt3Gi/vkj97L6Hmz2D+kfx9a+XGVPtgTUqfDFjLVcQgZdwQCHIiPijBrROFInJOkZyag5eHtISjn9j0bzH6Tb4NS6cT6Fvv0guXkxlw4ZofvxmpvRBRCK5zaQZ6Js4fvw43t7euLu7o1QqGTt2LNu2bWt0zPbt23n88ccBaNu2LSUlJeTm5jZHuBLJv6LUV1GnrmwyXltXiVJPqrNr7eTL5UspZGQ0rpO9ffthzC0t6Nkzkm3bFhAY6E5+fgnjxr2Dl5cjBw7Ur482Na2fpfX0dMDPz4U33/weqP9AfulSGkeOxBMV5c/Jk8sQBGgdak/J1Z2sXziC87sXkxz7BesWjmbPyrnEbl5M3rV4DI3t0dMzwtOlE/bWgWgRyRZquSqr5qisEH+vAYR6P4uxsg+zX9/E7Dd+uO2vS7dOIdh4WPGjXhLlYh06UeSsWMBJg0Lcg50wlCvZQ+MOjhliBVc1pfTuFg7Aiq9eoO+kTiwyOsdUWQwX3Wp45LGeiICMxgm/ACgEGRpN45rQkr/n6GDFmZiPGdy7DedPJ+JoacqZmI/p3b1Vc4cmkbQ4UgJ9E5mZmbi4uDQ8dnZ2JjMz8x+PJ1Rz8AAAIABJREFUych4cNeRSu59rv5tKSq7Rm7B5YYxnajjXOI2fCP6NGNk9wZ9lTGh3R6me4/Z/PzzEdLSclixYicTJn6EytSB6JizuLqOxc5uBJ6ejxAV5c+JE8uwtDThlVe+ISEhg507j7J1aywDBrTj0qVrVFZW8+ab3+PiYktkpB+dO0+noKCUAQPacunSNSZP7IeoU3Mk9iPOnf2CAzGLKc6Io2OogCDIsLH0xdW+NXmFCQgyOTKZHAebQESlHuGBownxHYpCrk9FVQH6SnuWfr2L95b8RGVlzW17XQRBYPeOt3Ec5MkreseZpjjIfs9i1q15meg97/HMrKEcVuTxIWc5KGaxWZ7CEtV5vvjkGYyN61ud6+kpWTj/cQrS11KRtZGLZ7/kq4+fxdfLkZPkNXq+8xRibG7U0KnwTwmJGTz0yLtYOY/F1edxnnvpS1at28++6LNotVKybWRkwKTH+rDsw6d57aWHcHaybu6QJJIWSfpO5yb+7Veg/7/QuFSrVHIvk8nkdHtoFvvXvoeTXQhGBtZk5J3F2MqW1r3GN3d494SwbuNIMrPnuZlrKSvKxdTKkTq1htkvdGfAgCguXbrGizO/ory8mpiYcwwb9jqHDp1n+vSRtGsXyMSJi1Eo5EybNowFC1bj4/Movr4u7NjxDg4OVqhU+sybt5LS0krGjOnGpk0HiIjw46efYujVqw3h4d7MmTOWL7/cjr2tJ/lFV3C0DcXMxJHikjQCPPtxOWUPWk0trg5tOHDyczJzzyEIMgI8++BsF8aXy4/y/ap9xG56HkvzW2s1/v+ZA99/MIqv3hlKbZ0GU5P6xJjKLF6f0oEZ49qweutxDh1KwtfRgfcefoQAb3vE4rQm19IDxD8aLi57fzSDHl3GNXUFXmpTUhUVHFRms/H9SVByrWFpSEp6AZ0HLqZblQOv6sLZXHmV5St+5cB3x6gx1FFjILJt5VTCApybPJ/kHqHUQzCWluRI7n9SJ8KbOHr0KPPmzWP37t0AvPvuu/V1VOfMaTjmqaeeolu3bowdOxYAf39/YmJisLOza3QtQRCI6PNEw2NHr3CcvKWv1SR3j1aj5sSub4k/ugOdVoO+ygQH7zDMbZyxdw/Byae19OHvb/y6fDpvvdqP8eN7N4zl5BTh6TmONWteRRRFOnYMxs7OElEUMTcfwq5d77JgwWouX76GoaEBhw59irm5MQDZ2YUEBj6JQiFn69a3eeaZj8nJKQLqKyk4OdkwadIAPvhgA5kZ5Rgb2qGQ62Nr6UNyxmGMVFZ4u3bhzKWNWFt6o9OqyS28zKBub6OvZ0JVdREG+ubExa+hq0k2b90HG8jSyqv4Pj6VywUVeFsa8WSgG55mxo2OefnQeQoSYQRe/C5mEks2LxKG4R9NW46KOWzXT+bEwz1QyqQvWO815j1sULRvhaCnLyXR96DoQ+eJPnS+4fH8ReukToQ3Ic1A30RERASJiYmkpqbi6OjI+vXrWbt2baNjhgwZwtKlSxk7dixHjx7F3Ny8SfL8p8i+T96NsCWSGzr40xLKMjIY2HkexoY25BddJTbua5x8WuPs26a5w7tniaJIakIco0e/22jc3t6SsDAv5HI5Q4Z0aBivqKimrk7NkCGv07dvBPHx3/HUU0uYMuVDhg3rRPfu4eTmFqPValm7di7btx/G1NSI/PxSvLwcSE7OpnfvNrzyynIefbQXhw5dJi9Tj6KyDAwNLJDLFBQUX6Vb1AuUlKVzKWUvrQNGI5MpSEjZz5XU/chkCnQ6DTKZgvQsLQ+5dcdcX3W3X7qbyqgs5WxBFqZ6+rS3c0MpkzPJLxj+7IguQmbJH/8pipwuyGR3aiGTRH8QIIZMxuDTkDwDtBPsOaDN4acEGV0cPO/+Td1hOlHkSG4axwozMJQr6evkg4eJZcPPi2qq2JeZSIWmjrY2rgRZ3vi9qLlkbgan/Xuwfq0PFKeBUq+5Q5L8Rddwa7qGd294PH/RumaM5t4nJdA3oVAoWLp0KX379kWr1TJx4kQCAgL46quvAJg6dSoDBgxg586deHt7Y2RkxHfffdfMUUskTVWWFXL1XAwjen6InrI+kbKx9KJD2CQO7/kW/8j+0uzz3xAEAUNjY7KyCvH4SzUIURTJyS1m795TDQm0KIq89dYPtGnjy/ffz8H7j/W7L7wwkt69Z6HTiTz77CfY21vw0EPdOHjwHJ9/vhUnJxuOHPkMNzd7duw4zJQpHxEe7oWTkzXbt8+jR/c55BZWkpp5HD2lCj2lEeWVuXi7dSUxLQalQkVpeRY1taUYqaywMncnwKsfgiAjPmkXTxzYwo/dR2GoaP6ERSeKvHf6d/amXyFIZkkxdSwQfuOjjoMJtrRvcny5upYXDm6jsKISUaOlgGo8MaWUOuxo+qHAVjSgoKbqbtzKXaXWaZl1ejelhloeebI3+bklTP12G1O8IhjlHsK+zEQWXohh8KD22DrYMHf9b4QY2vJmaA/kwr0zG59Z4gnv7MG8h01zhyKR/E+kBPof9O/fn/79+zcamzp1aqPHS5cuvZshSST/WXFOKpbmrg3J859sLLypKM1Fq6lDrtAjM/EUV+NiQBTxCO2Mi1+UlFgDfhH9eGnWctavexXFHw0pfli1l8pqgbUbj3LydArt2/myd+9p8nILiYv7Bnv76zODMpkMS0tTfvppHrm5RURFPcP6DbHI5AIzZoxm/vwnGl7nwYM7EBKyBUtLE0pLq3BysmH9hlcZNOhVNBod+XmlCIKME+dX073tDGRyBSqVBeWVuXi5jiIr7zwdWk1uuF7H1pM5eGwJ21MvMda7+TvRbUo+z7mMDBboojivK+IsBYA+zx7cyu6BE1EplI2O//DMAczL5Dyni+A0+WwjhQDRAm/MOEsBPf5Se1ot6rhAEc/cIBG/3/2Uch59D1NO7Xu/4Xdw2vPDaR0ymWAzOxaejybm8KeEhXkB8M67k+jV9UW2plxkpGdIc4beRGaJJ5mbmzsKieR/IyXQEskDwMTSjtKyLLQ6DXLZ9f/tSyuy0DcwRiZXErN+MVlXzuLt3AVBEIjd+Cm2HgH0eORVhAd8PWmr3hOJXvM6Hl6P07dvBBcvXiPhSha9n3gPcxtXUi/GciQhC9vgh0hJ+QS1WtNwriiKLF26hWHDOgJgZ2fJwoWTmL94P4XZKTzxRJ8mH1Lc3e05ePAcgwfXz2wvXbqV4uIKugyMZOWc/ix6chdrki+wee8MTFRWHDm9HKVCRXFZOkqFiq37ZlNdW4JMpkBAwNTYnk2pGRTWVJJfW4WhQp8Odi50sHNr1BnwbticdJ7BWle+Jp4atHTFETU6irXXmH5oO192HdHweqh1WvZlJfKurh0yQcBGNMAWFS9zBCeM+YmrKEQZbbChiFo2y5NpZeOEj1nLqzzxW2EKixY/15A8Q/3vyYiRnfl27wkGDmzXkDxDfd3y1+Y9ytwpy+65BFoiaQmkBFoieQCYWTtj4+LLqfi1tAkYg1yuR01tOccu/EBw55FkJp4i68pZBnSah1JR38DD170Hu2Lnk3bpCO5BHZv5DpqXUs+AXk+8T+61eBLSEzDziWT00PbI/5gt9Qrr1nCsurqUNpHTmP78MFxdbViz5jcyMvKJiVnScIyHhz01lSXYu4ewefMhZs263q66traO7dsPo9Xq0NNTMGr0AqIPXqLd5CXMeskYs5j9jPNpxTifVmRWlnKhKIeLRbmsS71IUUkqSoUKlYEZZqaOhPkNp66uggOnvqBKYcCq5It4uXTGUN+c3y8cxv3qeT5p1x89+d17KyhR15BOJXXomEOrhuUF7UV73iw5wcmCDCJt6kuDqnVatKKIAoGPxTgyqMAXc0zRI50Kerr4EF9ZwfqSJEwV+gz1DGKCf+Rdu5e7SaPToVLpNxk3MlZRrdVgbWfe5GfW1mZUaeruRngSyQNHSqAlkgdEz0dfZ//qhWza9yImJvUz0v6R/WndaxwHNn6Ej0vXhuQZQCHXw9elO1fP/P7AJ9BQvxba3i0Ie7egmx4X3PkhbFyD2bB7N+WFp6ksukbile8wMbleSm7z5sNYOQfhEzGIBQtnoFTKGTOmOxkZ+cycuQyNVo6lUxCvv7MHS5dwIp+cgFJlRIR5GSX784H6LpJORmY4GZnR18WPap2OHemJdI18nmNx3+FoG8Kpi+spLksjxGcI565sY1C3tzExqu/0F+jVnwPHPmBj8gXG+YTfsdft/wuzcuB4di59cWm0NldfkNNF58DvGVcbEmhDhR4eRuZ8XRGPEQoW0R65IEMURbaSQnp5CSt6PHTXYm9O7S2c+fqLHXTqdH02ubS0gvXrfme6Z3uWbojh3UWTMTS83gxp9co9RFo43ehyEonkfyQl0BLJA8LA0JQBU96jvCiHitJ8LGxdMTAyA0DU6ZAJ8ibnCH8kK5L/xs4tEDu3QESdjt0rZvDEkx8x/63xWFiY8P3KPSz/dg+Dn1mGiaU9/SZ+xNerV/LGvDWoDI1xC+3FyNmPIZcryKosQydq8O5Ww7zWZVS/sLF+E9YNhFnacbpGQW7hZdSaasoqcnCyDaGqpggEAQebIC4l76GoJBVDlSV+Hj3x9RrIL5fX3NUEelJgWyZmb0B7g8bfWkFE+f+WlDwd0oGXjvzMYjo2JNyCIDBYdGdO+VHSK0pwMW46+9rSPOwRxpR9Wxk17A0em9CXwsIyFr+zlp42nvRx8eFI8TV6dJrB628/jrW1GWtW7mXz2mhWtB/Z3KFLJC2SlEBLJA8YE0t7NOoaDm3+lOzkc+irTLDzCORKUgw+bt2Qy+uXJWh1GpIyDtB64GPNHPH9S5DJ6PXYe5zd/wNdu79KbU01bgFRDJj6KSZ/bHSzcvSi2yPzm5z71+R5sXM+cY8nAX9fms3KwJC66isUajW4OETQPvxJMnLOYqSyoqIqn6y88/h79CI8YCQlZZkcPPkFni6dqNPd3e593mZWPBkUyZb4c0SJdij/SIqrRA2x8hzec2lcUjHMygGFIMdUbLy5UCHIsBQMKK6tfiASaFM9A5a3G87WlHgWT/8eQ5mSSTZhdP2jXN/ckO7sSLvEvKe/olJTR5S5Eyvaj8RGdXua6EgkksakBFoiecAU5aSy7fMXCHTvQ1DkLKpqijh7ZQuiTMfuwwvwde0OCCSmR2Ns54BHSOfmDvm+ptRXEdl/KpH9p/7zwX/IqCgGRKZPKKdjzO/EfWLyj+dE2rigqYuhoqac1oGjAbC29KLwdDJqTTXh/iMxNbZDFEV83bthbxPAzph5jPMKvtVbu2WP+rYmvjCXhQWnaa+xQ4OOg/Jserj6EPr/KmiYKPWxNTDicnUJAVg0jBeKNeSKVXiZWt3t8JuNsVKf8d6tGE/TJlxyQcYw9yCGud98iZFEIrk9pARaInnAnNrzA4HufQn2GQiAqbEdPc092bp/FkF9HiUv5SKiKNKq/3g8w7oikzVd2iG5c7Iqy/hr8nxx8z8nzwAKmYxP2w/g8QNb0WjrN44Z6Jng79mH8wnbKKvIwdLMFY2mlsqaYjq1noKJkS0hzVDyTS7IeK/9AA7nphGdkYRCJmeBa39aWTk2qUgiCALPhnTg/VO/M0brjT8WXKOcDfKrPObbGiOpGYdEImkGUgItkTxgspPjCGn7SqMxpUIfJ7swlEo9ej/+JgClBZnEH9mOQmmAR3An9A3/XSIn+d95d6uha10Jcf8yef6Tj5k1T3iHsvvKdqzbzkAmyHCwDuRy8l76d34DM5P6RjA5+fHEnPwcY6UKK33DO3EL/0gmCHSyd6eTvfs/HtvL2QcDhZIVF4/zQ8UV7A2MecIviiFugXc+UIlEIrkBKYGWSB4w+ipjqmqKMDFq3AmsqqYIfUNTRFHkyI5lJBzbhYtDa+rUVcRu/YyuD83C+y9tXiX3psd8W3Ps8M/sO/AGjo7tuZoWg5mJEzEnPsNA3xQft264O7XF2S6crOzjZFeXIy+SEWhhe083zblRsl2tUfPd5ZPsvpaAWqels707k4LaYqsybp4gJRLJA0NKoCWSB4xfu4HEHd1Cj6gXUcjrv/7OzDtHUVk6boHtSTl/kLS4wwztvgh9vfoNSMWl19izcRFWDl6UFWUhlytw8AxrqIMsuT3+XPs8yFMNt1i+10Cu4OtOQ4jNSWVPRhznaopxc26Hq0MbyivziLu8haLSNIrL0tEJcr5Nz6Wo7CS2enp83K7/fbPpTKPT8eyBLeiXiUzS+aGPnIPpWTyRs541vR/Goplm1iUSyYNBSqAlkgdMaKcRFKZfYcv+WTjbhVFZU0RR6TX6Pfk2CqU+l4/+QrDXwIbkGcDCzBUXu9Zs/HAitja+aDS1VFQX0Gv8XJx9I5rxblqOJhsH/+Pyjb+SCzK6OHiyJe0KYf7DCfIewLWsk1xI/JmyimwqqvKxtw6gf+fXkcuViKKOc5c38fLJfazoPPT23dQddCgnhYqKGp7TtWropjgab6o0WjYknWNqULtmjlAikbRkUgItkTxgZHIFPcfPpSgnhazkOAwMTXEP6ohCWd9EpbaqHEOrpmXBDPXN8XLpSLuwJwHIKbjEnpXzGDNnJUYPUCWEOyGjogiA1S/W13q++De1nv+rU/npDAh5jpTMo5y+uJ62YU9gZ+nDT3um0y7siYaShYIgI8RvONv2RN83dZVP5WXSSmPVpBV5G5010bnpUgItkUjuKNk/HyKRSFoiS3sPgjsMwzu8R0PyDODo04rU7BONjtXptKRmHcfV4fpss711AK4OESSc/PWuxdzSZFWWkVFRhHe3GraMT79po5RbYaDQo7q2jLjLW+jYeip2lr7EX/0VQZARe+YbEtNi0P1RB1oQ5BgZmJNVWX7bnv9OMtc3oFhW22S8gBrM9AxucMaDoVxdy96MRPZkXKGsrqa5w5FIWixpBloikTQS0mUUm05N4eTFtXi5dEatriIuYStKhQo9pRGJaTEYqaxwsAnE1NCeqpKC5g75vtSkUcqsmzdKuZHE0gKWxp/gdEE6hgp9hrn5M9GvDXry+j/tg139+P3SBqqqi7A0c+PX2HdRGZjj7tSO6ppSLiT+QmbuWdyd2nE6fiPVtSW8eGwnvZx8mB3a6Z4sEXeuMJsfLp8iqaSAfF0lXXDERTAmRSxjA0lcoQS9fDnzT+zjhbBOD1QyvTP9Mh/EH6J92wDkMhnv7Y9hmn8HRki1oSWS204QpT69d4UgCDz9YUxzhyGR/CuVpQWc3reaa5eOoVDqo2doQkVeLqKowd46gOKyDLTaOvQMjAnt8xC+bXo3d8j3nazKMjy7VjKvdR3VL2z4zzPPyWWFTDiwFX+/4Xg4taO6poTzlzfiSCWftu+PIAjUaDU8d/gX4gqzCPIZSE5BPKXl2TjZhmJm4khW3jkKi1NRKPXpEjENOys/auvKOXtxLSa1mXzdacgduvtbE511lbdP7GOw1h1vzNhPBofJwVMwJU0s52F8aIc9VWjYIUsly6iGH3qNbWgB3pIllxXy1IntHDj8CYGB7gAkJWXSse00PgrrT4CFbfMGKLnvtNn8KVKK+PekGWiJ5AFRlJPKxUNbKM3PxNLJk+BOwzG1dLjhsUZm1nQeOb3h8f6176FXq6RLxLSGxioXk3ZxIWkHnqFd7kr8ksaWJ5zB13sAAZ59ADDQN6VT5Av88tssLhTnEmJp31CRY/bxXzmadgCdqKVT66k42YUCEOwziC37XqJN0Fjsrf0brhPVajI/753BpeK8eybx0okiH509wBRtIP5CfUfCx/GnjWjLl+IF+gmudMYRADP0GKfzYWH1aWJzUunicPuWxdyrdmReZsrUQQ3JM4C3txPPTx/J9jWn7pl/R4mkpWj5H8slEglpl46ybenzyIsFfKw6ockqZ9NHU8m7dukfz9XptCSfiyEq5NFGXQkDvfqiUBhQWpBxJ0Nvkf5cvgEiRnklt3SNuKIcnOxbNxqTyRQ42IVzvii7YUwQBBZE9EIpqjHQM2lInv/8WZ26Ejsr/8bXEWTYWflytazwlmK7E3KqyqlWq/Gj8QbHYMESGQLBYuONrIIgEKCx4HJJ/t0Ms9mUaGvx8nVqMu7p7UCJtulacYlE8r+RZqAlkhZOp9NyYONHdGn9DPbWAQA427fCwsSFQ5s/Y8T0L25+vlaDRl1LcvphrqRFU11bgrWFF2G+QzEytCT+yA6MzKxx9o3A1sX/pteS3J61zwAW+oZUVOZjbtI4aaquysXC2gmNTsfh3DSuVRTjbmLJMFcfduQVNbmOsaE1RaVpONoGN4yJokhRaRpOnp3+c1x3ikqhpFbUokaHHtc/yGlEHSIiWVTiiWmjc3Ll1bR6QJqqhBjZsnldDBMnDmg0vnn9AUL+X9MkiUTyv5MSaImkhSvKSUGGvCF5/pO7UzuOnf+BmqoyDAxN/+ZsUCj1URmakZF7lq6R0zA1siMjN46Yk5+jVldhaGCNTr+G3Qe24OTbmu4Pv9xoplrSmE7UXK/1/Mmt13oe4xHAsssbsbH0Rl+vPknMyDlDUUkKgaGtGPXbOrQKEywsvChKj0OpraSsspSyihxMje0brmNu4sLxuO/o2eFlTIxs0Oo0XLyyDVM5hFvdeInP3ZBdVc6W5Auklxfja2HDUPcgQi3s+bUoncG4kyaWs5VkLlKEgMAWIRlf0RxbQYUoipwinyShlN5OPs12D3dTfxc/1h/+ickTFtM60heZTMa5s1c5cTCeaR1HNXd4EkmLIyXQEkkLJ5Mp0Gk1iKLYqFWzKGr/GLv5Sq7y4lw06lp6dp2JUqkCwMO5HQhwJn4jXVo/BUC43wj2HnufhBO7CWg78M7dUAsQYQ4l+/OBW0+gB7n6k1hWxJZ9M3G0DqC6toTqqnw+aT+AhXGHsHbqTIjfcOrUlRSVXiMt4wiuskv8FvsOAT5DMTdxJDsvjvy8Mwxy9WN7zFxMDK2oqCnB29SKz9sPuKOtvUVR5FheOtGZV5ELAr1dfQm3ql/DfCo/g5cO/0Jb0RZXnTHn867xY+JZ3mnbj3dP7edMXQG52kpG4c1kgiijjq/Fi7zJcTzlZlSiQSMX+aT90HuyksidoFIoGe0czKfr9nPxUhoA5+Ku8qxvO0z+UqZSIpHcHlICLZG0cBZ2bihVKtKyTuDuFNUwnpC6H3v3YPT/4Svu/PQE7GwDGpLnPznbtSL29DcNjxUKfYK9BnH5+K9SAv03sirLbtu1BEHgxZCOjPMK5UxhFiZKZ6JsXSitq+FicQ7DIwcSl7CVS1d3Y27iTGV1IXXqSuaEdOBw/nEys6oINrdiYfeHcDQy5dnAdlwtK8RCX4WTkdlti/NGdKLI3GO7uZibQwetHXWCjleu7aSXmw8zwrow/8Q+Jmj9CBWsQYCOOgd+12aw/OIxNvQdz7QDW2lTZE13oX75iiEK5ooRzJUdp19AIKGWDgRb2jdpstKSJZUW8k3qSY6d+IKgIA8ALlxIoXun6UTYOOMlNTuSSG4rKYGWSFo4QRDo/sgr7PzmZbIKzmFp4k5ucQIFpVcZ8uwn/3i+ysSC8orcJjPY5ZW5qPQbJ1r6esaoa6pu+z20BH9d+2yUV0LSbWqYYmdoQj9Dv4bHFeo6DJQqEtOiSUz9HV/3nrg6tMHS3J34pJ3/x959x1dZ3Q8c/zx3Z9/c7L13CCHsvaeiAq66tXW21tmfbW2rtuKss1qrVivW1gqCAgrK3nsmhAySELL3vrnJHc/z+yMYCGEpkIRw3q+Xf+TkGedJ5OZ7z/2e75ePj6xl6dRbugWXThotySb/Uy9/SawryyO3soo/OtLQSh3pPpMcQfz52F7iPH2x2xwMoGvAN5ZAvmzYQpvdTlVrC9fQNfdbJUmkYEJWIKUXU096y7LSLB785bWdwTNAcnIEDzx0Dcu+zuYx99G9ODtB6H9EFQ5BuAL4hSZw0/99gndyIi3ODYQMHcFNTy3A6BN8znP9w5ORtGpyjq7prAlqs7ezM30B0aFjuxxbULKNkIShl+QZLmclLfWduc8vFq86vnHw0gh28cBmt7Dv8EJ8TDE4ZCub9rzDlr3/ICpkHHaVnj3VvVs5ZVVhDhMcAZ3BM4CzpGWM7M/2ikI4y8KxgoKXwZlKLN2+V61qw9vgfAlm3PfVO9qIiev+7zkmPoR6e/eflSAIF0asQAvCFcLZzZNBE3/2o8+TJImZv5jPyn/+niMlG3Fz8aeqJhu9iwdltZl4VUSg17lSULqV8vpM5t75q0sw+8tXSUs9oHRuHMxc8tPzns9HdkMV7Q4Zd1d/JEki1H8wwb4D2bzvfZasfhxQeOvwTt50MwEKVRYzoW7GHs2TdSgKmtOs32gUFS5qHSq1ikx7HcknrUJvoZx4Dx/cdQZujB3I3/duJcbhgbvUkeO8X6mmkGYmBkb12HP0JYnO3iz9cgu33z6ty/jXCzeR6CyqcAjCxSY6EfYQ0YlQuNwpskx5YQaW5jp8guNx9fQla+e3HNm1Cmt7KyEJwxg44Uac3Uy9PdU+o6Slo2zcZ483YXlk0Y/uNvhjHaqr4P6ty4iNmE6g7wDqm4o5mP0VsmJnxMC7CQ0cit3eTnr2YopKtiErMh7OJhpba7g+YgAPJ43okbzhb45l8fmBvTzpGNh5P5vi4C/qffxuxGRUksRT21cwSvEjVHYlV9XIQVUt742fS4yHN4qi8I/MHXyed4BYlSdNWGlW2Xh11FU9lobS17TY2rlj65fcePdkfv3oPADeeuNLFn2yjk9HX4+r2Ego/EiiE+HZiQC6h4gAWrgStTbX0VhTirspABcPb6Cj+kJVURaFmVuRVGqiUydi8o84x5UuPyUtdZ21nmvmr7rkwTPAXZu+wj1kJpEho6moOUxx+T7Kqg/hY4plVOo9nTns63a83tFxcMDtaDR6LG0NbNn1JvMCA7g7bvAln6dNdvDrzUtpbmhlrMMfGzIb1eVE+/rywogZSJJEqbmRJQX/9AK6AAAgAElEQVSH2F1ZTLXFTKO9jRBnD+5KGMLM0I5643VtrRyoLcNVqyfNOwiN6srOSqy2mHk/bxfry/IBmBgYxf3Rw/BxcunlmQmXIxFAn50IoHuICKCFK4nDbmXTl29SkL4BD/dAGpvKCY0fzvgbn2T7sr9TlLmDiMCRyIqdgpJtJI+bx5Bpd/T2tC+qkpY6PnuiBf3X6y552gZAm8PO+OXvc9Os99lxcAE19XlEhY5DVhzkHdtIsF8qQwfcRrO5iu+2PM+8qa+jVms7z69vKmbL9pdYPfOuS1q+7gc22cH3xbmsKzqCRqVialgck4Oiu6yAL8w/yKeH9nCLI4Yo3Mmnif+qj3Bb0mBujk695HMUhCuZCKDPTuRAC4Jw0e1Y/j4tpeXMnfwaOq0zNnsbOzM+4buPn6a1robZ4+ej1RgASIycybebnyEscQQ+wbG9PPPLl1qSkCQVx8r2UNdYyFXj/4xG0/GxfXzEVL7d+CdCa7KQFRmjW1CX4BnA6BZMU3srNtmBTn3p/zRoVWquDkvg6rCE037fJjv4MHMXjzoGECx1lFpMwsRDjiReP7yLeZED0IqGPYIg9JIr+/MuQRAuOpu1jezdKxkx4C502o6KCFqNgWHJd1B5LJOooLGdwTOAk8GD6JBxHNm7premfNH9kPuMIh9vmHLpaVVqxgdEcThvBV7GCFZu/gv/XnY3X699isLS7USHjudY2W7cXf2pbTiKzd7W5fzq+jy8nd37TFBa0dqMRpE6g+cfBEmu6BT1Ra2pLQiC8GOJFWhBEC6qdnMTGrUeZ4Oxy7hO64xB74Es27udo9U40Wa7/OtHn1zr+dk0a49sHDzZ/6WMYfb3n2G21BHgm0xsxCTcXPzYe+hzXJy8qG8ooLJqP05qDdv2vMvQgXfj7GSirvEYu/d/wIOxaT2SvnE+3HUGWhQbFsWOk6Rhn1LNSo5RihnJIbGzqogwN8/enqYgCFcoEUALgnBRObl5IiPT2FyOh9uJhhatbQ20W5spr80kKfpEm2hZtlNQto2RIx/qrSlfNLLi6Nw4ePDOPKDngmeAdocNBwreHqEY3YIoq8ygpmEpowb9nA073+aBhMGM848kxMWDtzJ3sHz9b5EkNQa1ml/EDWZORFKPzvdsPHQGRvuGsbiqgAjZja85yq3EEosHR2nms0N7sToc3Bab1ttTFQThCiQ2EfYQsYlQuJLsX/tfcrd/z+jUe/F0D6GxuZztGR8TkJBKxdEMtA4DsSETcMg2sgpX4+xtYsY985FOU0WhuiSXw9uXY66vxjc8gcSR1+DcR1ceS1rqj9d7vvgbB9sddvKbanHXGQg+Q6vt+7Ysw+aZSmr8XAAamkrZk/k5DU3FyLKNd0fMINU7sPN4q8NOs82KUW9ALfW9jL4maxtPbv2GzPpKniCVKOnEc1cqrbyk3s+Kq3+OoQdytgXhSiM2EZ6deNURBOGiS530MyS1hrXrX8dutaBSaxkwbh5pU27DYbdxeMdysg+uQ6VWkzBhNnFDZ5w2eM7Zu4rtX71LfPgUfNyHUJqbwcItdzPsqnsJihqIh/e5Oyn2LOX4fxfXF/np/D1rFy4GT1rbmwhzNfLS0MkEOLt3HtNia+dQbRnzhj4NQH7RFvZmfk502ARCAwZTULKNZ/dv4JNxczDqnQDQqTV49eHg011nYP6IGdz4/WdEKV3fNPhJznhIOo411xNnFI1CBEHoWX33lVMQhMuWJEmkTriRlHHzsLaZ0emdUR0P1FQ6NQPH3cDAcTec9Ro2axtbl/yNaSOfwtM9BICwwKGk5yxlzzcfs0ux4xeeyOTb/oiul9s3n5z7PMSoHN846IasKCwsSOeLo1k0tJtJ8vTngfjBp232YZMdZNVXoVWpiTP6dJZzW1eaz4d5h5g85hk83AKQZQdZ+St4YOs3LJlyc+fKsU12oFKpUau1WG1mdh/6jJlj/4SHW8eKc0zYBHan/4sPsvfwm5QxfSbX+VxctXpsyLQoNlylE5VDbIqDerkdz+NvBgRBEHpS3/vMThCEfkOlUmNwdu8Mnk/lsNuoLcunpb6q2/fKC9Ixugd1Bs8/iI+cgqWtkTmT/oq6VcOmL1+/JHM/XycHz68GV5N3597OjYMvp2/ms6IiEgb8gmnjX0DxG8svt31Lem15l2usKs5l+soF/O7Adh7dvY5rVv2n85hP8zNISfxZZz65SqUmKWY2DrULOyqLOq9h1DkR4OJBcfk+Sisz8POK7wyeoeNNjdEjgq+Lchj61TtMXvEv/nF4JzbZcal/RBfESaNlUkAUX6rykRUFRVHYqJTye3ZgUxw8t2s1B2rLenuagiBcYcQKtCAIvSJz+3J2r/gnOp0rbZZGnFyNBMcPISp1IgERKahUKhSle3Anyw5UkoRarWVo0i0sWfM4ba1NGE5KZzgf9ZWFNNaWYXB2x9xUi5OLBwERKadNJTmTkpZ6QDme97yeg2+dyHuuaG1mRVEu10x9vbOcX2z4JFQqLe9kreaDMdcAkFVfxQvpWxg34jd4GSNQFIXiin38evs/+WrqLZSZG4kxhne7t6cxgtKTSrlJksRvB4zi8Z0f4e2d2C13sawqgwPZixk56F6C/QbS1FLJ94f+Ten+jfxl8KQf8ZPrWXZZZnZEIn9r3spTLdtxVjTYFJm7SSBAceFQbS2Pb1nOy6NmMdQn5NwXFARBuAhEAC0IQo8rzNzKvu8/Zcrw/8PTPRiHbCcjdzl5ezdwLH07vuHxTLzltzSbq6iuO4KPKabz3MP53xEaOBRJkjpK4xk8sDTXn3cA3dbaxOoFz1JXdhQ3Fz9q649i0Luh1RpwqGRm3PM8XgHnrp5xavB86qbBjLoKArxiO4PnH4QGDGZJ+icAyIrCgryDxETOwsvY0c5ckiRCAwZTVr6Lb4uyiXT3orImG9fQE3m+iqJQVZNFVMjoLtce7BPMR2Ov46OcvayrLKDZXI2bS8d5B3O+ZnjKnYQGdLTqNroHMXrYYyxd/Qhl5iEEuvy4NyCXUrO1nY3lBWTUVbC+JA8XRYMDBUmtotpu4SVG4i7pABhLIHqHmncPbuOTKTfhUGQy6ipod9gZYPLHWaPr5acRBKE/EgG0IAg97uC6LxgcfyOe7h2bANUqDQPjrqO0Yj8D4+dwqGAFR/atZeLPfsva/8wnMmgUHq4BlFQcoLGlnOmjfw9As7kSq82M22lyis9k/ecv4yKbmDD5l6hUGuwOK1v3fYBB7463MZIVHz7FLU9/jvosm+t+CJ4/e7wJyyOLyDxNrWej3oDZUoss28kqWE1+8RZstlZMHmE4qfX8+8gBFhzZT5PVgkZTDJJEUvRMpOM5ze7u4ZS2ZvOL2FSe3PU/nAyeBPgkYbW1kpG9CF+dhjTvoG73jfbw4sVh0/giP4N/bHmO6IhpuDj7UlOfT5BvSpdjtRo9AaYYshuq+kwAvb40j2f3rCEMNwocjTxCCrGSEUVRWMZRMqjrDJ5/kIYPHzQd5kBNGU/v/A69Q4UTGkqUFh4ZMJq5kQN66WkEQeivRAAtCEKPa6wtwysmosuYJEl4eUbQYqklKWImOXtWc+2v3uL6Jz4ke9dKjubsoamplDGD7sfJYKSmPp/tGZ8wcOLNaLT6s97Pbmvn4IaF5O7+HnNjDRHBI2mztuBsMKJR6xg24Da+XvtbBifdzJGSDZTk7CYsceRZr/noPS3ol24g7wyNUtK8g8C+ge+3vIBGo2dEyp0Y9B4cLdlGZU0m/z56hNEjnsLkEUpDUwnb9v8Th8PKwPg5ANTWHOKqID+G+ATzTOo43jz0MVvbzMiKg/EB0fx21FVn3Qh4U9QAkj19WXIsi9rGDNx0zjS2lON1UjqIoig0tJTj49Q3WqjXtrXy7J41PO5IYTuVROJGrNTRkEeSJAYo3myhAkVRujx7He24qLQ8vm05d9rjSJW8AShXzLyRsYMwN08G+/S1ii2CIFzORAAtCEKP8/QLo6o2F1fnU9ISao8QHjQCALu1o9W0uymAYTPuYej0uzm8Yzk7135KU305bkY/Uif9jKRR1571Xnabla//9jAqq4oRCXeh0zqRV7SJ7zb/mVnjnsWgd8fJYEStUmOzteLm4o+5qfaCn1Etqfh14nD+fHAr86a9iVrdUUFiYPwcZNlOs7kak0coAEb3YMYN/RXfbPgjMeGTyCtcg6WlhKnBYwCYGBTFhMBIGqwWDGotThrtGe97siSTH0kmPwA+ydnHkkP/ZuzwJ9BpnVEUmaz8lXioJZI9z38F/1L6viSXQXgTLrmzXClkFF3nFYYrDmT+RRajlQBiMCKjsFiVT4LJF3W9vTN4BgiQXJjpCOWLIwf7ZQAtKwoWuw1njfayqaoiCP2FCKAFQehxgybfwtp/P4+Lsze+plhs9jYO5ixBo9Hj5xXP1oMfEprUdQVYkiSSRl5D0shrOjYSqtTnvE9B+iY2LvwrGkmHJKnYtv9DRqfdx5DkW7DZ28g+uobU+LnUNhSiUmvRaJ0prz7E4NC7znjNH9I3zqfec3VbC5FBwzqD5x+EB49k4+6/dX7dbm1BURRUKg1fr36MZFMAYa6ezFnzOX7O7twWmcTU4Bg89T+9XN/tsamUWjazbPVj+HlG0miuxFOr4a0RM/tM8NVkbcPDocWOTCDOZFFPiuLNMo6ykVKsyKiRaMTKx2TRjoxKJTHAK4AwD0/MNY1wyqME4sz+1tLeeaBLxKHIfJK3j4WFGbTarBidnLgjYhDXhyX3md+lIPR3IoAWhD6uqbaMnB3fYmmowSc6hehBk9HqDL09rQsSHDuY0fN+zfZvPqCtpRGHw4qXMYLEqJlsPfABda3FTBj72zOefz7Bc01ZHhsX/pWJQx7BxxTdUd2ifC/rd77BNZNeJCxwGBm5S6moyWL7gY+JCR3P5r3v4h+RjFdg1GmvWdJSB3DW3OeTuesMtLfVoigKhaU7yTm6BrOlDhcnEyBhaW9i58FPKK8+jFajx25v4/qIJL4pziMxfh6jk5NpaC7l1cOfU2Ju4u64wed87jNRSyqeTh3PvXGDyW6owtuQTILRt08FXJHuJr7gAKsoRkZBh5pcGvFCzxB8qaedh0hGK6lRFIXvKWa3oYY3xsxmbVke/yo8xtX2rukdmVI9icdX4fuLv+fs5IirmS27/0ZcXCh79+Zy+83zkQsVbopIOfcFBEG4YKKVdw8RrbyFn6IgfROb//sCo2Rf/B169umaqHRRMfvR93ByNfb29C6YIsu0NFZTnLOLgv0bsVkthCaOIHnMdeidLqwV9sYv/oq+xcCAmNkd91IUGpqK2Ze1CF9THM4GIzvTP0Wt0WK1WnBy8SBhxFUMnnoH6tOkSJS01BM9wcKrwdXUzF/VWev5bBraLVz9/b/xNMVS11iITuuCq7MPHq4B5BVtxKD3IDRgMClxc9Bq9FTX5bF+5+tEh00kLfFEo5mW1hq+2/B7Vsy4E7dz5Htfrtocdm7+/jNS2jyZSRhaVKzkGN9TxBuM4Um28TSD8ZNOrMLLisIf1Lv467jZRHt4cevqz4m0uHCVHIYTarZSwVJNIQsm3URIP/j3Ah0VSq5d9yk5Bf/Gz8/UOZ6ens+0cU+wbOIdaH5EKUZBOBPRyvvsxAq0IPRRNmsbmz5/kd/YkgiX3EGCSTb4T1M+e5a/z9ifPdXbU7xgkkqFm6cfiSNmkzhi9kW9dmNNCbE+EwCobShk6773ccg2VJKGqtpcNFonJt/2NJEp4877mrMjHWDlvILn74pzee3QNrRaZ6rqcnE2GElLuIE2azPpOV8TGTKGoyXbSUu8qXPF1McUzZABt5F/bHOXa7k6e+PlHkRWfRXDfPtnreM1JUcw2XXMk06s/vsoTiTgiQYVZmz40LXroEqS8JGcqGtvRavy5f0Jc3nz4BaeKt2OXZEZbArivdS5/SZ4BihqaSA81K9L8AyQkhKFXZFptLbh1cudOQXhSiACaEHoo0qP7CVE5d4RPJ9khiOIP6Vv6BcB9E9haWkga+cKakuO4OYdQOKI2bh7BXR+v6ooi+1L36P8WAblhRmE+A+msuYwQwfcRnjQCCRJorahkDU7XsXZ3XSWO3WnnEfeM8De6hJeztjGmGGP4e0ZhSzbyS5YzZ7M/3Ht5Jfw9oxkzfZXsdo6qmqopRMvxb6mWPZmftH1voqCua2x364+A2TXVxFnN3bJYfZETyUWJMAfZ15iLw2KFS8MTCaYeDwpkBtJMPp2HK935rlh03hGmYqC0tnmvD/xdXLlWEkVZrMFF5cTbyhKSqqxO2TctKLutSD0hP736iII/YQiy6iV7vmpGlTIitwLM/rpHA47beZG5AtsG11fVcTCV++h5nAWfpoY2ovrWPzGfRTn7AagobqYbz/4P8I9h3HL1f/khulv42zoWH0MO958BcDLGE5KzLVkbFrS7R6lefvZ8L9XWP3pn8nduwqH3XY891lhiFGmZv6qc85zQV46SfHX4+3ZsZqqUmlIjJ6Jq7MXxeX7MHmEYdC5I0lqpFN2vdXU56OW1Cgn/Y7zijbgppaIN/rQXwW4uFOqNncZC8GVZmwsIIdmbIwkgCdJZQrBLCKfv0h7mBOehOmUFVeVJPXL4BnAx8mF4b6hPPzg21gs7QA0NZm57+5XuTY0Ed1Z6pcLgnDxiH9pgtBHBUYPYqPcQKXS2iXvc51URkTCqF6c2fmTHXZ2f/cvMrctQ5EdaPVODJp8C8lj5v6kzWtbF79NYvh0EqNmHB8ZTZBPChv+9zK3/vEL0jcsIjZ0EtGhYwFQ61wZknwL1fV5FJXvIzxoWOe1TB6hFB3b1+X6O755n7w9a4kLnYRW60/G6iUc2PwVt3z0FM+PBMsji7qlbyiKwrbKY3xbko9NlpnoH8KxlkZS47qneXgZI2k2V6AoMm3WJly0ejJylpAcey1qtY7ahkIOZv4XV0nmy+8eBkmNWpJQK1Y+GjunT234u9hmhcbzUdYu9ivVxGHkc46wl2rUSOymkl+QyCCp4w2EL86EKm48y27uTRreyzPveb9PGs/zWzcQ4n8DsVHBHM4tYkpQNA8lju3tqQnCFUME0ILQR+mdXBl+7S95Yel7TLMH4K8Y2K9t5JDOzDWz7+/t6Z2XrV+/Q23+EWaNeQY3Fx/qGovYsuF9AAaMnXfO85tqyzm4/n9UHD2E3sWD8qMHGTPzvi7HBPgkoVHpqSnJpaYkl9TwuV2+L0kSwX4DqW881iWArqjNwis4uvPr2vJ8sneu5Jrx89HrXAGIDh3L+r1vELFpGXn/CAK6B8/PH9jElppKIiOmoVHp+EfBeix2K5XVhzvrPP+gqi6XpOhZHDm2EbUi8+m4ebyUsZUl36/BSeeC4mhnRmAkK0rzSYmfh69XLDX1BWRmLyK7oZqIH5lycjnx1Dvx5uhreHrHdzS3t5GEiVcYhRYVv2YzA/HucrwXBjwlA3/atQp/JzemhcWS6hXYS7PvWa5aPS+lTqe8tYkycxNh40fjbXDp7WkJwhVFBNCC0IcljJyNV1A02VuWcqChGu/o4cwddc1lUYGjzdxI7t5VzJn0amdAavIIZUzqfaxf+xZJo687azm6+qoilv7tYaKCxzI05lZaWqtp0BeSkbusS4WKDh0rs66efjQ0leLnFdfluzUNBdhlO0dLdqBSqWm3msk5tpbrrn2385ijGVuICBzROVcASVIRFzKFj/+xiPHjurfNTq+rYGNlCTMmvIBW25GPGhk6hvVbXyA9ZzEe7sEE+CThcFjJyF1Gi7mavIJVNDcX86+x1xLm7sl7o6+mps1Ms62dIGd35q79HyMG/4oAn0QAPN1D8HQP5vXdbzA1OKZfV1hI8Qrg9TGzeWD9Yu6RE9BIKuyKjAqJFmy405Hf61Bk/kEmdtlBQKUam9TC74pXMC08jscGXjmrsAHO7gQ4940W7IJwpREBtCD0cb6hCfjektDb0/jRGqqLcXcL6BKQApg8wrC1W7BaWjC4eJzx/N0rPyYhfBrJMVcD4GOKwt8nka/WPEl8xBScnTwBqKg+jF1uwzs4luSxc1mz4Dn8vOIwugehKApF5XuoqM1GdthpaalEp+1oaZ068WY8fUPPeP8fKHTrzdFpbVkBYaHjOoNnAJWkIipiGprcL9i+9++02dtRqdQYtXrmBEcQZ/RhctCELt0EvQ0ueBtcqGhtptlmw9+76+/b2zMKSaWjqKWeSHevc875clbU0kC0yohG6XijoJFUDFf8WEw+dyrxqCSJXVRRQxvPMxzN8VznCY4gnivcw9SQGJJNfaOzoiAI/ZcIoM+irq6Om266iWPHjhEeHs7ChQsxGruv/IWHh+Pu7o5arUar1bJr165emK0g9C2unn40N1dgt7ej0ZyoHtFsrkZSqdGe4yPnkty9DBr3ly5jTnp3fE2xbN73HjFhE6hvKiK/eAtT7vgTKpWawKiBDL3q56xa/iKuLn7YbK20280oDgc+nlHode5U1GQSFTKWzC1LiRw4Hu/AjjSOiAFjWb7pMZJjrsag66hBrSgy+QXfc0NgxGnnqDp+zKlaWqsoa21iUMINhAUOw2ypIyPrCyrb23gi7Mxvhpw0WuwOKw6HtcvPzCHbabNZcNb0/woLoa5GCpUmZEVBdTzn+0aieYV9PMV2UiVv9ivVXE9UZ/AM4CJpGeXwY03xERFAC4JwyfXfzwIvgpdeeompU6eSm5vL5MmTeemll057nCRJbNiwgf3794vgWRCOc/XwIShmMLsyP8Nu76gW0G5tYUfGv0gadQ3qc1QL0OoMtNvM3cZl7LiHhlJuzUEb5MHcx/5BSNzQzu8nDL+K25/5ktE3/opxNz+Ow9rO5BFPMH3M00wY9jDXTn6Fypos/L3iydr+Ted5XgGRJIy8mm83P0dm3gpyC9ezbuvzGOUm5kYknXaOU4KiKCzahPWkecqyndyC1QyMm0t85DScDEa8PSMZN/wJdteUkddYe8Zn9tAZGOQdTOaR5V3Gc/K/I8rdC3/nC2suczmI8fAm3MPE56ojtCsdVVuaseJQwdUxiaQNiCLAzR31af58qelI+RAEQbjUxAr0WSxbtoyNGzu6B955551MmDDhjEG06NYjCN1N/NlTrP/fKyxe8xhubv40NZcTO3g6Q2fcfc5zY4dMIz33a8amPYTq+EpjRU0WDS1lzL7xTTRnqYms0eoJjEolc9tSAnyS8D+eTwwdq9gpcddxOH8lqsau1wgafz2G8ASM5rU4HSthdmQUk4Oi0Z4hVzvZ5M/MoAhWbfwjEWGTUat1FBVtxG5vIzRwSJdj1Wodgb4pHKwtI9rjzGkYzwwaz31blrG25hAmUxwN9XnY22p4f8w15/yZ9RevjLqKP+9ew5PV2zCq9LQoNu6KH8LtsWlIkoRepear9IMMdvh0rlK3Kw52qCt5Lii1l2d/4YpbGthRVYRBrWV8QATuOkNvT0kQhFOIAPosKisr8fPzA8DPz4/KysrTHidJElOmTEGtVnP//fdz77339uQ0BaHP0hlcmH7Xc7Q0VtNSX4nRJ+Ssec8nS5t6OysLf8s3m/5IsE8qLW01lFdnMu2u584aPJ/M0lKPu0tAt3FXZx9a2xqIjZzZOVbSUg8o/OW5EEZvDCNzSfJ53ePJAaOZHBjBiuJsrLKDGxMG8G7WHlpaa3B17lq3ubW1Gs+AqDNcqYOPkyuLJt/MloqjHG2uJzQ6hnEB084YxPdHHjoDr42+mtq2VurbLQS7emA46ROLq0Lj+e5YDq81HmSMwx8rDjaoyxkSEEKad/fNnpcLRVH4W852vinJZvbskTTUm3lj/Wc8nTyByUHR576AIAg95ooPoKdOnUpFRUW38fnz53f5WpKkM9Zg3bp1KwEBAVRXVzN16lTi4+MZO/bK2QkuCOfi6uGDq8ePawKi1RmY/dAblB7ZR0VhBia3WCYNfBr9j0hj8A9PZsvON0lV5nWuYgMUle9Bxk7C8FkAxxulwGePN2F5ZBGZ59Gq+weSJJHmHdQlcKuxmFlw+AtMI59Cd3yDYVH5XppbyhjjP/mc19SoVEwIjGLCec+if/IyOJ+2LbVOreGdcdexqiSX9UX5aNUqHgkbx7iAiMu6Vva6snx2Wys5UvgfPD07/j8/cCCPiWMeJcXkj4+T6zmuIAhCT5EUkXtwRvHx8WzYsAF/f3/Ky8uZOHEi2dnZZz3nueeew9XVlSeeeKLLuCRJDJl2V+fXgVGpBEUPuhTTFgThOEWWWfb3x9BadaTEXode50p+0SYO5X3LVfe/iuIbhqzYiZ7QxqvB1dTMX9WtUcpPISsKLxzYxPelRwjyScJiqaW1tZq3Rs4i0dPvIjyZ0B89vn8l9z1/I7fc0vVN1j13vIzHATO3x6T10syEK8Ge6hL2Vpd0fv1B9i6RnnoWV/wK9Nlcc801LFiwgKeeeooFCxZw3XXXdTumtbUVh8OBm5sbZrOZVatW8cwzz5z2ekOnnzvvUxCEi0dSqZh138vsXf0p6/e8gc3aRkjsUOY9/gEWF2OX4Pngb/I4tVHKT6WSJP4waDx3xaZysLYMd60/I/xCr6g0jPOhKApVlhZ0ajWe+u4rzVeaZnsbQUHe3caDw30p2332xRtBuFBDfIIZ4hPc+fUH2aIowtmIFeizqKur48Ybb6SoqKhLGbuysjLuvfdevv32WwoKCpg7t6Pzmd1u59Zbb+V3v/tdt2tJksSDr23s6UcQBOEMysxNRI4382yaFcsjCy/KyrNw/nZXF/PK3g3UtrViRybew5c/Dp1MyGXQJOhSeSdrO/oxfrz3weOdY3a7g4EJ9/BLv8GM8Dt33XJBuFgGL3lbrECfhViBPguTycSaNWu6jQcGBvLtt98CEBkZyYEDB3p6aoIgCJcNm+xAURR0xzcCFjTV8tS2FdzhiGUg3jhQWN9Qwn0bFrNkxh1dmsxcSW6OSOHuRYtxdXPi7p/PpLHRzPxn/6GkvkIAACAASURBVI2nVcMw35Denp4gCCcRAbQgCFecMnMTsmIHFFyqGrD09oT6qYrWZl7bv5EtVceQUUjzDOTxQeNYeOQgk+QgBkkdG0tVSEwjlFy5ie9Lcrku/PR1t/s7b4MLH42cy4Jv9zFzwVoMGi1TfaK4ddDMznJ9giD0DSKAFgThivJD8Hwpcp+FE1rtVu5d/yVDrd68oYxGg8SWunIe2LiEICd3ZimBIIFFsVOGGXd0RNndKDhLo5krga+TK79JGtfb0xAE4RxEAC0IwhVFVuw8ek8zozeu5+Bb/b+zX2/5rigHf4cT1xIBxxdPJxJMhWyhSLFQQBP5ShPfU4Q3TtTThhY1txpEpQlBEPo+EUALQj9jbqwhP30Ddms7oQnD8Q4UDRhONcQIDeuqARFAXyrZdVUk2I2dwfMPEmRPKlTtrJSK8FT0PMcwTJIBuyLzFQWsKsrljtjBl3U9Z0EQ+j/VuQ8RBOFykb1zBYteuBW+/Rq3775n1duPsPnzl1Bkuben1ieUmZt6ewpXjABXD8pU5m7jpZKZKKM3HjoDtxGLSepoU62RVMwjisZWC4fqT9/1VRAEoa8QAbQg9BPNdRXsWPI2f7AP5Of2GG5RonjRlkZD+k7yDqzr7en1upNzn12qGkTZukvs6rAE9kk1ZCgncpoLlSbWqkq4ITqFelsboad8AqCSJEIlN/FGRxCEPk+kcAhCP5G3fy3D8SFAcukcM0garrYG8N3W5cSkTenF2fWukpZ6QBG5zz3Ix8mFV0ddxZ92rWKxowAtampp46lBE4k3+hLp4kl2cz2DONHi3a7IHFEaiHL36sWZC4IgnJsIoAWhn7C2teJjV3fLOXVFi72ttXcm1QecGjxnLhHBc08Z7BPMsll3cbi+Crssk+Tp21kL+hdJw3lh91pcHFpi8KAJG4tUeSSa/Ij2EAG0IAh9mwigBaGfCI4bwo4t3zDbGoZGOpGdtVVTQ8CAyb04s95T0lIHwGePN2F5ZBGZIm2jx6klFQNM/t3GxwdGYkmz8V7GduqtFhRgZnAsj6WKEm6CIPR9IoAWhH4iMCoVt4gE/lpwmKtsATihYbO6imwXG9eNnXfO8+srj1F5LBMnV0+C44aiVl/eLw8lLXWdtZ5rHlklcp4vgKIoLD+WxcIjB6huayXB6MsvkoaRfJrA+MeYERLH9OBYGq1tOGu0navTgiAIfZ14tRKEfkKSJCb//Hmyti9n4Y4V2K1tBKeM5doJN2Fw8TjjeQ6HnU2fPU9J1k6SJC8KpXa2aOykzribsoytmOsr8ApLIHnyLXj6hvbgE124ZwfbsX+9XwTPF+jdQ9tYX3CEuY4I/HHmUHUdv968lNdGX80g76ALurYkSRj1ThdppoIgCD1DBNCC0I+o1RqSx8whecyccx7bXFfBsewdlObsQZWdxV/tQ9FJagB2tlfyyeI3uYloQvDiUG0Oy9IfIH7sXIr2rqGhqRovUyApM+4ietCVmR5ypahvb2Vhfjrz5eFYcbCRMqqwEO1w540Dm/l0ys29PUVBEIQeJwJoQbgC7V3xERkbF5Iq+eBus5BDPbupYjQBAAyX/FilFOGNgSjJgyjFA5NVx5frvuAhJYkIYjlS08gnX7yJra2VhJGze/mJuvoh9xlFFg1TLlBGXSVRaiPlspl3OcQo/BmEN7k0kN5UQVZ9FQmevqc9t9nazt6aErQqNUN8gtGLFA1BEPoJ8WomCFeY4pzd5G/+mhftQ3CXdCBBuWLmRfYRrXjgJzkD4I0Tzdg6zxuBPwuUHKLxQC2pSMLEr2xa3ljxEXHDZ6FSqc957+a6CjLWfU5V3kGc3EzETZhHeNLoi/ZsJ9d6fjbNiuWRRSJ94wJ56AzUyRY+JYd7SCBV8gY6/n8IV9x5ee96PplyU7fzFuWn807GViJVHliRqaCVvwybxij/8B5+AkEQhItPNFIRhCvMka3LmGUN6AiejwuQXBiNP9upAMCi2MminmhO5E5bsKNBQjqpTl6Y5IZis9LadKJZxpnUVxXx1Wu/IGDnQe6uNjG5wMKez15m3/efXLRnkxVH58bBvDv3iuD5Ihhg8sehgTYcDKRrebnR+FPQUkd9e9cyiftrSvnw0E7+IA/hUUcK/+dI5UF7Ik/v/I5qS0tPTl8QBOGSEAG0IFxhrOYmjOi6jRvRU0c7uUoDr7CfCNzwkTo2dymKwlcUMARfVNKJALpZsWJV7OidXM95333L32dWux83KhFESx6MlPz5vTWZg+s+p7W5/qI939WRduzb91+0611pFEXp8rVKkngqbQJ2ureDV7qNdPgyL4PpjmB8pRObA2MkI2mKD98cy76Y0xUEQegVIoVDEK4w/onD2FH6LYPsJzrAyYrCVqmSarWNHDcVgQNmcGTnSv6m5BBi1ZKuN1MtmxnsMCHLCipJwq7I/E9TSFTyOLR653Pet+jIXh5UBndp9OIh6YlVe1GWt5/oQZMuwtMpnDmsu7Jl1Vfx39x9HG2sJ8LDxC2xg7rkLuc2VPO39K3srCnGoNIwIySOhweMxk2nZ4x/BCYnZ/Zbakg7qXPgNiqIdvPC85Tff01rCwknHfcDX9mJGrECLQhCPyACaEG4wiSMuoavty7l05Y8Jjr8seJgpaYcxT+Yu379DqrjG72GTL+bvAPrKK2vIj44lvFhSaz7+A/8pnIPoSoP8uw1OJn8GTZkCoosI6lO/4GWw2HnaMYmJFnmGwqZroTgfdLKZCt2NHrDBT3TybnPQ4yK2Dh4is3lR3l21yqmyyHMUULJMzfyq4qveG74dMb4h1Pc0sADm5ZwjT2M2xmHWbaxvPgYD9YtYcGUm1BLKv40bCqPb1lOtlJPmOxGnrqJdKmW94bM7Xa/FJ9A0psqGKCcSPlQFIVDmjpu9Y7qyUcXBEG4JCTl1M/rhEtCkiQefG1jb09DEABoba7n4JrPKDq4CZVaQ8TQqQyc+DO0Z6nH67BbOXpoC2X5ByjL2oW9qZF4jRclmLG7ujD9wddw8/Trco61zcy3f3sY57omhlo9qMLCbqq4gziGSL5kK/W8q8/j1j8vQa3pnlZyPk4Onl8Nrubgb/J+0nX6K1lRuG7lJ9zSFkWCZOocz1Tq+MJQwNcz7+TFfeuxFbVwHSdyxhVF4UXNfn45dAzjAjrGK1ub+eroIY41NRBt9OK6iGS8DN0/fai2tHDrms+ZYA9gnBKIFZmVqiKKnFr595SbRcMUQbgMDF7ydreULuEE8SomCFcgZzdPRs55mJFzHj6v4+uriljx7qME2LSEWnU0KE2oUHGrIww3tHxrK2bdx3/k2ic+6HLevu8+IbymjXvtyUjHc6fHK4G8yD52qOvIUjUw5e75Zw2eZYedgvRNFB/YiEqjJXLYdIJjhyBJEiUt9YDCo/c0M3rjeg6+1X9Xne2yzNHmOpw0WoLP0hjnVKXmRtpsduLx7DKeiCcWm5XS1iYyayuYq4R1Sa+RJIlEu5HMusrOANrP2Y0Hkkae854+Tq58NPEG3k3fxm+rdqCRVEwLjuXp5JkieBYEoV8Qr2SCIJyVoiis/9efuMbszSQ6us4pRPEFeXzOEe6Xkpglh7C2ejf1lYV4+oV3nluwbw1P2qM7g2eAUMmNCJWJ1rSB3Dz7IQzO7t3u2VBdQs62ZZhryqiuKMCl2cJkmy/tkoO1h5/nWOpYwmbdgyTRGTxnLum/wfPqklxe278JraLCotgJdHHnueHTiHAznfNcg1qDVXHgQEFzUoTsQKFdcaBXafBxcqWipZVoPFAUhQKaqKGNo6oWkp0iftKcQ1yNvDRq1k86VxAEoa8TVTgEQTirhqoi2htqmKAEdo5JksQ1hHOAGqyKA5UkYVI7Y2lp6HKuw2FHR/f60E4aPYFRg04bPBdmbmXpa/fis3UnYw7XEF5npdVmJhFPphLCM9YUSvZvoL44m88ebyLpmYX9OnhOry3n5b0buM+WwPOOYbzqGElas5GHNn6FxX6iTreiKKf9uNXHyZUody/WSaVdxtdJJcR6eOPj5MKNsQP5Rn2MAqWR59nLhxxmK2XkyfVsLMnvch9BEARBBNCCIJyDtb0VF5WuS/k6AMPx9UwbMvVKO+X2JrwCo7scE5Y0ig2qii5j9Uo7WY4aQuKGdruXw25j039f5FFbAjfJkYyWAviVNICxBLKQfACcJA3j7N4EshH90g39vtbzf3L2M8sRSpTUkbahkiQmEkyg7Mza0jzq21t5dtcqxix9j+FfvcMjm5eSf0pd7meHTWW9roy31RksU47ytjqDDboK/jR0KgCj/MK4LWEwf2U/GiRasFGEGS0qSmvqeevg5h5/bkEQhL5MBNCCIJyVV0Ak9Uo7pUrX8mMHqcEbA8do4nXdYVLG39itHnTarJ+zzbmJj9RHOKjUsEw5yh9Uu/EKiaOmNA9F7lpbuKIwA28MREtdc3ynEMIBqpGPr7AqdA3m+7Pi5gbCT1NRJNTuQmFTHfeuX0xbmZmX5BG8wzhCqnXcv2Ex5a3NnceGuBpZMuNO5gxMxRTnxdzUVBbPuIMQV2PnMeMCI1CjQo+aPzOMN6Ux/B9pOFBYfiwLm+zokecVBEG4HIgAWhCEs9Jo9Qy9+gFe02axVSmnRGlhDcV8KGVRqbbxqVcdMdfeS9rMe7qd62r0Zc5vPsY2cTL/MdWzUipmGH4MO9rE/k9fZMXfH8Nuaz/nHE4Ol1sVO5s01Yyd3X0Fuz+KMpo4IjV2G8/XNNMq23CyqrhZjsZd0qGX1EyVQhgu+/K/Iwe6HK9TqRnoFcBNUQO5KjQBwymb+Y61NGBD5j6SMEkdZQUDJRfuJREZBbPNeukeUhAE4TIjNhEKgnBOiaNm4+blz/o1n9NUV4wpMIqrpj2Jb0j8Oc91cjWSOGYO6ev/xx+UNIJwBQlmtcu8XZLNoU1fkjr5VgD8w5NZq1goUJqIlE7kR6+mmBBcWU0xa7RV3HDbUG5Yuo/MxktXU9gmO5CQ0JyhvnVPuTUujYcqviLA4UwKXliRWSUV06ixISkSSXZjl02aAEmyiXU1J1Jn1pfm89qBjdjsDtoUO/FGX54ZOpVAlxM/Y41KhQkDrpK2y7WCJVckRRLlrARBEE4iAmhBEM5LSNzQ0+Ytn4+jh7aQrPImSDqR4qGWVFxtC+SjnSs7A2i1RseYm5/ijf++yHiHH8GyE+naJjJUDRhD4smKcOa/t08nZX0hZZcoeD7aVMfrBzaxq6YESYKxvuE8MWg8/s69s1Ex3ujLiyNn8td9G/m4LRs7MmmmQN4fMo/VJUfYraqiSbbiirYzT72CVnydO37WGXXlzN+zlvscCcRixI7CmvoSHti4mMUz7kCr6tjkGeXmRaNkpU2xY5BO/GmoUlrRqFS46vQ9//CCIAh9lAigBUG45Bx2Kwal+0quHjX2U1IDIlPG4ekXRva2ZeTVlGOKHMONI66mRpF57J4WxlnrOPh198oeF0O1xcx9GxYz3R7M7YzFoSisrirmF+sX8cW023DR/rRmLxdquG8oC6ffRk1bKwa1BjednnaHnWONdeyUK9lHNc5omK2EE4uR79RFvBxzNQD/zt7HLEcocVJHHWgtEjMJ5bC9nvVl+UwLjgXAx8mFsf4RLKjI4Q4lDidJQ5Ni5V+qbH4WPbAz0BYEQRBEAC0IQg8IiRvGsuUfYlbCcDkpRWCzuoqQAaO7He/pF9a9yUtLPQqXNo1gcUE6g2QvpkohnWPXEkGJzcy7mduQFAmDRsOM0DhiPLwv2n1tsoNPc/ay9Ohhmm3tDPIK4IHkkcQafTqPkSQJHyeXzq//vHsNNZUNvMxIPCU9R5Um3iEDi2TniQHjGOTdUXbwWFM9Y4iiVDGzjyoUIA0fwu2uFJ1SdvBPQ6fw/J61PFWxA2+VgRrZwrXhSdyXNOKiPasgCEJ/IAJoQRAuOaNPMLEjrmL+rjVcZfXHiJ4dmloOOVm4bspt5zy/pKUOgCFGmZpHVgGXpnTd4dpKBsmeXXYtOhSZWsVCydE8xij+tEgyDxYs4ba4NO6KH3JR7vuHHd9RWd3IPY5YTOjZXVXF/RuX8OGE64n28Op2fHlrE1srCnlFHole6lgZjpDcuU9J5L/6AuZGDug8Nszdk69bCzhKMyPwQ4XE6xxAJ6kZ7dI1h91Jo2X+iBnUtbVSaWkh2NUDN61I3RAEQTiVCKAFQegRw6/7JYXRA9mwZSnW1mb8EycxZ+xcnE4qpXaqMnMTsmInekIbz6ZZsTyy6JLWffZ3caes1kwaJ1Z+t1KBCom/KMPQSB1pKBMdQfw5Zw/jgyLPqxvg2eQ2VLO/uowXHMPRHr/+FEJwyAr/PLyTl0Z27+ZX0FRHpNoDvdI1rSIWIyVtjTgUGfXxa40MCOOtiiL+wnDcpY4UlKuUMP6o7ML1DMGxyeCMyeB8Qc8lCILQn4kAWhCEHiFJEhEDxhIxYOx5HX9y8PxqcDUH78zjUqw8OxQZRemoQnFDdAr3lywm2WEi/HgVkE2UMZOwzuAZwFPSM0L2Y03JEe5NGH5B9z9YV04yps7g+QeDFB9er00/7TmBLu4Uy83YFbnLvIppwVvn3Bk8A+TUVzOVkM7gGcBF0jJTCWVVUS6j/cMvaP6CIAhXIhFAC4LQ55S01AMKj97TzOiN6zn41vlXwMioK2d5wWEarW0M9w9lZmg8Thptt+OqLS28fmATGyqO4lBkhnkF89igcfxu8ERe2rcBbwzYkal1WNCdZgOkVpGwOeRu4z+Wp96JWqmt23gNFow6w2nPiXAzEWP0ZlF9PtfLkWglNQ1KO/9RH+HW2EHUt1vYXnkMtaSi2dqGz2le6p3RUGUXtZ0FQRB+ChFAC4LQp5SZmzg5eM5ccv7B86c5e/ksex8T5UDCFB3fVh9iYV46H064HreTyrC1Oezcu+FLUto8eU0ZhQYVm2vLuH/DYv4z5WesvOoeMuoqUKtUHKguY3NOHskOU2e95XbFwS51Na8GXtjqM8BY/whekTawX6lmkNSROmJR7CxVF3J99KAznvfiyFn8aef3/KZ2O94qJ6rkVq6PSEEjqbh25SckqEzIKGQ56nCXdIxXgjrL3MmKwg51FdcHnfn6giAIwpmJAFoQhD4nekIb460NHPwRwXNlazMfZe3mOXkonpIeJBjl8Ofj1mw+zdnLLweM6jx2VXEuXjY984jq3DA4hRBq5DYW5qfz8IDRDPYJBiDWw5vVxbm803qIUQ5/2rCzRl3KyMBwkjz9LvhZ9WoNr4+ezRNbv2GVUoonOjLlOmYExzE3IvmM53noDLw19loqWpupaTMT5uZJYXMdT2xezjPyELwVJwBKFTPz2cMrqv3MkkNRIbFBXYbKVcOM4yXsBEEQhB9HBNCCIPQLmyqOkip5dwTPx0mSxCQ5iE9L8roE0IfrKom3G7v2CAcSZRNbT+rgB+Cs0fHPiTewtDCTzaVH0as1PBQxmsmB0d06AP5UySZ/vrnqbnZUFtFka+P3XoEEuXic17n+zm6dTV6+ys9kshyMt+TU+f0gyYVJBFNnktlg6yhjNyk0jnkRA9CpxZ8AQRCEn0K8egqC0Gf8kPt8daQN+8b9wPmvQEtIcJo60Qrd4mT8XdzIUNV1O7xMMuPv0v2eThotN0encnN06mnvLSsKElxQQK1VqRkbEPGTzweoa2slWXHt9sBeih6NQc0z46Ze0PUFQRCEDt13xgiCIPSCUzcO/pjcZ4BxAREcUGqpU05syFMUhXWqEiaHxHQ59uqwBPZLNRxSajvHipUWVqtKuDFm4Hnfc1dVMXes+R/Dv3qHScs+4M2Dm2l32H/UvC+mNN8gDqhruowpisIBTR1pfsG9NCtBEIT+R6xAC4LQ635olPLZ401YHllE5k+o9ezr5Mp9icN5IWs34+VAPBQde9TV2Jwl7ohL63Kst8GFV0ZdxR93fo+zfBQdaiqVVp4cNJ7E88xr3l9Tym+3r+BWRwyPkkito50vj+bzu+aVvD5m9o+e/8UwJyKZRfnpfNGex0QlEAcKq1QlWPQy04Jjzn0BQRAE4bxIiqJc2t64AtDx0e6Dr23s7WkIQp9yaq3nmvmrLrhRSlZ9FcuOZtJkbWe4fyjTQ2LRnyHX1y7LZNRVYJMdpHgFYPgROcEPbljCgDo3RksBJ66nyPxOvYN3J8y9qK2+f4xqi5kPM3ewobwAlSQxNTiGXyQOx+MMJfEEQRBOZ/CStxEh4pmJFWhBEHpFt0Ypv7k4jVISPH1J8PQ9r2M1KhWDvAN/0n1yGqv5GeFdryepiJdMZDVU9VoA7ePkwu+HTOb3TO6V+wuCIFwJRA60IAi9JnpCO8+m2aiZv6q3p/KjeRtcKMfcbbwCM74G116YkSAIgtBTRAAtCILwE9wcM5Av1QU0KR3d/BRFYR0lWLUKQ33Fhj1BEIT+TKRwCILQ435I3+ioI3d55tjNiUimtKWRpwt2EqnyoJY2nPQ63h59HWpJrE0IgiD0ZyKAFgShR3XLfb7z4uQ+9zRJkng4ZQy3/X979xfa5Xn2AfwKTra3fctswaapEUKNLl2mqS6rjOImrGln/UN7NA9ekHky9oelE2SH7cHqJmMHHT0oK2N0CKUL6DrY5iL0z6BTslXfHGi31bJCEheNNWlMqtjo/R5o8maamN+TP7/nl9/z+UDAxMd49ea2/fbxuq/7cxvi1OC5uPvT/xUPLrt33i5XAaByCdBAWV1LYxOznrufzzbruRLd/ek74pH7GvIuA4Ay8veMQNm1LosYen0g7zIAYFYEaKBszowO510CAMyZAA2UxeTe5zvPDc35whQAyIseaGDB9Y4MRkSqqt5nAIrLG2hgQd0cnk8eFJ4BWNwE6Gl0dHREc3NzLFmyJI4fPz7tc4cPH46mpqZYvXp17N+/v4wVQuXrHbkQESkO7BmO5md+IzwDUBUE6GmsXbs2Dh06FF/5ylemfebq1avxve99Lw4fPhynTp2KV155Jd59990yVgmVq3dkMBo3X45D/9MTl9o79DwDUDUE6Gk0NTXFmjVrbvtMV1dXNDY2RkNDQyxdujR27twZr732WpkqhEqX4tkvjsXY0RPCMwBVRYCeg76+vli5cuXE5/X19dHX15djRQAALLRCT+Foa2uL/v7+W76+b9++2L59+4y/3pW9MLXrvc8Rka7duDBF7zMA1aPQAfrIkSNz+vUrVqyInp6eic97enqivr5+2uf/+qdfTfz4/lUPxYrG9XP6/aHSTJ71/NP6gTjf3ql9A2AR+NtAb7wz0Jt3GYtGoQN0qVJKU369tbU13nvvvfjggw/i/vvvj1dffTVeeeWVab/Plx7/5kKVCBXhWro6EZ67956OCOEZYDFoXV4frcv//yXgL/7elWM1lU8P9DQOHToUK1eujGPHjsXWrVtjy5YtERFx5syZ2Lp1a0REfOpTn4oXXnghHn/88fj85z8f3/jGN+LBBx/Ms2zI3bYHrh8cBIBqVZOme73KvKqpqYlv/+ytvMuABdU7csGFKQBV4IsHfz7t38CjhQOYB5N7n1uXJQcHAahqAjQwJzcfHOzepfcZgOomQAOz1jsyGBFpom2j+3lvnQGofgI0MCvjs571PANQNAI0kNn1N88RB/YMx6X2jjhp1jMABWKMHTArP9g9Gnee+8hFKQAUjgANzEqKZN4zAIUkQAOZjB8cvP4BAMWjBxoo2fjBQb3PABSZAA2U5Mzo8MSs5/PtnXqfASgsLRxAZsIzAEUmQAMluZau5l0CAFQEARqY0fXe5xTPbrgS55/rzLscAMiVHmhgWmdGh+NaGovGzZfj2Q1X4lJ7h/YNAApPgAamNDk8/7R+ILp3nY4I4RkABGjgFuOznp/efTEeeeuN6H7+rrxLAoCKoQca+A9nRodjcng+eVB4BoDJBGjgFo2bL8fmTz6KodcH8i4FACqOAA1MKSVXdQPAVARoYELvyGBcS2Ox7YFPYuzoCRM3AGAKAjQQEbceHNT7DABTM4UDuHFRSsSBPcNxqb0jTnrzDADTEqChwG6e9Xy+vVPbBgDMQICGAvuPi1L2uigFAEqhBxoK7tkNn8T55zrzLgMAFg0BGgAAMtDCAQU0ufc5wrxnAMhCgIaCufngYPcuvc8AkIUADQVzLY1NzHruft6sZwDISg80FFDrsoih1wfyLgMAFiUBGgrkzOhw3iUAwKInQENBTO59vvPckAtTAGCW9EBDAfSODEZE0vsMAPPAG2iocjeH55MHhWcAmAtvoKGK9Y5ciIiIA3uG41J7R5zUtgEAcyZAQ5XqHRmcmPV8vr1TzzMAzBMtHFDFtj0wFmNHTwjPADCPBGgAAMhAgIYqdL33OUXrsmsuTAGAeaYHGqrI5FnPep8BYGEI0FBFrqWrE+G5e+/piBCeAWC+aeGAKjN+cBAAWBgCNFSVdOMDAFgoWjigCkzufW5dlm4cHHTjIAAsBAEaFrmbDw5279L7DAALSYCGRax3ZDAiUjy9+2I88tYb0f28t84AsNAEaFikrs96jonwfPKg8AwA5SBAwyJ0/c1zxIE9w3GpvSNOmvUMAGVjCgcsUj/YPRp3nvvIRSkAUGYCNCxSybg6AMiFAA2LzPjBwYjkwhQAyIEAfRsdHR3R3NwcS5YsiePHj0/7XENDQ6xbty7Wr18fDz/8cBkrpGiuHxxMcWDPcDQ/8xsHBwEgBw4R3sbatWvj0KFD8a1vfeu2z9XU1MSbb74Z99xzT5kqo2hunvV8vr1T7zMA5ESAvo2mpqaSn01JPyoL45aLUva6KAUA8qSFYx7U1NTEo48+Gq2trfHSSy/lXQ5VZDw8P737Yvy4p/NGeAYA8lT4N9BtbW3R399/y9f37dsX27dvL+l7vP3221FXVxcDAwPR1tYWTU1NsWnTpvkulYJq3Hw5vnplKLr1OwNARSh8gD5y5Micv0ddXV1ERCxfvjyeeuqp6OrqmjJA//VPv5r48f2rHooVjevnoEoYgwAACU9JREFU/HsDAMzV3wZ6452B3rzLWDQKH6BLNV2P88cffxxXr16Nu+66K0ZHR6OzszOeeeaZKZ/90uPfXMgSqTLj7RsAsNBal9dH6/L6ic9/8feuHKupfHqgb+PQoUOxcuXKOHbsWGzdujW2bNkSERFnzpyJrVu3RkREf39/bNq0KR566KHYuHFjbNu2LR577LE8y6YK9I4M6n0GgApVk4yPKIuampr49s/eyrsMFoHxN88H9lyMT7/2hlnPAJTdFw/+3ISx2/AGGipQ4+bL8d8DH8XQ6wN5lwIA3ESAhgrl//wBoDIJ0FBBxnuftz3wSYwdPeG2QQCoQAI0VIjekcGISPH07ovxyFt6nwGgUhljBxWgd+RCREQc2DMcl9o74qQ3zwBQsQRoyNH4xI3GzZfjp/UDcb69U9sGAFQ4ARpyNDk8X5/1LDwDQKXTAw05e3bDJ3H+uc68ywAASiRAAwBABlo4IAeTe58jzHsGgMVEgIYyu/ngYPcuvc8AsJgI0FBm19LYxKzn7ufNegaAxUYPNOSgdVnE0OsDeZcBAMyCAA1ldGZ0OO8SAIA5EqChTCb3Pt95bsiFKQCwSOmBhjLoHRmMiKT3GQCqgAANC6x35EJExER4PnlQeAaAxUyAhgU0Hp4P7BmOS+0dcVLbBgAsegI0LJDekcGJWc/n2zv1PANAlXCIEBbQtgfGYuzoCeEZAKqIAA0AABkI0LAArvc+p2hdds2FKQBQZfRAwzyaPOtZ7zMAVCcBGubRtXR1Ijx37z0dEcIzAFQbLRwwz8YPDgIA1UmAhnmVbnwAANVKCwfMg8m9z63L0o2Dg24cBIBqJEDDHN18cLB7l95nAKhmAjTMQe/IYESkeHr3xXjkrTei+3lvnQGg2gnQMEvXZz3HRHg+eVB4BoAiEKBhFq6/eY44sGc4LrV3xEmzngGgMEzhgFn6we7RuPPcRy5KAYCCEaBhlpJxdQBQSAI0ZDR+cDAiuTAFAApIDzRkMH5wUO8zABSXAA0luHnW8/n2Tr3PAFBQWjiYF32nq7eV4ZaLUvaenlV4/ttA7wJUV12s0e1Zn5lZo5lZo9uzPpRCgGZenHn/f/MuYUGMh+end1+MH/d0Rvfe07P+Xu/4l/KMrNHtWZ+ZWaOZWaPbsz6UQoCGGTRuvhxfvTLkohQAICIEaAAAyKQmpWSYbRnU1NTkXQIAQMlExOmZwlEmNiEAQHXQwgEAABkI0AAAkIEAzax0dHREc3NzLFmyJI4fPz7tcw0NDbFu3bpYv359PPzww2WsMH+lrtHhw4ejqakpVq9eHfv37y9jhfm7cOFCtLW1xZo1a+Kxxx6LoaGhKZ8r2j4qZU98//vfj9WrV0dLS0ucOFG9c9inM9Mavfnmm/HZz3421q9fH+vXr48f/ehHOVSZn927d0dtbW2sXbt22meKvIdmWp+i7x9KkGAW3n333fSPf/wjbd68Ob3zzjvTPtfQ0JA+/PDDMlZWOUpZo7GxsbRq1ar0r3/9K125ciW1tLSkU6dOlbnS/Ozduzft378/pZTST37yk/TDH/5wyueKtI9K2RO///3v05YtW1JKKR07dixt3Lgxj1JzU8oavfHGG2n79u05VZi/P//5z+n48ePpC1/4wpQ/X/Q9NNP6FH3/MDNvoJmVpqamWLNmTUnPpoIeoCxljbq6uqKxsTEaGhpi6dKlsXPnznjttdfKVGH+fve738WuXbsiImLXrl3x29/+dtpni7KPStkTk9dt48aNMTQ0FGfPns2j3FyU+uemKHtmKps2bYq777572p8v+h6aaX0iir1/mJkAzYKqqamJRx99NFpbW+Oll17Ku5yK09fXFytXrpz4vL6+Pvr6+nKsqLzOnj0btbW1ERFRW1s77X/Ai7SPStkTUz3T21uc29NKWaOampr4y1/+Ei0tLfHEE0/EqVOnyl1mRSv6HpqJ/cNMjLFjWm1tbdHf33/L1/ft2xfbt28v6Xu8/fbbUVdXFwMDA9HW1hZNTU2xadOm+S41N3NdoyLMB59ujZ577rn/+Lympmba9aj2fTRZqXvi5rdjRdhL40r5Z92wYUP09PTEHXfcEX/84x/jySefjH/+859lqG7xKPIemon9w0wEaKZ15MiROX+Purq6iIhYvnx5PPXUU9HV1VVVwWeua7RixYro6emZ+Lynpyfq6+vnWlZFud0a1dbWRn9/f9x3333x73//O+69994pn6v2fTRZKXvi5md6e3tjxYoVZasxb6Ws0V133TXx4y1btsR3vvOduHDhQtxzzz1lq7OSFX0PzcT+YSZaOJiz6frEPv7447h48WJERIyOjkZnZ+dtT4RXs+nWqLW1Nd5777344IMP4sqVK/Hqq6/Gjh07ylxdfnbs2BEvv/xyRES8/PLL8eSTT97yTNH2USl7YseOHfHrX/86IiKOHTsWy5Ytm2iFKYJS1ujs2bMTf+66uroipST8TFL0PTQT+4cZ5XN2kcXu4MGDqb6+Pn3mM59JtbW16etf/3pKKaW+vr70xBNPpJRSev/991NLS0tqaWlJzc3Nad++fXmWXHalrFFKKf3hD39Ia9asSatWrSrcGn344Yfpa1/7Wlq9enVqa2tLg4ODKSX7aKo98eKLL6YXX3xx4pnvfve7adWqVWndunW3nYRTrWZaoxdeeCE1NzenlpaW9OUvfzkdPXo0z3LLbufOnamuri4tXbo01dfXp1/+8pf20CQzrU/R9w8zq0nJMVMAACiVFg4AAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADIQIAGAIAMBGgAAMhAgAYAgAwEaAAAyECABgCADARoAADI4P8AJAa/Nc/tyogAAAAASUVORK5CYII=" } }, "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Training a Neural Network" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Clearly, a linear classifier is inadequate for this dataset and we would like to use a Neural Network. One additional hidden layer will suffice for this toy data. We will now need two sets of weights and biases (for the first and second layers):\n" ] }, { "cell_type": "code", "execution_count": 84, "metadata": {}, "outputs": [], "source": [ "# initialize parameters randomly\n", "h = 100 # size of hidden layer\n", "W = 0.01 * np.random.randn(D,h)\n", "b = np.zeros((1,h))\n", "W2 = 0.01 * np.random.randn(h,K)\n", "b2 = np.zeros((1,K))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The forward pass to compute scores now changes form:" ] }, { "cell_type": "code", "execution_count": 85, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(300, 100)\n", "(300, 3)\n" ] } ], "source": [ "# evaluate class scores with a 2-layer Neural Network\n", "hidden_layer = np.maximum(0, np.dot(X, W) + b) # note, ReLU activation\n", "print(hidden_layer.shape)\n", "scores = np.dot(hidden_layer, W2) + b2\n", "print(scores.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Remember $X$ has shape $(num\\_examples, D)$, $ W $ has shape $(D, h)$, thus \n", "`hidden_layer` has shape $(num\\_examples, h)$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that the only change from before is one extra line of code, where \n", "we first compute the hidden layer representation and then the scores based \n", "on this hidden layer. Crucially, we have also added a non-linearity, which \n", "in this case is simple ReLU that thresholds the activations on the hidden layer \n", "at zero.\n", "\n", "Everything else remains the same. We compute the loss based on the scores \n", "exactly as before, and get the gradient for the scores `dscores` exactly as \n", "before. However, the way we backpropagate that gradient into the model \n", "parameters now changes form, of course. First let us backpropagate the \n", "second layer of the Neural Network. This looks identical to the code \n", "we had for the Softmax classifier, except we are replacing $X$ (the raw data), with the variable \n", "`hidden_layer`):\n" ] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(100, 3)\n", "(1, 3)\n" ] } ], "source": [ "# backpropate the gradient to the parameters\n", "# first backprop into parameters W2 and b2\n", "dW2 = np.dot(hidden_layer.T, dscores)\n", "db2 = np.sum(dscores, axis=0, keepdims=True)\n", "print(dW2.shape)\n", "print(db2.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- `hidden_layer.T` is the transpose of `hidden_layer`, and thus has \n", "shape $(h, num\\_examples)$. `dscores` has shape $(num\\_examples, 3)$. \n", "And `dW2` thus has shape $(h, 3)$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, unlike before we are not yet done, because `hidden_layer` is \n", "itself a function of other parameters and the data! We need to continue\n", "backpropagation through this variable. Its gradient can be computed as:" ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(300, 100)\n" ] } ], "source": [ "dhidden = np.dot(dscores, W2.T)\n", "print(dhidden.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`dhidden` has shape $(num\\_examples, h)$. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we have the gradient on the outputs of the hidden layer. Next, we have to backpropagate the ReLU non-linearity. This turns out to be easy because ReLU during the backward pass is effectively a switch. Since \n", "\n", "$$\n", "r=\\max(0, x)\n", "$$\n", "\n", "we have that \n", "\n", "$$\n", "\\frac{dr}{dx}=\\mathbb{1}(x>0)\n", "$$\n", "\n", "Combined with the chain rule, we see that the ReLU unit lets the gradient pass through unchanged if its input was greater than $ 0 $, but _kills_ it if its input was less than zero during the forward pass. Hence, we can backpropagate the ReLU in place simply with:" ] }, { "cell_type": "code", "execution_count": 88, "metadata": {}, "outputs": [], "source": [ "# backprop the ReLU non-linearity\n", "dhidden[hidden_layer <= 0] = 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And now we finally continue to the first layer weights and biases:" ] }, { "cell_type": "code", "execution_count": 89, "metadata": {}, "outputs": [], "source": [ "# finally into W,b\n", "dW = np.dot(X.T, dhidden)\n", "db = np.sum(dhidden, axis=0, keepdims=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We are done! We have the gradients $ dW,db,dW2,db2$ and can perform the parameter update. Everything else remains unchanged. The full code looks very similar:" ] }, { "cell_type": "code", "execution_count": 90, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "iteration 0: loss 1.098716\n", "iteration 1000: loss 0.496002\n", "iteration 2000: loss 0.396590\n", "iteration 3000: loss 0.390806\n", "iteration 4000: loss 0.388821\n", "iteration 5000: loss 0.389472\n", "iteration 6000: loss 0.401825\n", "iteration 7000: loss 0.392470\n", "iteration 8000: loss 0.452559\n", "iteration 9000: loss 0.422789\n" ] } ], "source": [ "# initialize parameters randomly\n", "h = 100 # size of hidden layer\n", "W = 0.01 * np.random.randn(D,h)\n", "b = np.zeros((1,h))\n", "W2 = 0.01 * np.random.randn(h,K)\n", "b2 = np.zeros((1,K))\n", "\n", "# some hyperparameters\n", "step_size = 1e-0\n", "reg = 1e-3 # regularization strength\n", "\n", "# gradient descent loop\n", "num_examples = X.shape[0]\n", "for i in range(10000):\n", " \n", " # evaluate class scores, [N x K]\n", " hidden_layer = np.maximum(0, np.dot(X, W) + b) # note, ReLU activation\n", " scores = np.dot(hidden_layer, W2) + b2\n", " \n", " # compute the class probabilities\n", " exp_scores = np.exp(scores)\n", " probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True) # [N x K]\n", " \n", " # compute the loss: average cross-entropy loss and regularization\n", " correct_logprobs = -np.log(probs[range(num_examples),y])\n", " data_loss = np.sum(correct_logprobs)/num_examples\n", " reg_loss = 0.5*reg*np.sum(W*W) + 0.5*reg*np.sum(W2*W2)\n", " loss = data_loss + reg_loss\n", " if i % 1000 == 0:\n", " print(\"iteration %d: loss %f\" % (i, loss))\n", " \n", " # compute the gradient on scores\n", " dscores = probs\n", " dscores[range(num_examples),y] -= 1\n", " dscores /= num_examples\n", " \n", " # backpropagate the gradient to the parameters\n", " # first backprop into parameters W2 and b2\n", " dW2 = np.dot(hidden_layer.T, dscores)\n", " db2 = np.sum(dscores, axis=0, keepdims=True)\n", " # next backprop into hidden layer\n", " dhidden = np.dot(dscores, W2.T)\n", " # backprop the ReLU non-linearity\n", " dhidden[hidden_layer <= 0] = 0\n", " # finally into W,b\n", " dW = np.dot(X.T, dhidden)\n", " db = np.sum(dhidden, axis=0, keepdims=True)\n", " \n", " # add regularization gradient contribution\n", " dW2 += reg * W2\n", " dW += reg * W\n", " \n", " # perform a parameter update\n", " W += -step_size * dW\n", " b += -step_size * db\n", " W2 += -step_size * dW2\n", " b2 += -step_size * db2\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The training accuracy is now:" ] }, { "cell_type": "code", "execution_count": 91, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "training accuracy: 0.93\n" ] } ], "source": [ "# evaluate training set accuracy\n", "hidden_layer = np.maximum(0, np.dot(X, W) + b)\n", "scores = np.dot(hidden_layer, W2) + b2\n", "predicted_class = np.argmax(scores, axis=1)\n", "print('training accuracy: %.2f' % (np.mean(predicted_class == y)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Which prints $99\\%$ We can also visualize the decision boundaries:" ] }, { "attachments": { "spiral_net.png": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtAAAAJACAYAAACkMVHfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd0VNXexvHvpJFeSYAkBEhCr6GDdFCK0hVpSlMREIFrgWuhikrxKghSBRtNEQSkSQtI770GCIQQQkkgpLd5/8jraG6AOF7IhPB81rprZfacc+Z3Dtfkyc4uBqPRaERERERERP4WK0sXICIiIiLyOFGAFhERERExgwK0iIiIiIgZFKBFRERERMygAC0iIiIiYgYFaBERERERMyhAi4iIiIiYQQFaRERERMQMCtAiIiIiImZQgBYRERERMYMCtIiIiIiIGRSgRURERETMoAAtIiIiImIGBWgRERERETMoQIuIiIiImEEBWkRERETEDArQIiIiIiJmUIAWERERETGDArSIiIiIiBkUoEVEREREzKAALSIiIiJiBgVoEREREREzKECLiIiIiJhBAVpERERExAwK0CIiIiIiZlCAFhERERExgwK0iIiIiIgZFKBFRERERMygAC0iIiIiYgYFaBERERERMyhAi4iIiIiYQQFaRERERMQMCtAiIiIiImZQgBYRERERMYMCtIiIiIiIGRSgRURERETMoAAtIiIiImIGBWgRERERETMoQIuIiIiImEEBWkRERETEDArQIiIiIiJmUIAWERERETGDArSIiIiIiBkUoEVEREREzKAALSIiIiJiBgVoEREREREzKECLiIiIiJhBAVpERERExAwK0CIiIiIiZlCAFhERERExgwK0iIiIiIgZFKBFRERERMygAC0iIiIiYgYFaBERERERMyhAi4iIiIiYQQFaRERERMQMCtAiIiIiImawsXQBTwqDwWDpEkRERET+NqPRaOkS8i0F6Dw04LOtli7hkdm3fj61WvaxdBn5mp5R7v7JM7qaEEdg42TaBqbQxNn1EVWWP4z+dCGjR3S3dBn5mp5R7vSMHkzPJ4uVRztLl5CvaQiHiIiIiIgZFKBFRERERMygAC0PhW9QNUuXkO/pGeVOz+jBmjSobOkS8j09o9zpGT2Yno/8HQajRojnCYPBUKDHQItYypM0BlpEJK9YebTTJMIHUA+0iIiIiIgZFKBFRERERMygAC0iIiIiYgYFaBF57IWF2rLqgjWh8XGWLkVERJ4ACtAi8ljzdXLFymBNWKgDqy7YEBofR3yhREuXJSIiBZh2IhSRx56vU9bqG2GhsawiGUinibNlaxIRkYJLPdAiUmBYGawBa0uXISIiBZwCtIiIiIiIGRSgRURERETMoAAtIiIiImIGBWgRERERETMoQIuIiIiImEEBWkRERETEDArQIiIiIiJmUIAWERERETGDArSIiIiIiBkUoEVEREREzKAALSIiIiJiBgVoEREREREzKECLiIiIiJhBAVpERERExAwK0CIiIiIiZlCAFhERERExgwK0iIiIiIgZFKBFpEAJC7UFIL5QooUrERGRgkoBWkQKDF8nVwA+n+fE/lvpCtEiIvJIKECLSIHi7+yBlcGGz+c5MWqfUSFaREQeOgVoESlwfJ1csTLYADbsv5Vu6XJERKSAUYAWERERETGDArSIiIiIiBkUoEVEREREzKAALSIiIiJiBgVoEREREREzKEDnom/fvhQpUoTKlSvf8/3Q0FDc3NwICQkhJCSEjz76KI8rFBEREZG8ZGPpAvK7Pn36MHjwYF5++eX7HtO4cWNWrlyZh1WJiIiIiKWoBzoXDRs2xMPD44HHGI3GPKpGRERERCxNAfp/ZDAY2LlzJ1WrVqVNmzacPHnS0iWJiIiIyCOkIRz/o+rVqxMREYGjoyNr166lQ4cOnD171tJliYiIiMgjogD9P3JxcTF93bp1awYOHEhMTAyenp45jt23fr7pa9+gavgFh+RJjSIiIiIPErr9GKHbj1m6jMeGAvT/KDo6Gh8fHwwGA3v37sVoNN4zPAPUatknj6sTERERyV2TBpVp0uDPFcfGTlhswWryPwXoXHTr1o2tW7dy8+ZNihcvzpgxY0hLSwOgf//+LF26lBkzZmBjY4OjoyOLF+v/cCIiIiIFmcGoJSTyhMFgYMBnWy1dhsgT42pCHIGNk2kbmEITZ1dLlyMi8lix8minVcYeQKtwiIiIiIiYQQFaRERERMQMCtAiIiIiImZQgBYRERERMYMCtIgUWGGhtqy6YE1ofJylSxERkQJEAVpECiRfJ1esDNaEhTooRIuIyEOlAC0iBVb2EG1DfKFES5ckIiIFgAK0iBRof4RoERGRh0UBWkRERETEDArQIiIiIiJmUIAWERERETGDArSIiIiIiBkUoEVEREREzKAALSIiIiJiBgVoEREREREzKECLiIiIiJhBAVpERERExAwK0CIiIiIiZlCAFhERERExgwK0iIiIiIgZFKBFRERERMygAC0iIiIiYgYFaBERERERMyhAi4iIiIiYQQFaRERERMQMCtAiIiIiImZQgBYRERERMYMCtIiIiIiIGRSgReSJEBZqz6h9RkLj4yxdioiIPOYUoEWkwPN1csXKYE1YqAOrLtgQGh9HfKFES5clIiKPKRtLFyAikhd8nVwBCAuNZRXJ1PSycEEiIvLYUg+0iDxRrAzWli5BREQecwrQIiIiIiJmUIAWERERETGDArSIiIiIiBkUoEVEREREzKAALSIiIiJiBgVoEREREREzKECLiIiIiJhBAVpERERExAwK0CIiIiIiZlCAFhERERExgwK0iIiIiIgZFKBFRERERMygAC0iIiIiYgYFaBERERERMyhAi4iIiIiYQQFaRERERMQMCtAiIiIiImZQgBYRERERMYMCdC769u1LkSJFqFy58n2PefPNNyldujRVq1bl0KFDeVidiIiIiOQ1Behc9OnTh3Xr1t33/TVr1hAWFsa5c+eYPXs2AwYMyMPqRERERCSvKUDnomHDhnh4eNz3/ZUrV9KrVy8A6tSpw+3bt4mOjs6r8kREREQkjylA/48iIyMpXry46bW/vz9XrlyxYEUiIiIi8igpQD8ERqMx22uDwWChSkRERETkUbOxdAGPOz8/PyIiIkyvr1y5gp+f3z2P3bd+vulr36Bq+AWHPPL6RERERHITuv0YoduPWbqMx4YC9P+oXbt2TJs2ja5du7J7927c3d0pUqTIPY+t1bJPHlcnIiIikrsmDSrTpMGfK46NnbDYgtXkfwrQuejWrRtbt27l5s2bFC9enDFjxpCWlgZA//79adOmDWvWrCE4OBgnJyfmz5+fyxVFRERE5HGmAJ2LRYsW5XrMtGnT8qASEREREckPNIlQRERERMQMCtAiIiIiImZQgBYRERERMYMCtIiIiIiIGRSgReSJtP9WuqVLEBGRx5QCtIg8ccJC7Vl1wZrQ+DhLlyIiIo8hBWgRyXfS01K4FXWBxLuxD/3avk6uWBmsCQt1YNUFG0Lj44gvlPjQP0dERAourQMtIvmG0WjkSOgSDm1aQCE7F5KS7+AXHEKTru9i7+j60D7H1ynrWmGhsawimZpeD+3SIiLyBFCAFpF84/TeNZzavopW9T/E1bkIaekpHDz1I7/NH0m7QV889M+zMlg/9GuKiEjBpyEcIpJvHNmyhNoVX8bVuQgAtjaFqFWpB7evR3DzaphZ17oecZrfl37Ohm/HcGLXKtJSkx9FySIi8gRSgBaRfONOTCSFPQKztVkZrPByL8WdG1f+9nWObvuZtbP/je1tG3ysgzi/YyPLvxhAStLdh12yiIg8gRSgRcSijEYjZw/8xvIvBmJjXYjrt85mez8zM50bt87i5u3/t66XEHeLfeu+ptVTH1C5TDuCAxrSrNYw3O39ObRp4aO4BRERecIoQIuIRe1dM5eDa7+ngu8zVA5uy67D84i5cwmAlNR4dh76GowGNv3wEYl3Y3K93qWTO/EvUg1nx8KmNoPBQLmSLbhwZNsjuw8REXlyaBKhiFhMYtwtju9YToemE7Ev5IJ/0RBsbR3YsHMCRiMYjRkEFKtJ+2afcuzcr/y+9HNa9hln6bJFROQJpx5oEbGYa+EnKFK4LPaFXExtZUo2pX2zCaRnpND5mS94qvqr2Nk6Url0WyLO7CUt5cFrNpeoUJ8r0YeJT7xpajMajZwO30hg1UaP7F5EROTJoR5oEbEYOwdnEpPv5GhPSb1LIVsn7GwdTG22NoUwGKxIT0vFtpDjfa/p5OpFrVb9WLfhI8qUaIajvQeXru0lOfMujZu/+0juQ0REniwK0CJiMb6BVUhJj+fS1X2U8K0FQGZmBvtPLMKvSNVsx0ZGH8HFoyj2Tm65XrdKo84UK1WJ03vWcD3xAkFPPU2ZGk9ja2f/SO5DRESeLArQImIxVtY2tOozjjVzR3AuYisujkW4euModk4uxN68wrlLW/FyL0X0rdMcC1tFi5c+xGAw/K1rexcvi3fxso/4DkRE5EmkAC0iFuVdvCw9PlhM+PEdJN6NoUKJjhQJqEBk2CGOhv7ImROb8ChWimf7T8SneDlLlysiIqIALSKWZ2NbiOCQZtna/EtXx790dQtVJCIicn8K0CLyj6SnpRBxZj9pKQn4BlXD2d3H0iWJiIjkCQVoETFbZNghNnw3GjdnP+ztXPj95ykU9itN0t3bGDPTKVGxPiHNu+Pg7G7pUkVERB46BWgRMUtKUjy/fTOShiEDKOZdEYDk1Lus3TaOQP96+BcN4dzlrSyfMpDOw2ZRyNEllyuKiIg8XrSRioiY5cLRrRTxKmcKzwD2di5Ur/AC0bfO4OVekrpVeuHlXJITu1ZasFIREZFHQwFaRMySFH8bJ3uvHO3Ojt4kp941vS5ZrDaRZw7kZWlmuRIfS6YxnbaB6ZYuRUREHjMK0CJilqIlKxF54wiZmRnZ2iOuHcTHs7TpdVLKHewcnPO6vFxdTYjjSnwswU2SGdY3gZpeNjin3H9nQxERkf+mAC0iZikWWAVXHz+2HZxO7J3LJCbFcvzcr5y5uImKwa2BrDHRJy+uo1zdNhau9t6CmyTTNjCdJs6uCs8iImI2TSIUEbMYDAZa9RvPwY0LCN03jdSURNy9/ck0pnP47HJsrAsREXWA8vXaElCujqXLva+aXjaQYukqRETkcaQALSJms7EtRO3Wfanduq+pLfFuDOHHt5ORnkatrq/iVtjPghXeX6YxI/eDREREHkABWkQeCkcXTyrUa2fpMu7rakIcmcYM0/ANffsTEZF/Sj9BROShykhP48rZ/aQkxVMssAouHkUsXdJfwnMSbQMzaOLsquEbBUhmZibXomNxdXHE2dnB0uWIyBNAAVpEHppr4cdZP38kLo7e2BdyY/vPX1C2VmvqdxiEwWCwaG3BTZIZU8uAc4qrRevICzGxdzlx6jJ+vl4Elixq6XIeqQU/hjLyk4XEJySRnJxKp+fq8cUnr+Lm5mTp0kSkANMqHCJyX8kJdwg/sYOr5w/nWLbuv6WlJrNu3vvUrdiLlvXeo3H1QXRsPonIkwc4s29tHlX8ZMvMzGT46G8ICnmNEeO+p37Ld2n9/Ghu3Lxj6dIeiZVr9vD++B9YsPB9rl9fTnj4Imwc7enSZ4KlSxORAk4BWkTu6cCG71kwvhtH1i5h+5IvWDi+O9cjTt/3+PDj2/F0LYl/0WqmNjtbJ6qV6cjJHavyouQn3n+mr2D7ntOcPfsdO3dNIyJiCZWqBdO17yQgK2AfPHKefQfPkZ5u2cmUJ09fZvX6fYRfjv7H15j05XKmTHmDevWydsX09HRl1qxhnLsQxcEj5x9WqSIiOWgIh4jkcOHoNs7sXEO7xh/j6OABwKWr+1gzZwQ93l+EbaGc40yT4m/j7OCdo93Z0Zuk+NhHXvOTKjU1DRsba6ysrJg251dWrByPj0/Wv5mdnS2ffvoqgYHd+WbBRsb/5ydsbKyxtbXh9u14Znw2gGdb1srTem/FxNGt32ROnbtC5Uql2H/gLC2bhTBnyhvY29uZda0Tpy/TsGHlbG3W1tbUr1+RE6cuU71q0MMsXUTERD3QIpLD8d+XUbVMR1N4BijhWwsvt1JcOLbtnucULVmRyOtHyMjMvjV2xLWDFC1V6ZHW+yQK3X6Mp1q+i7NfFzxKdmPQOzO4HHGDKlUCsx1nY2NNhfIlGPreHKZMHcyp099y7Pg8Fix8n76Dp3L67JU8rbvPoClUrBpEePgi1q6bwKVLi0hMTee9cd+bfa2gUkU5cOBstjaj0ciBA2cJKlWwx36LiGUpQItIDolxMbg5F8vR7upYhMS4W/c8xyegPIWLl2br/i+JuXOJpOQ7nDr/GycvriekRc9HXfJ9XYmPJdOY/v9L1xUMe/af4cW+Exn2dheSktZx6tQ3xCen413YjS1bDmc7NjExmd17TtG27VM8+2xd02TORo2q0r9/W2Z9sy7P6r4ccYPd+8/y6aevYmNjDYCjoz1fThvM7G/W8f3izaSl/f1/p6Gvt2PIm9M4dy7rl4Dk5FTee28u7q5O1Ktd7pHcg4gIKECLyD14B5TlyvUj2dqMxkyu3jyOd/H7B5Nneo+mWOUQth6ezsqt73HTeIl2gz7Hs2jJR1xxTlcT4rgSH0twk2SG9U2gppdNgdm2e+KUZYwe3ZsXXmiCjY01fn7ezP/mXaytrejbZyLbtx/DaDRy+XI03bqOw6+YJ40aVclxnZCQYMIvX8+zuq9eu0WpkkVzDNXw8/PGxsaar+av49kuY0lJSbvn+SkpaRw8cp7zF6MA6NGlCa++/Az1679B5Up98ffvwpEDZ1n+w3sWX/VFRAo2jYEWkRxCmvdgxbTBFLJ1JtC/PsmpcRw+sxwHdw/8gkPue561jR01W/aiZsteeVjt/QU3SaNtYHqBW/f54NHzTPpiYLY2a2trOnRsSELsXfr2mUDUtVhsba3p2+NpGtUpz5bNh3jtteeynbN582EqVyiRrc1oNAI8kgBavkxxzoVFEhV1i2LFvEzte/acwsvLjd+3T6FN638zf8FGXu/bOtu5837YwHtjv8fHx52bt+IIKlmUb74awrCB7Xm9TytOn7uCt5cb/n6FTees3XCAr7//jRu34qhfuxyDX3sO3798rojIP6UeaBHJwbNoSZ7rP5nIxOMsWvs6q38fjVNAUZ59bQIGg4HEu7Gc2b+Oswc2kJJ419LlPnH8fQtz4kR4jvYTxy/SqV09Tu+dQcTxeUSf/Z5J4/rQt2cLduw4zqRJS0hMTCYlJZWZM1ey9KetvN6nFZC1KkbHnuOxL9IZt4CuvDpkGjdvxT3Uut3cnBjYrw0dO3zIgQNnyczMZMuWQ/ToMZ6RI1/C1taWgYPas2LNnmzn/bb5EGMnLWHjpskcOz6PiIgldO7ShFbPjyYtLR0Hh0KEVAnKFp4//uxHhrw3h2c7NGD0uD4kpGdSp8XbXLx07aHek4g8mdQDLSL35F28LM+9PjlH+9FtP7Nv3dcU86mE0ZjJ7z9/zlMdBlOudut7XMVyMo0ZgGWXanvYkpNTWbLsd06evsSQIdOpXr0M/v7eGI1G5sxZTUTEddo8XRODwZBtIxF3N2c2rRjHsPfmMmrUNxgMUK92Odb/PBrfYl5EXLlB8/YfMnxEN75f9CEJCcmMH7+A5u0/YN/mz7Czs31o9zD2ve54zXShdavh3LwVR4UKJRg7tg/duzcH/uwB/6spM1cyfnw/KlfOmiBpY2PNsGHP88vy3/l1/T5CqgTy7aLNRN+4TZ0aZWhUvxKTp/3CyZPzTT3dzZpVx9PThXETlzBv+pCHdj8i8mRSgBaRv+1a+HEObfyB5xqNw9kxq7fvzt0o1q8cj09AeYuMdf5vf2zbDUbaBqZT08vmsR++kZGRwahPFjF97mrS0zNYs+YTdu48QeXK/ahUqSQXLkTh6uzAmh9HYWt772/rwYG+rFo8koSEZDIzM3Fx+XM8+Jezf6V7j+YMG/Y8AC4ujkyd+gbNmv6LZat20bVzo4d2L1ZWVgwb2J5G9SvSoed4fv99Ch4eLqb7nPblL7zYvn62cy6ER1O9eukc16peowwr1+7htaHT6d69ORWqBrFo+XY+/OgHmjcLyTZMBKBfv9bUqjngod2LiDy5FKBF5G87vXsN5Us+YwrPAG4uxQgu3pgz+9ZSr61lw8kf4Tm4SRJtAzMKzNjnD8cvYNeBs/Ts2QJ7ezsaNapKo0ZVef31duzff4aFCzfh5mBHuTL+uV7Lyck+R9v+w2G8P/LlbG0Gg4E2z9Zh36FzDzVA/6FGtWDq1ypPpUp9GTq0M66uTsyYsRJ3F0d6/39v9B8qlivOtm1HqVChpKnNaDSyNfQw4eHRrP9tIrVqZU1ufeONjrRpM4Kb99h98e7dRBwc/t5a06mpadyKuYuXp8tD7YEXkYJBY6BF5L6SE+5wK+o8aSmJQNZmKU4OOSdhOTt4kRyfP7aLzpo4+P/hOR+5FRPH0H/PIaBSX/wr9GHg2zO4Fp37BjMJCcnM+mYdPyx4n7i4RCpWLGl6z93dmRYtavDMMzWJvHrv5QX/Dr9iXpw6dTlH+8mTl/At6pmj3Wg08vvOE4ybtJgvZ/9K9PX730dqahobQw/z67p93LmTYGo/dPQ8W7Yf5c03OxIWdpXt249RrlwAkVdv8t+jOPq+9DQjR85n3bq9GI1G7tyJ5+23Z3LnTgLVqgWZwjNkBf///GcAe/aeZs+eU6b2zMxMPh6/gK6dGj7wWWRmZjJ24mL8K/ahepNh+Ffsw9iJi8nMzHzgeSLyZFEPtIjkkJaSxLal/yH8+HYcHT1JSrpNpQYdKRpclcuH91PSr47pWKPRyKXoA5Rr/KwFK87fIqNu0bD1CFo8XYONmyZjY2PNtGm/0OjZf7Nv02fZxivf61xPDxf8/b2pXr00GzYcoE+f7OPNf/ttP9UqB97nCrl7rVdLevT/jJYta1KmTHEANm06yKpVu/h017Rsx6alpdO17ySOn75Mx04NOX/8IqM/XcjXX75Jh2frZjt287YjvNT/cwICfHBxceTlAZ/z0fs9GfhKG6bMWMXw4d14++0Xs53TovlbLP81a9hIXFwiA9+ewZoNB3BxceTFF8eCETKNRto8XYMP3urCj6t25bgfDw9XrKwMtGkzgo4dG1CqVDFW/LIDG2srZkx47YHPYtQnCwndeYLdu6cTFORHWFgkvXt9Snp6BmPf6/FPHq+IFEAK0CKSw5bFEzDeTqVTi/9gZ+tIQtItth38ioCQp4hNuMLe4z9QrmRzMjMzOXlxHakkEhzSzNJl50sLf9rKq0OmUadOOWbPfsu0PNznnw8i6uot5i3YyLCB7e97frEiHtyKucu1azH07t2Kzz9fyscfL2Dw4I4YDAZmzFjJb+v38enWL3Kt5fadeNLTMynslb13vmH9inzwVhfq1h1EtarBxMcnERl5kx/nv4uPt3u2Y6fNWU18cirHT8wzDW04ePAsLVq8TeOnKuHh7gzAjZt3eLHvJJYsGUnz5tUBOH8+kqZN/0XF8gGcPBvBwKGdc9T4VIPKnDwTAUDvQV/g4e3O5cuLcXFxJDw8ig7tP6Rbp4a8O6Qzd+4k8K8P5hEWFklwsJ/pGrNmraJz2/pMGNObBT+FsmPXSaKjY7gaFUNgyGu81qslo4Z3zTE0IzExha++XsORI3MpXtwHgOBgPxYu+oCQkNcYMfR5HB0L5fqcRaTg0xAOEckmPvY6V87up27l3tjZZk00c3Lwol7lvpzcuYL2b0zFpogTG/ZOZvPBL3AKKEr7QVOwsVWw+G9hF64y9L25tG5dm27dmudYW7ljp4bs2H3qPmdncXFx5OWuTenbZyJpaemEhn7O7t0n8fRsj4dHO3ZsPczmlePxLux232ucvxhFmxfGEFCpH8HVX6Nui7fZtfd0tmNe692K8CNzeXtgez75sCcXj8yhacOcm68s+CmUDz7smS18Vq9ehubNqvPL6t2mth9+DKXtc/VM4RkgKMiPESO6M2v+OoJLFWPvf9UAsH/faUoH+nL+YhQ7955mxoyhpgmPJUsWY/43w/nq6zUYjUbc3Jz4dNTLNGkyjMmTl7BixQ5eeWUSc+esZsx73SlaxIPgwGLsPxLG3K/fITX1N3bvns6RU5foP3R6js+OiLyBl6erKTz/ISCgCB7uzkRG/fNhMiJSsKgHWkSyiYuJws3VDxub7IHY3dWPtJQkbAs50LDzUBp2HmqhCnOTf5au+35JKC+99DS2tjZcuhSd4/2LF6OIu5tIv8Ff4u7myEsvNr3nUIyJY3rz9ofzKVPmZfx8vYi8eotXez3DpLF9cHTMOSnwr+Ljk2je/gPeGNyJZSs/ws7Ohp9+2kr7Hh+xc/1EggN9Tce6uDjS+ukaD75eQjKenjnHl3t4uhCfkGx6HXUthrLliuc4rly54ixZuIlPRr5M516fUrNmGerVq0h6egYzZ67k+PGL/PT1O/y+6yRVKgdSqFD2SX/VqgUTefUWaWnp2NnZ8lrvVlSpWJK5328gdONBatcozYHQz02/UHw0+UdmzfoXTz9dE8jqUf5p6WgCAroSfjmakgFFTNcuVsSTGzfvcOPGbbz/0vN+/Xost2LuUtQne2+8iDy51AMtItm4eftz+84V0tKSsrXH3LlMIQdnbO0cLFTZg2WtwJEOGLOWrssHbsbEUaJEEXr3bsmcOas5deqS6b2LF6OYNGkJMXcSqP1URVy83GjTZSzTZv+a4zp2drZMnfAaFw7PZuHst7hweDbTJw/INTwDLFy6lerVy/DOOy/i4FAIa2trunZtxquvPse0OavNvqenm1Tju+9+y9YWF5fAypU7eaZpNVNbreqlWbN6T451nVev3kOtkNLUq12OLye8xotdxlI6uCf+/l1Y+P0G1v88BgeHQpQN9uPQ4TCSkrIvo7J372lKBPhkW66vbq1yzJ06mFWLP+TDd7pm640/dOSCKTz/wdHRnrp1ynP0eHi2dldXR3p0acwr/SYRG5u1QVBs7F1efWUyPbs0ybb0n4g82fLHTxkRyTecXL0oWfEpdhyZQ51KvXCwd+PO3Sh2HplEUOppAAAgAElEQVRLtWbdMFjlv9+7r8THAkaG9U2gppcNzin5I+g0rFuBL2b/yptvduKzz16nfv3BNGxYmbS0DEJDD1O8uDf7D8w0De3o27cVISGv0fG5evj55lztxN3NGXc3Z7NqOHE6gkaNcw7FaNy4KpMnLDL7noYP6cxTrYaTnJxKt27NiIq6xfiPFtC5XT3Klv5zGb0Oz9blk8+XMmjQFN5/vwcuLo7MnbuGBT9sZM/GrA16OrerT/s2dTh99gpOToUoVaKo6fwSAT483aQafXpPYOqXg/Hx8eDkyXBeeuljWjWvwUv9/8OGLYdxcXGgZ5cmjBj6PPb2OZeoK+6ftWtjtWrBprbMzExOnrxEcf/COY7/bFw/3hwxm8DAHpQqWZSL4dfo0rEBk8b2MftZiUjBlf9+EoqIxTXu8hauAf6s2DKCnzcOY93Oj7BxciDs4GY2L/qUm5HnLF2iydWEOIKbpOW78AzQqW09wsIi6dnzY6pWDWLDholkZGRy8OBZevRoQbNmIdnGRZcoUZR2beuzcu2eB1z1/jIzM0lKSsnW6xtYsggH9p/Ncez+/WcILFk0R3tu/P0Ks+u3idgZYfCgKXz15XKG9n+OLyf0z3acra0NG5aPJSMplbJle+Hl1YGRI+dTqJAN6zYdMNVoY2NNpQolsoXnP8ydOhhneztKluxO4cIdaNhwCEWLejJ/wQYqVy/NwUOz+XnZWI6ejqDTSx/fcxfDQa88y+A3phITk7UteXp6BmPGfEdRH/ccw2XS0tJZtW4v9oXs6N+7JaPefZHNKz6icf2KbNt5nPT0/DM8SEQsSwH6b1i3bh3lypWjdOnSTJgwIcf7oaGhuLm5ERISQkhICB999JEFqhS5P6PRyLXw4+xc8RU7V0wn6uKxe4aNP9jYFqLh88N4efTP1G3/OmDAz7USVQPa45TkyqoZb3HpZM7lwyzNUuH54qVr9Bv8JcUr9qFc7QGMnbiY5ORU7OxsqVOjDHFxiXTqNIrOnUdTqlRRDh+ew7lzV/Dz885xLStrKzIyzFtzODU1jRGjv8Wn9Et4lOxOlafeZOmKHQC89GJTNm48yMKFm0xrGW/bdoSpU5cxsO8/237dt5gXk8b1Yd/m/7Bh+Th6vtg0xwRJAC9PV0qVKELpYD927vySu3dXs+THUUybu4YpM1fm+jmOjoUIKlmUZk2rsW/fDK5d+5lKlUoxbNgLjBjRHX9/b6pVC+anpaO5ePk623aeIDMzk1Vr99Jv8Je8OmQa5csUp2bVQIKCetK0yTBKlerGlo0H+Omb4dlqvns3kSbPvccXs1ZRorQfCekZ9BrwBc3af8Dydfv48JOFlKn5OkeOXfxHz0xEChYN4chFRkYGb7zxBhs3bsTPz49atWrRrl07ypcvn+24xo0bs3Jl7j8QRPKa0Whk+7KphB/dTpB/AwzApm/HEVCxLg2fH3bP4PMHGzt7Dvz2PU9VfQX/olnjW4sWLoeXWym2/zyFgHJ18smQDsv1DF6JvMlTLYfTv39bto2ZQmxsPB+N+54OPcazdulohg1szytDprF27adUqFCSzMxM5s9fx/HjFwko7oPRaDT9G1y7FsPKlTsZOfR5s2p49c1pxMYnsX//TEqWLMrmzYfo3ftTbG2tad+mLqt/HEnfN6YyYsRsHOwLkZKSypwv3qBKpVKP4pGYJCWlMHnacvbu/YrA/5+sWLduBX5aOprGjYYysF+bXHf5W7pyB19MHUypUsUA2LHjOPPmvZvtGBsba9q2q8/vu04wc95aTodF8sqrz5KZaWTE2G+pFVKaU3umc+z/N4apUC4gx+d88vlSSgX58sOC9zEYDEybtpzy5Uuwfv0EXF2z1uletGgT7bp/RNjBWffdMl1Engz6DpCLvXv3EhwcTMmSJQHo2rUrK1asyBGgH9SbJ5KXjEYjV88f5sKRrWAw4OJZlEvHdvJco3HY2WZNACwf2JI1O8Zy5ex+ipetdd9rxd++TkpCHH5FqmZrL1q4PJnp6dy+eQUPn5xhJK/8Mfa5bWB61sTBPN62Ozk5lTZdxvBClyaMHtPb1P7T0tFUq/oKm7cdpUWTanzwVhcaNx5KiYAi3LoVh7ubE6sWf8iAf31F+3Yf0K17M6KiYpg6dRn/GtieEgE+9//Q/3Lx0jXWbjrIpUuLTJMKmzevzqxZ/2LUh/Np36Yu1asGcWjbF5w5F0lqahoVywdgbW39sB/HPWqLxsvT1RSe/1CuXAAOjoW4cvVWrsNIMjIys4VsHx8PwsOvUaNGmWzHXQq/hqeLI2cvRLFn71em1TteeaUNNWu8zuFjF2n5lyX1/ttPK3bw87Kxpl9m5sxZzZdfDjaFZ4Bu3Zrz1fQVrN90iOda/fnfTWJiCouXbePkmQiCShal+/ONH7g5jog8/vJD11G+FhkZSfHify7F5O/vT2RkZLZjDAYDO3fupGrVqrRp04aTJ0/mdZkiQFZ43rpkEqELJmB32xa7WBuOblqCg60rtjZ/rthga+tAmYCmnDuw8YHXs7axIyMzjUxj9h5eozGT9PQUi639fDUhjivxsQQ3Sbbo2Ofho78l9k4CXbs2zdae1SP6FC07jaRBq+EE+Htz6ejXTP3kVVYu/IADoZ9Tr1Y5dqybSKsmVfl58RZOHjnP9zOG8e9/vWBWDUeOh1OvbvkcK3K0aFGDQ0fOm14bDAbKlfGnSqVSeRKeAYr4uBN9PZa4uIRs7bdu3eH27XgK32M5vP/WtlVtZs5Yaeqk6NevNaNHf8utW39uHb9lyyE2bzlM7O14Bgxsl23pO0dHe1559VmW/Zp9yJHRaGTH7pN8t3gzBw6HkZqanm2TlOjoWIKCsgd/gKAgX6Jv/Ll1+YXwa1Sq/wa/rNtHkeI+bN1zigp1B3H0uIZ6iBRk6oHOxYP+vP2H6tWrExERgaOjI2vXrqVDhw6cPZtz0o7Io3b59B6izh7h2YZjsf3/dZzLlmrBr6EfEHXjOL4+lU3H2ljbkZmR/sDrObp4UNi3NGcubqRCUCtT+7lLW3Hz9sfFo8gDzn60gpuk0TbISE1Py4Tn5ORUvlu8mRo1yhARcSPH+5cvRzN58gACAnzoNfALvpsxlKebhmQ7xsnJngH92jCgX5t/XEeAvzcnTlwiMzMTq78Mpzl2LGuIiCV5ebryXMtaDBkynZkzh1KokB3JyakMeXMandvVx9U193+3YQPb0+S59+jcaRTdezTnxo3bXL8eS2BgD1q3qs3Nm3c4duwiS+a9w7wfNt7ze7aVVfa2a9GxdOgxnrj4JGrUKMPoTxdhY23FrJmrmPzZAADq1avAL7/sYNCgDqbzkpNTWf/bft4Z0M7U9sY7s3h9QDuGD+9maps/fy393vySvZs++1s/Q0Tk8aMAnQs/Pz8iIiJMryMiIvD39892jIuLi+nr1q1bM3DgQGJiYvD09Mx23L71801f+wZVwy84+w9Tkf/V+UNbKBPQ1BSeAWxt7Clb6mnCI/eYAnRmZgZhV36nWuvuuV6zSdd3WPnVMK7FnMbbLYibcReJiQun7YDPHtl9PA5u30nAzs6GAQPaMWbMdzz9dA3TBiO7d59kzZo9fPHFILy93TEajXw0+UdTgM7MzGT63DXM/mY9UdEx1KlRhvff6kL9OuUf9JH3FFIlkCLebowe/S0jR76MjY01N2/e4c3BUxn0yj8P5g/LV5Nf5+UBnxMQ0JVqVYM4dDiMxvUrMXP6kL91voe7M9vXfsr8hZv4fv463Fwc+X7mMMqV9id0+3FcXBxo1bw69vZ2xMTGM2HqMl5++RnTsI+kpBTmzlnNxNG9Tdfs+8YUmj1dg48/fgWDwUBGRgY9e37CnLmrib4eS9euzShXLoDhw2djZ2dDp04NuXz5Ov8eMYdmDSubxlDH3o5nx55TLFuZfeJ4r14t+fDD+YRdiKL0PXqxRfKj0O3HCN1+zNJlPDYMRg3efaD09HTKli3Lpk2b8PX1pXbt2ixatCjbGOjo6Gh8fHwwGAzs3buXLl26EB4enu06BoOBAZ9tzePq5Umz8fuP8DL6U6Zk9iEFZy5u4ljYr9Qo3wUMBs5e3oyduyut+33MtfBjRF86iYOzB0FVm2Bnn3PsZlpKEqf3rSX8+A5s7ApRvs5zlKhQz2K9a1cT4ghsnEzboHRqehryvAc6IyODVev28eqbX7Jh42R+/DGU2bNX07p1bWJi4tiy5TDff/9vnn++MQCJicl4eLQjOXoZAENGzOHA0fNMnPQ6pUv7sXr1boYPn83Sb0bQsH5Fs+uJuhZDt1cmcz78GsFBvhw+cp5XXn6GCaN7ZeuVtqSwC1c5dz6KsqX9/tHyeX9HRkYG3fpN5mLEdV597TkyMjKZNXMV1SqWZP5XQzAYDFyJvEn1JsO4cmVJtqEeMTFxBAS8SOPG1UhPz6BwYTfq16/I2rV72br1CLa2Nrw1qD3vvNnJNIEw+nosFeoO4ubNX3I85zKlX2LpN8OpXLHkI7lXkUfNyqOd5nc9gHqgc2FjY8O0adNo2bIlGRkZ9OvXj/LlyzNr1iwA+vfvz9KlS5kxYwY2NjY4OjqyePFiC1ctT6pSVRpwcPV3BAc0xMoq6z/vjMx0zl7eQmBIIy7fOgxGIxWatqdU5Uasnv0uCbeu4+ddhaikw+xeNYtWfT+iWGD2jTeuhR9n39p5+PpUxhlPdi37iqNbf6LNq59abBy0pcTejqdlp1FY2VhTt14FOnYcyQ8/vMfrr7dl/vx1zJ9/lMGDO5rCM8CZMxH4FsvaGOVK5E1++HELFy4sxN09a1OUPn1aY2trw5iJi9j4S85lMNPS0h+46kOxop6E/voxp85EEBUdQ5WKpSjslfv44rwUHOibbdvwR8Ha2ppFX7/NyrV7WbF6DwYDfPReD9o8U8P0y96NW3coVtQzxxbhHh4uWFlZkZmRyfr1E03tgwZ1YMiQaThaW/HeW12ynePj7U6pEkVYvnw7nTs3MrX//vtRUlPTqHCPrcxFpGBQD3QeUQ+05IXMzAzWff0+CTduULZEczBmcurCbyQm38bFqyjPDZiMveP/DzP4dRax587ToPoArAxZvWeR0UfZdXwePT/8EWubrD+Bp6Um88PYF2hc/Q2KFC6X9TnGTH4/MJ3C5cpRu80reX6fph7owJQ8n0D4+r++AlsbZs7MWgJw/vy1jB79LRERN/D388JoNDJk6Au89dYLGAwG4uISaPvc+7RoWIkP3unKslU7+e6nraxYOT7bdePjkyhcuANJ134Gsia5zZq/jsnTlhN+6ToBxb15a1AHBr7SRuNq/wdJSSkEVO7Hrl3TKP2XnRN37jxO924fYV/IlvYdGzJ8eFfs7e2YO3cNH49fwL7Nn+Hvl3Pnwt93nqBzr08YNKgjjRpVZv/+s3z22Y/M/nwQ7drUyctbE3mo1AP9YPnjb3si8lBYWVnTss84kjPucvrCBq5EH6ZicBs6P/05nvbF2bH8S9OxZ/atp0qZDqbwDOBXpArODt5Ehh00tUWc2YuHWwlTeAawMlhRuXR7zh7YkDc39hdXE+LINKYD6Xkeno1GI4uWbmXUqJdNIbZPn9ZcurSYKpVL8cOst9i6+hN++G495cr1onmztyhW7HnOhV3hP1+tYOzExRT2ciU8PDrHD6bw8Gv4eLubXn/+1QpmzF/HwkUfkp6+gZ+WjubrBRuZOGVZnt1vQeTgUIj333qBdm3fZ/36fdy6dYfly3+na9dxjP/gJTavHE/UpWj8/Lrg5taWjev2smnFuHuGZ4CG9SuybfWnXI+8wdhR33D62EXW/jRK4VmkgNMQDpEC5vaNCEjPpE2zkRj+Eo6rlu3Iso1v0ST9Haxt7EhLTcLeLuef+QsVciE1+c9lx9JSkihkm3NcdCE7Z9JSkh7NTdzHH+s+W2rpOqPRSGJiCh4eLjne8/R0JSExmVIlinJw6xc0aDUcG1trDh6cRdmyAUREXKdTx5G4ujiSmpLK11+v4ZVXngWyVnd4951Z9O3ZAoCUlDQmfPEzv2+fQtmyWRPWatYsy9KfR1OnziDe7P8cDg5P1tCZh2nogPZ4e7nx/r/ncCH8GhXKBjBtQn/atq4NwLczhzH/qyEYjca/teRfuTL+TJ/0+qMuW0TyEQVokQImNTkBe3vXbOEZwM7WCYxGMtLTsLaxwz+4Ohev7KR8UEvTMUkpcUTfOEWzwPdMbX7BIWxfNoXklDjsC/0ZuC9EbH/gJiwP29WEuP9fui7dYkvXWVlZ8XTTEL7/fgOvvfacqf3ixSgOHwmjfu2sycVnwyIJj7jOpe1TTWOXixf3Yeasf9G500jW/DiK9j3GM3vWr5Qu7cemzYdo3qgK/x6WtQNhROQNnJzsTeH5D0FBfnh6uHAp4gblymRfDUjM06NLE3p0aXLf9x/15MuY2LvMnLeWrTtO4OHuRO/uzWnVosYj/UwReXgUoEUKmMJ+pbkbf524+ChcnYuZ2q9EHcTdO8C0ykatNv1Y+dUwUtMT8S8Swt2E6xw99wuVG3bG0dXLdJ6zuw+VnurEb7s+pXJwO5wdvbl87QAXru6gw+BpeX5/ljb+w560en40kZE3ad26NqdPX2bsmO8YPaK7aV3j8xevUaVyYI6Jf9Wrl+ZyxA3KBPtyas90Nm87SlR0LO8P6Uz5sn9OOPP2ciMm9i4xMXGmpfEAbt+O5/qN2/h4u+XNzcojEX09lgatR9CgQWWGvv0CV6/eYsi/59Jt/xlGj8h9aUkRsTxNIswjmkQoeenY9uUc3rCQkLKd8XAL4NqNkxwNW8nTL4/Cv8yfvVy3b1zh8OZFRF88joOLBxUbtCewSpMck9SMRiMXjm3j1I5VJMXfpmhQZao1eREXz0ezHNm9ZE0cTDBt2/0we6AvR9xg0c/buBufRPPGVWjSoPIDJ+qdDYvk8xkrOHjkAh7uToRUCqRF02o0aVAJa2trLl66Rp0W73D58uJsQy1+//0or/abzKm9X+VaU7/BX5Kamcncr9/B3j5rA5LX+/8HY2o638wY+lDuWyxj6L/ngJ0tU6a8YWq7ceM25cr14sCWz83ayl3kUdEkwgdTgM4jCtCS18JP7uTY1p+5G3MNL98gQpp3wyfA/I068oNHOfb5hyVbGPreXLp0aYq3txtLf9pKmaBi/Dh/+AOXjktPz+D1f33FyrV7ada0GufPX+XOnQR++eE9KpQLoFu/SRitrZj65WCKFPHkyJHzvNhlDMOHdKZ39+a51pWQkEyfQVPYuvM41aoGceToBZ6qXZ5vvhqCi0veD1+Rh6dMzddZ/ss4KlUqla29d69PqVstmP59Wt3nTJG8owD9YBrCIVJAlaxQn5IV6ud6nDEzk9jrlzEYDLj7BOS7JdIeZXi+Fh3LmyPmsHPnl5QvXwKADz7oydMt3mHcpCWMfa/Hfc/9bNovXIy4QXj4QpycHACYN28tHXp+zKk905k37U3eHjmfsmV7YW9vh421Fe+/1eVvhWfI2ub7x2+Gc/HSNc6GXaV0kO8j24BE8paNjRWJick52hMTk7Gz049lkceBeqDziHqgxdIy0lM5f3Qr1y+dxtndmzI1nyE2OpzQJZPJTEvDaMzE1t6Rpl2HU7RUJUuXa3IlPpZh/ZIeyY6DX81dw96j5/n2u39na9+wYT8vvjiOT0e9zKu9Wt7z3NI1+rN4yUhq1iybrb1G9deYOLoXzRpVBSAxMYXbd+Lx8XbHxib3FR3yI2N81P98DcNfxuM/yW7fiad87YHUrlOeX34ZZ5qsePJkOE899SbnDszEyzN/bYIjTyb1QD+YftUVeQIkxd9mxfQh2Buc8S1ciRtXTnJww/dkGjNpVH0Avj5ZOw9GRB1g7dfv0eWdeTi53Xvd24IkMSkFVzfnHO1ubk4UK+bJiDHf0rJZdQKKe3PnToLpPYDrN24TFJRzZ72gIF+ir982vXZ0LISj4+O75JwxPgpjagrpuw7942vY1AuB+CiFaGD6nDU0bRrC1ahb1Kv3Bs8/34grV27y9ddrGDagncKzyGNCAVrkCbB71SyKupSlZsXupiEaUb512brvS4p5VzS1BfjWJOrWSU7tXk3Nlr0sWXKeaNW8Oi2fH8XHH/fLNq54/vx1tG1bj5iYu0yb8ysHjpxn74GzGI1GSpUowpv921K3VjlWrNhB795/jleNj09i85bDfPJ+T0vczj923x7mtFSMRiM3x/9G5O3Af3z9ihwyhej7eVLC9W9bDjFyTG+aNKnGr7/uYuvWI3h5udK7d0syMtXbJ/K4UIAWKeCMRiNhRzbTsdmkbOObi3lXxMmhMNdvnaWodwVTu6drCW7diLBEqXmuUoUSdHyuLrVqDWD06F74+LizYMEmtm07yo4dU5kwYTEzZ61i4qTXWbdhEkajkblzV/PW8Dn4+LjzzjuzMBqNtG//FOfPX+Xdd2bR8bl6j9VYZWPsJQAyDhzP8V5mSiq3N9/4n8IzwIllLvht/g33Zt5YFbLL8b51jUoYn5AeaidHe2Jj72JjY02HDg3o0KEBAG+8MRWnx/gvFSJPGgVokSdAZkY6NjY5fzhbW9uSkZmere3G7TAKV8wfq3VcTYjL+sKYwaP6dvXlhP483+tTPvxgHkWLedG8eQi7dk3D3t6OBQs28vQzNRkwoJ3p+EGDOnLgwDnCw6+RnpzKjws3M3TodIr4uPPqyy0ZNrDdAz4t/zDGR0FaKmk7D3J78437Hve/huds19l84d5vro2k8PvPQOwlDB4lHsrn5Vfdn2/ExAmLefbZujg62gNw/nwkixdvZs/GyRauTkT+LgVokQLOYDBQvEwtzl0KpULQn8MN4uKjiL1zmYTEm2RmZmDEyPnL24i8cZQGtf9lwYqzXE2II9OYTnCT5Ee6bbfBYGDOlDdo2HoEJQJ8qFevIhs27GfSxCV4uDnx/PONc5zTtGk11qzZw9q1ezl3YBaFvfLnuNU/QvL9/BGeH1ZIzs0DP2d8Vg+17X8vHGNrV6B6pru/0JjNvx+jQoU+dOnShLi4RH76KZSPP3zpsfrLhciTTqtw5BGtwiGWFHMtnJXTh1DKtx6+3pW5c/cqJy6uoXy957hy5gCx0RcxAoV9S9PohWF4Fi2V6zUfpb+G5zG1Hv7qG/dy+048M+et47cth3B2sqfHC004fOwCqQYD//nPwGzHDhs2HWdnB779dj1bVo7Pl8Hn7/Qw51Vw/rv83LP3UGcF6uoFLkQD7Dt4jrUbD+DoUIgX2j+lzVMk39EqHA+mAJ1HFKDF0uJjr3Ns+89cDz+Nk3thKjboQLFSlYGsVToMBgP2Tvlji+hHuXSdOS5eukad5m/zw4L3eeaZmgCsW7eXl1/+lNmz32LY0GmEHZyFtXX+Wp7urytnnFjmYuly/icVO93Fpl4IBrv7jw8uaOFaJD9QgH4wDeEQeUI4e/hQr+2Ae77n4Oyex9U8HkqVKMriee/wUu8JGKwMFCpkC0D//m15882pTBjVK/+F59hLD2XljPzixDIXKnL/JfSepCXyjp0I50xYJGWCfKlSybJ/JRJ50ilAi0i+kjVx0PhIJw7+HUePX2TMhMVs2X4MVxcHKpYL4FbsXa5Fx7J/90m+njqYp5uGWKy+eylo4fkPD+pFNy2RV4AnIN65k0DXfpM4ceYytWqWZf+Bs5QL9mPxvHfxcM+5jrmIPHpWli5AROQPeTVxMDcnT1/m6U4jafpMTc6e/Y5Vv36Mrb0dRX08uHhkLmuXjrZ4eDbGR2UF5r/8L23nwQIXnnNzYpkLN8f/htFoNC3JV9C88e4sAgKLcfHiIpYtH8eFCwsJKhvAwLdmWLo0kSeWxkDnEY2BFnmwP8LzsL4JFg3PAL0HfEH5qoGMGNHd1JaWlk7ZMi+zcM5b1Pmv7bvz2oMmCD5J4fmv/NwvPFaTDm/fiWf3vjO4uTpRu0ZpFv60jfkLNxITG0+DeuV5541OlAjwIfZ2PIHVXuXSpUW4/WXXzLt3Eyle/EXCDs7S7oXySGgM9INpCIeI5AuZxgyG9UumpqdlwzPAzn2n+feol7O12dra0LpNHXbtO23xAA1ZG5/k5RJ0+d0f60y7cxDb+tXz9cYsk6Yu45PPl1I9JJjo6Fhu3rxD0aKejBnbB39/b376aSv1W77LtjWfYDQa8XB3zhaeAVxcHCns5crNW3EK0CIWoAAtIvJfPN2dOX/+KuXKBWRrP3XqEk+FBFuoKslN5O1AIpdBRQ7m28mFy3/dxdc/bOTo0bkUL+5DWFgktWsPZOu2L0whuUaNMtjb2/HJ50uZ8dkAkpNTOXHiIhUr/jlx8NSpS9y9m0TJgCKWuhWRJ5rGQIuIAAkJyURdiyEzMxMXZweGD59NbOxd0/u//rqL3btPUba0vwWrlL/jj3HRD9pExlJmzFvLuI/6Urx41rrPW7Ycol27+jl6mHv2bMGGLYextbXhg7dfpFPHUWzadJCkpBS2bDlEp44jee+tF0wrw4hI3lIPtIg80eLiEhn2/tcsXbEde3s7nJ0ciI9P5Lm29QkOfolmzaoRFRVDePg1mjatxi+rd1O9apCly5bH1JXIm1So8OdqIS4ujty8eSfHcTdv3sHVJWso08BX2uDq4sBbw6bzf+zdd1xW5fvA8c95BnvvvTcqiIC4995ZaWm/hqYNS60sK/umVjYsM9N2liPNkTs1J7g3TgRBhrL3eJjPOL8/UIxwNBAUz/v16vWKm3POc50nguvcz3Vfd/yldPx8nJk+ZST/N7pnk8UtkUjqk2agJRLJAyU3r5i9+8+SlJwJwKhnPgGlnJSUFeTmrmfFyhkIMhleXo6cOfM9Dz3UhbffHkNKygrkchmxZ5Pv8ApNQ1d9782u3otEVVZzh1BPu1Aftm8/Vvf14MEdOHIkjoMHz9eN1dSomT1rKWMf7V43NnZUD2L3fUFl9m+cObBASp4lkmYmzUBLJPexmqpy4o9tI/vyOQxNLV2pNW4AACAASURBVAjsMBgbZ9/mDuuek5dfwrrNh1ixdh+nz6XQNtSbhEvp+Hg6knIlhy3bP0ahqN0QJSoqiO+/f5WJE+fx+uujeeyxXgCcOJFAdPQZhg6IbM5bqev1LC0gvL2MYi8sDt17tdDTXhpB7xHvYGdnyejRPSguVhHW1pe+facxeFAULq52bNp4kOAAV6a+MLS5w5VIJLcgzUBLJPepitIC1nz2LOknj+GoF4CiWM6Wb17j4tHf//O1RVFEo65uES2Mlv26F/+I59l96AKePs4YGCjp2KkVaWkrCY8MQKlU1CXP13XtGkJRkYqgoKeZNu0bxo6dQ9++rxPg70rXjsHNch/X+z4/iL2e/626HtE11fdMj+g2rTzZtGIGP/+4FUPDAQQHP42vpz1xRxbRs2MwNiaG/LxoMuuWvYWenlTfLJHcq6QZaInkPnV8+0+4WLUhPPixujEPp0i2bpyNV5uu6Bveeve2WxFFkbgjW4jdtZzy0nwMDM1o3e1h2vZ4DEF2/z1vJ6dm88qMHzl8eGFdR43CwlI6dXqZjh2D+WTuRJYs3UF8/JV6HTdOnEjAyEifrl3boFTKCQpyw8TEkOg9sTz+cLfmuh2pdd2/kFHsBR/swHZGv+YOpU5URAC7N76PVqtFJpMhCAIAE57q38yRSSSSv+v++4sokUgASDm3H3+P3vXGzEwcsbP252rC8X91zfMHN3Bm50o6t57ImME/0ivyNVKORXN06/eNEXKTW7E2hscf71UvObayMuP110exZMkO9PSU+Pg4MXPmEqqqamuKL1/OYPLLC5nx6ijU5VUsWrSRuXNXo6msJnrLBxgbGzTX7UhaGLlcXpc8SySS+4s0Ay2R3K8EAbhZiYX4r/4o63RaTu1cRs/wqViZ1yaclmYudG03iU3Rb9K21xj0DU3ucJV7S3FJOY5ONg3GHR2tKSoqo6RERVJiBmbGBri6PoqtjQWZWQVMeW4IU14Yet8lNzVaDQB68qb71V5SU8XWK/FkqkoJsLKlt7Mv+k34+n+XKIr3VC20RCK5v0kz0BLJfcqrdVcupuyoN1ZclkFuwSVc/CL+8fWqVMXoNOq65Pk6IwMLTIztKMlP/0/xNocenVuzZnU0Wq223vivv+4hJMSbRx6exaiRXZnx2igEBAwM9OjcqRXzv9lMn+Hv8Ma7P7Pv4PkGteBFxSqSkjOpqVE35e3cUkppIS/GrKfrpm/ouukbpuzfRHp5w9Zot5JbqeKbC4d57cBmvjh74G+fe64wmxHbl3DoQiLq5DLWno5l1I7l5Faq/u2t3BUZxV73XC20RCK5v0kJtERynwrv/zTZxRfZe+ILktL2cTp+HTsOf0Tnhyb/q5liPUMTdKKWisqieuMaTTXl5XkYm9s2VuhNpn/vMKwtTBgx/H8cOHCO06eTeO65eaxZs48lS/6gfVsfZr/5OI8+/TErVs7gVOx3bPn9Q+Ljl5ByNZeruUU8O2URTz0/H51OR0lJOU9MnIdX6LP0GzkT99bjWfjdlma9x4KqCibE/IZngQFfil34QuyMXb6MZ/eupaym+o7nxxfn8tjOFSQnZuOfa0RecgFP7PqV43lXb3ueThSZcWQ7YzW+jNcFMkBwZ4q2DaFVVnwaG9NYt9do6pJoUZSSaIlE8p9JCbREcp8yMrXk4Ve/w6N9V/JIQ7DTY9ikL/APr10spdWoSTq9h8Obv+Hs/t+ousOsokKpT0DEAI6eX4paU1V7Da2a43ErcPYNw9jM+q7fU2OTy+VsWjmDju38eOnFL3hs1GxMlArOH/qS77+YxKFjFwntMhlzc+N6tc0ODla8//44SksrOHP2By4mZbBq3X4ef/ZTDM2MSUtbSXLKCnbv+ZRFP25l2a97m+0ef0s+R4jWij6CK3qCHANBwSA88Naasjkt7o7nf3RiLw9pPBkj+hIp2POI6M0zWn8+OL77tl1YEorz0Gl0hFH/waqf6MaBnNS6cpJ7yfUkWiKRSP6re69QTSKR/G1KfSNadRpOq07D641XqorZ9NUUlKIBTtatyEo+yck/fqb/Mx/g6NXmlteLGjKR6FWfsm7XK1hauFFcmoGDRzA9R799t2/lrjEw0OONKSN5Y8rIurEvv9vCgu+28NFHzxIS4s3u3bEMH/4/li2bTt++teUv3t5O5OQUYWioz9RXHmHxd1uIT8pg09aP6treBQd78vU3U5n80pc8MbpHs9xffEEOQTpL+Eu5tr/WgvjC3NueW1RdwWVVAS8TWG+8NdasVCeRUlaI1y0enKp1GgxouAhODxkioG0BLRAlEonkVqQEWiJpgQ5v/gZ7Ez/Cgx+vS3DSc86wa9l7jHnnV2Qy+U3Pkyv06DXmLVQleRTnXMHM2hEza6emDP2uKy+vYtbHKzlyZBG+vi4A+Pq64OxszRtvfE+fPuEIgsDmzYeIiqpNLE1MDCgurSAywr9Bz+gOHYJJSGy++nAnU3PS80uIwK7eeIasHA8TB4qqK/n2whF2pyeiRaSroxfPB0dhb2SKgIBWJ950KaoOEeGvWfmfBFnYUUg1aWIZ7sKNlomHyaaVuR2Ginu3h7G0oFAikfxXUgmHRNJCiDodaRePELP6U5Jid+NgE1RvdtDFPgQ9hTHZqedvc5VaJua2uPi1a3HJM8CZ8yl4eTrWJc/XDRoURWJiOsnJmXz66Sp++GErr7zyCKIo8vXXm4ho683xEwloNPUXJB45Eoevt3NT3kI9I71bs0+WRZJ4o0QnTizkqJBLf7cAnt27loIrRbyuacs7mnZo0yt4Zu8aSmqqyKlUoUDgAPW3uz5FPqJMwMPU8pavqydX8FpoNxbIz7GDK8SJhazjMuvkKUxt2/Wu3e9/JS0olEgkjUGagZZIWgCdTsvOJbMoykjF27kTrX2HcPzccnIK4utttKJUGKBV1zRjpM3PwtyYnNwidDodsj9tDlNYWIZarSEw8GlsbGp7RcfFpTFp0gIOH47juFyGn7czL774BXPnTsTMzJj4+Cu88Px8pr00otnux9PUincj+vDByd2Yikp0QJVMy0eRAzhbkIVptZwxOt+6h6mH8aZEU8P6lPNUazW0xZaNpJAsluKLOWmUcZQcHJSmN23jJ4oixTWV6MsVDHDzx8XEnFWJp4lX5WCir4eVyojx0WtxNDBlbEAYwz2C77l2gNc3V7HoaYuym540Ey2RSP4xKYGWSFqApNN7KM1KZ2Cn/yGX13507u/Zmy3R7+DuGIGtlQ+lqiwKi9M4s3c1+9fOx8LOldCeo3HyDm3m6JtWoL8r9rYWLFy4gZdffgioTQrfeOM7rK3MGDO2N6GhPvz66x62bTvOoEHtKSgoJTzcn/NnLlNWWIqb22hsrM0oK6vkzVce5qnHezXrPXVz8qKjgztxRbnIBAiytEcuyHjz8FZCtFYNEtgQrTX70pOJcHDFWFAwS4zkIFkkUIwdhozCl1i94gavsyk1jm/PH6FUXYUoQEc7d6aH9eD99v3ZnBbHV6cP8bjWFz/MSassY+nZ4xRXVfJ04D9vq3i3ZRR7YUFZc4fxr11Nz+Pt95ax/vcjAIwYFMWc//0fLs4N+55LJJLGJyXQEkkLcPnUXgLce9clzwD6esb4uHXh4uU/KChJ5VziJgRkuBq3wcZ1BHmFSez8eRadRr6MT2jzLIBrDoIgMG5Mb954ZzFr18YQEuJNdPQZTEwMUCgVODhYMmZMb8aMubHLo0pVSVlZBUdPJFCQvILKqmryC0rxcLNHX//eqPVVyuSEWNefST1dkEUoVg2OzaWS9NJiEssKkAH9cWOA4A6AVtQxX36Owe6t6o5X67S8enALJ/PSeY5gWmNNlahlc04qL+xbz7Leo/nm3BEmaIPwEswA8MeSSdpWvHfpJKN9Q+/pmuj7TXGJim6D32LsE31IWbgCgPnzf6Pb4Dc5FT0fc3PjZo5QImn5pARaImkBRFGHcJOFgYJMTp4qBbm1EQgivdpPxdbKFwALU2fMTRw5sOlrvNp0veXCwpbo8PEE5swZh4uLHamp2Tz0UBe6dw9l/vy1bNp0iFdfHVXv+IsXr9C2rQ+CICCXy7C2MsPayqyZor81nShyIi+d/KpysivKqKip5hDZdBQdcLu20C9XrGAXV5kktsFQlPORcIpZshN0F50wFBVEk0GRtprtqQlE2LriaWbF4ovHScjLYSTetBFqZzgNUfCI6M2cqlPsSE+kUqOuS56vsxEMsRYMSC0rItDSrkG8kn/np+W7iGwfyHvvPVM39sEH47iclMGSX/fw8sQhzRidRPJgkBJoiaQF8GzThfjoLbg7RSITaut6NZpqkjMO0mvMWxib27J54ZS65Pk6O2s/RI2WssIszG1cbnbpFkmn06Gnp2TYsE71xhUKOefPpZCcnImXV+0Cyj/+OM6OHSdwdLSmfbg/I8bO4XJqFsEBbrzywnA6dwhqjlto4IqqmJf3b0ShFlFrtGRTgRum9MWVucTiKpogIJBKGY/gXZfsjhC9iDUr5o+Sq3hiykN4IUPgt5LLPL5rBR7GlmRXlWGAHH8s6r2mIAj4aM3IqihFjY5SsQYzQa/u+2pRS4GuCit9wyZ9L1oynU7HD8t28taMsQ2+N2Bge3ZuPdoMUUkkDx6pC4dE0gL4teuL3NSAnUc+IvnqQRLTotl26H0cfUNw8g5Fz8CIGnUFWm39BYQabQ1qTSVK/QfrI9+hA9vz3bdb6m3FXVVVw4IF6+gQEUDbthOIjHyeVq2eYezYD/DxdmLlit1cupzB2Kf6sWnzHAYN68Sjz3zM5m3HmiZodQ266psvABVFkVcObKZ7lT2DNe6UUkMwVmRTgT+WfEJHjFFijIJP6EB34UbXEBdMyK4oo5vMideEtojAci4xFE8+oSMDy50p1dZgjyFpN6kZTpWp8DC1pI+zD6tkSWhEHVA7G75BSKG1lQP2RqYNzrtnqGsQVVl3Pu4esWbDQUrLKklIaLhTZEL8VRztb905RSKRNB5pBloiaQHkCiWDJnxC0uk9pJ49gEwup/3wCXgEdUQQBIxMrbB3DeJC0lba+N/YdOVC0u/YuwVjdJt2ZS3RiMFRfPfzH7RtO4FXXnkEnU7H119vwtXVjkPHL3Js12ecOnuZ/YfjkMsFWgd58L85v/DHjrmEhHgDEBDghoeHAy9PWsDg/hF3rdOEqMoCdQ3qQ6co3pNX20Himj0Zl/kp7hjJqkJMRCXdceZLzuGGKQFY4oop8znDaHwIxYa9ZGD0l1/75yhAXybHR22OiMg6knmWIAKE2p+JNtjgJprgiRnrScZNNMVVMEEniuwinWJFDV0dPDmSmcZ5XQGvchAv0YxUylDI5fwS8ThxRTksjz9FamkR7mYWjPEPo5WVwy3vOamkgGUJJ0koysPJ2JTRfm2JtHNt9Pf2wjpTgjmFokPb+6Yv9Kr1+3np5RF89tkaRo7sSmioDwCxsYl8/c0mDv3xSTNHKJE8GKQEWiJpIeQKJf7h/eq28v6r7o+9zuavXyWj4Dw25l7klySjEasY/PxnTRxp85PL5Xh62GPjYMnu3aeQyWTMmDGW4cM7M3Hi56zbcpjpUx9m9MjafsZnzqVgZWVWlzxf16tXGHkFpWRlF+LkeJe2Or9J8qzR6fj+4lF+u3SWJ0R/KrDlIFkIgoBKVOOFKcmU8hzBWKHPL1yigCpE4BcuMUz0xAAFR8hmJ1dxVVqQVlOGr2hOKTUNSjVG4s23XKA99nxKLCaiklJqMFbq8V23h7lQlMuhzFQ+ogOFVJFJBYPxYJUuiV+SYtmcfIH+OjciRA+Sy0uYnLuRdyJ6093Ju8Htrks5z7zT+xggujEaT66Wq3inYDvPte7ACK9WDY7/ry6sM8V5zw5s3u57XyTRlZU1BAS4sXDhy/Tu/RqtWnkiiiKnTyfRpUMQAX4PTimWRNKcpARaImnhRJ2O9MSTXE04jk/bnhiZW6OursTDrhtu/pHI5A/mr4H0jHxeePkhBg/uUG88JMSLC6eT6o2ZmxtRUFiKWq1BqbzxfqlUlVRXqzExvrs1vn9Onv+4msAnJ6Op1Kl5nTA8BTOKxGqWEU+FqMYXcyrRkEgx+8ikC450w4mDZLGCS1ShZRqH0CLihwXPEsTiinhyZCo8tKboEFGhxpQbtcyBWCLIBLKMqqlUadBXKhjp3ppJrTshF2T8mniGjloHDAUFzpiQLpazjATSdSrSE8/wIq0JFqxAAC/McNWa8GlsDF0dvZD9aeZ+S9pF5sfu4wn8iBJqZ6g9McNba86n5w4ywD0Ag7vw85pR7IXF4ViU3aIa/dqNrX/vMBb/uJUNG99n8OAoYmLOoFZreH3at0x5flhzhyeRPDAezL+cEskDQqtRs33x25TlZuFuH0m5Jovz6etoP6S2vONBFtLKk507TzZIoHf8cYLUlCw82oyjqFgFCDw8rBNeHvbMn/8b06bVdugQRZHZs5fSv1dbzMyMmiTmX5POsOjsQXrizF4y8Ly2ENBS0MdIVDCPMwzCnWUkEIoNMWTyG8kIgAYdznJTxuuCeEYMRIeIQpAhiiJaUcd7kf347HQMyioZK0lkvBiETBAQRZFtwhUCzG35rsfDt4hMrFtQs0/MZCtpPIE/lugxl9MEUb9EyA8LNBodV1XFuF8rH6rQqPno1F406BpsS+4sGGMjGHKxKJe2Ni1vd8x/YvwTfVmxJoaHRvyPZycMory8is/nrSUk2IOeXds0d3gSyQNDSqAlkhbswqGNqIsrGNR5JjJZ7f/uAR69+X3TLNwCozAxt23mCJvPC+MGEt7jFQIC3Hj66f6o1RrmzVvD3ujTzJr1FFFRgezadYoFC9ZRrdORk1vCooXr2LB+P+ERARw8cA6dVse2Ne82SbwppQUsOnuQ4XiSQTkadFSIGoyE2v+urphQQBVruUwlWs5TSBVa1GhRCHLW9H2CUTt/oUyswVTQQ0btzG8iJdjoG9PV0ZMujp7EFeXwaWwMb6mO4idYcIUy9A30+KL90FvG1sPFh5lX/qCn1oWNpPAybXAXTCkRa1CjQ4OIkhszzVpEqkRNvd7QM45ux1rUJ5dKqtBi/Kc17qIoUi6qMZJ6SWNsbMDuje/z/dIdfPbJr+jrKZnwf30Y+2j3e27HR4mkJZMSaImkBUs6uYvWXkPqkmcAU2N73BzbkXw2hjZdbjWj2PK5ONuwc/1sXn/3Z6ZOXYQgCBgY6LFu3Sz69AkHoGPHVnh5OfL997/j5+/KyRMJJCVlkJSUydQXhuHiZM2v6/bTqX0g7a4t5mpsOp2OBaeTWBB7GWMUeGBGEdXYY8RaLjNW9EMmCLTCmi2k4YYJg/FAhZrdpOOFFSXGWhyNzRjuEcyitAs8rvXBBRPiKWKp/BKTgjshCAIC0MrKgZ97jSKuKIekkgKcjc0Is3G+bXLWzsaZSCc33s84iVqnw/1az2lzQQ8P0ZRdXGUA7nXHr+cydoYmKK61XLyqKuZY7hVsMSQMWzaTyijRp+41D5GNgb4SP3Nplz0AExNDpr4wjKkvNCzZyMou5OvF2zh15jLOTtZMfKo/YSENa80lEsl/IyXQEkkLplHXoFToNxhXyPXRqm/eEu1B0jrYg21rZ1JVVcPhY/FMf29ZXfJ83ahRPRg//lM+//xFHBws+emnN9iw4QBjx84hItyfwCAPPv96E+Eh3vzy/avo6TXuLOm02evZfjYLJ9EIPeQkUER77DlIFhmomM5hvEVzLlNCBWo8MCWWfAyQMwZftsvTGeLdGoApIV1YamDEosTT5KsrcDe0YHKrLvR39W/wukGW9gRZ2jcYF0WRlLJCytQ1+JnbYKhQIggCzwV3QJDB72kJqEQ1JkLt+/AkAcwlllNiHt6YcYxcNIi4VpsyfPsS2lg5cr4oG2/RnAzKaXstgZ7DSQJFK1IpJVko5ceoR+7+DOu1lnb3+kLCW0lITKfn0BkMH9GZiS8O48KFNAaNms1n7z3D4490a+7wJJIWRUqgJZIWzL1VRxIv76u3gUqNupK0rOOEPPR4M0Z2bzEw0MPc3JiSknJEUayXqJWXVyGXy0lOzsTBwQpBEBgxogujRvVg3br9DBnakYSEJYwY/j8+/XI9b736aN25oihSU6NBT0/xr5K/wqIyflx1mPc0EfxOGpWo2cFVHDGmH25s4woBWFCBmgo0tLdxJbooi9ZYYaRTsFyWSFs7F0Z61ibQMkHgqYBwngoIRyvqkAv/bCuAK6pi3jy8jfwKFeaCPrliBROD2mOk0OOzM/tojTWmKFlGAs+KQSgEGdYYECLYkG5SydnqIrqqnRkqeiDTCpwUc1mSn8Ag3DlJHs8SxFecpy3WiMBRsimmhjW9x+JqanHH+P6L4j15WHAKZcew+zaJnj5rCa++9iivXvsZHDq0E4MGtadXr1cZMTgKQ8OGD9MSieTfkRJoiaQFC+n2COtPv8iB2G/xdIqiqkZFXPI2vEO7U5STxsH1C6lSFePg1ZqQHqMws7r/kobG0raNF3KZwOrV0Ywa1QOoTYDnzPmFHj1C+e67LURHz687vlOnVpSUlPPLL7sQBIH33n+GIYPfwtPdnt7dQ4k5eJ7Zn/xK/KV0LCyMmfhUf/73+uh6XTzuJC7+Ks56JpjV6NFTdGYWx+mHK5tIIZ8q1Gg5QwF+mONnYcOCrsMprq5kd0YSKk0NT9h2vOksMvCPk2e1TsuL+9bTo9qRHmIrZIJAjljBZxdOUImGl3St+ZE4PDEln0pe5SAuogk5sko8za2YEtyFmYd31CbP1x4m9pPFaHyIxJ4/uEIFGt4jkkNkU0AVBoKCp3zb3PXkGWo7cWSsg+D7NInW6XRs3XGSX1b9r954mzbe+Pm6cOhYPL26hTRTdBJJyyPtRCiRtGAGxuY8NPVrbFsFczF7NxmV54kc9ixKfSOObfoBT4tIInzHIOapWTf/eYrz0ps75GYjCALLvn2FKZMXMnDgdGbPXkpk5AssXryNgwfPERrqw48//l53/P795+jQIYjly99i7txV2NlZUFyiYvHKPfi0ncDzr37F/AUvUVOzg0OHFnLyXArPv/r1P4rJxdma7Jpy1KIOe8GIIXiwi3QKqQJE3DDjHcKpQccgj0AALPQNGenVmif92t0yef43DmanYqZR0guXugTYXjDCS2dKpM6Og2QRjh0vCm14R4jgDcIIxYZKUcO8zkMo16hxFkzqta3LpgJvzMmhAjU6fiSOrzlPBuUcJQdHKwvGBUU02j38HRfWmaI+dKpJX7OxyOUyqqvVDcarq9UoFfJmiEgiabmkBPoOtm/fTkBAAL6+vnz88cc3Pebll1/G19eXkJAQYmNjmzhCieT29A1NCes1hmEvfcGA8XOwcwvgwqGN9I2ajqdzFLZW3rQLepQAt16c+OPn5g63WYWFeLN2yXSOH0ugrKyCnj1D+eGH10hPX83KlTNYtmwnlZVVfPXVRnbsOMHTT/cnKMgDjUbLzz9vJzDQHWcXW86d/xFBkHHiRAKCIODr68KatTNZu/Eg4d2nMnzMB39rC3APN3vCQn34lUQ0oo4+uOCLOaboMRJv+uPGapK4Qhn93RrWMTemzIoyXHQNt3w3QI45esSSTy9ubOLhJBjTR3DFT27Jsdyr+JnbkKgrplrU1h3jgBFJlLCMBB7Cm/l0oTeueGNGP9woqq6sW2gouT2ZTMbIoR359NPV9cajo0+TlVVIh8iAZopMImmZpBKO29BqtUyaNIldu3bh7OxMREQEQ4cOJTAwsO6YrVu3kpSURGJiIkePHuX555/nyJEjzRi1RHJ7GUmxONm1wkDftN64l2snfj8wq5miundU16gJCHBl7tzn6o0bGOhRXl6Fnd1IIiL82bVrLtbW5qSmZlNdrWbBgvV8990rvPnmD7i7O/D222N4//1l+Pu7Mnx4Z0xMDGnb1pcBAyJxcrJm0hvf8tj4ubi72vHCuIFMfLofJ2KTqKnRENnODwOD2o1MtHKRI7Jc9pGHKGpRKPSRibBHk4EFelhigEyux6g9a7HUN2S0ZyADXP0bfcGdr7k1q2WxiJr6NeJ6yDks5IAI4k3OExEBAWdjc7o4evJN9gUe0/pggyF+mLOSS4jAaziiEGSEX+sBrRNF9lYeJr28BFeTu1/C0RJ8PPNJug95m/PnUujXP4ILF1JZvTqaFd+/+o9KhyQSyZ1Jj/a3cezYMXx8fPDw8ECpVDJ69Gg2btxY75hNmzbx5JNPAtC+fXuKi4vJyclpjnAlkr9FqW9Ijbq8wXh1TTlKPYNmiOjeEtbGm/MXUklPz6s3vmnTIeztLOnXN5yNG98nKMiDvLxixoz5AG9vJ/btq62Pvr6pipeXI/7+rrz77s9A7QP5xYtpHD4cR2RkAMePf40ogqe3Ez/9ugfnoKeYMPUrXp+1BLdWz/Dw/81hwpSFHDx6CSMTB/T0jPFy7YyDTRBaRLKEai7LKjkiK8DfdxDtIl7BznskX166wBcXDjf6+xJu44KFsRHLZZcoE2vQiSKnxXyOyXKxNzPDEDk7uFLvnHRRRbJYSpS9KwD/i+hNmJc7HypimUg0sUZF9HMPQAd1famvEwAFAlrxZmm55GacHK2JjZnPkD7tOHcqEScrM2Jj5tOnR9vmDk0iaXGkR9LbyMjIwNXVte5rFxcXjh49esdj0tPTsbdvvNpDiaQxuQW0J2b1p+Tkx2NvU/uxrk7UcTZxI37hfZs5uuZnbm7MtJdG0K/v63z8yQRat/Zk165TTJ/+PX7eTuyNPo2b22j09BRUVFQxfvwg5s59joKCEt5883uysorYuvUIGzYcZODAKGbNWkp5eSUffrgCV1c7IiL86dJlCnv3fsbAge05dSqRqVMfZsaMH9kb/Tk2NuacPp1Er16vMWpUNwRBhq2VHwKQlXcBSwt3ZDI5dla+5JekEuo7hCDvfpSV56GqyMfQxJnVKedxNS1jXJAnxkpF3Tbg/PegMQAAIABJREFU/4UgCCzoMozPTu9jeuYRdKKIm5EFc0IHEG7rwk/xJ1iWcJJMsYJI7MgTqtgvz2J62x4YKWpn05UyOZNad+LFVh1R67ToyRWIosiFgmxOqHKJ5MbvzXMUYKinh/tfZp9Ty4r45vxhjuVdxVCmpKuzF8HW9tgaGBNu6/KPF0fe0n3a5tHY2IDx/9eX8c0diETSwkkJ9G383Y9Axb/MkEi7QUnuZTKZnO6PTmPPyo9wtm+NsYEN6bmnMbG2I6z32OYO754wferDeLjZ8eH7y7mSnoeXhwNqtYbRY3ozcGAkFy9e4fVp36JSVRITc5bhw9/hwIFzTJkykqioIMaNm4tCIWfSpOG8//5yfH2fwM/Plc2bP8DR0RpDQ31mzlxCSUk5o0Z157ff9hEe7s/atTH07t2O0FAf3nhjNN98swkHOy/yCi/hZNcGc1MniorTCPTqT3zKDrSaatwc27HvxCIycs4iCDICvfriYh/C8qwjrC65wM7d0wnRaDkzLek/vy9megbMiuzL29qe1Oi0mChvtEWbENSesX5t+f1KPGdyM7E2tORHz854mlk1uI4gCOjJFXX//lZ4Tybv30SqqMJXZ0aKUMY+eRYftxtY7/dpRnkJ4/euoafWiRliO9ZxmY0pF7iYkkGFQkOFXMu8TkPws/hvO2zeaGkHKPXuq24cEomkaUgJ9G04Oztz9erVuq+vXr2Ki4vLbY9JT0/H2dn5ptc7/sdPdf/u5B2Ks4/0sZqk6Wg1ao5vW0zckc3otBr0DU1R2JhgaGtDlx5TcfYNkx7+rhEEgcce7sZjD9duPtFr2AwWLnyZsWP7AODt7Ux4uD9eXmN4552xiKLIjz++hr29FaIoolJVsW3bh8yYsRhHRyuMjAzYsOE9LCxMAHjssZ68994yFAo5M2c+ydatR8nOLuT8+RRmz16Ks7Mt48cPRC6Xk12YgYmRPYUladhZ+VJclkF2fhzhwY8Te3ENJ+NWodOqEQSBwd1no69nSkVlIS4O7Th5cSW9R+9k6EejmLnEnMrJaxplNlpPrqhLgP/MSKHHI15teMSrzT+6XmsrR5b1Hs2apLOcLC7Aw9ySJT5dcfvL7POyhFN00jowCA/2kkEOlXxGR4wEJWjhiCabyQc2sWngUyhl/77rREaxF+xJvu/7Qksk/0T0gXNEHzjX3GHcN6QE+jbCw8NJTEwkNTUVJycnVq1axcqVK+sdM3ToUBYuXMjo0aM5cuQIFhYWtyzfiOj3dFOELZHc1P61n1Oans6gLjMxMbIlr/AyB898h7NvGC5+7Zo7vHuWKIrEHDjPth2f1Bt3cLAiJMQbuVzO0KEd68ZVqkpqatQMHfoO/fqFExf3E8899zkTJnzG8OGd6dEjlJycIrRaLStXzmDTpkOYmRmTl1eCt7cjyclZ9OnTjjff/IEnnujNgQPx5GboUViajpGBJXKZgvyiy3SPnExx6VUupuwkLPARZDIFCSl7uJS6B5lMgU6nqd3CPRvito5gbLQZy794hJDckpveZ/4HOxoluf670stLOJ2fiZmePh3s3XE2NmdKSJebHiuKIqfyM9ifmcJ4MQAEiCGDUfjWJs/XRAkO7NNlczgnja6O/+1erifRVvrnkUeF/adrNRadTscfu2PZGX0aE2MDHn+4GwF+NyZ1cvOKWbPhIKVlFfTuHkpEmO9triaR1Ne9c2u6d25d9/Xsj39txmjufVICfRsKhYKFCxfSr18/tFot48aNIzAwkG+//RaAiRMnMnDgQLZu3YqPjw/Gxsb89NNPd7iqRNL0yksLuHw2hod6fYae0hAAWytvOoaM59COxQREDJBmn29BEAQsLU3IzCzA0/PGLKQoiuTmFrFr18m6BFoURWbNWkq7dn78/PMb+PjUfho1efJI+vSZhk4n8uKLX+DgYMmjj3Zn//6zLFq0AWdnWw4f/hJ3dwc2bz7EhAnzCA31xtnZhk2bZtKzxxvkFJSTmnEMPaUhekpjyspz8HHvRmJaDEqFISVlmVRVl2BsaI21hQeB3v0RBBlxSds49uM7RE34mLHzzPDprnfT+5z5xSMEb4zmwjrTm36/sehEkY9O7WXn1UsEy6wooob3hd3M6zSEVlYODY4vU1czef9GClTliBot+VTihRkl1GCPYYPj7UQD8qsq7uo9NIeaGjUPPfEhmTlFjH6sJ/n5JXQb/CYz33iM58cNZO3Gg0ycuoghgztgY2vBqGc+oVP7QH7+ajJyudQDWiJpbFICfQcDBgxgwIAB9cYmTpxY7+uFCxc2ZUgSyT9WlJ2KlYVbXfJ8na2lD6qSHLSaGuQKPTIST3L5TAyIIp5tuuDqH9mkifXmywKIGsKtKzCpNmqy172TJx/ryZvTv2f5L2+juLYhxbJlOxGA37cc5tzZZNqF+7Nn9ykyswo4ffp7HBxu1P7KZDKsrMxYu3YmOTmFREa+wMYNB0AQmDr1EWbPfqrufR4ypCOtW6/HysqUkpIKnJ1tWbX6LQYPfguNRkdebgmCIOP4ueX0aD8VmVyBoaElZeU5eLs9TGbuOTq2fbbuep3CniX6xALK4g5jHdaH5BiTBvenE7WMjTZgyjM96MTeu5pE/5Z8jrPp6byvi+ScrpDT5AP6vLh/A9sHjcNQoax3/Gex+7AolfOSLpxT5LGRFAJFS3ww5zT59PxT72m1qOM8hbxwk0T8fvfNT9vRiCLHT3xT9zP44ovDadduIlHh/kyYsojo6M8JCfEG4IMPxtG3zzR+WLqTiU/3b87QJZIWSWpjJ5E8AEyt7CkpzUSr09QbL1Flom9ggkyuJGbVXPb9Og/DMmOMyk05uGYBe375AFGna5IYXUwsSYo24PPFxpwo0KDSv3dmEWe/OYaSwlIC/J/k+efn06P7VGa8/SPrlr5J3JFFvPB0f2xNDZk8YTBVVTWo1TfeZ1EUWbhwPcOHdwLA3t6KOXPG0z48AGMjA556qm+DhxQPDwfOnLlMUJA7AAsXbqCoSIWNZwT/N2s97QdNoFJXxrqdU9HXM+VQ7A8oFYYUlV5FqTBkw67XWbHlWX7d+jyrtr5AZUUxcUd+50rMKhI2fUXyjiWor8ThaGiCk7EZLiaWyAQF8xeb8qZrX0Lm+ty193Jd0jmGaN34jjj2kkEbrInEHmOtnCkHNtVblK3WadmVmcgInScyQcAWA+wwZDqHKaGGtVxmn5hJuajmqqjiK/l52to642tuc9fiby6r1x/gtWmj6pJnqP05eeihrsyZt4bBg6Lqkmeo7Vv+5luPs3x1dDNEK5G0fNIMtETyADC3ccHW1Y+TcStpFzgKuVyPquoyjp5fSqsuI8lIPEnmpdMM7DwTpaK2s4KfR0+2HZxN2sXDeAR3apI4XUwsySwv5fPFxvh0r2JWxL0xE21kpM/vq9/l6IkETsQm0btjEIP7RaCnVztb+vCwG+9PYXEZHTpM4sUXh+Pqassvv+wmPT2PmJjP647x9HQgN7+ETu0DWb/+AK+9Nqrue9XVNWzadAitVoeenoKHH3mf6P0XGTZ5MRY2tSUhId0fJaT7o5QWZJFzJY7ctHguHNpAYXEqSoUhhgbmmJs5EeI/gpoaFftOfoWi2oDzMb/h7doFpaGMI799xTnb3xgw7gPkCj2cjM3ILC8lKdqAad1tmTv37tRFF6uruEo5Neh4g7Z1bec6iA68W3ycE/npRNjWtgZV67RoRREFAvPFM6Sjwg8LzNDjKip6ufoSV65iVXESZgp9hnkF80xA4279rauu4V4ogKhRazA01G8wbmSkT3pFNS6eDRc52tiYU6aqbIrwJJIHjiD+tQeb5K4QBIHnP4tp7jAkD7CqilL2LJ9DTtoFTE1rZ6QDIgbQYdgL7FszD+MKM4K863/Um5CymyJ5Jr3GzmjSWDPLS9GJGny6VzHES0t3E7Mmff3/6uiJBJb8uodLSZkkJGVw8eJPmJre2Ab79de/pbqsgvFP9KXX8Bm89dYYRo3qQXp6Hq+++jXnL2Zi6uiDoUzAyjWUwKjB6BvevqwiZs08Ek/soFvEyxw98xPebl3Izo+nqDSN1r5DOXtpI4O6zcLU+NpOfzote45/jkf7roR0faTetdJVhQAsf6W00Tp3XDf98FYuZeXQD1eihPqlFlvFNOSexrzetnvd2OM7fkFfBcYoeIZA5IIMURTZQApXLar4seejjRbbzThbJGPzdu2nBIKl+119rduZ+dEK0rIK+XnJ9LqxkhIVgYFP8fkH45n27k/Exy/ByOjGZkiTJy9EodPx6XvPNEfIkvuczHJogza9khukBLqJSAm05F5RVpiNqiQPSzs3DIzNAdi78mPMqq0J8OpT79hLqXspEK7S+4l3mjzO2iRai0/3yvsyiYbargm9h7+DvaM17/zvCSwtTVmyZAcLvviNwzvm4u5mx9nzKcyct5KYfXHIlMbYBnal64AJyG/SJu524o9vJ2nfDixMXEhM3YOTXRsszVxJvBKDn0dP8goTMTSwoLA4FSNDK/w9e6HVqjl7ZRMjX/m2wfXSVUWAyJRnyugU03h10UklBYzbvZpR+NBJqD9ruplUDDxNeS20W93Y/qwUXju8hbl0wly4sQBSI+p4Q36En3o9ete3+r4XkuiiYhVdB04nKNiDsU/0oaCglM8+XU2vrm2Y/+GzPP3CfC6lZPP2jLHY2Jjzyy+72LTxIIf++ARHh4a9uCWSO5ES6NuTSjgkkgeMqZUDGnUVB9YtICv5LPqGpth7BnEpKQZf9+7I5bVlCVqdhqT0fYQN+r9mifNGSYEhn0eL8Ewp4daKe6Kk4++SyWRsXvkO73+6ioEDpqMqr6J/rzB+3/EOKVYGpKhKwcMazymT0YYaAAIuJpb/6rWMzayprClBW6zG1TGcDqFPk559GmNDa1QVeWTmniPAszehgSMpLs1g/4mv8HLtjFajvun1XEwsSVcVMX+xKTTi4kIfc2ueDo5gfdxZIkV7lNdKOCpEDQfl2XzkWr+lYoi1IwpBjplYf3GhQpBhJRhQVF151xPojGIv+GAHVgOckUc1z8YqlhYmHNz+CT8s28FXC9ZjamLI+2+PYeiA9gD88OVL/LxiN598uKK2jV23ECl5lkjuImkGuolIM9CSe0VhdiobF00myKMvbo4RVFQVcvrSeqq1KhQo8XPrAQgkXo3GxN6Rfk/PRvYfNqVoDNdnQ6c+U37fJdF/pdKv4N3jIknRtQnzdTJBjpPxv59l12k1rJgzBm1VDT2jpmJl7k5VTRkbdk3DzMQRT+cOmJnYI5Mpsbf2o7Q8h60xs2jddSRRgyfc8rp/LqeZ65LXKDsaakUdbx7eRlJ+Hh009mjQsV+eRQ83X14L7VpvUaUoijy0fSmjKr0IFG48XBSIVcySn2DrwGcwVt68NV9jcrZIvpZAh0mbqkgeCNIM9O1JM9ASyQPm5I6lBHn0o5XvIADMTOzpZeHFhj3TCO77BLkpFxBFkbYDxuIV0q3Zk2e4+eLCP7tfEupoVSmbz8pJijb8zwnzX8nkCgaMn8OGBS+h0dYAYKBnSoBXX84lbKRUlY2VuRsaTTXlVUV0DpuAqbEtdh6Bt73unxcXxjxjQaeHyv7zTLRckPFRh4EcykkjOj0JhUzO+24DaGvt1KAjiSAIvNi6I5+c3MsorQ8BWHKFMlbLL/N/fmFNkjxLJBLJX0kJtETygMlKPkPr9m/WG1Mq9HG2D0Gp1KPPk+8CUJKfQdzhTSiUBni26oy+0d3dYONO/lzS8S5Vf/qOyBCv0nu6RlqlX8GJAg2bkxUkRRs0evJ8nbWjNyE9RnH+7O90j3gZmSDD0SaI+OSdDOjyP8xNa2dOs/PiiDmxCAMDU4z+RsmIk7HZtU8BGq8nuEwQ6OzgQWcHjzse29vFFwOFkh8vHGOp6hIOBiY85R/JUPegRotHIpFI/gkpgZZIHjD6hiZUVBViamxbb7yiqhB9IzNEUeTw5q9JOLoNV8cwatQVHNzwJd0enYZPaI9mirrW9SQ6OeZGRwudqGUzleBVWyN9M009Q/3XHtYnCjR8vtiY/1Lj/HeF9hhNxqVTbDs4G3f7CJLSYjA3dSbm+JcY6Jvh694dD+f2uNiHciXnFGVFOchkcmxdA+7p3ShvlmxXatT8FH+C7VcSUOu0dHHwYHxwe+wMG24WI5FIJI1JSqAlkgeMf9QgzhxZT8/IV1DIaz/+zsg9S2HpVdyDOpBybj9pZw4xrMfH6OvVJqpFJVfYseZjrB29KS3MRC5X4OgVgvwvu8Y1hb/O3F6fld5MFQg3qdcTtUDTzVBfn21GqC192XxZICm6aZJnAIVSnyEvzOPKxSNcjt1LRVURHi5RuDm2o6w8lzPx6yksSaOo9CoCcCl6O0WlVzA0s6T/uA8wNrO+6zE2Bo1Ox4v71qNfKjJe548+cvZfzeSp7FX80ucxLPUb96Epo9gLi+oyZDXVoMqS6qAlkgeclEBLJA+YNp0fouDqJdbvmYaLfQjlVYUUllyh/9PvoVDqE3/kd1p5D6pLngEszd1wtQ9jzWfjsLP1Q6OpRlWZT++xM3DxC2/Gu7mRUCdFF/F59K2OapouHjcWCNYmzNfdrZKNW5HJ5HgEdyLh6HZCAh4i2GcAVzJPcD5xC6WqLFQVeTjYBDKgyzvI5UpEUcfphPXsWjqbYZO+aLI4/4sD2SmoVFW8pGuL7NrM+SP4UKHRsjrpLBODoxr9NS+sMyWYWBQd2kpJtETygJMSaInkASOTK+g1dgaF2SlkJp/BwMgMj+BOKJS1u5xVV5RhZN2wLZiRvgXerp2ICnkagOz8i+xYMpNRbyy5J2Ytbze7m64q4vPFxkx9phwoveO1/k2ifTcXCP5bGUmnaNf9EVIyjnDqwirahzyFvZUva3dMISrkqbqWhYIgI8RvGL/tfoWS/HTMbVxuer35i00If6U7wUQ3Wl/of+tkbgZtNdZ1yfN17XQ2ROdcvSsJNNQm0c57dmA7ox+ilERLJA8sKYGWSB5QVg6eWDl4Nhh38m1LaspxHGxvLNDS6bSkZh4jsvXYujEHm0DcHMNJOPEHYT0fb5KY/60bXTxM8Ol+877HN2gBDeHWf28b8RsLBO+t5BlAqW9IZXUpZ+LX0ylsItbm7sRd3o4gyDgY+z0ezlF4u3ZGJpMjCHKMDC0pK8y+aQJ9vS/02HlmTGnEvtD/loW+AQmyfPhL1U4+VZjrGdz8pEakPXkeeVTYXX+df6qkpJwde2PR6UT69myLpYVUDy6R3A1SAi2RSOpp3fVhfjs5gRMXVuLt2gW1uoIzCRtQKgzRUxqTmBaDsaE1jrZBmBk5UFGc39wh/y03FiDeuS3f59Gam7bLu5mmXCD4VwWZlzm2bTGZl2PR0zfGP3IA7fqMRa6orW33C+/H6YtrqagsxMrcnT8OfoihgQUezlFUVpVwPvF3MnJO4+Ecxam4NVRWF7P9pxl4h/Sg0/CX0DOo/wBx/UGksTdX+SfOFmSxNP4kScX55OnK6YoTroIJKWIpq0niEsXo5cmZfXwXk0M6N0kyfa9Yvmovk9/8no4dgpHLZTz/6td8PPNJnn2yX3OHJpG0ONJGKk1E2khFcj8pL8nn1K7lXLl4FIVSHz0jU1S5OYiiBgebQIpK09Fqa9AzMKFN30fxa9fnzhe9j9zYRrzqjscmRevTHMlzYXYqGxe+TGufIXg6R1FZVczpS+uQmekzYPyHCIKARl3N1u+nk51yjmDfQWTnx1FSloWzXRvMTZ3IzD1LQVEqCqU+XcMnYW/tT3VNGScvrqJaWcGQ5+fd9LXTVUXXtvje06QJdHTmZd47voshWg98MGcP6RwiGy/BjDSxjMfwJQoHKtCwWZZKpnEVS3uPRn5tt8PGci9uqhIXf4UeQ98mOvpzgoI8AEhKyqBLl8lsXjmDdqE+zRug5L4jbaRye9IMtETygCjMTuXCgfWU5GVg5exFq84jMLO6+R9/Y3MbuoycUvf1npUfoVetpGv4pLqNVS4kbeN80ma82nRtkvib0s3a5d2KTGjYGaQpxO5aTpBnPwK9+gJgoG9Gt3aT2Bj9JrlXLmLvHlTbkeP5eexcOpOkS/vRiRo6h03E2b4NAK18B7N+12u0Cx6Ng01A3XU6hIxjw55p5KVfwtbFr8nv7WZ0osi80/uYoA0i4NqOhE8SQDvRjm/E8/QX3OiCEwDm6DFG58ucylMczE6lq6NXc4beJH76ZTfjxw+qS54BfHycmTRpOIt/2SUl0BJJI2vcx3KJRHJPSrt4hI0LX0ZeJOBr3RlNZhm/zZtI7pWLdzxXp9OSfDaGyNZP1NuVMMi7HwqFASX56Xcz9GbjZGz2t/9pDtmpF3BxaFtvTCZT4GTXhuy0C3VjgiDQ8/G3EOUiBnqmdcnz9e/VqMuxtw6ofx1Bhp21P4XZyXf3Jv6B7IoyKtVq/Km/wLWVYIUMgVZi/YWsgiAQqLEkvjjvrsSjq64BdQ2iKuuuXP+fyi8owdvbqcG4l5cjeXklzRCRRNKySQm0RNLC6XRa9q2ZR9ewFwjxH4GLQ1vaBY0mPHA0B9Z9eefztRo06mqSrx5i3c7X+GXLeP44+CE5+fEYG1kRd3gzp3YvJ/dqfBPcjeQ6QxNLVOUNk0NVZR6GJhbotBpS4w5xJmYVGUmx+Ef2QyZr+KGjiZENhSVp9cZEUaSwJBUzq4YJWXMxVCipFrWo0dUb14g6REQyKW9wTo688q5sqpJR7EXxnjzUh07dM0l0h8gANqw/0GB8w/oDdGwfcJMzJBLJfyGVcEgkLVxhdgoy5DjYBNYb93CO4ui5pVRVlGJgdOtZVIVSH0Mjc9JzTtMtYhJmxvak55wh5sQi1OoKjAxs0OlXsX3fepz9wujx2PR6M9WSuyOo01BO71iJrZUP+nq1SWJ6diyFJanYuQaw+pOnUaCPjbkXlw7+gVamoVSVRakqGzMTh7rrWJi6cuTsz/Tp8AamxrZodRrOJW5GYWiAg2fr5ro9sirKWJ98nqtlRfhZ2jLMI5g2lg78UXiVIXiQJpaxgWQuUIiAwHohGT/RAjvBEFEUOUkeSUIJfZx970p8GcVeZKyDYE6h7BjW7C3txjzSnQXfbuHZZz8lLMwXmUzGuXPJnDqVyLdzn2u2uCSSlkpKoCWSFk4mU6DTahBFsd5WzaKovTZ2+w+iyopy0Kir6dXtVZRKQwA8XaJAgNi4NXQNq/3jHOr/EDuPfkLC8e0Eth90925IAoB/eD8Ks5JZv+d1HG2DqKwupryqgP7j5rB/7XzcbcJp4z+cGnU5hSVXSMk4gkyuZMfhD2nlPQQLUycy8s6RUXAe//C+bD0wE2MjGyoqC7F29GLgsx/f1a29RVHk/9m778Ao6/uB4+/nRu6yL5e9dyAkJGHvPQQHAm6lrrpqrVrqrFvrqq1Vq7WOn5XW1ooVB052kA1hhQCZZO+dXMaN5/n9cSRwBMLIBL6vv7jnnvF9QnL3ue99vp/P9soiNpTkopYk5oTFkeJtn/FOqyrmoS3fMU7xI0x2I72ykP9k7+XFcfN4OW0de8zVVNhMXE0Md5JAI2beVzJ4hh1EqT0xYcWqVnhzwpW4ap367B4A6tdVYdQNfEk7V1c99915GQ8//Q8yMvIB2L8/j1eeuRlPz9Pn8guCcHZEAC0IFzgv/3C0zs4UlO4kInhs5/bM/HUERCSiO81X3FVFmfj7xXcGzx1C/EewefcHnY81Gh2J0ZdzeMdPIoDuB5IkMXHBvSRNvZqyvP04ObsREjeadlMDlUWHmTr7HvZlfsWh3B8xuIdgaq3BbGlhwsL7KM1Ko6h4D75hQ7j62vfwMAYydv4vqa3Ix9nVgId3386kyorCk9t/JKOinIk2f8ySzOOF3zM7PJbfJk/l+Z1ruN02hCTJBySYJAey3lbMhxnbWX7JEu7b+BWjan2YIQUD4IKGJ5XRPKnawbz4YSQZA0k0BnRpsnIhO3CwgGdf/ZTt298hIcFe3/3AgSPMmvU7pk9OIiE+bIBHKAgXFhFAC8IFTpIkZtz4ON9/8Bil1fsxukdQUZdJdUMuC359+rbNzu5eNDVXdJnBbjJV4KzzdNhX5+SGpe30tZOF3uNm8CN25OzOx+1tJpycXMgu2EB2/nriImYRFjgKoyGCgznfs2/9p9zw2L+QVI7fPGh1zviHxZ94+pNSSWre+MiNb6fP5bXXqtj3cM5ZjXldaQ5ZFZU8ZRuJVrKn+8y0BfN8QRpDvPywWmwMx3FR4BSC+F/9JtqsVipbmllA8AljkkjCiKxAUh9/ABiMPvr3Gu6++4rO4BkgMTGSu+66nI/+vZo//+GXAzg6QbjwiEWEgnAR8A+L57pHPsYncRjNLvWEjhnPdY8uw+B78pbNxwuISETSqsk8sqazJqjF2s72/cuICZvisG9e8RZC48f0yT0IZ8bTOwiLtZXdB5fja4zFJpvZuOttNqX9nejQqahkFSW5e3t0jSBXD1SShpwNeh4u9iVm2SiCDWdesWNVfibTbYGdwTOAi6RlshzA1vJ86GbiWEHBW+9CBa1dnqtSteGjP7sW7BeKiso6YmODu2yPjQ2horJ+AEYkCBc2MQMtCBcJF3cvRsy44ayPkySJ+Xe8yA8f/p7s4lTcXQOorD6MztWT0poMvMsj0Tm5kVeymbK6DBbfcl8fjF44U9XF2dgsFjzcApAkibCAUYT4JfPz7vdYsXopCgrbv30fr1++CIqCqaEKT9/Q06bynKijfF/Ohlp2RVmYNNOXkhVndqxNUdCcZP5Go6hwVTuhUqvIsNaSeNws9CbKGOrpi4eTnmvjkvlb2mZibZ54SPYc5z1KFfk0MSMo+qzuozfI7WbUFnO/X/d440YPYeU3W/jFL+Y6bP/m681MHXdm3ywIgnDmRAAtCMJpefqEcN0jH1OWn05rUy1TQpbi5uXHoe3fcWjHKsztLYTGj2Xxzffh4m4c6OE2YJSYAAAgAElEQVRetCoKD/Ht33/HsKhLCPIbTl1jEak730ZWrIxPvo2woDFYre3sy1zB8ldvQ1FsuLn50tRcScLEKxl36Z1dUjtOT6LbKeOTmBEaw6c1aYyz+XfmKVsUG1vUFTweOovZYXE8uvV7Jir+hMluZKka2Keu4d1RiwGYExxLbn0NT+RsJ07lRSNmmlQW3pi4AGeN9izH3zMl9VGwLg8Du9FOBLROA1KN49YbZvH2B9/xyCPvcf/99p/Tm29+wb59ufzfX37d7+MRhAudCKAFQTgjkkpFUFSyw7aECQtImLDglMe0NNXSUF2ChzEQV08fwF59obLwEPkZm5FUamJSZmAMiDzlOYQzt/WrvzEm4SaiQicdbdtdipOTK77GOMKDxiJJEk5aZ5qaKwjxS2bs8F+g0ehobatnQ9pfcXJ2Z+SsG/t8nJeExvFd/iH+XL+PKbYALMikqssY5hfAOL9QJEnik9nXsyLvAJsriqhqNdFitfDkth+5NX4088OG8qvECVwXk8zemlLctDpG+gSjOevgv3c4BtEDU43Dw8OF1G9f4qmX/k1y8h0ALLxsPKnfvoSHx8WZ1iIIfUkE0IIg9Dqb1czG/71B3v4NeHoE0dBYRtjQcUy79iG2fvM3CjO2ERk0AVmx8s2mB0mcehWj59480MM+r1kt7VQUHWTm8PvZsudDqutyiA6bSlToJHIKUtmZ/gljhi+hyVRJdf0RrprzOmq1fbbWWW9g/PBbWbvxdUbMvKFPy9cBaFVq3ppyJT8VZbGuMBuNSsVd4ROZFRzTee1gV0/8XdyoNZn4hS2WaDzINTXyzp7NNJjbuD4mBaPehZnBg6NFdUcQ7Ttx4MYQGGDkw7d+w4dv/WbgBiEIFwkRQAuC0Ou2rXyP5pIyFs/6M05aFyzWNranf8yPHz1BS201V0x7Ea1GD8CwqPl89/MzhA8bj29I3ACP/PwlqdRIKhUFpbuobcjnsmnPo9HoABgaOYfvUp8mrPoQsiJjcA/uDJ47GNxDaGtpQLZZUGv6tnYy2IPoy8PjuTz85Pm5FtnGBxk7eNA2nBDJnp+dgJF7bQm8fnAHV0UNRysa9giCMEBEFQ5BEHqVxdzG4Z0/MH74rThp7V8dazV6xibeTEVBBtHBUzqDZwBnvScxoVPJTlszUEO+IKjVGiITppCR+z3ehkh++PkF/vXNbXy19lHyS7YSEzaNgtKdeLgFUFN/BIu1zeH4qroc3Dx8UanPPof4jY/c2DxtBgmLm3rrdihvaUKjSJ3Bc4dgyQ0nRU2pqbHXriUIgnC2RAAtCEKvajc1olHrcNEbHLY7aV3Q6zyRZWuXY7QaZ6yWti7bhbMzadF9NLdUU1S2G0/3IMYm/YKxSTeTeWQdVbXZFFaksWrrK2ic9Py8+++0tNYCUNtQwJZ9/8eIOUvOOn0jxM0LkHjjI3ceD51L8mu9k1Lh4aSnWbHQqth/X3YrVbyo7OJeJZVGWzvbKwt75TqCIAjnQqRwCILQq5zdvZCRaWgqw9P9WDWClrZ62s1NlNVkkBBzaWegJstW8kq3MGHCvQM15AuG1dKOItvw8orA4B5MaUU61fVfM3HEL9mw4y1Gzb2FiISJePoEs/3b9/km9QlUkhq1VsfIOUsYNv7yc7puiJsXpaZGcjboMS01kLC4iYwV7j26F08nPZP8wvmiMo9I2Z2vOMJNxBGHJ0do4pMDaZhtNpbEDWwL7RNZtuxGO21gKnEIgtB/JKWjM4LQpyRJ4ld/Th3oYQhCv9iz9j9kbf2JSSl34uURSkNTGVvTPyIwPoXyI+lobXriQqdjky0cyl+Ni4+Rebe/eNISalXFWRzcuhJTXRV+EfEMm7AAF3evAbirgWW1tFNbfgS9iwce3kEn3Wflu0sxqkNJGWovY1bfWMKujE+pbyzCJluZf+dLBEYlde5vs5ppb21G7+qJqhfyiYuba/nkd83ovlrX4wAaoNHcxkObvyWjroLfkUK0dKzzZYXSwivqPXx/+S/RqwfPXFCwIQ+fJ+YiSRKSV/hAD0cQzpnKawEiRDy1wfOqIwjCBSNl5g1Iag1r17+O1dyKSq1l+NSrGDl7CTarhYPbVnJ43zpUajXx069gyJh5Jw2eM9NWsfXLdxgaMRtfj9GUZKWzfNNtjL3sToKjk/H0OX0nxQtB+qYv2fXjR7g4e9Ha1oDBN5RZS57E3RjQuY+5zUR5QQZT594DQG7hJtIyPiUmfDphgaPIK97Chs9eY9H9b6N3tQeiao3ToK7b7eGk58Xx87j2p0+IVhzbxvtLLnhKThQ01THE4DtAI+yqpD4KXlyFYaYv2omIIFoQLlAigBYEoddJkkTK9GtJmnoV5jYTTjoXVEdnCVVOapKnXkPy1Gu6PYfF3MbmFX9l7oRH8fIIBSA8aAz7M79m17cfsUOx4h8xjFlLnsJpkLZvVmSZA1u+4uDmr2lprsc/NJ5R827BP6xr5Qmb1UJVcSZqtRaf4NjODxR56RvZt+ZTLpnwBJ7ugciyjYO5P/Dtew9z3aMfd84c26wWezqGWovZYmLngU+YP+VpPN3ts9Wx4dPZnr6MXav+yaSF9/V5qbre4qbVYUGmWbHgJh1b4GhRbNTJ7XjpnAdwdCc3GEraCYLQt8QiQkEQ+oxKpUbv4tEZPJ/IZrVQU5pLc11ll+fK8vZj8AjuDJ47DI2aTWtbA4tm/gl1i4aN/3u9T8beGzZ9+VcyN37PmNgbuXzycwTqh/D9e49Qnn/AYb/sPev413NXk/rvP7H6H8/yn5eXdO6zf/3njBx6bWc+uUqlJjH2cjSKlqLMnZ3n0Lt64mEMpKhsNyUV6fh7D+0MnsH+ocboEU7mjh/4+0MzWPb0Inb++A9stq6LOgcTZ42WmYHR/E+Vi6woKIpCqlLC79mGRbHx3I7V7K0pHehhCoJwkREz0IJwnik1NSIrNodt9koI55eMrSvZ+f2HODm50dbagLObgZCho4lOmUFgZBIqlQrlhPsEkGUbKklCrdYyJuFGVqxZSltLI3oXj7O6fl1FPg01pehdPDA11uDs6klgZNI5tLI+uea6SrLTVrFo1p86y/nFRcxEpdKy8/uPuOJee+BfVZTJ5i/eZObY3+JtiERRFIrKd/PDh7/n+sf+SWNtGd5xEV3Ob/SMoKm2rPOxJElMWvwbfvroKfyNQ7vkLpZWprP38BdMTLmDEP9kGpsr2JX+H5pqy5l54+O9cs8AS/7sxidLpxO87nP7TGwPWWWZKyKH8demzTzavBUXRYNFkbmNeAIVVw7U1LB000penXgpY3xDT39CQRCEXiACaEE4jxQ31wEKD95+rN7uGx+5U9xcS4jb4M1lPVF+xmZ2//RPZo97BC+PEGyylfSsleSkbaBg/1b8IoYy48bHaDJVUlWbja8xtvPYg7k/EhY05mhbahf0ek9am+rOOIBua2lk9bJnqS09grurPzV1R9Dr3NFq9dhUMvNu/wPegT0P/CoKM/DzHdoZPHcICxzFjvR/AvYUj70bljM0Yi7eBns7c0mSCAscRVHFbjLTVmEMiKCi+jBuYcfyfBVFoaLmMPEBjm3Ug6JTWHDfm+xZ+2/y9m+kyVSFu6v9uH2ZXzEu6RbCAkcBYPAIZtqo+1mxdimNtWV4GHteNSLEzUhxcx1LXvfgweeuZVLq+nNaTNhkbie1LI/02nLWF+fgqmiwoSCpVVRZW3mFCXhI9mYvUwhCZ1Pzzr4tfDz7OmyKTHptOe02K8ONAbj0Q1MYQRAuPiKAFoR+1JPmD/LRerifLG1E9/UG6tdV2R+/eQ3P7nYiZ0MtKunYn3SQ69nNyPanfes+Y9TQa/HysC8CVKs0JA9ZSEn5HpKHLuJA3vdk717LjBseY+2/XyQqeCKeboEUl++lobmMSyb9HoAmUwVmi8lhMd3prP/0VVxlI9Nn/RqVSoPVZmbz7vfR6zzwMUTx/QePcuMTn6LuYWUHvasnLS01yLKVQ3mryS3ahMXSgtEzHK3OhX2py9m77r+0tTSi1eiQJImEmPlIkn0G3MsthOaackbMvonVy57FWe9FoG8CZksLezO/QO/hSWBUcpfregdGMXvJUxzY9BU//fQiQyJm4ebsS3VdLsF+SQ77ajU6/HyGUF2c3SsBNNi/DSluruONj9wZfQ4z0etLcnh21xrCcSfP1sADJBEnGVAUhW84Qjq1ncFzh5H48n7jQfZWl/LE9h/R2VQ4o6FYaeaB4ZNYHDW8V+7tXCjNZaKknSBcgEQALQj9pGP2OGb6uTcMeXakmdYHPienPgo4OrN3SxqvvRbDw9MdKxHkbLCikjSDMpBuqCnFOzbSYZskSXh7RdLcWkNC5Hwyd63myvve5OrffcDhHT9wJHMXjY0lTB5xN856A9V1uWxN/5jkGdej0eq6vZ7V0s6+DcvJ2vkTpoZqIkMm0GZuxkVvQKN2YuzwJXy19jFGJVxPdvEGijN3Ej5sQo/uMTAqGYvSzk+bXkKj0TE+6Rb0Ok+OFG+hrPogB1O/Zubo32L0DKO+sZgtez7EZjOTPHQRAOV1h4lNnEtwzAimXvsQ21e+R0taLYpiIyJxCpcufrXbhYCJkxfiFzaEQ9u+o6E5Hb2zBw3NZXgbIjr3URSFxqZSXDy9e3SvJ7IH0bXA2S1UrGlr4dlda1hqS2IrFUThTpxkb8gjSRLDFR82UY6iKA73Xks7riotS7es5BbrEFIkHwDKFBN/Sd9GuLsXo3z7t2JLSX0Uhi270UwYASKIFoQLjgigBaEfHJ96Mc1cf87nqX5g1Uln8/Y9nMPLi/fY36yPSr3dcDS9o27Q5Uh7+YdTWZOFm4tjWkJlTTYRweMBsJrtHzQ8jIGMnXc7Yy65jYPbVrJ97T9prCvD3eBPyswbSJh4ZbfXslrMfPXX36AyqxgffytOWmdyCjfy48/Pc+nUZ9HrPHDWG1Cr1FgsLbi7BmBqrOnxPapUasZedgebPn+Dq+b8BfXRFtnJQxchy1aaTFUYPcMAMHiEMHXMfXy74SliI2aSlb+WxtZyolNmAhA1fAqRiZNpMzWgcdKjddKf8rrH8wuLx+9oxY896z5l147/MGPMgzhpXVAUmYO5P6JxdsE/bFiP77c3/FScxQh8iJA8WKnkMxHHbxbCccOGzD84xCQlkFgMyCh8ocol3uiHus7aGTwDBEquzLeF8Vn2vn4PoAEyVrgTvG4VPk/MhbqCXi9pJ8syJlMbbm7O501VFUG4UIgAWhDOQEcA3BOfLG2k9YHP2dejhVWnPjZjhTusyOl8PGlxE9w+Y1AG0SNm3cjaf/0BVxcf/IxxWKxt7MtcgUajw997KJv3fUBYguMMsCRJJExYQMKEBfaFhGfQ+CNv/0ZSl/8JjeSEJKnYsucDJo28i9GJN2KxtnH4yBpShi6mpj4flVqLRutCWdUBRoXd2iv32dJYTUTIuM7guUNEyARSd/6183G7uRlFUVCpNKxYs5SAsAQMvmF89srNuBn8GT5tMdHJM3B2M5x4iTOWPP1ammrLWLH2IXy9Y2hsLkfn6sa8X/5h0ARfjeY2PG1arMgE4cIh6khSfPiGI6RSghkZNRINmPmIQ7Qjo1JJDPcOJNzTC1N1Q5dJ7yBc2NNSMjA3xLG60L5PXtJr57TZbLzyly94+4PvaDa14m1059EHruKe2+cPmv9LQbjQiQBaEE7j+Nnj0ecYv7hW1p9y9vi01zc18M2Rg9S0mkjyDWRuSBzOGu1pj8tY4U7CuuV88uY1LHndY1AtNAyJG8Wkq+5n67fv09bcgM1mxtsQybDo+Wze+z61LUVMn/LYKY8/k+C5ujSH1OV/YsboB/A1xtirW5SlsX77X1gw82XCg8aSnvU15dWH2Lr3I2LDpvFz2jsERCbiHRTdK/epc/agpb0eRVHIL9lO5pE1mFprcXU2AhKt7Y1s3/cxZVUH0Wp0WK1tJEy4kuy01STFXsnI0VdR31TC9q/ep6G6lJGzbjrnsahUaqZevZSRc35BdXEWLh7e+IYMGVQBV5SHkc/YyyqKkFFwQk0WDXijYzR+1NHOvSSildQoisJPFLFTX81fJl/B2tIc/pFfwOVWx/SODKmOYUb/Abyr3vf7F/7FtrRsUjf+hSFDwkhLy+KWW17BZpO5765za8cuCMLZEa28+4lo5T1w7AHwubL/eXTMHp+rcy3nta4khxd2rWGCEoCvrCddU0eDk4UPZ1yNl+7MmocEG/Jw7lxoqOf4KbqBnpVWZJnmhiqKMneQtycVi7mVsGHjSZy8EJ1zz1pBp372J3TNeobHXmG/lqJQ31jE7kOf42ccgovewPb9/0St0WI2t+Ls6kn8+MsYNedm1GfwAeVMtJka+eQP1+NriKa2IR8nrStuLr54ugWSXZiKs86TsMBRJA1ZhFajo6o2h3Xb/0Js+HRGDjvWaKa5pZpvNz7FkqeWo3N265Wx9aWOD50dfzdn8vvfZrNy/U+fkNTmxXzC0aLiBwr4iUL+wmQeYgtPMAp/6djvvawoPKnewZ+mXkGMpzc3rf6UqFZXLpPDcUbNZsr5WpPPspnXEdqD2fueCjbk4fvkJb2SwlHf0ExUyl0cPvwx/v7HPhDv35/L/HmPkb//QzSanrdlFwTRyrt7YgZauKAdmz1uPsczKEwzn/vscU+0Wi28sGstD9qSiJA8QIKZthA+bcvmnfQtPDl69hmdp6Q+qnOhYertBjoC6G/zNORsUAZ0oaGkUuHu5c+w8VcwbPwVvXruhupi4nynA1BTn8/m3e9hky2oJA2VNVlotM7MWvIEUUlTe/W6HbJ3r2HrN+/ipHWmsjYLF72BkfHX0GZuYn/mV0SHTuZI8VZGDruuc8bU1xjDmOE3kVvws8O53Fx88PIMpaook5C4UX0y3t5iXzx4rFpMzhn+3awpzsZodeIq6djsv6/iTDxeaFBhwoIvjl0HVZKEr+RMbXsLWpUf701fzBv7NvFoyVasiswoYzDvpiwe0OC5t2XnlhEVGeAQPAMkJUVjtlipqW3E32/wpGsJwoVKBNDCBevEN/JztW+FO93lHveVnVVFhEpu9uD5OHPlUF4oSTvjALrDvodz7HnRR02bMGJQLzQ8ldbmeg5t/56a4mzcfQIZNv4KPLyPVTioLDzE1q/fpawgnbL8dEIDRlFRfZAxw5cQETweSZKoqc9nzbbXcPHom5SW0ty9bPnyHaaP/g0+XtHIspXDeavZlfFfrpz1Cj5eUazZ+hpmiwlZsaE+rvygnzGOtIzPHM6nKAotbfXoXHo2K9/Xjv+bO1Yt5swcrqtkiNXgkMPshY4KWpGAAFx4hTTqFTPe6JlFCEPxIk9uIN7gZ99f58JzY+fyjDIHBQW1NLia7fZGSbuQIG+O5JdjMrXi6nrsA0VxcRU2mw2D5+D/hkIQLgQigBYGrZ4u3IuZ3nZC2bfzi01R0NA1ANCgwqbI53ROh6YWK3L6baGhzWbF0mbCydntjPKXT6WuspBv3nmQQJ8EAozx1BUV8sXWu5j9i6cJHTKG+qoivnv/EUYOvY4Zwx/Aam1jf+bXAIQfbb4C4G2IICn2StI3riAgItHhGiU5e8jetRqLuY3whPFEJ88465SOfeuXkxy3EB8v+2yqSqVhWMx8iiv2UlS2m4jgseidPGg3m5BOWPVWXZeLWqVBUeTOmtA5halo9c74BMd2udZgYO+OaSVmehuvhVSx75YczvZDZ6CrB9vVlXDcr3YobjRhYRmZNGFhNqEk4EURzSwnFyRYFJGAUe+YzqSSJM62hF5fs2zZjXbiyB4H0YEBRuZMT+H++9/m7bfvx9lZR2OjiXvufp3bl8xBp+ud9CNBELonAmhhUDp+Jsu1suGcz3Mub+SDxWjfEJ5VVlOhtDjkfW6QSpgWGNnNkWeurxcayjYrO3/8BxlbvkGRbWh1zoyYdSOJkxef0+K1zV+8xbCISxgWPe/olkkE+yax4b+vctNTn7F/w+fEhc0kJmwKAGonN0Yn3khVXQ6FRwPXDkbPMAoLdjucf9u375Gzay1Dwmai1QZwYM0KDm5ZyeX3/OmUtaYVRaHw8HZydq9FtloIT5xEfVUxiYnzu+zrbYiiyVSOosi0mRvR6d3Yl/UVSbFXoFY7UVOfT9rh5ah0Gj7/6X4kSYUkqZBVMgt//eagWvDX4cQSjfsezjntMSdzadhQ/u/QDvYoVQzBwKdkk0YVaiR2UsEdDGOEZC976IcLYYo7z7KTOxPG9eLd9I2S+ihYl4eB3qkL/f4b9/HL37xFWNj1DB0SyoGMfK6+chIvPrmkF0ctCEJ3RAAt9ImeLtw732ePe4O7VseDwyfzx/QtzJaD8VOc2a+uJVPdwEfDrzn9Cc5QSX0UwQ983qWjYW/kRW/+6m1qcrO5dPIzuLv6UttQyKYN7wEwfMpVpz2+saaMfev/S/mRA+hcPSk7so/J8+9y2CfQNwGNSkd1cRbVxVmkRCx2eF6SJEL8k6lrKHAIoMtrDuEdEtP5uKYsl8Pbf2DBtBfROdm/Bo8Jm8K6nW9wcOtKkqZe3WV8iqKw8X+vU3p4N3FhM9GonEhf/TmWdhPlVYc66zx3qKzNIiHmUrILUkGChQ+8w5YVb/H56gfR6dywyRaiUqaRm7aO5CGL8POOo7ouj31ZK6gqycHLP+K0P7P+dHzwPCl1/dF0p3PjpXPmjUkLeGLbjzS1t5GAkT8yES0q7udnkvFx2N8bPV6Snqd3rCLA2Z254XGkeAf18I76Tkl9FCUrIIE9aKeN79G5PDxc+HzZYxQUVpJfVMmQmGAC/M+PFCxBuFCIAFrodR2zxw/e3nSaPU9tmrn+vJ497i2LohKJM/iyIi+dfS1NJPmG8UzUcLx0zqc/+Cwcv9Dw4em+5GzQU2pq7FEQ3WZqICttFYtmvtYZkBo9w5icchfr175JwqSF3aZz1FUW8vVff0N0yBTGxN5Ec0sV9bp80rO+cahQYWefmXXz8qe+sQR/7yEOz1bX52GVrRwp3oZKpabdbCKzYC0Lr3ync58j6ZuIDBrfOVYASVIxNHwWB/euPmkAXZGfQVHGdi6f8jxarf3/JCpsMqu3vsq+rBUYPIIJ9E3AZjOTnvUNzaYqMvPXUm8qYcF9b+HlG8pld79GS2MN7a3NuHsHsfzVW5g84h4Cfe3NTbw8QvHyCCH1m3eISZ6OqoctxnvLifnOGb3wQTfJO5DXJ1/BPeu/4HY5Ho2kwqrIqJBoxoIH9hbeNkXm72RglW0EVqixSM08XvQ9cyOG8NvkKT0ex/kiPMyP8DC/gR6GIFyUBscrsTBolZoaz2p/WbECPS/71rNmIxeWBKM/Cf1Ux7ajo+HmXsiLrq8qwsM90CEgBTB6hmNpb8Xc2oze1fOUx+/84SPiI+aSGGuva+trjCbAdxhfrnmIoZGzcXG2j6u86iBWuQ2fkDgSpyxmzbLn8PcegsEj2J5eUbaL8prDyDYrzc0VOGldaGguI2XG9Xj5hZ3y+h26y8LPS99IVPCkzuAZQCWpGBI+i3TpOzbt+TsWcyuSSo3exYOYMbPxDY0lKmm6QzdBFw9vXDy8aa6rxNxqIsAn3uE6Pl7RqCQN9VXFGAMiTjvmvnR8vnPHt0S9WaGmsLmeGJUBjWLP/9ZIKsYp/nxBLrcoQ1FJEjuopJo2/sA4NEfzxKfbgnkufxdzQmNJNAZ0dwlBEIQeEwF0N2pra7nuuusoKCggIiKC5cuXYzB0LYcUERGBh4cHarUarVbLjh07BmC0va9jhilmettZHXeui4iEwSFjhTuTWH/c4sJzy4t28/Knqakcq7UdjeZY/nCTqQpJpUard+32+OKsNEZMfcFhm7POAz9jHD/vfpfY8OnUNRaSW7SJ2Tc/jUqlJig6mTGX/ZJVK1/GzdUfi6WFdqsJxWbD1ysanZMH5dUZRIdOIWPT10QlT8MnyJ7GETl8Cis3/pbE2MvRO9lTERRFJrNgDVHjpp90jJIkIZ9kQWdzSyVNtWWMjL+G8KAxmFpr2ZP5Ba2NtQwZPe8kZ7LT6PRYbWZsNrPDz8wmW7GYW3A6w9rffaU3FgueTpibgXylEVlRji4GhGuJ4Y/s5lG2kiL5sEep4mqiO4NnAFdJy0SbP2uKskUALQhCnxMBdDdeeeUV5syZwyOPPMKrr77KK6+8wiuvvNJlP0mS2LBhA0bj4Ojy1lMnzjCdzSI+69Y97HtzcJfaEk6vNxYXunn6Ehw7ih0ZnzA2YQkajY52czPb0v9BwsQFqE+TiqB10tNuMXXONHeQseIRFkaZORP34AAWX/13PH2CO5+PH3cZsSNnU1l4CNlm5cePnmTW+N8RcDQlorW9kVWbXiLAeyiHtn7LlKseBMA7MIr4CZfz/abn7IsINc4cKd2K2k3PsPEn7+4WlTydn3Y9ybDoS3DS2j8QyLK1s0X40Kg5ADjrDcwc8wBfrnuEmrI8vANPHnTqXTwIikzmQM53pAw9lst9KO8nvAIicfMauK/ru+Q799HfeaynDxGeRj6tz+ZqORqdpKYJMzYVXB49DD8Xd8rz2lE3da1Qo8ae8jHoWcwDPQJBEHpIBNDd+Oabb0hNtXcPvOWWW5g+ffpJA2hg0HXr6Zg9PleOM0xnQwTPF4qOxYVfPjGXh4t9ydng+Dt1JgsNZ9zwKOv/+0e+WPNb3N0DaGwqI27UJYyZd9tprx83ei77s75iysh7UR2daSyvPkR9cylXXPvGKatiAGi0OoKiU8jY8jWBvgmdwTPYZ7GThizkYO4PqBoczzHusjsJjR9L9q7VNLbXMHzuNUQlTz9lsO8fFk/0iBl89/OzxIZOR6N2IrdkM1ZbG2GBjg1P1Gongv2TKM8/cMoAGmDadQ+x8t2llG85iK8hhuqGI7RaGrjiV6+f9mfWV45f1zApdb1jOcQ+8MeJl/H8zjU8VLUFg0pHs2Lh1qGj+UXcSCRJQqdS8+X+fYyy+Z0+XoMAACAASURBVHbOUrcrNrapK3guOKVPx9ZT9euq8JmgQF3BKTsT5h4pY9W6Pbi46FgwfxxeBlHbWRAGG9HKuxteXl7U1dmrSSiKgtFo7Hx8vKioKDw9PVGr1dx9993ceeedXfbpr1beJ84enyvd1xv6/E1SOH8kvxaDye9YvnJHW/AzrdbR3FBFc10FBt/QbvOej2cxt/HDB49hqq0mxDeF5rZqyqoymHvrc4TEjjyjc+xa9TGtedVdFh1W1eayYedbpMy5gZTp153RuU5FURTK8vaTs3stNquZiMRJ7PzhH4yKubZLLvPqba+SNO/603Y/lG1WCg5to66iAE/fECISJp12xr6vnLhYsD87cta0tVDX3kqImyf64+7fbLPym5+/xtTQymRbAGZsbFCXkRQYxDNj5gzKcn/HCzbkYZjpi3biSNA6dZa0UxSFR5/5mI8/XccVV0ygscHEuvV7eP+N+7hqwcQBHrVwsRGtvLt30c9Az5kzh/Ly8i7bX3zxRYfHkiSd8kV58+bNBAYGUlVVxZw5cxg6dChTppz7SvDeKAF3brPHxxPBs3DMibV9X17cdFYLDd08fXHz9D2ra2qd9Fxx718oyd5NeX46Rvc4ZiY/cVbd+AIiEtm0/Q1SlKs6Z7EBCst2IWMlftylZzWmk5EkiaDoZIKikzu3mRpr2LPxf8zyfAinowsMC8vSaDCVEz7s9CXMVGoNkYmTiUyc3OPxnasT850Hop29t94Fb33XvG8ntYa3py5kVXEW6wtz0apVPBA+lamBkYM+eAbHutDHl7RbsXIrP67bQ1bWP/Hysv+e792bw8yZS5kwZghBgd4DNWRBEE5w0QfQq1evPuVz/v7+lJeXExAQQFlZGX5+J88/DAy0zx74+vqyaNEiduzYcdIAeudP/+j8d1B0CsExIxye73jDAnjw9uazvhc7pUfNDAThTHRdaNg3XQwlSSIkbhQhcaNOv/NJBMeMxNnLm5/T/kZS3EJ0Tm7kFm4kq2Adl939GjrnvvmgmDBhAbWleXy57iEC/RJpaa3F1FbN/DteQq1x6pNr9qaOfOfOD+MPD75FwVqVmsvC4rksLP70Ow9CJfVRGHAs9bns07U8/vubOoNngJSUGBYtmsKnX2zkd/ct6u9hCheRDZvS2bApfaCHcd646APo7ixYsIBly5bx6KOPsmzZMhYuXNhln5aWFmw2G+7u7phMJlatWsUzzzxz0vMFT3ZsHHFiibjjZ3usqXvOedw9aWYgCGfqZAsNVdKxl5TeaMTSU5JKxaV3vUra6n+yftdfsJjbCI0bw1VL3z+jEnY9ue7Ua5aSMvN6yo6ko3fxIGTImAFLwzgb/bVYEOwpC5WtzTip1XgNcIWRAXPcgsLaumaCg3267BIc7ENdvak/RyVchKZPHs70ycM7Hz//6n8HcDSDn8iB7kZtbS3XXnsthYWFDmXsSktLufPOO/nuu+/Iy8tj8WL7anmr1cpNN93E448/3uVckiTx29RPu73e5VEWMXssnHeCDXn4HF1oeLyzyZEWBofj8537eh3Ezqoi/pi2gZq2FqzIDPX046kxswh161oq9EJ1Yi70E6+vpqHNzN/+9mDnPlarjREpd/LnF25jzowR3ZxNEHqXyIHungig+4kkSVhz3+p2H+vWPWLhnnDeSn4txuGxvXKHCKLPB33dHMUi21AUBaejM/B5jTXcsf5/3GyLIxkfbCisl4pZpy1jxbybcdZoe+3ag93xQXR5XSsTFr7GtdfN4Lbb5tHQYOLll/6DubWd75Y/jUrVtXSfIPQVEUB3TwTQ/USSJNIW3z/QwxCEfpOwuInN0+w50o6kPsmXFs5Nl+YovfgNWHlLE3/ek8qmygJkFEZ6BbF0xFSWZ+9DKWrlCiIc9n9bfYDLkoazMCKh18ZwvkhY3IRmwghKa1p49YOf+WFNGi7OOm64aiq/vfdK9PrBnzsvXFhEAN09EUD3ExFACxejhMVNtF85HbBXRthVz9GAWgTRg8Hx+c69nT7WYjVz3U//ZozZh7lKKBokNlHGSk0hwc4eXNoURLxkpFWxUooJD5zYRSVO0e4sTe6+zN+FqiMdSpKkU9aIFoT+IgLo7g3+FS2CIJy3Mla4E7zu887Hk2b6Mnrp9HPubij0zPELl2XFhsNiwV5OH/uxMJMAmzNXEtnx+YkZhFAut1KotJJHI7lKIz9RiA/O1NGGFjU36c+sxveFqKQ+Cl5chc8Tc6GuwKFGtCAIg4uYge4nYgZa6C9Vrc2sKcmhzWZlkn84cYazq7/c14INeTi/ec1ZN2MReqZjgWDM9LbObX2R79zhpV1r0RdamCWFOGzfq1Sz3qOc3KZavBQdvyUZo6THqsh8SR5H3Fv4ZPYN50U9575yqkYrgtCfxAx098QMtCBcQL45cpA/79vISMkHvazm08N7mBocye9HzepseTzQSuqj4JY0Xnsthoen2xcalpoaRRDdR05cIKj7ekPnc/XLqvqsOUqgmycZqgI44f23RDIRbfChur2FJe2xGCU9ABpJxVVKNE+17OBAXQXDjQF9Mq7zgUOjlYkjUZrLRBAtCIOMCKAF4QJR1tLI6/s28rg8gkDJFYCFtkheK9nHKv8s5oUOGeAROtr3cA4vL95zXEfD2nM+l5jFPub4hkwdHLuTHp+q0XdVfy4Pj+dfmWmkyD4Ml+wd9PKVRtaqi3knZhE/FmcRdsL1VZJEmOROqanxog6gwR5El6yABHajmTACRBAtCIOKCKAF4QLxY1EWY/HrDJ4B9JKGS2whrMw7OOgCaDjW0XD00unnfI6OVBAxi+3YQfDZkccadLhWNvR7fXlfZ1dem3gZT+9YxRe2PLSoqaGNR0fMYKjBjyhXLw431TGCYylGVkUmW6kn2kO0rO5gX0dwLC9aLC4UhMFBBNCCcIFosZhxlTWdC7Y6uKHFdFy3s8EmY4U7rEg75+Nfey2G1NsNnS3Fe9tgqxbS/T0eWxSY0YcdBM/UKN8Qvrn0Vg7WVWKVZRK8/DprQd+RMI6Xdq7F1aYlFk8asfC5KodhRn9iPEUAfbyOxYX2vGhEXrQgDAIigBaEC8Q4/zCeyz3M5bYINNKxhgvbVJVMCLpwZ632PZzDpMVNcPsMunx66KE3PnLrbFE+GGa3j5Wdaz7p86MNcp93EDxbakl10nSMaUFRtI608G76VurMrSjA/JA4fptycZawOx2RFy0Ig4uowtFPRBUOoa8pisJDW76lsrqBS2yhOKNhs6qcXKcmls26DoPOudvjjzTVkl5TjpfOmfH+YWhV6n4aee/oqFzQmzQTRjh0VDwbvRFwO5ads+c1d7TZPpn6dX2zKFBRFFYWHGJ59l6q2lqIN/hxR8JYEnshT1lRFBrMbbhotJ2z00L3OpquSE46EUQLfUZU4eieCKD7iQighf5gkW18eeQA3x85TJvNypSgSG6MG4FXN8GzRbbx3I7VbC0vIEHlTTWt1KnM3DFsLFtK8ylvaSLe6MeSIaOIcB9c6Qz9Ifm1GFKdDHybd3btpXtaoq+/y8515+30zazPy2axLZIAXDhALV+r8/nzpMsZ4RPcr2MR7DqDaNF0RegjIoDungig+4kIoIXBpqylkU3l+WyvKKSsoo4H5SScJPus83alnI/J5DpiCMWNDKmW9apSrooZztrCHErbGolw8eK2YaO5ZBAuTuxtHcHK2Th+5vpsgugTW2sfr78XAgLUtbdw5Q/LeFEehxkbqZRSSSsyCm0eEv+cfX2/j0mwE50Lhb4kAujuie/LBOEi9F7GNj7N3kuK5EOTzUwBjeykkknYvw4eJwWwSinCBz3RkifReOJl0/G/zP38igQi8SDb1MCbuzfRYrGwKCpxgO+ob9kXOp5d8NqTEn190Vr7XKXXVhCtNlAmm3iHA0wkgBH4kEU9+xvLOVRXSbyX30mPbTK3k1ZdjFalZrRvCDqRotGrOhYXGucHox4vFhYKQn8Sr2aCcJHZVlHINzkZvCCPxUNyAgnKFBMvs5sYxRN/yQUAH5xpwtJ53HgCWEYmMXiillQkYOQe2zD+dnAbCyKHoT5u4eKplLU08knmbvZWlWLUu3BNbBJTA/s3HaG/OJboO5vFjQq6rzf0emvtc+XppKdWbuWfZHI78aRIPoD99yFC8eDVtPV8PPu6Lsd9nruft9M3E6XyxIxMOS28MHYuEwMi+vkOBEEQep8IoAXhIvN13gHm2ELswfNRgZIrk5QAtlLOQqJoVawcoo6rie7cpxUrGiSk44LBcMkdq02mutWEv0v3AV9+Ux13rP+cCTZ/FivhVDW38se69WTFVnPHsLG9f6ODgL2G7+dnfVxOP+c4d2e4MQCbBtpsNpJxLC83iQA+b86lrr0FL51L5/Y91SV8cGA7T8qj8VPs+ffZSj1PbP+R5XOX4Ovs1q/3cKGT282cX0t+BeH8JwJoQbjINLa3MwRDl+0GdJRgIkup579kE4k7vpI9+FEUha/IYzR+Di3BmxQz7YoNdyfdaa/7bvoWZtuCmU84SBCDJ8NsXjyZtYPFUYkY9S6nPcf5qL8X/PWUoihIx/0fqySJR0dO56mtP3Xd9xTn+F9OOpfYQvCTji1ejZUMjFR8+bbgMLcNHd3bw75odZS385nQLhqtCEI/EgG0IFxkxgSGsqshjxHysZJvsqKwVSqnWmqnRNfKpKBovss/xLtkEGhz5pCmgUqlhRGyN7KioJIkrIrM56pcZgVF46Jx6uaKdtuqinhBGeuQzeAp6YhXGdlVXczckLi+uF3hqEN1lfwnazdHGuqI9DRyY9wIh9zlrPoq/rp/M9uri9CrNMwLHcJvhk/C3UnH5IBIjM4u7GmtZuRxnQO3UE6Mu7fD7DNAdUsz8XQtKegnO1PdevIa1sK5E41WBKH/iQBaEC4yV0UO56u8A3zSnsU0OQgzNlapivHycOPz6beiUdlzme8aNo5VxdlUtDQx1SuBJK8AHtv2A483bidCcidTrsPfxY1rQ+M6g+qTscg21pfmgqzwLflcooTic9zMpAkLevXZlYgTzs7PZUd4dscqLpFDWaSEkWNq4L7yL3lu3CVMDoigqLmeezauYIE1nF8wFZNsYWVRAb+qXcGy2dehllQ8PXYOSzet5LBSR7jsTo66kf1SDe+OXtzlekm+QexvLGe4cizlQ1EUDmhqucknusv+Qs+JRiuC0L9EGbt+IsrYCYNJbVsLHx/exYaSPLQqFXPD47g5bhTOmlMHsmablQ1leaRVlbC9vIDmtnbi1UaKaUar0/LmlCsJPCEPutnSzj0bViC3WkmxelNJKzup5GaGMFry47BSx/uaQ/xw2e2iiUYfkRWFhT98zI1t0cRLxs7tGUotn+nz+Gr+Lby8ez2WwmYWcizdRFEUXtbs4ddjJncu9KxoaeLLIwcoaKwnxuDNwshEvE+SelPV2sxNaz5lujWQqUoQZmR+UBVS6NzCv2ZfL/6v+5hotCL0BlHGrnviVUwQLkJGvQtLU6ay9AzbJuc31XHvxhX42vQEWV2QseGGlmut0bij5QdbIY9v/Z6PZzlWY/jg4A6MJg23y4mdebXTlCBeZjc7pUoyVQ28Ov7SbgMqqyyzrjSH1OJctCo1l4QPYbxfmEOe7sXAKsscaarFWaMlxNXzjI8rMTXQZrEyFMcmOMPwotVipqSlkYyachYr4Q7pNZIkMcxqIKO2ojOA9ndx556ECae9pq+zG/834xre2b+Fxyq3oZFUzA2J44nE+SJ47gf2xaur8HlirsiLFoQ+Il7JBEHolqIoPLHtB+a2BzNDCgHJvu0zcviUbO6WEpinhLG+eRtHGmuJ9Dg2y7m6KIv75ASHYDdMcidG8iQwxIdXkxbi6aTvcs3C5nq+zDtAWXMj2Y3VqNpkpspBtGPjpbK1TAyJ4LGRMy6aIHp1cRZ/3rMRraKiVbES5OrBc+PmEuluPO2xerUGs2LDhoLmuAjZhkK7YkOn0uDr7EZ5cwsxeKIoCnk0Uk0bR1TNJDpHntOYQ90MvDLx0nM6Vui5jrzojiBa5EULQu86feFWQRAuavnNdVS3mJjGsZbNkiSxgAj2Uo1ZsaGSJHxUemrbWxyOtSgyTicpsOWq0jLaL/ikwfPGsjxuWfsZ1bk1hJVrMbZoaZDbGYYXc6RQnrCNZGNxHvtry3v/Zgeh/TVlvJq2gbss8fzBNpbXbBMY2WTg3tQvabUeq9OtKMpJv271dXYj2sObdVKJw/Z1UjFxnj74OrtybVwy36oLyFMa+ANpfMBBNlNKjlxHanGuw3WE80dJfRTVL67CsmU3WMwozWUDPSRBuGCIAFoQhG6ZLGbcJKcuiwT1R+czLcjUKe2U2JqJMzhWXpgSEEmqVOqwrU5pJ0OuZbxf16+VLbKN53eu4T5bIlcTzSQpkPuk4UwhiOXkAuAsaZgo+7OueOC79PWHf2fu4VJbGNGSPW1DJUnMIIQg2YW1JTnUtbfw7I5VTP76XcZ9+TYP/Pw1uY01Dud4duwc1juV8pY6nW+UI7ylTmeDUzlPj5kDwET/cJbEj+JP7EGDRDMWCjGhRUVJdR1v7vu53+9b6B0l9VHUr6sSQbQg9DIRQAuC0K1YTx/qaKNEcSw/to9qfNBTQCNvqdO5MXYE7lrHetB3J4wjTVvNx9Jh9inVfKMc4WlpO0O9/MhsqEI+YcZ0b00pPuiJkRxzfGcTyl6O7X8xLWspaqongq5NasKsruQ31nLn+i9oKzXxijyet5lKaJUTd2/4grKWps59Q90MrJh3C4uSUzAO8WZxSgpfzLuZULdj9cCnBkWiRoUONc8zljekyTzCSGworCw4hEW29cv9Cr2vpD6KjBXuWLbsRjG3iyBaEHqBCKAFQeiWTq3hN4mTeFOdzmaljGKlmTVKER9ykGpVO1+6FnFz0hjuGjauy7H+Lu78e86NxMeF8q1LMT9SyFj8Catx4k/b1/PrjV/SbrOecFTXvObjt7QoVraqKpgVEtO7NzpIRRuMZEsNXbbnappokS04m1VcL8fgITmhk9TMkUIZJ/vx3+y9Dvs7qdQkewdyXXQyl4XFoz9hMV9Bcz0WZO4iAaNkT60Jkly5k2HIKJgs5r67SaFfZKxwp/rFVfYguq5goIcjCOc1sYhQEITTWhSVSICrO//J3MNaUxkxnt68H381w7z8T3usl86Za6KT+HfWbp5gFMG4gQSXWsN5pz6DT3P2cusQe2e6ZGMgVbSSpzQSJXl0nmM1RYTixhqKSFWXMT00muHGgD67X7Cnk0hInXWxB8pNQ0Zyb/mXBNpcSMIbMzKrpCIaNBYkRSLBauiymDJBNrKu+liO+PqSXP68NxWL1UabYmWowY9nxswhyPXYz1ijUmFEj5vkWMowRHJDUiRRzuoCIZquCELvEAG0IAhnZIJ/OBP8z60c1obSXBIlb4Ilt85taknFfFson+cf7gygndQanhg1kz/sWssUOZAgxYUMdR0ZUh0pxgBadU48HT6XMb4hfVaB40hjLa/v3ciO6mIkCab4RfC7EdMIcOmaRtEfhhr8eHnCfP60O5WP2g5jRWakMYj3Rl/F6uJsdqoqaZTNuKHtzFMvpwU/F/vPOr22jBd3reUuWzxxGLCisKaumHtSv+CLeTejVdkXeUa7e9MgmWlTrOilY28NlUoLGpUKtzNo1y6cH0TTFUHoORFAC4LQ58yyDf1JqnHoUHdJ4ZgZHEOkh5Evcw+Qa2pipE8kz0dcjkHn3OX43lbVauKuDV9wiTWEXzAFm6KwurKIO9Z/zmdzl+CqPX3L8r4wzi+M5ZcsobqtBb1ag7uTjnablYKGWrbLFeymChc0XKFEEIeBH9WFvBp7OQD/OrybS21hDJHsdaC1SMwnjIPWOtaX5na2UPd1dmVKQCTLyjO5WRmCs6ShUTHzD9VhbohJ7gy0hQtDSX0UJSsggd1oJowAEUQLwlkRAbQgCH1ugn8476Vv4yolCtfjUgQ2q8qZHNS1znCku/GMm7z0pi/y9jNC9maOFNq57UoiKbaYeCdjC5IioddomBc2hFhPn167rkW28c/MNL4+cpAmSzsjvAO5J3GCQ1UTSZLwdXbtfPz8zjVUV9TzKhPwknQcURp5m3RaJSu/Gz6VET5BABQ01jGZaEoUE7upRAFG4kuE1Y3C5nqHcTw9ZjZ/2LWWR8u34aPSUy23cmVEAncljO+1exUGF9F0RRDOjQigBUHoc2FuBq6IHMYf8/cy1xaCAR07VZVkaxv5eOjcgR5ep4M1FYyQvRxWLdoUmRqlleIjOUxWAmiWZH6Vt4IlQ0Zy69DRvXLdJ7f9SEVVA7fb4jCiY2dlJXenruCD6VcT4+ndZf+ylkY2l+fzR3kCOsk+MxwpeXCXMoz/6PJYHDW8c99wDy++asnjCE2Mxx8VEq+zFydJzSTXoQ7nddZoeXH8PGrbWqhobSbEzbNLZRXhwnNi0xURRAvC6YkAWhCEfvFg0mRSfIP4OvcATeY6xgaE8XRMMl79kJpxpgJcPSitMTGSYzO/mylHhcQLylg0kn1B4QxbMM9n7mJacNQZdQPsTlZ9FXuqSnnJNg7t0fPPJhSbrPDhwe28MqFrN7+8xlqi1J7oFMe0ijgMFLc1YFNk1EfPNSEwnDfLC3mBcXhI9hSUy5RwnlJ24HaK4Niod8God+nRfQnnF7G4UBDOjihjJwhCv5AkiRlB0bwx5Ur+b9a13J0wflAEzzZFxirLAFwTk8RaVQn5SmPn8xspZT7hncEzgJekY7zsz5ri7B5ff19tGYkYO4PnDiMUX/bVnLxeb5CrB0VyE1ZFdtheRDM+Ti6dwTNAZl0VcwjtDJ4BXCUt8wljVWFWj8cvXDhE0xVBOHNiBloQhAtKem0ZK/MO0mBuY1xAGPPDhuKs0XbZr6q1mdf3bmRD+RFsisxY7xB+O2Iqj4+awSu7N+CDHisyNbZWnJSucw1aRcJik7tsP1teOmdqpLYu26tpxXCSVudgzxGPNfjweV0uV8tRaCU19Uo7/1Znc1PcCOraW9laUYBaUtFkbsP3JC/1LmiotIrazoIjsbhQEM7M/7d397FVX2eewJ9r8xJIvcGUd5uE8lYnjnHMMKFpQoaWmIQQUJJmp8yMNNFWO4qm7eafbqeadmejHTWoaKTZ0SirzTTb3U003TSiMk22LxQqyrYluEwxnWaASTMhdMwlJgTjlkLDm8/+QXEg4Jcftu+17/18pEi+l8P14eiEfHP8PL+TSx7uWRC5XC52P/x4sacBJe25V3fH3/9zW3yke1bckMbF7sq34+SE7nhm+SNRdclj2N45fy7Wbfn7WPROddyXbowxURE/jMPxnTHt8dV7/iCqx0+IVzo7orKiIn569HDsfPVf4k/P1/c8Ou90Oh9PVP5D/NWyB+LWQT6P+vT5c7Hm2/8r/vDs/GjKXSgd+U06F39T+bN4pKEpHrmknvlSvzzzTvznH383fnbszZhSMSHe6j4Vj8xdFFMmTIz/vrc1bq6YHN2RYv/5zvg3MS6eTEt7HnPXnVL818qfxSO3NcXam24Z1PwpXTWTDsTU/3Svco4yVVG91vPf+yBAF4gADcPryKkT8ftbvhr/pft3ozp3ISynlOJ/VvxzfHBeTXyq4cM9Y186uC9e/Nk/xuPnLw+nX8u9FrPmTYv/0HBnz3unzp2JP/n+12PCqVx8+PyMeCfOxfcq83HbrNr4iyUrhuR51P/U2RGf2fHNeH+6LqpjXOzt7oz7Zn8w/mzx8p7Q25uOUyfi7XdOxk1V1XHwRGd85of/Nz53vimm5C6Ux+TTyXgyfhI3VlTF/d03RkXkYnvl4Tj7vor48vKPxbhKP4jk6momHYjJq2qi8kOLBegyJED3zd+cQEn4QccbcVtuSk94jrjwP64f7a6J5w79y2UBel/nkag7N+mKW8Nv6Z4cOy65wS8iYuKYcfE/PvJv48WDe+OH+TdifOWY+OQH7owVs+YP2WUut06eEd9c/e+i9ci/xq/OvhOff/+sqLn+hgH93hkTq3ouedn0+t5Y0V3bE54jImpy18dHozY6J3fH9rMXHmP30Rs/GB/7QIPwDHCN/O0JlIRc5CLiytOSFFfk5JhxfVW8UtF5xfDDuZMx4/orbxycMGZsrJt/W6ybf9tVv3d3SpGLGFSgHltRGctmXvlM7Cw63zkVt6b3XfEHfn8aH2Ouq4wn7m4e1OdTXvJdc2PS6RNRcea0Wmh4D0/hAErC3TM/ED9Nx6IzvduQl1KKbRWHYsXsBZeNfeCmm2NP7u34p3Ss57329OvYWnEofn9B44C/56632uOPv/e1WLrpqfjoS1+Ov/nHH15xs2IhLZ5WEz+tfPuy91JK8dMxnbF4em2RZsVotrelKt5+ckukM6cjHf9FsacDI4Ya6AJRAw3D76s/3xP/e/8/xO/9tonwJ5VH4+zEXHx5+ceueObxT44eir/48XdjYndljIvKOJJOxX9s+r24/8a6Xj79cnvezsdndnwz/uj8gmiKKXEsTsfXK16P66dcH39915rh+OP168SZ0/GH3/s/0XC6Oj6SZsX5SLGl4lDkJ/wmnrtnXYxXssE1qpl0IKZ8YeWFn7JoKiwLaqD7JkAXiAANhbH/+Fvx0ht741dnTsfSGTfGvbMX9hocz3V3xyudHXG2+3wsev/MuC5DwPzT7S3R0FkVd+beDRLnUnf8eWVr/LflDw/pVd9ZHP3NyXhmb2tsf/NAVORy0Vy7IP79LUvjhl4eiQcDVTPpwG8vWlksRJcBAbpvjiOAknJz9bS4uXragMaOqaiIpimzrun7vPrLo/EHMefyz8tVRF1ucuzveqtoAXrqhOvj80tWxOdjRVG+P6Ur3zU3YtuBmOQZ0aAGGuBaTLnu+ngzTl7xfkecjGnXva8IM4Lhl++aG3tbquLczj0X6qLdVkiZEqABrsG6BY3x9coD8at04Ta/lFJsi0NxZmyKvJPrqQAADiNJREFU352mYY/SprmQcqeEA+AaPPSBWyP/61/GFw78OOZW3BDH4p2YMH5c/O2dD0ZlztkEpS/fNTfiyS2/rYsOddGUFU2EBaKJEErT8dOnYt/xt6J6/IS4edK0IbtcBUYLzYWlSRNh35xAAwxC9fiJceeMOcWeBhRNvmtu5Fsi6jUXUkb8nBEAGDR10ZQTARoAGBL5rrkXQnRKQjQlTQkHADBkNBdSDpxAAwBDKt81N7q2HY2zL7dFnD3jedGUHAEaABhyFy9dOftym0tXKDkCdC82btwY9fX1UVlZGW1tbb2O27x5c9TV1cWCBQtiw4YNBZwhAIx8mgspRQJ0LxoaGmLTpk1x99139zrm/Pnz8elPfzo2b94c+/bti+effz72799fwFkCwMh3sbnw7MttkY7/wmk0o54A3Yu6urpYuHBhn2N27doV8+fPjzlz5sTYsWNj3bp18eKLLxZohgAweqiLppQI0IOQz+dj9uzZPa9ra2sjn88XcUYAMHKpi6ZUlPVj7Jqbm6Ojo+OK99evXx9r1qzp9/e7shcAstvbUhX1scfNhYxaZR2gt27dOqjfX1NTE+3t7T2v29vbo7a2ttfxf7evtefr35laG0um9j4WAErZ3paqqNm2JaZ8YWXE8V9ErvqmYk+prG3/0Sux/UevFHsao0ZZB+iBSild9f0lS5bEa6+9FgcPHoxZs2bFCy+8EM8//3yvn/PYLR8arikCwKjj0pWRY/ldDbH8roae13+54WtFnM3Ipwa6F5s2bYrZs2dHa2trrF69OlatWhUREYcPH47Vq1dHRMSYMWPiqaeeinvvvTduueWW+PjHPx4333xzMacNAKOK5kJGo1zq7XiVIZXL5WL3w48XexoAMGLVP3wixtzRFLlx451EF1lF9dpefwKPE2gAYIRw6QqjhRpoAGDEUBfNaOAEGgAYUdRFM9IJ0ADAiHNpiHbpCiONAA0AjEgXby48t3OPEM2IIkADACOa5kJGGk2EAMCIp7mQkcQJNAAwKmguZKQQoAGAUeNiXbTmQopJgAYARh110RSTAA0AjEr5rrkXQnRKQjQFpYkQABi1NBdSDE6gAYBRTXMhhSZAAwCjnuZCCkmABgBKhuZCCkENNABQUtRFM9ycQAMAJUddNMNJgAYASpK6aIaLAA0AlLS9LVVxbuceIZohI0ADACVPcyFDSRMhAFAWNBcyVJxAAwBlQ3MhQ0GABgDKiuZCBkuABgDKkrporpUaaACgbKmL5lo4gQYAypq6aLISoAGAsndpiFYXTX8EaACAeLe50KUr9EeABgC4hOZC+qOJEADgPTQX0hcn0AAAV6G5kN4I0AAAvXDpClcjQAMA9ENdNJcSoAEABiDfNfdCiE5JiC5zmggBAAZIcyERTqABADLRXIgADQCQkebC8iZAAwBcI82F5UkNNADAIKiLLj9OoAEABklddHkRoAEAhoC66PIhQAMADKG9LVVxbuceIbqECdAAAENMc2Fp00QIADAMNBeWLifQAADDRHNhaRKgAQCGkebC0iNAAwAUgLro0qEGGgCgQNRFlwYn0AAABaQuevQToAEACuzSEK0uevRRwgEAUAT5rrmRb4mojz0x5o6miPeEaKUdI5cT6D5s3Lgx6uvro7KyMtra2nodN2fOnFi0aFE0NTXF7bffXsAZAgCj3cXmwrP/r7XnH42GI5sT6D40NDTEpk2b4rHHHutzXC6Xi+3bt8fkyZMLNDMAoJRcPI2+qGbblpjyhZURx3+h0XAEEqD7UFdXN+CxKaVhnAkAUE4uf1rH4ki/flOIHkGUcAyBXC4X99xzTyxZsiSeeeaZYk8HACgBntYxcpX9CXRzc3N0dHRc8f769etjzZo1A/qMHTt2xMyZM+Po0aPR3NwcdXV1sWzZsqGeKgBQZt5tNGzraTR0El18ZR+gt27dOujPmDnzwkaeOnVqPPTQQ7Fr166rBui/29fa8/XvTK2NJVNrB/29AYDSt7el6rK66Fz1TUP6+dt/9Eps/9ErQ/qZpazsA/RA9VbjfOrUqTh//nxUVVXFyZMnY8uWLfHEE09cdexjt3xoOKcIAJSw4bzFcPldDbH8roae13+54WtD8rmlSg10HzZt2hSzZ8+O1tbWWL16daxatSoiIg4fPhyrV6+OiIiOjo5YtmxZ3HbbbbF06dJ44IEHYuXKlcWcNgBQotRFjwy55PERBZHL5WL3w48XexoAQImof/hEjLmjKXLjxg95XXRF9VpPGOuDE2gAgFHo4gUsLl0pPAEaAGCUynfNvRCiUxKiC0gTIQDAKDaczYVcnRNoAIBRTnNhYQnQAAAlIN81N/a2VMXZl9su1EUL0cNGgAYAKCGaC4efGmgAgBKjLnp4CdAAACUo3zU3YtuBmBRtMfbDiy8/jRaoB0WABgAoUfmuuZFviajZ9t2e9y6cSi+O9Os3hehrJEADAJS4fNfcd79uiaiPthhzR1OEEH1NNBECAJQZjYaD4wQaAKAMaTS8dk6gAQDKlAtYro0ADQBQxlzAkp0ADQBA7G2pinM790Q6c7rYUxnx1EADABARF0J0zbYtxZ7GiOcEGgCAHpc+8o6rE6ABACADARoAADIQoAEAIAMBGgAAMhCgAQAgAwEaAAAyEKABACADARoAADIQoAEAIAMBGgAAMhCgAQAgAwEaAAAyEKABACADARoAADIQoAEAIAMBGgAAMhCgAQAgAwEaAAAyEKABACADAZoh8ZOjh4o9hRHPGvXPGvXN+vTPGvXPGvXN+jAQAjRDYre/cPpljfpnjfpmffpnjfpnjfpmfRgIARoAADIQoAEAIINcSikVexLlIJfLFXsKAAADJiL2bkyxJ1AubEIAgNKghAMAADIQoAEAIAMBmmuycePGqK+vj8rKymhra+t13Jw5c2LRokXR1NQUt99+ewFnWHwDXaPNmzdHXV1dLFiwIDZs2FDAGRZfZ2dnNDc3x8KFC2PlypXR1dV11XHlto8Gsicef/zxWLBgQTQ2NsaePXsKPMPi62+Ntm/fHjfccEM0NTVFU1NTfPGLXyzCLIvnE5/4REyfPj0aGhp6HVPOe6i/9Sn3/cMAJLgG+/fvT6+++mpavnx52r17d6/j5syZk44dO1bAmY0cA1mjc+fOpXnz5qU33ngjnTlzJjU2NqZ9+/YVeKbF89nPfjZt2LAhpZTSl770pfS5z33uquPKaR8NZE9861vfSqtWrUoppdTa2pqWLl1ajKkWzUDW6Pvf/35as2ZNkWZYfD/4wQ9SW1tbuvXWW6/66+W+h/pbn3LfP/TPCTTXpK6uLhYuXDigsalMGygHska7du2K+fPnx5w5c2Ls2LGxbt26ePHFFws0w+J76aWX4tFHH42IiEcffTS+8Y1v9Dq2XPbRQPbEpeu2dOnS6OrqiiNHjhRjukUx0H9vymXPXM2yZcuiurq6118v9z3U3/pElPf+oX8CNMMql8vFPffcE0uWLIlnnnmm2NMZcfL5fMyePbvndW1tbeTz+SLOqLCOHDkS06dPj4iI6dOn9/of8HLaRwPZE1cbc+hQ+dyeNpA1yuVy8fLLL0djY2Pcf//9sW/fvkJPc0Qr9z3UH/uH/niMHb1qbm6Ojo6OK95fv359rFmzZkCfsWPHjpg5c2YcPXo0mpubo66uLpYtWzbUUy2awa5ROTwfvLc1evLJJy97ncvlel2PUt9Hlxronnjv6Vg57KWLBvJnXbx4cbS3t8fEiRPjO9/5Tjz44IPx85//vACzGz3KeQ/1x/6hPwI0vdq6deugP2PmzJkRETF16tR46KGHYteuXSUVfAa7RjU1NdHe3t7zur29PWprawc7rRGlrzWaPn16dHR0xIwZM+LNN9+MadOmXXVcqe+jSw1kT7x3zKFDh6KmpqZgcyy2gaxRVVVVz9erVq2KT37yk9HZ2RmTJ08u2DxHsnLfQ/2xf+iPEg4Grbc6sVOnTsWJEyciIuLkyZOxZcuWPjvCS1lva7RkyZJ47bXX4uDBg3HmzJl44YUXYu3atQWeXfGsXbs2nn322YiIePbZZ+PBBx+8Yky57aOB7Im1a9fGc889FxERra2tMWnSpJ5SmHIwkDU6cuRIz793u3btipSS8HOJct9D/bF/6FdxehcZ7VpaWlJtbW267rrr0vTp09N9992XUkopn8+n+++/P6WU0uuvv54aGxtTY2Njqq+vT+vXry/mlAtuIGuUUkrf/va308KFC9O8efPKbo2OHTuWVqxYkRYsWJCam5vT8ePHU0r20dX2xNNPP52efvrpnjGf+tSn0rx589KiRYv6fBJOqepvjZ566qlUX1+fGhsb0x133JF27txZzOkW3Lp169LMmTPT2LFjU21tbfrKV75iD12iv/Up9/1D/3IpaTMFAICBUsIBAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGQjQAACQgQANAAAZCNAAAJCBAA0AABkI0AAAkIEADQAAGfx/tc5ZkLrmw0YAAAAASUVORK5CYII=" } }, "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 2 : Keras for MNIST\n", "\n", "Keras is a deep learning API for Python, built on top of TensorFlow, that provides a convenient\n", "way to define and train any kind of deep learning model. Keras was initially\n", "developed for research, with the aim of enabling fast deep learning experimentation.\n", "\n", "Through TensorFlow, Keras can run on top of different types of hardware - GPU, TPU, or plain CPU — and can be seamlessly scaled to thousands of machines. Keras is also a popular framework on Kaggle, the machine learning competition website, where most deep learning competitions have been won using Keras." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let’s look at a concrete example of a neural network that uses the Python library\n", "Keras to learn to classify handwritten digits. \n", "\n", "The problem we’re trying to solve here is to classify grayscale images of handwritten\n", "digits (28x28 pixels) into their 10 categories (0 through 9). We’ll use the MNIST\n", "dataset, a classic in the machine learning community, which has been around almost\n", "as long as the field itself and has been intensively studied. It’s a set of 60'000 training images, plus 10'000 test images, assembled by the National Institute of Standards and Technology (the NIST in MNIST) in the 1980s. You can think of \n", "“solving” MNIST as the “Hello World” of deep learning—it’s what you do to verify \n", "that your algorithms are working as expected. As you become a machine learning practitioner, you’ll see MNIST come up over and over again in scientific papers, blog posts, and so on." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Loading the MNIST dataset in Keras\n", "The MNIST dataset comes preloaded in Keras, in the form of a set of four NumPy\n", "arrays." ] }, { "cell_type": "code", "execution_count": 97, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras.datasets import mnist\n", "(X_train, y_train), (X_test, y_test) = mnist.load_data()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`X_train` and `y_train` form the training set, the data that the model will\n", "learn from. The model will then be tested on the test set, `X_test` and `y_test`. The images are encoded as NumPy arrays, and the labels are an array of digits, ranging from 0 to 9. The images and labels have a one-to-one correspondence.\n", "Let’s look at the training data:" ] }, { "cell_type": "code", "execution_count": 98, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(60000, 28, 28)" ] }, "execution_count": 98, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train.shape" ] }, { "cell_type": "code", "execution_count": 99, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "60000" ] }, "execution_count": 99, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(y_train)" ] }, { "cell_type": "code", "execution_count": 100, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)" ] }, "execution_count": 100, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_train" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us display an the fourth digit:" ] }, { "cell_type": "code", "execution_count": 101, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAANpElEQVR4nO3db6xU9Z3H8c9HtxpDS4TlSpCSvbXyhKwpbSaySbGyaRbUaLAmEokSTIj0ASY2qXENakqMGt0sbWpcmtBVSrUrmrQKD0yRJY3YJ4TRsAqarmggFdF70ZhSo7LY7z64h+aKd35zmf/l+34lNzNzvnPmfDP64cyc35nzc0QIwJnvrH43AKA3CDuQBGEHkiDsQBKEHUji73q5sRkzZsTw8HAvNwmkcvDgQR09etQT1doKu+0rJP1U0tmS/jMiHiw9f3h4WPV6vZ1NAiio1WoNay1/jLd9tqT/kHSlpHmSltue1+rrAeiudr6zXyrpQES8FRHHJW2RtLQzbQHotHbCPlvSH8c9frta9jm2V9uu266Pjo62sTkA7ej60fiI2BgRtYioDQ0NdXtzABpoJ+yHJc0Z9/ir1TIAA6idsO+RNNf212yfI+kGSds60xaATmt56C0iTti+VdJ2jQ29PRYR+zvWGYCOamucPSKek/Rch3oB0EWcLgskQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1Ioq0pm20flHRM0meSTkRErRNNAei8tsJe+eeIONqB1wHQRXyMB5JoN+wh6XnbL9lePdETbK+2XbddHx0dbXNzAFrVbtgXRsS3JF0paY3t75z6hIjYGBG1iKgNDQ21uTkArWor7BFxuLodkfSMpEs70RSAzms57Lan2P7KyfuSFkva16nGAHRWO0fjZ0p6xvbJ1/mviPhtR7oC0HEthz0i3pL0jQ72AqCLGHoDkiDsQBKEHUiCsANJEHYgiU78EAYDbPfu3cX6448/Xqzv2rWrWN+3r/VTK9avX1+sX3jhhcX6iy++WKyvWLGiYW3BggXFdc9E7NmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnG2c8ATz31VMPabbfdVly32aXCIqJYX7RoUbF+9Gjja5HefvvtxXWbadZbadtbtmxpa9t/i9izA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASjLMPgBMnThTre/bsKdZvueWWhrWPPvqouO7ll19erN9zzz3F+sKFC4v1Tz/9tGFt2bJlxXW3b99erDdTqzGp8Hjs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZB8ATTzxRrK9atarl1168eHGxXvotvCRNnTq15W03e/12x9HnzJlTrK9cubKt1z/TNN2z237M9ojtfeOWTbe9w/Yb1e207rYJoF2T+Rj/C0lXnLLsTkk7I2KupJ3VYwADrGnYI2KXpA9OWbxU0ubq/mZJ13a2LQCd1uoBupkRcaS6/66kmY2eaHu17brterPrnQHonraPxsfYVf8aXvkvIjZGRC0iakNDQ+1uDkCLWg37e7ZnSVJ1O9K5lgB0Q6th3ybp5LjGSklbO9MOgG5pOs5u+0lJiyTNsP22pB9JelDS07ZXSTokqfzD5OTuvvvuYv2BBx4o1m0X62vWrGlYu++++4rrtjuO3sz999/ftdd++OGHi3W+Nn5e07BHxPIGpe92uBcAXcTpskAShB1IgrADSRB2IAnCDiTBT1w74N577y3Wmw2tnXvuucX6kiVLivWHHnqoYe28884rrtvMJ598Uqw///zzxfqhQ4ca1ppNudzsMtZLly4t1vF57NmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnG2Sfpww8/bFjbsGFDcd1mP1FtNo7+7LPPFuvtOHDgQLF+4403Fuv1er3lbV9//fXF+h133NHya+OL2LMDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKMs0/S8ePHG9bandaq2SWRR0bKc3Bs2rSpYW3r1vIl/ffv31+sHzt2rFhvdg7BWWc13p/cdNNNxXWnTJlSrOP0sGcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ5+kc845p2HtggsuKK7bbJx8eHi4WG82lt2O2bNnF+vNpnR+5513ivUZM2Y0rF1zzTXFddFZTffsth+zPWJ737hl62wftr23+ruqu20CaNdkPsb/QtIVEyz/SUTMr/6e62xbADqtadgjYpekD3rQC4AuaucA3a22X6k+5k9r9CTbq23XbdfbPYccQOtaDfvPJH1d0nxJRyStb/TEiNgYEbWIqA0NDbW4OQDtainsEfFeRHwWEX+R9HNJl3a2LQCd1lLYbc8a9/B7kvY1ei6AwdB0nN32k5IWSZph+21JP5K0yPZ8SSHpoKTvd6/FwXD++ec3rDW7rvvVV19drL///vvF+sUXX1ysl+Ypv/nmm4vrTp8+vVi/4YYbivVm4+zN1kfvNA17RCyfYPGjXegFQBdxuiyQBGEHkiDsQBKEHUiCsANJ8BPXDliwYEGxPsinCe/atatYf+GFF4r1Zj+/veiii067J3QHe3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJx9uQ+/vjjYr3ZOHqzOj9xHRzs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZk1uyZEm/W0CPsGcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ09u+/bt/W4BPdJ0z257ju3f2X7N9n7bt1XLp9veYfuN6nZa99sF0KrJfIw/IemHETFP0j9JWmN7nqQ7Je2MiLmSdlaPAQyopmGPiCMR8XJ1/5ik1yXNlrRU0ubqaZslXdulHgF0wGkdoLM9LOmbknZLmhkRR6rSu5JmNlhnte267fogz3kGnOkmHXbbX5b0a0k/iIg/ja9FREiKidaLiI0RUYuI2tDQUFvNAmjdpMJu+0saC/qvIuI31eL3bM+q6rMkjXSnRQCd0HTozWPXCn5U0usR8eNxpW2SVkp6sLrd2pUO0VVvvvlmv1tAj0xmnP3bklZIetX23mrZWo2F/GnbqyQdkrSsKx0C6IimYY+I30tqNBPAdzvbDoBu4XRZIAnCDiRB2IEkCDuQBGEHkuAnrslddtllxfrYyZE4E7BnB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkGGdP7pJLLinW586dW6w3+z18qc6Vi3qLPTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJME4O4rWrl1brK9atarl9R955JHiuvPmzSvWcXrYswNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEpOZn32OpF9KmikpJG2MiJ/aXifpFkmj1VPXRsRz3WoU/XHdddcV61u2bCnWd+zY0bC2bt264rqbNm0q1qdMmVKs4/Mmc1LNCUk/jIiXbX9F0ku2T/4X/ElE/Hv32gPQKZOZn/2IpCPV/WO2X5c0u9uNAeis0/rObntY0jcl7a4W3Wr7FduP2Z7WYJ3Vtuu266OjoxM9BUAPTDrstr8s6deSfhARf5L0M0lflzRfY3v+9ROtFxEbI6IWETWuOQb0z6TCbvtLGgv6ryLiN5IUEe9FxGcR8RdJP5d0affaBNCupmG3bUmPSno9In48bvmscU/7nqR9nW8PQKdM5mj8tyWtkPSq7b3VsrWSltuer7HhuIOSvt+F/tBnU6dOLdaffvrpYv2uu+5qWNuwYUNx3WZDc/wE9vRM5mj87yV5ghJj6sDfEM6gA5Ig7EAShB1IgrADSRB2IAnCDiThiOjZxmq1WtTr9Z5tD8imVqupXq9PNFTOnh3IgrADSRB2IAnCDiRB2IEkCDuQBGEHkujpOLvtUUmHxi2aIelozxo4PYPa26D2JdFbqzrZ2z9ExITXf+tp2L+wcbseEbW+NVAwqL0Nal8SvbWqV73xMR5IgrADSfQ77Bv7vP2SQe1tUPuS6K1VPemtr9/ZAfROv/fsAHqEsANJ9CXstq+w/QfbB2zf2Y8eGrF90Partvfa7uuP76s59EZs7xu3bLrtHbbfqG4nnGOvT72ts324eu/22r6qT73Nsf0726/Z3m/7tmp5X9+7Ql89ed96/p3d9tmS/lfSv0h6W9IeScsj4rWeNtKA7YOSahHR9xMwbH9H0p8l/TIi/rFa9m+SPoiIB6t/KKdFxL8OSG/rJP2539N4V7MVzRo/zbikayXdrD6+d4W+lqkH71s/9uyXSjoQEW9FxHFJWyQt7UMfAy8idkn64JTFSyVtru5v1tj/LD3XoLeBEBFHIuLl6v4xSSenGe/re1foqyf6EfbZkv447vHbGqz53kPS87Zfsr26381MYGZEHKnuvytpZj+bmUDTabx76ZRpxgfmvWtl+vN2cYDuixZGxLckXSlpTfVxdSDF2HewQRo7ndQ03r0ywTTjf9XP967V6c/b1Y+wH5Y0Z9zjr1bLBkJEHK5uRyQ9o8Gbivq9kzPoVrcjfe7nrwZpGu+JphnXALx3/Zz+vB9h3yNpru2v2T5H0g2StvWhjy+wPaU6cCLbUyQt1uBNRb1N0srq/kpJW/vYy+cMyjTejaYZV5/fu75Pfx4RPf+TdJXGjsi/KemufvTQoK+LJP1P9be/371JelJjH+v+T2PHNlZJ+ntJOyW9Iem/JU0foN4el/SqpFc0FqxZfeptocY+or8iaW/1d1W/37tCXz153zhdFkiCA3RAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kMT/A38cJNEbCe0NAAAAAElFTkSuQmCC\n", "text/plain": [ "<Figure size 432x288 with 1 Axes>" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "digit = X_train[4]\n", "plt.imshow(digit, cmap=plt.cm.binary)\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 102, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "9" ] }, "execution_count": 102, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_train[4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And the test data:" ] }, { "cell_type": "code", "execution_count": 103, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(10000, 28, 28)" ] }, "execution_count": 103, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_test.shape" ] }, { "cell_type": "code", "execution_count": 104, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "10000" ] }, "execution_count": 104, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(y_test)" ] }, { "cell_type": "code", "execution_count": 105, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([7, 2, 1, ..., 4, 5, 6], dtype=uint8)" ] }, "execution_count": 105, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_test" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before training, we’ll preprocess the data by reshaping it into the shape the model\n", "expects and scaling it so that all values are in the $[0, 1]$ interval. Previously, our training images were stored in an array of shape $(60000, 28, 28)$ of type `uint8` with values in the $[0, 255]$ interval. We’ll transform it into a `float32` array of shape $(60000, 28 * 28)$ with values between $0$ and $1$." ] }, { "cell_type": "code", "execution_count": 106, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(60000, 784)\n", "(10000, 784)\n" ] } ], "source": [ "X_train = X_train.reshape((60000, 28 * 28))\n", "print(X_train.shape)\n", "X_train = X_train.astype(\"float32\") / 255\n", "X_test = X_test.reshape((10000, 28 * 28))\n", "X_test = X_test.astype(\"float32\") / 255\n", "print(X_test.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In addition, we need to _one-hot-encode_ the labels: " ] }, { "cell_type": "code", "execution_count": 107, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "5\n", "[0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]\n" ] } ], "source": [ "from tensorflow.keras.utils import to_categorical\n", "y_train_cat = to_categorical(y_train)\n", "print(y_train[0])\n", "print(y_train_cat[0])" ] }, { "cell_type": "code", "execution_count": 108, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "7\n", "[0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n" ] } ], "source": [ "y_test_cat = to_categorical(y_test)\n", "print(y_test[0])\n", "print(y_test_cat[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The workflow will be as follows: First, we’ll feed the neural network the training data, `X_train` and `y_train`. The network will then learn to associate images and labels. Finally, we’ll ask the network to produce predictions for test_images, and we’ll verify whether these predictions match the labels from `test_labels`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Define Networks\n", "\n", "Building the neural network requires configuring the layers of the model, then compiling the model.\n", "\n", "\n", "#### Setup the layers\n", "\n", "The basic building block of a neural network is the *layer*. A layer extracts a representation from the data fed into it. Hopefully, a series of connected layers results in a representation that is meaningful for the problem at hand.\n", "\n", "Much of deep learning consists of chaining together simple layers. Most layers, like `tf.keras.layers.Dense`, have internal parameters which are adjusted (\"learned\") during training." ] }, { "cell_type": "code", "execution_count": 109, "metadata": {}, "outputs": [], "source": [ "from tensorflow import keras\n", "from tensorflow.keras import layers\n", "\n", "model = keras.Sequential([\n", "layers.Dense(500, activation=\"relu\", input_shape=(784,)),\n", "layers.Dense(50, activation=\"relu\"), \n", "layers.Dense(10, activation=\"softmax\")\n", "])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This network has three layers:\n", "\n", "\n", "\n", "* **\"hidden\"** `layers.Dense`— A densely connected layer of 500 neurons. Each neuron (or node) takes input from all 784 nodes in the previous layer - by specifying an `input_shape` to the first layer in the Sequential model - , weighting that input according to hidden parameters which will be learned during training, and outputs a single value to the next layer. \n", "\n", "* **\"hidden\"** `layers.Dense`— A densely connected layer of 50 neurons. Each neuron (or node) takes input from all 500 nodes in the previous layer, weighting that input according to hidden parameters which will be learned during training, and outputs a single value to the next layer.\n", "\n", "* **output** `layers.Dense` — A 10-node *softmax* layer, with each node representing a class of clothing. As in the previous layer, each node takes input from the 50 nodes in the layer before it. Each node weights the input according to learned parameters, and then outputs a value in the range `[0, 1]`, representing the probability that the image belongs to that class. The sum of all 10 node values is 1.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Compile Networks\n", "\n", "\n", "Before the model is ready for training, it needs a few more settings. These are added during the model's *compile* step:\n", "\n", "\n", "* *Loss function* — An algorithm for measuring how far the model's outputs are from the desired output. The goal of training is this measures loss.\n", "* *Optimizer* —An algorithm for adjusting the inner parameters of the model in order to minimize loss.\n", "* *Metrics* —Used to monitor the training and testing steps. The following example uses *accuracy*, the fraction of the images that are correctly classified." ] }, { "cell_type": "code", "execution_count": 110, "metadata": {}, "outputs": [], "source": [ "model.compile(optimizer='sgd',\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "code", "execution_count": 111, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_1\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " dense_3 (Dense) (None, 500) 392500 \n", " \n", " dense_4 (Dense) (None, 50) 25050 \n", " \n", " dense_5 (Dense) (None, 10) 510 \n", " \n", "=================================================================\n", "Total params: 418,060\n", "Trainable params: 418,060\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 3. Fit Network" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We’re now ready to train the model, which in Keras is done via a call to the model’s\n", "`model.fit` method — we fit the model to its training data.\n" ] }, { "cell_type": "code", "execution_count": 112, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/5\n", "469/469 [==============================] - 7s 15ms/step - loss: 1.0895 - accuracy: 0.7293\n", "Epoch 2/5\n", "469/469 [==============================] - 7s 14ms/step - loss: 0.4560 - accuracy: 0.8832\n", "Epoch 3/5\n", "469/469 [==============================] - 7s 14ms/step - loss: 0.3601 - accuracy: 0.9021\n", "Epoch 4/5\n", "469/469 [==============================] - 7s 14ms/step - loss: 0.3174 - accuracy: 0.9122\n", "Epoch 5/5\n", "469/469 [==============================] - 7s 15ms/step - loss: 0.2906 - accuracy: 0.9188\n" ] } ], "source": [ "history = model.fit(X_train, y_train_cat, epochs=5, batch_size=128)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Two quantities are displayed during training: the __loss__ of the model over the training\n", "data, and the __accuracy__ of the model over the training data. We quickly reach an accuracy of $0.92$ on the training data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Evaluate Network\n", "\n", "On average, how good is our model at classifying never-before-seen digits? Let’s\n", "check by computing average accuracy over the entire test set." ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "313/313 [==============================] - 2s 5ms/step - loss: 0.2665 - accuracy: 0.9258\n", "test_acc: 0.9258000254631042\n" ] } ], "source": [ "test_loss, test_acc = model.evaluate(X_test, y_test_cat)\n", "print(f\"test_acc: {test_acc}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Make Predictions\n", "\n", "Now that we have a trained model, we can use it to predict class probabilities for\n", "new digits—images that weren’t part of the training data, like those from the test set." ] }, { "cell_type": "code", "execution_count": 114, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([7, 2, 1, 0, 4, 1, 4, 9, 6, 9])" ] }, "execution_count": 114, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "predictions=model.predict(X_test[0:10])\n", "np.argmax(predictions, axis=1)" ] }, { "cell_type": "code", "execution_count": 115, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([7, 2, 1, 0, 4, 1, 4, 9, 5, 9], dtype=uint8)" ] }, "execution_count": 115, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_test[0:10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each number of index $i$ in that array corresponds to the probability that digit image\n", "`X_test[0]` belongs to class $i$. This first test digit has the highest probability score (0.9956499, almost 1) at\n", "index 7, so according to our model, it must be a 7:" ] }, { "cell_type": "code", "execution_count": 116, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([2.0822265e-05, 2.8483171e-05, 3.3001334e-04, 1.0540996e-03,\n", " 3.7455920e-05, 8.7879715e-05, 4.8982071e-07, 9.9719810e-01,\n", " 2.7312219e-04, 9.6961117e-04], dtype=float32)" ] }, "execution_count": 116, "metadata": {}, "output_type": "execute_result" } ], "source": [ "predictions[0]" ] }, { "cell_type": "code", "execution_count": 118, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "7" ] }, "execution_count": 118, "metadata": {}, "output_type": "execute_result" } ], "source": [ "predictions[0].argmax()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can check that the test label agrees:" ] }, { "cell_type": "code", "execution_count": 119, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "7" ] }, "execution_count": 119, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_test[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 3 : Toy Neural Network with Keras" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2.7.1\n" ] } ], "source": [ "# Import TensorFlow \n", "import tensorflow as tf\n", "\n", "# Helper libraries\n", "import math\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "\n", "print(tf.__version__)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Define Your Network\n", "\n", "The `tf.keras.models.Sequential` class is a wrapper for the neural network model that treats \n", "the network as a sequence of layers. It implements the Keras model interface with common \n", "methods like `compile()`, `fit()`, and `evaluate()` that are used to train and \n", "run the model. We'll cover these functions soon, but first let's start looking at the layers of the model.\n", "\n", "#### Layers\n", "\n", "The Keras Layer class provides a common interface for a variety of standard neural network layers. You can add a layer to a model using the \n", "model's `add()` method. For example, a simple model with a single hidden layer might look like this for the spiral dataset:" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "model = tf.keras.Sequential()\n", "# From Input to first hidden layer\n", "model.add(tf.keras.layers.Dense(100, activation= tf.nn.relu, \n", " input_shape=(2,)))\n", "# From first hidden layer to output layer\n", "model.add(tf.keras.layers.Dense(3, activation=tf.nn.softmax))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Compile Network\n", "\n", "\n", "Once we have our model built, we need to compile it before it can be run. Compiling the Keras \n", "model calls the backend (tensorflow, theano, etc.) and binds the optimizer, loss function, \n", "and other parameters required before the model can be run on any input data. We'll specify the \n", "loss function to be `categorical_crossentropy`, \n", "and specify `adam` as the optimizer (which is a reasonable default when speed is a priority). And finally, \n", "we can specify what metrics we want to evaluate the model with. Here we'll use `accuracy`.\n" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(300, 2)" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.compile(optimizer='adam',\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])\n", "X.shape" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see the resulting model architecture with the following command:" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_2\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " dense_4 (Dense) (None, 100) 300 \n", " \n", " dense_5 (Dense) (None, 3) 303 \n", " \n", "=================================================================\n", "Total params: 603\n", "Trainable params: 603\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Fit Network\n", "\n", "The model is trained with the `fit()` method, through the following command that specifies the \n", "number of training epochs and the message level (how much information we want displayed on the screen \n", "during training).\n", "\n", "Before starting, we need to one-hot-encode the labels.\n" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[0 0 1]\n", " [0 1 0]\n", " [1 0 0]]\n", "[[1 0 0]\n", " [1 0 0]\n", " [1 0 0]]\n" ] } ], "source": [ "def convertToOneHot(vector, num_classes=None):\n", " result = np.zeros((len(vector), num_classes), dtype='uint8')\n", " result[np.arange(len(vector)), vector] = 1\n", " return result\n", "print(convertToOneHot([2,1,0], 3))\n", "\n", "# One-hot-encoded labels of spiral datset\n", "y_cat = convertToOneHot(y,3)\n", "print(y_cat[:3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "or alternatively" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[1. 0. 0.]\n", " [1. 0. 0.]\n", " [1. 0. 0.]]\n" ] } ], "source": [ "from tensorflow.keras import utils\n", "y_cat = utils.to_categorical(y, 3)\n", "print(y_cat[:3])" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/1000\n", "3/3 [==============================] - 1s 3ms/step - loss: 1.0828 - accuracy: 0.3133\n", "Epoch 2/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 1.0702 - accuracy: 0.3733\n", "Epoch 3/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 1.0585 - accuracy: 0.4300\n", "Epoch 4/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 1.0472 - accuracy: 0.5100\n", "Epoch 5/1000\n", "3/3 [==============================] - 0s 16ms/step - loss: 1.0359 - accuracy: 0.5367\n", "Epoch 6/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 1.0246 - accuracy: 0.5500\n", "Epoch 7/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 1.0135 - accuracy: 0.5533\n", "Epoch 8/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 1.0028 - accuracy: 0.5600\n", "Epoch 9/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.9918 - accuracy: 0.5500\n", "Epoch 10/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.9819 - accuracy: 0.5500\n", "Epoch 11/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.9713 - accuracy: 0.5533\n", "Epoch 12/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.9609 - accuracy: 0.5533\n", "Epoch 13/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.9512 - accuracy: 0.5533\n", "Epoch 14/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.9410 - accuracy: 0.5533\n", "Epoch 15/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.9311 - accuracy: 0.5533\n", "Epoch 16/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.9219 - accuracy: 0.5500\n", "Epoch 17/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.9123 - accuracy: 0.5533\n", "Epoch 18/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.9033 - accuracy: 0.5500\n", "Epoch 19/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8944 - accuracy: 0.5533\n", "Epoch 20/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8854 - accuracy: 0.5533\n", "Epoch 21/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8767 - accuracy: 0.5533\n", "Epoch 22/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8686 - accuracy: 0.5533\n", "Epoch 23/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8600 - accuracy: 0.5533\n", "Epoch 24/1000\n", "3/3 [==============================] - 0s 7ms/step - loss: 0.8524 - accuracy: 0.5533\n", "Epoch 25/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.8446 - accuracy: 0.5533\n", "Epoch 26/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.8372 - accuracy: 0.5533\n", "Epoch 27/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8299 - accuracy: 0.5500\n", "Epoch 28/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8230 - accuracy: 0.5500\n", "Epoch 29/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8165 - accuracy: 0.5500\n", "Epoch 30/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.8102 - accuracy: 0.5500\n", "Epoch 31/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.8041 - accuracy: 0.5533\n", "Epoch 32/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.7979 - accuracy: 0.5533\n", "Epoch 33/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7924 - accuracy: 0.5533\n", "Epoch 34/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7870 - accuracy: 0.5533\n", "Epoch 35/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7817 - accuracy: 0.5533\n", "Epoch 36/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7769 - accuracy: 0.5500\n", "Epoch 37/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7721 - accuracy: 0.5500\n", "Epoch 38/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7673 - accuracy: 0.5533\n", "Epoch 39/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7627 - accuracy: 0.5500\n", "Epoch 40/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.7585 - accuracy: 0.5500\n", "Epoch 41/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7545 - accuracy: 0.5533\n", "Epoch 42/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7504 - accuracy: 0.5567\n", "Epoch 43/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7466 - accuracy: 0.5567\n", "Epoch 44/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7430 - accuracy: 0.5600\n", "Epoch 45/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7392 - accuracy: 0.5600\n", "Epoch 46/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7359 - accuracy: 0.5600\n", "Epoch 47/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.7324 - accuracy: 0.5600\n", "Epoch 48/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.7291 - accuracy: 0.5600\n", "Epoch 49/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.7259 - accuracy: 0.5633\n", "Epoch 50/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.7229 - accuracy: 0.5633\n", "Epoch 51/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.7198 - accuracy: 0.5633\n", "Epoch 52/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.7170 - accuracy: 0.5667\n", "Epoch 53/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7141 - accuracy: 0.5700\n", "Epoch 54/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7113 - accuracy: 0.5733\n", "Epoch 55/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.7084 - accuracy: 0.5733\n", "Epoch 56/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.7059 - accuracy: 0.5733\n", "Epoch 57/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7032 - accuracy: 0.5733\n", "Epoch 58/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.7005 - accuracy: 0.5733\n", "Epoch 59/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6979 - accuracy: 0.5700\n", "Epoch 60/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6956 - accuracy: 0.5700\n", "Epoch 61/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6930 - accuracy: 0.5733\n", "Epoch 62/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6904 - accuracy: 0.5733\n", "Epoch 63/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6880 - accuracy: 0.5733\n", "Epoch 64/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6856 - accuracy: 0.5733\n", "Epoch 65/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6834 - accuracy: 0.5733\n", "Epoch 66/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6809 - accuracy: 0.5733\n", "Epoch 67/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6787 - accuracy: 0.5733\n", "Epoch 68/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6762 - accuracy: 0.5733\n", "Epoch 69/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6741 - accuracy: 0.5733\n", "Epoch 70/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6716 - accuracy: 0.5800\n", "Epoch 71/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6694 - accuracy: 0.5800\n", "Epoch 72/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6673 - accuracy: 0.5833\n", "Epoch 73/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.6650 - accuracy: 0.5833\n", "Epoch 74/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6627 - accuracy: 0.5833\n", "Epoch 75/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6605 - accuracy: 0.5867\n", "Epoch 76/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6584 - accuracy: 0.5833\n", "Epoch 77/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6561 - accuracy: 0.5900\n", "Epoch 78/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6541 - accuracy: 0.5867\n", "Epoch 79/1000\n", "3/3 [==============================] - 0s 7ms/step - loss: 0.6517 - accuracy: 0.5900\n", "Epoch 80/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6496 - accuracy: 0.5933\n", "Epoch 81/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6473 - accuracy: 0.5967\n", "Epoch 82/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6453 - accuracy: 0.5967\n", "Epoch 83/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6430 - accuracy: 0.5967\n", "Epoch 84/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6409 - accuracy: 0.5967\n", "Epoch 85/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6388 - accuracy: 0.5967\n", "Epoch 86/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6366 - accuracy: 0.5967\n", "Epoch 87/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6344 - accuracy: 0.6000\n", "Epoch 88/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6325 - accuracy: 0.6000\n", "Epoch 89/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.6301 - accuracy: 0.6000\n", "Epoch 90/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6279 - accuracy: 0.6033\n", "Epoch 91/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6259 - accuracy: 0.6033\n", "Epoch 92/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6236 - accuracy: 0.6067\n", "Epoch 93/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6216 - accuracy: 0.6133\n", "Epoch 94/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6194 - accuracy: 0.6133\n", "Epoch 95/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6173 - accuracy: 0.6167\n", "Epoch 96/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6150 - accuracy: 0.6233\n", "Epoch 97/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6129 - accuracy: 0.6300\n", "Epoch 98/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6107 - accuracy: 0.6300\n", "Epoch 99/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6087 - accuracy: 0.6333\n", "Epoch 100/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6064 - accuracy: 0.6367\n", "Epoch 101/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6044 - accuracy: 0.6367\n", "Epoch 102/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.6022 - accuracy: 0.6400\n", "Epoch 103/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.6000 - accuracy: 0.6433\n", "Epoch 104/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5979 - accuracy: 0.6433\n", "Epoch 105/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.5957 - accuracy: 0.6433\n", "Epoch 106/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5937 - accuracy: 0.6500\n", "Epoch 107/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5915 - accuracy: 0.6533\n", "Epoch 108/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5895 - accuracy: 0.6600\n", "Epoch 109/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5871 - accuracy: 0.6633\n", "Epoch 110/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5849 - accuracy: 0.6633\n", "Epoch 111/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5829 - accuracy: 0.6700\n", "Epoch 112/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.5806 - accuracy: 0.6733\n", "Epoch 113/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5786 - accuracy: 0.6767\n", "Epoch 114/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5767 - accuracy: 0.6800\n", "Epoch 115/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5744 - accuracy: 0.6800\n", "Epoch 116/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5722 - accuracy: 0.6867\n", "Epoch 117/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5701 - accuracy: 0.6867\n", "Epoch 118/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5678 - accuracy: 0.6867\n", "Epoch 119/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5657 - accuracy: 0.6867\n", "Epoch 120/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5635 - accuracy: 0.6900\n", "Epoch 121/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5615 - accuracy: 0.6900\n", "Epoch 122/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5593 - accuracy: 0.6900\n", "Epoch 123/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5572 - accuracy: 0.6867\n", "Epoch 124/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5551 - accuracy: 0.6867\n", "Epoch 125/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5528 - accuracy: 0.6900\n", "Epoch 126/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5506 - accuracy: 0.6900\n", "Epoch 127/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5485 - accuracy: 0.6900\n", "Epoch 128/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5465 - accuracy: 0.6967\n", "Epoch 129/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5442 - accuracy: 0.7000\n", "Epoch 130/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5421 - accuracy: 0.7000\n", "Epoch 131/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5399 - accuracy: 0.7033\n", "Epoch 132/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5378 - accuracy: 0.7067\n", "Epoch 133/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5357 - accuracy: 0.7067\n", "Epoch 134/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5336 - accuracy: 0.7067\n", "Epoch 135/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5314 - accuracy: 0.7067\n", "Epoch 136/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5295 - accuracy: 0.7067\n", "Epoch 137/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5271 - accuracy: 0.7100\n", "Epoch 138/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5249 - accuracy: 0.7100\n", "Epoch 139/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5231 - accuracy: 0.7100\n", "Epoch 140/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5208 - accuracy: 0.7133\n", "Epoch 141/1000\n", "3/3 [==============================] - ETA: 0s - loss: 0.5236 - accuracy: 0.72 - 0s 3ms/step - loss: 0.5185 - accuracy: 0.7133\n", "Epoch 142/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5164 - accuracy: 0.7133\n", "Epoch 143/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5143 - accuracy: 0.7133\n", "Epoch 144/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5120 - accuracy: 0.7133\n", "Epoch 145/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5102 - accuracy: 0.7167\n", "Epoch 146/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5079 - accuracy: 0.7167\n", "Epoch 147/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5058 - accuracy: 0.7233\n", "Epoch 148/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.5037 - accuracy: 0.7233\n", "Epoch 149/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.5017 - accuracy: 0.7233\n", "Epoch 150/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4995 - accuracy: 0.7200\n", "Epoch 151/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4974 - accuracy: 0.7233\n", "Epoch 152/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4953 - accuracy: 0.7233\n", "Epoch 153/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4933 - accuracy: 0.7233\n", "Epoch 154/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4913 - accuracy: 0.7300\n", "Epoch 155/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4892 - accuracy: 0.7333\n", "Epoch 156/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4871 - accuracy: 0.7333\n", "Epoch 157/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4850 - accuracy: 0.7333\n", "Epoch 158/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4831 - accuracy: 0.7367\n", "Epoch 159/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4810 - accuracy: 0.7433\n", "Epoch 160/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.4789 - accuracy: 0.7400\n", "Epoch 161/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4768 - accuracy: 0.7433\n", "Epoch 162/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4751 - accuracy: 0.7433\n", "Epoch 163/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.4729 - accuracy: 0.7433\n", "Epoch 164/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.4710 - accuracy: 0.7467\n", "Epoch 165/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4688 - accuracy: 0.7467\n", "Epoch 166/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4669 - accuracy: 0.7500\n", "Epoch 167/1000\n", "3/3 [==============================] - 0s 11ms/step - loss: 0.4649 - accuracy: 0.7500\n", "Epoch 168/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4629 - accuracy: 0.7500\n", "Epoch 169/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4611 - accuracy: 0.7533\n", "Epoch 170/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4591 - accuracy: 0.7533\n", "Epoch 171/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4571 - accuracy: 0.7533\n", "Epoch 172/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4552 - accuracy: 0.7567\n", "Epoch 173/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4533 - accuracy: 0.7600\n", "Epoch 174/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4513 - accuracy: 0.7600\n", "Epoch 175/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4495 - accuracy: 0.7600\n", "Epoch 176/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4475 - accuracy: 0.7667\n", "Epoch 177/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4458 - accuracy: 0.7667\n", "Epoch 178/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4437 - accuracy: 0.7667\n", "Epoch 179/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4421 - accuracy: 0.7667\n", "Epoch 180/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4400 - accuracy: 0.7667\n", "Epoch 181/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4382 - accuracy: 0.7767\n", "Epoch 182/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4363 - accuracy: 0.7733\n", "Epoch 183/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4345 - accuracy: 0.7767\n", "Epoch 184/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4327 - accuracy: 0.7767\n", "Epoch 185/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4309 - accuracy: 0.7767\n", "Epoch 186/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4291 - accuracy: 0.7767\n", "Epoch 187/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4272 - accuracy: 0.7767\n", "Epoch 188/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4254 - accuracy: 0.7767\n", "Epoch 189/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4237 - accuracy: 0.7767\n", "Epoch 190/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4219 - accuracy: 0.7767\n", "Epoch 191/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4201 - accuracy: 0.7800\n", "Epoch 192/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4185 - accuracy: 0.7833\n", "Epoch 193/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4166 - accuracy: 0.7833\n", "Epoch 194/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4150 - accuracy: 0.7867\n", "Epoch 195/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4132 - accuracy: 0.7833\n", "Epoch 196/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4114 - accuracy: 0.7867\n", "Epoch 197/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4098 - accuracy: 0.7867\n", "Epoch 198/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4081 - accuracy: 0.7900\n", "Epoch 199/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4064 - accuracy: 0.7933\n", "Epoch 200/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4047 - accuracy: 0.7933\n", "Epoch 201/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.4031 - accuracy: 0.7933\n", "Epoch 202/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.4015 - accuracy: 0.7967\n", "Epoch 203/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3997 - accuracy: 0.7967\n", "Epoch 204/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3982 - accuracy: 0.7967\n", "Epoch 205/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3965 - accuracy: 0.8000\n", "Epoch 206/1000\n", "3/3 [==============================] - 0s 9ms/step - loss: 0.3950 - accuracy: 0.8000\n", "Epoch 207/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.3933 - accuracy: 0.8000\n", "Epoch 208/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3918 - accuracy: 0.8000\n", "Epoch 209/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3901 - accuracy: 0.8000\n", "Epoch 210/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3886 - accuracy: 0.8000\n", "Epoch 211/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3870 - accuracy: 0.8033\n", "Epoch 212/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3855 - accuracy: 0.8067\n", "Epoch 213/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.3839 - accuracy: 0.8067\n", "Epoch 214/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.3823 - accuracy: 0.8067\n", "Epoch 215/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3808 - accuracy: 0.8100\n", "Epoch 216/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3794 - accuracy: 0.8100\n", "Epoch 217/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3778 - accuracy: 0.8133\n", "Epoch 218/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3766 - accuracy: 0.8133\n", "Epoch 219/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3749 - accuracy: 0.8133\n", "Epoch 220/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3733 - accuracy: 0.8133\n", "Epoch 221/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3718 - accuracy: 0.8133\n", "Epoch 222/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3704 - accuracy: 0.8133\n", "Epoch 223/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3689 - accuracy: 0.8133\n", "Epoch 224/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3676 - accuracy: 0.8167\n", "Epoch 225/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3661 - accuracy: 0.8167\n", "Epoch 226/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3647 - accuracy: 0.8200\n", "Epoch 227/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3633 - accuracy: 0.8200\n", "Epoch 228/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3618 - accuracy: 0.8267\n", "Epoch 229/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3603 - accuracy: 0.8267\n", "Epoch 230/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3590 - accuracy: 0.8267\n", "Epoch 231/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3577 - accuracy: 0.8233\n", "Epoch 232/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3563 - accuracy: 0.8233\n", "Epoch 233/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3549 - accuracy: 0.8233\n", "Epoch 234/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3535 - accuracy: 0.8233\n", "Epoch 235/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3522 - accuracy: 0.8267\n", "Epoch 236/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3509 - accuracy: 0.8267\n", "Epoch 237/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3498 - accuracy: 0.8267\n", "Epoch 238/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3484 - accuracy: 0.8300\n", "Epoch 239/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3468 - accuracy: 0.8333\n", "Epoch 240/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3458 - accuracy: 0.8300\n", "Epoch 241/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3443 - accuracy: 0.8300\n", "Epoch 242/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3429 - accuracy: 0.8333\n", "Epoch 243/1000\n", "3/3 [==============================] - 0s 17ms/step - loss: 0.3417 - accuracy: 0.8333\n", "Epoch 244/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3403 - accuracy: 0.8367\n", "Epoch 245/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3391 - accuracy: 0.8333\n", "Epoch 246/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3379 - accuracy: 0.8367\n", "Epoch 247/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3366 - accuracy: 0.8400\n", "Epoch 248/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3353 - accuracy: 0.8400\n", "Epoch 249/1000\n", "3/3 [==============================] - ETA: 0s - loss: 0.3375 - accuracy: 0.83 - 0s 4ms/step - loss: 0.3341 - accuracy: 0.8433\n", "Epoch 250/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3328 - accuracy: 0.8433\n", "Epoch 251/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3316 - accuracy: 0.8433\n", "Epoch 252/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3304 - accuracy: 0.8433\n", "Epoch 253/1000\n", "3/3 [==============================] - 0s 21ms/step - loss: 0.3293 - accuracy: 0.8433\n", "Epoch 254/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3282 - accuracy: 0.8433\n", "Epoch 255/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3269 - accuracy: 0.8433\n", "Epoch 256/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3258 - accuracy: 0.8433\n", "Epoch 257/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3245 - accuracy: 0.8433\n", "Epoch 258/1000\n", "3/3 [==============================] - 0s 20ms/step - loss: 0.3232 - accuracy: 0.8433\n", "Epoch 259/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3221 - accuracy: 0.8433\n", "Epoch 260/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3209 - accuracy: 0.8433\n", "Epoch 261/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3197 - accuracy: 0.8467\n", "Epoch 262/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3186 - accuracy: 0.8433\n", "Epoch 263/1000\n", "3/3 [==============================] - 0s 18ms/step - loss: 0.3175 - accuracy: 0.8433\n", "Epoch 264/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3163 - accuracy: 0.8433\n", "Epoch 265/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3151 - accuracy: 0.8433\n", "Epoch 266/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3141 - accuracy: 0.8467\n", "Epoch 267/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3131 - accuracy: 0.8500\n", "Epoch 268/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3117 - accuracy: 0.8467\n", "Epoch 269/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3108 - accuracy: 0.8567\n", "Epoch 270/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.3096 - accuracy: 0.8567\n", "Epoch 271/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3085 - accuracy: 0.8567\n", "Epoch 272/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3074 - accuracy: 0.8567\n", "Epoch 273/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3064 - accuracy: 0.8600\n", "Epoch 274/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3052 - accuracy: 0.8600\n", "Epoch 275/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3041 - accuracy: 0.8600\n", "Epoch 276/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.3032 - accuracy: 0.8600\n", "Epoch 277/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3020 - accuracy: 0.8633\n", "Epoch 278/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.3010 - accuracy: 0.8633\n", "Epoch 279/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2999 - accuracy: 0.8667\n", "Epoch 280/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2988 - accuracy: 0.8667\n", "Epoch 281/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2978 - accuracy: 0.8667\n", "Epoch 282/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2968 - accuracy: 0.8700\n", "Epoch 283/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2958 - accuracy: 0.8700\n", "Epoch 284/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2948 - accuracy: 0.8700\n", "Epoch 285/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2938 - accuracy: 0.8700\n", "Epoch 286/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2927 - accuracy: 0.8700\n", "Epoch 287/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2917 - accuracy: 0.8700\n", "Epoch 288/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2907 - accuracy: 0.8767\n", "Epoch 289/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2897 - accuracy: 0.8767\n", "Epoch 290/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2888 - accuracy: 0.8800\n", "Epoch 291/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2878 - accuracy: 0.8767\n", "Epoch 292/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.2867 - accuracy: 0.8767\n", "Epoch 293/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2858 - accuracy: 0.8800\n", "Epoch 294/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2849 - accuracy: 0.8800\n", "Epoch 295/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2839 - accuracy: 0.8800\n", "Epoch 296/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2831 - accuracy: 0.8800\n", "Epoch 297/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2819 - accuracy: 0.8800\n", "Epoch 298/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2810 - accuracy: 0.8800\n", "Epoch 299/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2801 - accuracy: 0.8800\n", "Epoch 300/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2792 - accuracy: 0.8833\n", "Epoch 301/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2783 - accuracy: 0.8833\n", "Epoch 302/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2772 - accuracy: 0.8867\n", "Epoch 303/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2764 - accuracy: 0.8833\n", "Epoch 304/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2754 - accuracy: 0.8867\n", "Epoch 305/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2745 - accuracy: 0.8867\n", "Epoch 306/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2736 - accuracy: 0.8867\n", "Epoch 307/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2727 - accuracy: 0.8867\n", "Epoch 308/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2719 - accuracy: 0.8867\n", "Epoch 309/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2709 - accuracy: 0.8867\n", "Epoch 310/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.2700 - accuracy: 0.8867\n", "Epoch 311/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2691 - accuracy: 0.8867\n", "Epoch 312/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2682 - accuracy: 0.8900\n", "Epoch 313/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2673 - accuracy: 0.8900\n", "Epoch 314/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2665 - accuracy: 0.8900\n", "Epoch 315/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2656 - accuracy: 0.8933\n", "Epoch 316/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2648 - accuracy: 0.8933\n", "Epoch 317/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2640 - accuracy: 0.8933\n", "Epoch 318/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2631 - accuracy: 0.8967\n", "Epoch 319/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.2621 - accuracy: 0.8967\n", "Epoch 320/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2613 - accuracy: 0.8967\n", "Epoch 321/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2605 - accuracy: 0.8967\n", "Epoch 322/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2598 - accuracy: 0.8933\n", "Epoch 323/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2588 - accuracy: 0.8967\n", "Epoch 324/1000\n", "3/3 [==============================] - 0s 7ms/step - loss: 0.2580 - accuracy: 0.8967\n", "Epoch 325/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2571 - accuracy: 0.8967\n", "Epoch 326/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2563 - accuracy: 0.8933\n", "Epoch 327/1000\n", "3/3 [==============================] - 0s 7ms/step - loss: 0.2555 - accuracy: 0.8967\n", "Epoch 328/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2547 - accuracy: 0.9000\n", "Epoch 329/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2540 - accuracy: 0.9000\n", "Epoch 330/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2531 - accuracy: 0.9000\n", "Epoch 331/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2524 - accuracy: 0.8967\n", "Epoch 332/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2515 - accuracy: 0.8967\n", "Epoch 333/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2507 - accuracy: 0.8967\n", "Epoch 334/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2498 - accuracy: 0.9000\n", "Epoch 335/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2492 - accuracy: 0.9000\n", "Epoch 336/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2485 - accuracy: 0.9000\n", "Epoch 337/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2476 - accuracy: 0.9000\n", "Epoch 338/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2468 - accuracy: 0.9000\n", "Epoch 339/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.2461 - accuracy: 0.9000\n", "Epoch 340/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2453 - accuracy: 0.9000\n", "Epoch 341/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2445 - accuracy: 0.9000\n", "Epoch 342/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2438 - accuracy: 0.9033\n", "Epoch 343/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2430 - accuracy: 0.9033\n", "Epoch 344/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2422 - accuracy: 0.9000\n", "Epoch 345/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2415 - accuracy: 0.9000\n", "Epoch 346/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2409 - accuracy: 0.9000\n", "Epoch 347/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2401 - accuracy: 0.9033\n", "Epoch 348/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2392 - accuracy: 0.9033\n", "Epoch 349/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2386 - accuracy: 0.9033\n", "Epoch 350/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2379 - accuracy: 0.9067\n", "Epoch 351/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2373 - accuracy: 0.9033\n", "Epoch 352/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.2364 - accuracy: 0.9033\n", "Epoch 353/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2357 - accuracy: 0.9033\n", "Epoch 354/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.2349 - accuracy: 0.9067\n", "Epoch 355/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2343 - accuracy: 0.9067\n", "Epoch 356/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2336 - accuracy: 0.9067\n", "Epoch 357/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2328 - accuracy: 0.9067\n", "Epoch 358/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2321 - accuracy: 0.9133\n", "Epoch 359/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2315 - accuracy: 0.9133\n", "Epoch 360/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2309 - accuracy: 0.9133\n", "Epoch 361/1000\n", "3/3 [==============================] - ETA: 0s - loss: 0.1736 - accuracy: 0.95 - 0s 4ms/step - loss: 0.2300 - accuracy: 0.9133\n", "Epoch 362/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2295 - accuracy: 0.9100\n", "Epoch 363/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2288 - accuracy: 0.9100\n", "Epoch 364/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2281 - accuracy: 0.9067\n", "Epoch 365/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2273 - accuracy: 0.9100\n", "Epoch 366/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2268 - accuracy: 0.9133\n", "Epoch 367/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2261 - accuracy: 0.9133\n", "Epoch 368/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2255 - accuracy: 0.9133\n", "Epoch 369/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2247 - accuracy: 0.9167\n", "Epoch 370/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2241 - accuracy: 0.9167\n", "Epoch 371/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2235 - accuracy: 0.9167\n", "Epoch 372/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2228 - accuracy: 0.9167\n", "Epoch 373/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2222 - accuracy: 0.9167\n", "Epoch 374/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2215 - accuracy: 0.9200\n", "Epoch 375/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2208 - accuracy: 0.9200\n", "Epoch 376/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2202 - accuracy: 0.9200\n", "Epoch 377/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2196 - accuracy: 0.9200\n", "Epoch 378/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2190 - accuracy: 0.9200\n", "Epoch 379/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2182 - accuracy: 0.9200\n", "Epoch 380/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2178 - accuracy: 0.9233\n", "Epoch 381/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2171 - accuracy: 0.9233\n", "Epoch 382/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2164 - accuracy: 0.9233\n", "Epoch 383/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2159 - accuracy: 0.9233\n", "Epoch 384/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2152 - accuracy: 0.9267\n", "Epoch 385/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2147 - accuracy: 0.9267\n", "Epoch 386/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2140 - accuracy: 0.9267\n", "Epoch 387/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2133 - accuracy: 0.9267\n", "Epoch 388/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2127 - accuracy: 0.9267\n", "Epoch 389/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2122 - accuracy: 0.9233\n", "Epoch 390/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2115 - accuracy: 0.9300\n", "Epoch 391/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2110 - accuracy: 0.9267\n", "Epoch 392/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2104 - accuracy: 0.9267\n", "Epoch 393/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2098 - accuracy: 0.9300\n", "Epoch 394/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.2091 - accuracy: 0.9300\n", "Epoch 395/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2086 - accuracy: 0.9267\n", "Epoch 396/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2080 - accuracy: 0.9267\n", "Epoch 397/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2076 - accuracy: 0.9267\n", "Epoch 398/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2070 - accuracy: 0.9300\n", "Epoch 399/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2064 - accuracy: 0.9267\n", "Epoch 400/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2057 - accuracy: 0.9267\n", "Epoch 401/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2051 - accuracy: 0.9267\n", "Epoch 402/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2046 - accuracy: 0.9267\n", "Epoch 403/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2040 - accuracy: 0.9333\n", "Epoch 404/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2034 - accuracy: 0.9333\n", "Epoch 405/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2029 - accuracy: 0.9300\n", "Epoch 406/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2023 - accuracy: 0.9333\n", "Epoch 407/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2017 - accuracy: 0.9333\n", "Epoch 408/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2013 - accuracy: 0.9333\n", "Epoch 409/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2007 - accuracy: 0.9367\n", "Epoch 410/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.2001 - accuracy: 0.9333\n", "Epoch 411/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1995 - accuracy: 0.9367\n", "Epoch 412/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1990 - accuracy: 0.9400\n", "Epoch 413/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1984 - accuracy: 0.9367\n", "Epoch 414/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1980 - accuracy: 0.9367\n", "Epoch 415/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1975 - accuracy: 0.9400\n", "Epoch 416/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1968 - accuracy: 0.9400\n", "Epoch 417/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1963 - accuracy: 0.9400\n", "Epoch 418/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1957 - accuracy: 0.9367\n", "Epoch 419/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1953 - accuracy: 0.9367\n", "Epoch 420/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1948 - accuracy: 0.9367\n", "Epoch 421/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1942 - accuracy: 0.9400\n", "Epoch 422/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1937 - accuracy: 0.9400\n", "Epoch 423/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1932 - accuracy: 0.9400\n", "Epoch 424/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1927 - accuracy: 0.9433\n", "Epoch 425/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1922 - accuracy: 0.9433\n", "Epoch 426/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1916 - accuracy: 0.9433\n", "Epoch 427/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1911 - accuracy: 0.9433\n", "Epoch 428/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1907 - accuracy: 0.9400\n", "Epoch 429/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1900 - accuracy: 0.9400\n", "Epoch 430/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1896 - accuracy: 0.9400\n", "Epoch 431/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1891 - accuracy: 0.9433\n", "Epoch 432/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1886 - accuracy: 0.9433\n", "Epoch 433/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1881 - accuracy: 0.9433\n", "Epoch 434/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.1875 - accuracy: 0.9433\n", "Epoch 435/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1871 - accuracy: 0.9433\n", "Epoch 436/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1866 - accuracy: 0.9433\n", "Epoch 437/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1860 - accuracy: 0.9433\n", "Epoch 438/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1856 - accuracy: 0.9433\n", "Epoch 439/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1851 - accuracy: 0.9433\n", "Epoch 440/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1846 - accuracy: 0.9433\n", "Epoch 441/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1841 - accuracy: 0.9433\n", "Epoch 442/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1838 - accuracy: 0.9433\n", "Epoch 443/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1832 - accuracy: 0.9433\n", "Epoch 444/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1828 - accuracy: 0.9433\n", "Epoch 445/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1822 - accuracy: 0.9433\n", "Epoch 446/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1818 - accuracy: 0.9433\n", "Epoch 447/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1812 - accuracy: 0.9433\n", "Epoch 448/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1809 - accuracy: 0.9433\n", "Epoch 449/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1804 - accuracy: 0.9433\n", "Epoch 450/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1799 - accuracy: 0.9433\n", "Epoch 451/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1794 - accuracy: 0.9433\n", "Epoch 452/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1789 - accuracy: 0.9433\n", "Epoch 453/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1786 - accuracy: 0.9433\n", "Epoch 454/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1781 - accuracy: 0.9433\n", "Epoch 455/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1775 - accuracy: 0.9433\n", "Epoch 456/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1771 - accuracy: 0.9433\n", "Epoch 457/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1767 - accuracy: 0.9433\n", "Epoch 458/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1763 - accuracy: 0.9433\n", "Epoch 459/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1758 - accuracy: 0.9433\n", "Epoch 460/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1753 - accuracy: 0.9467\n", "Epoch 461/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1748 - accuracy: 0.9467\n", "Epoch 462/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1744 - accuracy: 0.9467\n", "Epoch 463/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1741 - accuracy: 0.9500\n", "Epoch 464/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1736 - accuracy: 0.9467\n", "Epoch 465/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1732 - accuracy: 0.9467\n", "Epoch 466/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1727 - accuracy: 0.9467\n", "Epoch 467/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1722 - accuracy: 0.9467\n", "Epoch 468/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1719 - accuracy: 0.9467\n", "Epoch 469/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1715 - accuracy: 0.9467\n", "Epoch 470/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1710 - accuracy: 0.9467\n", "Epoch 471/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1706 - accuracy: 0.9500\n", "Epoch 472/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1701 - accuracy: 0.9467\n", "Epoch 473/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1697 - accuracy: 0.9500\n", "Epoch 474/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1692 - accuracy: 0.9467\n", "Epoch 475/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1688 - accuracy: 0.9500\n", "Epoch 476/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1683 - accuracy: 0.9533\n", "Epoch 477/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1680 - accuracy: 0.9533\n", "Epoch 478/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1675 - accuracy: 0.9533\n", "Epoch 479/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1672 - accuracy: 0.9533\n", "Epoch 480/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1667 - accuracy: 0.9533\n", "Epoch 481/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1663 - accuracy: 0.9533\n", "Epoch 482/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1659 - accuracy: 0.9500\n", "Epoch 483/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1656 - accuracy: 0.9533\n", "Epoch 484/1000\n", "3/3 [==============================] - 0s 12ms/step - loss: 0.1652 - accuracy: 0.9533\n", "Epoch 485/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1646 - accuracy: 0.9533\n", "Epoch 486/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1642 - accuracy: 0.9567\n", "Epoch 487/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1639 - accuracy: 0.9533\n", "Epoch 488/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1634 - accuracy: 0.9533\n", "Epoch 489/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1630 - accuracy: 0.9533\n", "Epoch 490/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1627 - accuracy: 0.9533\n", "Epoch 491/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1622 - accuracy: 0.9533\n", "Epoch 492/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1618 - accuracy: 0.9533\n", "Epoch 493/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1616 - accuracy: 0.9533\n", "Epoch 494/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1610 - accuracy: 0.9567\n", "Epoch 495/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1607 - accuracy: 0.9533\n", "Epoch 496/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1603 - accuracy: 0.9533\n", "Epoch 497/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1598 - accuracy: 0.9567\n", "Epoch 498/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1595 - accuracy: 0.9567\n", "Epoch 499/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1592 - accuracy: 0.9567\n", "Epoch 500/1000\n", "3/3 [==============================] - 0s 22ms/step - loss: 0.1587 - accuracy: 0.9567\n", "Epoch 501/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1583 - accuracy: 0.9567\n", "Epoch 502/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1580 - accuracy: 0.9533\n", "Epoch 503/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1575 - accuracy: 0.9533\n", "Epoch 504/1000\n", "3/3 [==============================] - 0s 21ms/step - loss: 0.1573 - accuracy: 0.9533\n", "Epoch 505/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1568 - accuracy: 0.9533\n", "Epoch 506/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1564 - accuracy: 0.9567\n", "Epoch 507/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1561 - accuracy: 0.9500\n", "Epoch 508/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1556 - accuracy: 0.9567\n", "Epoch 509/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1553 - accuracy: 0.9567\n", "Epoch 510/1000\n", "3/3 [==============================] - 0s 11ms/step - loss: 0.1549 - accuracy: 0.9567\n", "Epoch 511/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1545 - accuracy: 0.9567\n", "Epoch 512/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1541 - accuracy: 0.9567\n", "Epoch 513/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1539 - accuracy: 0.9567\n", "Epoch 514/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1534 - accuracy: 0.9567\n", "Epoch 515/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1530 - accuracy: 0.9567\n", "Epoch 516/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1527 - accuracy: 0.9567\n", "Epoch 517/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1523 - accuracy: 0.9567\n", "Epoch 518/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1519 - accuracy: 0.9567\n", "Epoch 519/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1516 - accuracy: 0.9567\n", "Epoch 520/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1513 - accuracy: 0.9567\n", "Epoch 521/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1509 - accuracy: 0.9567\n", "Epoch 522/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1506 - accuracy: 0.9567\n", "Epoch 523/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1503 - accuracy: 0.9567\n", "Epoch 524/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1499 - accuracy: 0.9567\n", "Epoch 525/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1496 - accuracy: 0.9533\n", "Epoch 526/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1491 - accuracy: 0.9567\n", "Epoch 527/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1488 - accuracy: 0.9567\n", "Epoch 528/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1484 - accuracy: 0.9567\n", "Epoch 529/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1480 - accuracy: 0.9633\n", "Epoch 530/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1477 - accuracy: 0.9600\n", "Epoch 531/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1474 - accuracy: 0.9600\n", "Epoch 532/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1470 - accuracy: 0.9633\n", "Epoch 533/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.1467 - accuracy: 0.9633\n", "Epoch 534/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1463 - accuracy: 0.9633\n", "Epoch 535/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1460 - accuracy: 0.9633\n", "Epoch 536/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1457 - accuracy: 0.9567\n", "Epoch 537/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1453 - accuracy: 0.9600\n", "Epoch 538/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1449 - accuracy: 0.9633\n", "Epoch 539/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.1446 - accuracy: 0.9633\n", "Epoch 540/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.1444 - accuracy: 0.9633\n", "Epoch 541/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1440 - accuracy: 0.9633\n", "Epoch 542/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1436 - accuracy: 0.9633\n", "Epoch 543/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1434 - accuracy: 0.9633\n", "Epoch 544/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1430 - accuracy: 0.9633\n", "Epoch 545/1000\n", "3/3 [==============================] - 0s 7ms/step - loss: 0.1426 - accuracy: 0.9600\n", "Epoch 546/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1424 - accuracy: 0.9600\n", "Epoch 547/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1420 - accuracy: 0.9633\n", "Epoch 548/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1417 - accuracy: 0.9633\n", "Epoch 549/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1414 - accuracy: 0.9633\n", "Epoch 550/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1411 - accuracy: 0.9633\n", "Epoch 551/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1408 - accuracy: 0.9600\n", "Epoch 552/1000\n", "3/3 [==============================] - 0s 11ms/step - loss: 0.1404 - accuracy: 0.9633\n", "Epoch 553/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1401 - accuracy: 0.9633\n", "Epoch 554/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1398 - accuracy: 0.9600\n", "Epoch 555/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1395 - accuracy: 0.9633\n", "Epoch 556/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1392 - accuracy: 0.9633\n", "Epoch 557/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1390 - accuracy: 0.9633\n", "Epoch 558/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1385 - accuracy: 0.9633\n", "Epoch 559/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1382 - accuracy: 0.9633\n", "Epoch 560/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1379 - accuracy: 0.9633\n", "Epoch 561/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1377 - accuracy: 0.9633\n", "Epoch 562/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1374 - accuracy: 0.9667\n", "Epoch 563/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1370 - accuracy: 0.9667\n", "Epoch 564/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1367 - accuracy: 0.9667\n", "Epoch 565/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1365 - accuracy: 0.9667\n", "Epoch 566/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1361 - accuracy: 0.9700\n", "Epoch 567/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1358 - accuracy: 0.9700\n", "Epoch 568/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1356 - accuracy: 0.9633\n", "Epoch 569/1000\n", "3/3 [==============================] - ETA: 0s - loss: 0.1524 - accuracy: 0.95 - 0s 3ms/step - loss: 0.1353 - accuracy: 0.9667\n", "Epoch 570/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1349 - accuracy: 0.9667\n", "Epoch 571/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1346 - accuracy: 0.9700\n", "Epoch 572/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1343 - accuracy: 0.9700\n", "Epoch 573/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1340 - accuracy: 0.9700\n", "Epoch 574/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1337 - accuracy: 0.9667\n", "Epoch 575/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1335 - accuracy: 0.9633\n", "Epoch 576/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1332 - accuracy: 0.9667\n", "Epoch 577/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1329 - accuracy: 0.9700\n", "Epoch 578/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1325 - accuracy: 0.9700\n", "Epoch 579/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1322 - accuracy: 0.9700\n", "Epoch 580/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1322 - accuracy: 0.9700\n", "Epoch 581/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1317 - accuracy: 0.9700\n", "Epoch 582/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1315 - accuracy: 0.9700\n", "Epoch 583/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1312 - accuracy: 0.9667\n", "Epoch 584/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1309 - accuracy: 0.9667\n", "Epoch 585/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1306 - accuracy: 0.9667\n", "Epoch 586/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1304 - accuracy: 0.9667\n", "Epoch 587/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1300 - accuracy: 0.9667\n", "Epoch 588/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1297 - accuracy: 0.9667\n", "Epoch 589/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1294 - accuracy: 0.9700\n", "Epoch 590/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1293 - accuracy: 0.9633\n", "Epoch 591/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1289 - accuracy: 0.9667\n", "Epoch 592/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1286 - accuracy: 0.9700\n", "Epoch 593/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1284 - accuracy: 0.9700\n", "Epoch 594/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1281 - accuracy: 0.9700\n", "Epoch 595/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1278 - accuracy: 0.9700\n", "Epoch 596/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1275 - accuracy: 0.9700\n", "Epoch 597/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1273 - accuracy: 0.9700\n", "Epoch 598/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1270 - accuracy: 0.9700\n", "Epoch 599/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1268 - accuracy: 0.9700\n", "Epoch 600/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1266 - accuracy: 0.9700\n", "Epoch 601/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1262 - accuracy: 0.9733\n", "Epoch 602/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1260 - accuracy: 0.9733\n", "Epoch 603/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1257 - accuracy: 0.9733\n", "Epoch 604/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1254 - accuracy: 0.9733\n", "Epoch 605/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1252 - accuracy: 0.9733\n", "Epoch 606/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1249 - accuracy: 0.9733\n", "Epoch 607/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1246 - accuracy: 0.9800\n", "Epoch 608/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1244 - accuracy: 0.9767\n", "Epoch 609/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1241 - accuracy: 0.9800\n", "Epoch 610/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1238 - accuracy: 0.9800\n", "Epoch 611/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1236 - accuracy: 0.9733\n", "Epoch 612/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1233 - accuracy: 0.9733\n", "Epoch 613/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1231 - accuracy: 0.9700\n", "Epoch 614/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1229 - accuracy: 0.9733\n", "Epoch 615/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1226 - accuracy: 0.9767\n", "Epoch 616/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1224 - accuracy: 0.9733\n", "Epoch 617/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1221 - accuracy: 0.9767\n", "Epoch 618/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1218 - accuracy: 0.9767\n", "Epoch 619/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1216 - accuracy: 0.9800\n", "Epoch 620/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1213 - accuracy: 0.9767\n", "Epoch 621/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1211 - accuracy: 0.9767\n", "Epoch 622/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.1208 - accuracy: 0.9767\n", "Epoch 623/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1205 - accuracy: 0.9767\n", "Epoch 624/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1202 - accuracy: 0.9767\n", "Epoch 625/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1201 - accuracy: 0.9767\n", "Epoch 626/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1199 - accuracy: 0.9767\n", "Epoch 627/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1195 - accuracy: 0.9767\n", "Epoch 628/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1195 - accuracy: 0.9767\n", "Epoch 629/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1191 - accuracy: 0.9800\n", "Epoch 630/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1188 - accuracy: 0.9800\n", "Epoch 631/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1186 - accuracy: 0.9800\n", "Epoch 632/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1185 - accuracy: 0.9800\n", "Epoch 633/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1181 - accuracy: 0.9833\n", "Epoch 634/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1179 - accuracy: 0.9800\n", "Epoch 635/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1176 - accuracy: 0.9800\n", "Epoch 636/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1174 - accuracy: 0.9800\n", "Epoch 637/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1171 - accuracy: 0.9800\n", "Epoch 638/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1169 - accuracy: 0.9800\n", "Epoch 639/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1168 - accuracy: 0.9800\n", "Epoch 640/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1165 - accuracy: 0.9800\n", "Epoch 641/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1162 - accuracy: 0.9800\n", "Epoch 642/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1160 - accuracy: 0.9800\n", "Epoch 643/1000\n", "3/3 [==============================] - 0s 7ms/step - loss: 0.1158 - accuracy: 0.9800\n", "Epoch 644/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1154 - accuracy: 0.9800\n", "Epoch 645/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1153 - accuracy: 0.9833\n", "Epoch 646/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1150 - accuracy: 0.9833\n", "Epoch 647/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1147 - accuracy: 0.9833\n", "Epoch 648/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1145 - accuracy: 0.9833\n", "Epoch 649/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1143 - accuracy: 0.9833\n", "Epoch 650/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1141 - accuracy: 0.9800\n", "Epoch 651/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1139 - accuracy: 0.9767\n", "Epoch 652/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1137 - accuracy: 0.9833\n", "Epoch 653/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1134 - accuracy: 0.9833\n", "Epoch 654/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1132 - accuracy: 0.9800\n", "Epoch 655/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1129 - accuracy: 0.9800\n", "Epoch 656/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1127 - accuracy: 0.9800\n", "Epoch 657/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1125 - accuracy: 0.9800\n", "Epoch 658/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1122 - accuracy: 0.9800\n", "Epoch 659/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1120 - accuracy: 0.9800\n", "Epoch 660/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1118 - accuracy: 0.9800\n", "Epoch 661/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1116 - accuracy: 0.9800\n", "Epoch 662/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1114 - accuracy: 0.9800\n", "Epoch 663/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1111 - accuracy: 0.9800\n", "Epoch 664/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1109 - accuracy: 0.9833\n", "Epoch 665/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1106 - accuracy: 0.9833\n", "Epoch 666/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1105 - accuracy: 0.9800\n", "Epoch 667/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1102 - accuracy: 0.9800\n", "Epoch 668/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1100 - accuracy: 0.9800\n", "Epoch 669/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1099 - accuracy: 0.9800\n", "Epoch 670/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1096 - accuracy: 0.9800\n", "Epoch 671/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1095 - accuracy: 0.9800\n", "Epoch 672/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1091 - accuracy: 0.9800\n", "Epoch 673/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1090 - accuracy: 0.9833\n", "Epoch 674/1000\n", "3/3 [==============================] - ETA: 0s - loss: 0.1040 - accuracy: 1.00 - 0s 3ms/step - loss: 0.1088 - accuracy: 0.9833\n", "Epoch 675/1000\n", "3/3 [==============================] - 0s 8ms/step - loss: 0.1085 - accuracy: 0.9833\n", "Epoch 676/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1083 - accuracy: 0.9833\n", "Epoch 677/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1080 - accuracy: 0.9833\n", "Epoch 678/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1079 - accuracy: 0.9800\n", "Epoch 679/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1077 - accuracy: 0.9800\n", "Epoch 680/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1074 - accuracy: 0.9800\n", "Epoch 681/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1073 - accuracy: 0.9833\n", "Epoch 682/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1071 - accuracy: 0.9800\n", "Epoch 683/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1069 - accuracy: 0.9800\n", "Epoch 684/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1066 - accuracy: 0.9800\n", "Epoch 685/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1064 - accuracy: 0.9800\n", "Epoch 686/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1062 - accuracy: 0.9833\n", "Epoch 687/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1059 - accuracy: 0.9833\n", "Epoch 688/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1058 - accuracy: 0.9833\n", "Epoch 689/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1056 - accuracy: 0.9833\n", "Epoch 690/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1054 - accuracy: 0.9833\n", "Epoch 691/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1053 - accuracy: 0.9833\n", "Epoch 692/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1050 - accuracy: 0.9800\n", "Epoch 693/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1048 - accuracy: 0.9800\n", "Epoch 694/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1046 - accuracy: 0.9833\n", "Epoch 695/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1043 - accuracy: 0.9833\n", "Epoch 696/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1042 - accuracy: 0.9833\n", "Epoch 697/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1040 - accuracy: 0.9833\n", "Epoch 698/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1038 - accuracy: 0.9800\n", "Epoch 699/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1036 - accuracy: 0.9833\n", "Epoch 700/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1034 - accuracy: 0.9800\n", "Epoch 701/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1032 - accuracy: 0.9800\n", "Epoch 702/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1029 - accuracy: 0.9833\n", "Epoch 703/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1028 - accuracy: 0.9833\n", "Epoch 704/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1026 - accuracy: 0.9833\n", "Epoch 705/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1026 - accuracy: 0.9833\n", "Epoch 706/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1022 - accuracy: 0.9833\n", "Epoch 707/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1020 - accuracy: 0.9800\n", "Epoch 708/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1019 - accuracy: 0.9833\n", "Epoch 709/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1016 - accuracy: 0.9833\n", "Epoch 710/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1015 - accuracy: 0.9800\n", "Epoch 711/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1013 - accuracy: 0.9800\n", "Epoch 712/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1011 - accuracy: 0.9800\n", "Epoch 713/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.1010 - accuracy: 0.9800\n", "Epoch 714/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1007 - accuracy: 0.9800\n", "Epoch 715/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1005 - accuracy: 0.9800\n", "Epoch 716/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.1004 - accuracy: 0.9833\n", "Epoch 717/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.1002 - accuracy: 0.9833\n", "Epoch 718/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0999 - accuracy: 0.9833\n", "Epoch 719/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0998 - accuracy: 0.9800\n", "Epoch 720/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0997 - accuracy: 0.9833\n", "Epoch 721/1000\n", "3/3 [==============================] - 0s 9ms/step - loss: 0.0994 - accuracy: 0.9833\n", "Epoch 722/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0993 - accuracy: 0.9833\n", "Epoch 723/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0990 - accuracy: 0.9833\n", "Epoch 724/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0989 - accuracy: 0.9833\n", "Epoch 725/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0987 - accuracy: 0.9833\n", "Epoch 726/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0986 - accuracy: 0.9800\n", "Epoch 727/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0983 - accuracy: 0.9800\n", "Epoch 728/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0982 - accuracy: 0.9800\n", "Epoch 729/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0980 - accuracy: 0.9800\n", "Epoch 730/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0978 - accuracy: 0.9833\n", "Epoch 731/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0976 - accuracy: 0.9833\n", "Epoch 732/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0974 - accuracy: 0.9833\n", "Epoch 733/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0973 - accuracy: 0.9833\n", "Epoch 734/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0970 - accuracy: 0.9833\n", "Epoch 735/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0969 - accuracy: 0.9833\n", "Epoch 736/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0967 - accuracy: 0.9833\n", "Epoch 737/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0965 - accuracy: 0.9833\n", "Epoch 738/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0964 - accuracy: 0.9833\n", "Epoch 739/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0962 - accuracy: 0.9833\n", "Epoch 740/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0960 - accuracy: 0.9833\n", "Epoch 741/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0959 - accuracy: 0.9800\n", "Epoch 742/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0957 - accuracy: 0.9800\n", "Epoch 743/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0956 - accuracy: 0.9833\n", "Epoch 744/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0954 - accuracy: 0.9833\n", "Epoch 745/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0952 - accuracy: 0.9833\n", "Epoch 746/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0950 - accuracy: 0.9833\n", "Epoch 747/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0948 - accuracy: 0.9833\n", "Epoch 748/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.0948 - accuracy: 0.9833\n", "Epoch 749/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0946 - accuracy: 0.9800\n", "Epoch 750/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0943 - accuracy: 0.9833\n", "Epoch 751/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0941 - accuracy: 0.9800\n", "Epoch 752/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0940 - accuracy: 0.9833\n", "Epoch 753/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0938 - accuracy: 0.9833\n", "Epoch 754/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0937 - accuracy: 0.9833\n", "Epoch 755/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0936 - accuracy: 0.9833\n", "Epoch 756/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0933 - accuracy: 0.9833\n", "Epoch 757/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0932 - accuracy: 0.9833\n", "Epoch 758/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0930 - accuracy: 0.9833\n", "Epoch 759/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0928 - accuracy: 0.9833\n", "Epoch 760/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0927 - accuracy: 0.9833\n", "Epoch 761/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0925 - accuracy: 0.9833\n", "Epoch 762/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0925 - accuracy: 0.9800\n", "Epoch 763/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0923 - accuracy: 0.9833\n", "Epoch 764/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0920 - accuracy: 0.9800\n", "Epoch 765/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0918 - accuracy: 0.9833\n", "Epoch 766/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0917 - accuracy: 0.9833\n", "Epoch 767/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0916 - accuracy: 0.9833\n", "Epoch 768/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0914 - accuracy: 0.9833\n", "Epoch 769/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0913 - accuracy: 0.9833\n", "Epoch 770/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0911 - accuracy: 0.9833\n", "Epoch 771/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0909 - accuracy: 0.9833\n", "Epoch 772/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0908 - accuracy: 0.9833\n", "Epoch 773/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0906 - accuracy: 0.9833\n", "Epoch 774/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0904 - accuracy: 0.9833\n", "Epoch 775/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0904 - accuracy: 0.9833\n", "Epoch 776/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0901 - accuracy: 0.9833\n", "Epoch 777/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0900 - accuracy: 0.9833\n", "Epoch 778/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0898 - accuracy: 0.9833\n", "Epoch 779/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0896 - accuracy: 0.9833\n", "Epoch 780/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0895 - accuracy: 0.9833\n", "Epoch 781/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0894 - accuracy: 0.9833\n", "Epoch 782/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0892 - accuracy: 0.9867\n", "Epoch 783/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0890 - accuracy: 0.9833\n", "Epoch 784/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0890 - accuracy: 0.9833\n", "Epoch 785/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0888 - accuracy: 0.9833\n", "Epoch 786/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0887 - accuracy: 0.9833\n", "Epoch 787/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0886 - accuracy: 0.9833\n", "Epoch 788/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0883 - accuracy: 0.9833\n", "Epoch 789/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0882 - accuracy: 0.9833\n", "Epoch 790/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0880 - accuracy: 0.9833\n", "Epoch 791/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0879 - accuracy: 0.9833\n", "Epoch 792/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0877 - accuracy: 0.9833\n", "Epoch 793/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0876 - accuracy: 0.9833\n", "Epoch 794/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0874 - accuracy: 0.9833\n", "Epoch 795/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0873 - accuracy: 0.9833\n", "Epoch 796/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0872 - accuracy: 0.9833\n", "Epoch 797/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0870 - accuracy: 0.9833\n", "Epoch 798/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0869 - accuracy: 0.9833\n", "Epoch 799/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0867 - accuracy: 0.9833\n", "Epoch 800/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0866 - accuracy: 0.9833\n", "Epoch 801/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0864 - accuracy: 0.9833\n", "Epoch 802/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0863 - accuracy: 0.9833\n", "Epoch 803/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0861 - accuracy: 0.9833\n", "Epoch 804/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0860 - accuracy: 0.9833\n", "Epoch 805/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0858 - accuracy: 0.9833\n", "Epoch 806/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0857 - accuracy: 0.9833\n", "Epoch 807/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0855 - accuracy: 0.9833\n", "Epoch 808/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0854 - accuracy: 0.9833\n", "Epoch 809/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0855 - accuracy: 0.9833\n", "Epoch 810/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0852 - accuracy: 0.9867\n", "Epoch 811/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0850 - accuracy: 0.9867\n", "Epoch 812/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0849 - accuracy: 0.9833\n", "Epoch 813/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0847 - accuracy: 0.9833\n", "Epoch 814/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0845 - accuracy: 0.9833\n", "Epoch 815/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0847 - accuracy: 0.9833\n", "Epoch 816/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0843 - accuracy: 0.9800\n", "Epoch 817/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0842 - accuracy: 0.9833\n", "Epoch 818/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0840 - accuracy: 0.9833\n", "Epoch 819/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0839 - accuracy: 0.9833\n", "Epoch 820/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0837 - accuracy: 0.9833\n", "Epoch 821/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0836 - accuracy: 0.9833\n", "Epoch 822/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0835 - accuracy: 0.9867\n", "Epoch 823/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0833 - accuracy: 0.9867\n", "Epoch 824/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0832 - accuracy: 0.9867\n", "Epoch 825/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0830 - accuracy: 0.9867\n", "Epoch 826/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0830 - accuracy: 0.9833\n", "Epoch 827/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0828 - accuracy: 0.9833\n", "Epoch 828/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0827 - accuracy: 0.9833\n", "Epoch 829/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0825 - accuracy: 0.9833\n", "Epoch 830/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0825 - accuracy: 0.9867\n", "Epoch 831/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0823 - accuracy: 0.9867\n", "Epoch 832/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0822 - accuracy: 0.9867\n", "Epoch 833/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0820 - accuracy: 0.9867\n", "Epoch 834/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.0819 - accuracy: 0.9867\n", "Epoch 835/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0818 - accuracy: 0.9867\n", "Epoch 836/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0816 - accuracy: 0.9833\n", "Epoch 837/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0815 - accuracy: 0.9867\n", "Epoch 838/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0813 - accuracy: 0.9867\n", "Epoch 839/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0813 - accuracy: 0.9833\n", "Epoch 840/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0811 - accuracy: 0.9833\n", "Epoch 841/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0810 - accuracy: 0.9833\n", "Epoch 842/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0808 - accuracy: 0.9867\n", "Epoch 843/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0807 - accuracy: 0.9867\n", "Epoch 844/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0806 - accuracy: 0.9867\n", "Epoch 845/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0804 - accuracy: 0.9867\n", "Epoch 846/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0803 - accuracy: 0.9867\n", "Epoch 847/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0802 - accuracy: 0.9867\n", "Epoch 848/1000\n", "3/3 [==============================] - 0s 21ms/step - loss: 0.0801 - accuracy: 0.9867\n", "Epoch 849/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0800 - accuracy: 0.9867\n", "Epoch 850/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0799 - accuracy: 0.9867\n", "Epoch 851/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0798 - accuracy: 0.9867\n", "Epoch 852/1000\n", "3/3 [==============================] - 0s 23ms/step - loss: 0.0796 - accuracy: 0.9867\n", "Epoch 853/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0794 - accuracy: 0.9867\n", "Epoch 854/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0793 - accuracy: 0.9867\n", "Epoch 855/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0792 - accuracy: 0.9867\n", "Epoch 856/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0791 - accuracy: 0.9867\n", "Epoch 857/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0790 - accuracy: 0.9867\n", "Epoch 858/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0788 - accuracy: 0.9900\n", "Epoch 859/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0787 - accuracy: 0.9867\n", "Epoch 860/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0786 - accuracy: 0.9833\n", "Epoch 861/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0785 - accuracy: 0.9833\n", "Epoch 862/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0783 - accuracy: 0.9867\n", "Epoch 863/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0783 - accuracy: 0.9867\n", "Epoch 864/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0781 - accuracy: 0.9867\n", "Epoch 865/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0780 - accuracy: 0.9867\n", "Epoch 866/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0779 - accuracy: 0.9867\n", "Epoch 867/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0777 - accuracy: 0.9867\n", "Epoch 868/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0777 - accuracy: 0.9867\n", "Epoch 869/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0775 - accuracy: 0.9867\n", "Epoch 870/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0774 - accuracy: 0.9900\n", "Epoch 871/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0772 - accuracy: 0.9900\n", "Epoch 872/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0772 - accuracy: 0.9867\n", "Epoch 873/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0770 - accuracy: 0.9867\n", "Epoch 874/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0770 - accuracy: 0.9867\n", "Epoch 875/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0768 - accuracy: 0.9867\n", "Epoch 876/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0766 - accuracy: 0.9867\n", "Epoch 877/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0765 - accuracy: 0.9867\n", "Epoch 878/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0765 - accuracy: 0.9867\n", "Epoch 879/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0763 - accuracy: 0.9867\n", "Epoch 880/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0763 - accuracy: 0.9867\n", "Epoch 881/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0761 - accuracy: 0.9900\n", "Epoch 882/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0760 - accuracy: 0.9900\n", "Epoch 883/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0759 - accuracy: 0.9867\n", "Epoch 884/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0757 - accuracy: 0.9867\n", "Epoch 885/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0757 - accuracy: 0.9867\n", "Epoch 886/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0755 - accuracy: 0.9867\n", "Epoch 887/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0754 - accuracy: 0.9867\n", "Epoch 888/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0754 - accuracy: 0.9867\n", "Epoch 889/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0752 - accuracy: 0.9900\n", "Epoch 890/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0751 - accuracy: 0.9900\n", "Epoch 891/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0750 - accuracy: 0.9900\n", "Epoch 892/1000\n", "3/3 [==============================] - 0s 12ms/step - loss: 0.0748 - accuracy: 0.9900\n", "Epoch 893/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0748 - accuracy: 0.9867\n", "Epoch 894/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0747 - accuracy: 0.9867\n", "Epoch 895/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0745 - accuracy: 0.9867\n", "Epoch 896/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0744 - accuracy: 0.9867\n", "Epoch 897/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0743 - accuracy: 0.9833\n", "Epoch 898/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0742 - accuracy: 0.9900\n", "Epoch 899/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0740 - accuracy: 0.9900\n", "Epoch 900/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0740 - accuracy: 0.9900\n", "Epoch 901/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0739 - accuracy: 0.9900\n", "Epoch 902/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0738 - accuracy: 0.9900\n", "Epoch 903/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0737 - accuracy: 0.9900\n", "Epoch 904/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0735 - accuracy: 0.9900\n", "Epoch 905/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0737 - accuracy: 0.9867\n", "Epoch 906/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0734 - accuracy: 0.9867\n", "Epoch 907/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0733 - accuracy: 0.9900\n", "Epoch 908/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0731 - accuracy: 0.9900\n", "Epoch 909/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0730 - accuracy: 0.9900\n", "Epoch 910/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0729 - accuracy: 0.9900\n", "Epoch 911/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0728 - accuracy: 0.9900\n", "Epoch 912/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0727 - accuracy: 0.9900\n", "Epoch 913/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0726 - accuracy: 0.9900\n", "Epoch 914/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0725 - accuracy: 0.9900\n", "Epoch 915/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0724 - accuracy: 0.9900\n", "Epoch 916/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0723 - accuracy: 0.9867\n", "Epoch 917/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0722 - accuracy: 0.9900\n", "Epoch 918/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0721 - accuracy: 0.9900\n", "Epoch 919/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0719 - accuracy: 0.9900\n", "Epoch 920/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0718 - accuracy: 0.9900\n", "Epoch 921/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0718 - accuracy: 0.9900\n", "Epoch 922/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0717 - accuracy: 0.9900\n", "Epoch 923/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.0716 - accuracy: 0.9900\n", "Epoch 924/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0715 - accuracy: 0.9833\n", "Epoch 925/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0713 - accuracy: 0.9867\n", "Epoch 926/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0713 - accuracy: 0.9867\n", "Epoch 927/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0711 - accuracy: 0.9867\n", "Epoch 928/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0710 - accuracy: 0.9900\n", "Epoch 929/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0709 - accuracy: 0.9900\n", "Epoch 930/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0708 - accuracy: 0.9900\n", "Epoch 931/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0708 - accuracy: 0.9900\n", "Epoch 932/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0706 - accuracy: 0.9900\n", "Epoch 933/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0705 - accuracy: 0.9900\n", "Epoch 934/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0704 - accuracy: 0.9900\n", "Epoch 935/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0703 - accuracy: 0.9900\n", "Epoch 936/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0702 - accuracy: 0.9900\n", "Epoch 937/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0701 - accuracy: 0.9900\n", "Epoch 938/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0701 - accuracy: 0.9900\n", "Epoch 939/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0699 - accuracy: 0.9867\n", "Epoch 940/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0699 - accuracy: 0.9900\n", "Epoch 941/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0697 - accuracy: 0.9900\n", "Epoch 942/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0696 - accuracy: 0.9900\n", "Epoch 943/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0696 - accuracy: 0.9900\n", "Epoch 944/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0694 - accuracy: 0.9900\n", "Epoch 945/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0693 - accuracy: 0.9900\n", "Epoch 946/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0694 - accuracy: 0.9867\n", "Epoch 947/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0691 - accuracy: 0.9900\n", "Epoch 948/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0690 - accuracy: 0.9900\n", "Epoch 949/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0690 - accuracy: 0.9867\n", "Epoch 950/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0689 - accuracy: 0.9900\n", "Epoch 951/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0688 - accuracy: 0.9900\n", "Epoch 952/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0687 - accuracy: 0.9900\n", "Epoch 953/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0686 - accuracy: 0.9900\n", "Epoch 954/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0685 - accuracy: 0.9900\n", "Epoch 955/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0685 - accuracy: 0.9900\n", "Epoch 956/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0684 - accuracy: 0.9900\n", "Epoch 957/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0683 - accuracy: 0.9900\n", "Epoch 958/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0681 - accuracy: 0.9900\n", "Epoch 959/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0680 - accuracy: 0.9900\n", "Epoch 960/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0679 - accuracy: 0.9867\n", "Epoch 961/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0679 - accuracy: 0.9900\n", "Epoch 962/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0677 - accuracy: 0.9900\n", "Epoch 963/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0678 - accuracy: 0.9867\n", "Epoch 964/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0675 - accuracy: 0.9900\n", "Epoch 965/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0675 - accuracy: 0.9900\n", "Epoch 966/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0674 - accuracy: 0.9900\n", "Epoch 967/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0673 - accuracy: 0.9867\n", "Epoch 968/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0672 - accuracy: 0.9900\n", "Epoch 969/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0671 - accuracy: 0.9900\n", "Epoch 970/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0670 - accuracy: 0.9900\n", "Epoch 971/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0670 - accuracy: 0.9900\n", "Epoch 972/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0670 - accuracy: 0.9900\n", "Epoch 973/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0667 - accuracy: 0.9900\n", "Epoch 974/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0667 - accuracy: 0.9900\n", "Epoch 975/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0666 - accuracy: 0.9900\n", "Epoch 976/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0664 - accuracy: 0.9900\n", "Epoch 977/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0665 - accuracy: 0.9900\n", "Epoch 978/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0664 - accuracy: 0.9900\n", "Epoch 979/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0662 - accuracy: 0.9900\n", "Epoch 980/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0661 - accuracy: 0.9900\n", "Epoch 981/1000\n", "3/3 [==============================] - 0s 6ms/step - loss: 0.0660 - accuracy: 0.9900\n", "Epoch 982/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0659 - accuracy: 0.9900\n", "Epoch 983/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0659 - accuracy: 0.9900\n", "Epoch 984/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0658 - accuracy: 0.9900\n", "Epoch 985/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0657 - accuracy: 0.9900\n", "Epoch 986/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0657 - accuracy: 0.9900\n", "Epoch 987/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0655 - accuracy: 0.9867\n", "Epoch 988/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0654 - accuracy: 0.9900\n", "Epoch 989/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0653 - accuracy: 0.9900\n", "Epoch 990/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0653 - accuracy: 0.9867\n", "Epoch 991/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0652 - accuracy: 0.9867\n", "Epoch 992/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0651 - accuracy: 0.9900\n", "Epoch 993/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0650 - accuracy: 0.9900\n", "Epoch 994/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0649 - accuracy: 0.9900\n", "Epoch 995/1000\n", "3/3 [==============================] - 0s 5ms/step - loss: 0.0649 - accuracy: 0.9900\n", "Epoch 996/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0647 - accuracy: 0.9900\n", "Epoch 997/1000\n", "3/3 [==============================] - 0s 4ms/step - loss: 0.0646 - accuracy: 0.9900\n", "Epoch 998/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0645 - accuracy: 0.9900\n", "Epoch 999/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0646 - accuracy: 0.9867\n", "Epoch 1000/1000\n", "3/3 [==============================] - 0s 3ms/step - loss: 0.0644 - accuracy: 0.9900\n" ] }, { "data": { "text/plain": [ "<keras.callbacks.History at 0x7f472477ad50>" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "BATCH_SIZE=128\n", "num_train_examples = X.shape[0]\n", "num_train_examples\n", "model.fit(X, y_cat, epochs=1000, steps_per_epoch=math.ceil(num_train_examples/BATCH_SIZE))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Evaluate Network\n", "\n", "Finally, we can use the following command to evaluate the model:" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "10/10 [==============================] - 0s 2ms/step - loss: 0.0643 - accuracy: 0.9900\n", "Accuracy on test dataset: 0.9900000095367432\n" ] } ], "source": [ "test_loss, test_accuracy = model.evaluate(X, y_cat, steps=math.ceil(num_train_examples/32))\n", "print('Accuracy on test dataset:', test_accuracy)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }