Skip to content
Snippets Groups Projects
mean_bias_reduction.ipynb 44.8 KiB
Newer Older
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train ML model to correct predictions of week 3-4 & 5-6\n",
    "\n",
    "This notebook create a Machine Learning `ML_model` to predict weeks 3-4 & 5-6 based on `S2S` weeks 3-4 & 5-6 forecasts and is compared to `CPC` observations for the [`s2s-ai-challenge`](https://s2s-ai-challenge.github.io/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Synopsis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Method: `mean bias reduction`\n",
    "\n",
    "- calculate the mean bias from 2000-2019 deterministic ensemble mean forecast\n",
    "- remove that mean bias from 2020 forecast deterministic ensemble mean forecast\n",
    "- no Machine Learning used here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data used\n",
    "\n",
    "type: renku datasets\n",
    "\n",
    "Training-input for Machine Learning model:\n",
    "- hindcasts of models:\n",
    "    - ECMWF: `ecmwf_hindcast-input_2000-2019_biweekly_deterministic.zarr`\n",
    "\n",
    "Forecast-input for Machine Learning model:\n",
    "- real-time 2020 forecasts of models:\n",
    "    - ECMWF: `ecmwf_forecast-input_2020_biweekly_deterministic.zarr`\n",
    "\n",
    "Compare Machine Learning model forecast against against ground truth:\n",
    "- `CPC` observations:\n",
    "    - `hindcast-like-observations_biweekly_deterministic.zarr`\n",
    "    - `forecast-like-observations_2020_biweekly_deterministic.zarr`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resources used\n",
Aaron Spring's avatar
Aaron Spring committed
    "for training, details in reproducibility\n",
    "\n",
    "- platform: MPI-M supercompute 1 Node\n",
    "- memory: 64 GB\n",
    "- processors: 36 CPU\n",
    "- storage required: 10 GB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Safeguards\n",
    "\n",
    "All points have to be [x] checked. If not, your submission is invalid.\n",
    "\n",
    "Changes to the code after submissions are not possible, as the `commit` before the `tag` will be reviewed.\n",
    "(Only in exceptions and if previous effort in reproducibility can be found, it may be allowed to improve readability and reproducibility after November 1st 2021.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Safeguards to prevent [overfitting](https://en.wikipedia.org/wiki/Overfitting?wprov=sfti1) \n",
    "\n",
    "If the organizers suspect overfitting, your contribution can be disqualified.\n",
    "\n",
    "  - [x] We didnt use 2020 observations in training (explicit overfitting and cheating)\n",
    "  - [x] We didnt repeatedly verify my model on 2020 observations and incrementally improved my RPSS (implicit overfitting)\n",
    "  - [x] We provide RPSS scores for the training period with script `skill_by_year`, see in section 6.3 `predict`.\n",
    "  - [x] We tried our best to prevent [data leakage](https://en.wikipedia.org/wiki/Leakage_(machine_learning)?wprov=sfti1).\n",
    "  - [x] We honor the `train-validate-test` [split principle](https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets). This means that the hindcast data is split into `train` and `validate`, whereas `test` is withheld.\n",
    "  - [x] We did use `test` explicitly in training or implicitly in incrementally adjusting parameters.\n",
    "  - [x] We considered [cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics))."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Safeguards for Reproducibility\n",
    "Notebook/code must be independently reproducible from scratch by the organizers (after the competition), if not possible: no prize\n",
    "  - [x] All training data is publicly available (no pre-trained private neural networks, as they are not reproducible for us)\n",
    "  - [x] Code is well documented, readable and reproducible.\n",
    "  - [x] Code to reproduce training and predictions should run within a day on the described architecture. If the training takes longer than a day, please justify why this is needed. Please do not submit training piplelines, which take weeks to train."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "xr.set_options(display_style='text')\n",
    "import numpy as np\n",
    "\n",
    "from dask.utils import format_bytes\n",
    "import xskillscore as xs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get training data\n",
    "\n",
    "preprocessing of input data may be done in separate notebook/script"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hindcast\n",
    "\n",
    "get weekly initialized hindcasts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33m\u001b[1mWarning: \u001b[0mRun CLI commands only from project's root directory.\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# preprocessed as renku dataset\n",
    "!renku storage pull ../data/ecmwf_hindcast-input_2000-2019_biweekly_deterministic.zarr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "hind_2000_2019 = xr.open_zarr(\"../data/ecmwf_hindcast-input_2000-2019_biweekly_deterministic.zarr\", consolidated=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33m\u001b[1mWarning: \u001b[0mRun CLI commands only from project's root directory.\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# preprocessed as renku dataset\n",
    "!renku storage pull ../data/ecmwf_forecast-input_2020_biweekly_deterministic.zarr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "fct_2020 = xr.open_zarr(\"../data/ecmwf_forecast-input_2020_biweekly_deterministic.zarr\", consolidated=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Observations\n",
    "corresponding to hindcasts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33m\u001b[1mWarning: \u001b[0mRun CLI commands only from project's root directory.\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# preprocessed as renku dataset\n",
    "!renku storage pull ../data/hindcast-like-observations_2000-2019_biweekly_deterministic.zarr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs_2000_2019 = xr.open_zarr(\"../data/hindcast-like-observations_2000-2019_biweekly_deterministic.zarr\", consolidated=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33m\u001b[1mWarning: \u001b[0mRun CLI commands only from project's root directory.\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# preprocessed as renku dataset\n",
    "!renku storage pull ../data/forecast-like-observations_2020_biweekly_deterministic.zarr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs_2020 = xr.open_zarr(\"../data/forecast-like-observations_2020_biweekly_deterministic.zarr\", consolidated=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# no ML model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we just remove the mean bias from the ensemble mean forecast."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/work/mh0727/m300524/conda-envs/s2s-ai/lib/python3.7/site-packages/xarray/core/accessor_dt.py:381: FutureWarning: dt.weekofyear and dt.week have been deprecated. Please use dt.isocalendar().week instead.\n",
      "  FutureWarning,\n",
      "/work/mh0727/m300524/conda-envs/s2s-ai/lib/python3.7/site-packages/dask/array/numpy_compat.py:40: RuntimeWarning: invalid value encountered in true_divide\n",
      "  x = np.divide(x1, x2, out)\n"
     ]
    }
   ],
   "source": [
    "bias_2000_2019 = (hind_2000_2019.mean('realization') - obs_2000_2019).groupby('forecast_time.weekofyear').mean().compute()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `predict`\n",
    "\n",
    "Create predictions and print `mean(variable, lead_time, longitude, weighted latitude)` RPSS for all years as calculated by `skill_by_year`. For now RPS, todo: change to RPSS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts import make_probabilistic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33m\u001b[1mWarning: \u001b[0mRun CLI commands only from project's root directory.\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!renku storage pull ../data/hindcast-like-observations_2000-2019_biweekly_tercile-edges.nc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "cache_path='../data'\n",
    "tercile_file = f'{cache_path}/hindcast-like-observations_2000-2019_biweekly_tercile-edges.nc'\n",
    "tercile_edges = xr.open_dataset(tercile_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this is not useful but results have expected dimensions\n",
    "# actually train for each lead_time\n",
    "\n",
    "def create_predictions(fct, bias):\n",
    "    preds = fct - bias.sel(weekofyear=fct.forecast_time.dt.weekofyear)\n",
    "    preds = make_probabilistic(preds, tercile_edges)\n",
    "    return preds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `predict` training period in-sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!renku storage pull ../data/forecast-like-observations_2020_biweekly_terciled.nc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!renku storage pull ../data/hindcast-like-observations_2000-2019_biweekly_terciled.zarr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts import skill_by_year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/work/mh0727/m300524/conda-envs/s2s-ai/lib/python3.7/site-packages/xarray/core/accessor_dt.py:381: FutureWarning: dt.weekofyear and dt.week have been deprecated. Please use dt.isocalendar().week instead.\n",
      "  FutureWarning,\n",
      "/work/mh0727/m300524/conda-envs/s2s-ai/lib/python3.7/site-packages/xarray/core/accessor_dt.py:381: FutureWarning: dt.weekofyear and dt.week have been deprecated. Please use dt.isocalendar().week instead.\n",
      "  FutureWarning,\n"
     ]
    }
   ],
   "source": [
    "preds_is = create_predictions(hind_2000_2019, bias_2000_2019).compute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>RPS</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>year</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000</th>\n",
       "      <td>0.463290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2001</th>\n",
       "      <td>0.501615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002</th>\n",
       "      <td>0.498100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2003</th>\n",
       "      <td>0.499914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2004</th>\n",
       "      <td>0.533146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005</th>\n",
       "      <td>0.486682</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006</th>\n",
       "      <td>0.492787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007</th>\n",
       "      <td>0.555934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008</th>\n",
       "      <td>0.507756</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2009</th>\n",
       "      <td>0.515228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010</th>\n",
       "      <td>0.498032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011</th>\n",
       "      <td>0.548217</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012</th>\n",
       "      <td>0.556501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2013</th>\n",
       "      <td>0.519008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2014</th>\n",
       "      <td>0.521487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2015</th>\n",
       "      <td>0.507068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016</th>\n",
       "      <td>0.520476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017</th>\n",
       "      <td>0.590591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018</th>\n",
       "      <td>0.604847</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019</th>\n",
       "      <td>0.546725</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           RPS\n",
       "year          \n",
       "2000  0.463290\n",
       "2001  0.501615\n",
       "2002  0.498100\n",
       "2003  0.499914\n",
       "2004  0.533146\n",
       "2005  0.486682\n",
       "2006  0.492787\n",
       "2007  0.555934\n",
       "2008  0.507756\n",
       "2009  0.515228\n",
       "2010  0.498032\n",
       "2011  0.548217\n",
       "2012  0.556501\n",
       "2013  0.519008\n",
       "2014  0.521487\n",
       "2015  0.507068\n",
       "2016  0.520476\n",
       "2017  0.590591\n",
       "2018  0.604847\n",
       "2019  0.546725"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "skill_by_year(preds_is)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `predict` test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/work/mh0727/m300524/conda-envs/s2s-ai/lib/python3.7/site-packages/xarray/core/accessor_dt.py:381: FutureWarning: dt.weekofyear and dt.week have been deprecated. Please use dt.isocalendar().week instead.\n",
      "  FutureWarning,\n",
      "/work/mh0727/m300524/conda-envs/s2s-ai/lib/python3.7/site-packages/xarray/core/accessor_dt.py:381: FutureWarning: dt.weekofyear and dt.week have been deprecated. Please use dt.isocalendar().week instead.\n",
      "  FutureWarning,\n"
     ]
    }
   ],
   "source": [
    "preds_test = create_predictions(fct_2020, bias_2000_2019)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>RPS</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>year</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020</th>\n",
       "      <td>0.520714</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           RPS\n",
       "year          \n",
       "2020  0.520714"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "skill_by_year(preds_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Submission"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts import assert_predictions_2020\n",
    "assert_predictions_2020(preds_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "del preds_test['weekofyear']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "preds_test.to_netcdf('../submissions/ML_prediction_2020.nc')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
Aaron Spring's avatar
Aaron Spring committed
    "#!git add ../submissions/ML_prediction_2020.nc"
   ]
  },
  {
   "cell_type": "code",
Aaron Spring's avatar
Aaron Spring committed
   "execution_count": null,
   "metadata": {},
Aaron Spring's avatar
Aaron Spring committed
   "outputs": [],
   "source": [
Aaron Spring's avatar
Aaron Spring committed
    "#!git commit -m \"template_test no ML mean bias reduction\" # whatever message you want"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
Aaron Spring's avatar
Aaron Spring committed
    "#!git tag \"submission-no_ML_mean_bias_reduction-0.0.1\" # if this is to be checked by scorer, only the last submitted==tagged version will be considered"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!git push --tags"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reproducibility"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             total       used       free     shared    buffers     cached\n",
      "Mem:            62         21         41          0          0          5\n",
      "-/+ buffers/cache:         15         47\n",
      "Swap:            0          0          0\n"
     ]
    }
   ],
   "source": [
    "# https://phoenixnap.com/kb/linux-commands-check-memory-usage\n",
    "!free -g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Architecture:          x86_64\n",
      "CPU op-mode(s):        32-bit, 64-bit\n",
      "Byte Order:            Little Endian\n",
      "CPU(s):                72\n",
      "On-line CPU(s) list:   0-71\n",
      "Thread(s) per core:    2\n",
      "Core(s) per socket:    18\n",
      "Socket(s):             2\n",
      "NUMA node(s):          2\n",
      "Vendor ID:             GenuineIntel\n",
      "CPU family:            6\n",
      "Model:                 79\n",
      "Model name:            Intel(R) Xeon(R) CPU E5-2695 v4 @ 2.10GHz\n",
      "Stepping:              1\n",
      "CPU MHz:               1200.000\n",
      "BogoMIPS:              4190.00\n",
      "Virtualization:        VT-x\n",
      "L1d cache:             32K\n",
      "L1i cache:             32K\n",
      "L2 cache:              256K\n",
      "L3 cache:              46080K\n",
      "NUMA node0 CPU(s):     0-17,36-53\n",
      "NUMA node1 CPU(s):     18-35,54-71\n"
     ]
    }
   ],
   "source": [
    "!lscpu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## software"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# packages in environment at /opt/conda:\n",
      "#\n",
      "# Name                    Version                   Build  Channel\n",
      "_libgcc_mutex             0.1                 conda_forge    conda-forge\n",
      "_openmp_mutex             4.5                       1_gnu    conda-forge\n",
      "_tflow_select             2.3.0                       mkl    defaults\n",
      "absl-py                   0.12.0           py38h06a4308_0    defaults\n",
      "aiobotocore               1.2.2              pyhd3eb1b0_0    defaults\n",
      "aiohttp                   3.7.4.post0              pypi_0    pypi\n",
      "aioitertools              0.7.1              pyhd3eb1b0_0    defaults\n",
      "alembic                   1.4.3              pyh9f0ad1d_0    conda-forge\n",
      "ansiwrap                  0.8.4                    pypi_0    pypi\n",
      "appdirs                   1.4.4                    pypi_0    pypi\n",
      "argcomplete               1.12.2                   pypi_0    pypi\n",
      "argon2-cffi               20.1.0           py38h497a2fe_2    conda-forge\n",
      "argparse                  1.4.0                    pypi_0    pypi\n",
      "asciitree                 0.3.3                      py_2    defaults\n",
      "astunparse                1.6.3                      py_0    defaults\n",
      "async-timeout             3.0.1                    pypi_0    pypi\n",
      "async_generator           1.10                       py_0    conda-forge\n",
      "attrs                     20.3.0             pyhd3deb0d_0    conda-forge\n",
      "backcall                  0.2.0              pyh9f0ad1d_0    conda-forge\n",
      "backports                 1.0                        py_2    conda-forge\n",
      "backports.functools_lru_cache 1.6.1                      py_0    conda-forge\n",
      "binutils_impl_linux-64    2.35.1               h193b22a_1    conda-forge\n",
      "binutils_linux-64         2.35                h67ddf6f_30    conda-forge\n",
      "black                     20.8b1                   pypi_0    pypi\n",
      "blas                      1.0                         mkl    defaults\n",
      "bleach                    3.2.1              pyh9f0ad1d_0    conda-forge\n",
      "blinker                   1.4                        py_1    conda-forge\n",
      "bokeh                     2.3.2            py38h06a4308_0    defaults\n",
      "botocore                  1.20.78            pyhd3eb1b0_1    defaults\n",
      "bottleneck                1.3.2            py38heb32a55_1    defaults\n",
      "branca                    0.3.1                    pypi_0    pypi\n",
      "brotlipy                  0.7.0           py38h497a2fe_1001    conda-forge\n",
      "bzip2                     1.0.8                h7f98852_4    conda-forge\n",
      "c-ares                    1.17.1               h36c2ea0_0    conda-forge\n",
      "ca-certificates           2021.4.13            h06a4308_1    defaults\n",
      "cachetools                4.2.2              pyhd3eb1b0_0    defaults\n",
      "cdsapi                    0.5.1                    pypi_0    pypi\n",
      "certifi                   2020.12.5        py38h06a4308_0    defaults\n",
      "certipy                   0.1.3                      py_0    conda-forge\n",
      "cffi                      1.14.4           py38ha65f79e_1    conda-forge\n",
      "cfgrib                    0.9.9.0            pyhd8ed1ab_1    conda-forge\n",
      "cftime                    1.5.0            py38h6323ea4_0    defaults\n",
      "chardet                   4.0.0            py38h578d9bd_1    conda-forge\n",
      "click                     7.1.2                    pypi_0    pypi\n",
      "climetlab                 0.7.0                    pypi_0    pypi\n",
      "climetlab-s2s-ai-challenge 0.6.2                    pypi_0    pypi\n",
      "cloudpickle               1.6.0                      py_0    defaults\n",
      "colorama                  0.4.4                    pypi_0    pypi\n",
      "conda                     4.9.2            py38h578d9bd_0    conda-forge\n",
      "conda-package-handling    1.7.2            py38h8df0ef7_0    conda-forge\n",
      "configargparse            1.4.1                    pypi_0    pypi\n",
      "configurable-http-proxy   1.3.0                         0    conda-forge\n",
      "coverage                  5.5              py38h27cfd23_2    defaults\n",
      "cryptography              3.3.1            py38h2b97feb_1    conda-forge\n",
      "curl                      7.71.1               he644dc0_8    conda-forge\n",
      "cycler                    0.10.0                   py38_0    defaults\n",
      "cython                    0.29.23          py38h2531618_0    defaults\n",
      "cytoolz                   0.11.0           py38h7b6447c_0    defaults\n",
      "dask                      2021.4.0           pyhd3eb1b0_0    defaults\n",
      "dask-core                 2021.4.0           pyhd3eb1b0_0    defaults\n",
      "decorator                 4.4.2                      py_0    conda-forge\n",
      "defusedxml                0.6.0                      py_0    conda-forge\n",
      "distributed               2021.5.0         py38h06a4308_0    defaults\n",
      "distro                    1.5.0                    pypi_0    pypi\n",
      "eccodes                   2.18.0               hf05d9b7_0    conda-forge\n",
      "ecmwf-api-client          1.6.1                    pypi_0    pypi\n",
      "ecmwflibs                 0.3.7                    pypi_0    pypi\n",
      "entrypoints               0.3             pyhd8ed1ab_1003    conda-forge\n",
      "fasteners                 0.16               pyhd3eb1b0_0    defaults\n",
      "findlibs                  0.0.2                    pypi_0    pypi\n",
      "folium                    0.12.1                   pypi_0    pypi\n",
      "freetype                  2.10.4               h5ab3b9f_0    defaults\n",
      "fsspec                    0.9.0              pyhd3eb1b0_0    defaults\n",
      "gast                      0.4.0                      py_0    defaults\n",
      "gcc_impl_linux-64         9.3.0               h70c0ae5_18    conda-forge\n",
      "gcc_linux-64              9.3.0               hf25ea35_30    conda-forge\n",
      "gitdb                     4.0.7                    pypi_0    pypi\n",
      "gitpython                 3.1.14                   pypi_0    pypi\n",
      "google-auth               1.30.1             pyhd3eb1b0_0    defaults\n",
      "google-auth-oauthlib      0.4.4              pyhd3eb1b0_0    defaults\n",
      "google-pasta              0.2.0                      py_0    defaults\n",
      "grpcio                    1.36.1           py38h2157cd5_1    defaults\n",
      "gxx_impl_linux-64         9.3.0               hd87eabc_18    conda-forge\n",
      "gxx_linux-64              9.3.0               h3fbe746_30    conda-forge\n",
      "h5py                      2.10.0           py38hd6299e0_1    defaults\n",
      "hdf4                      4.2.13               h3ca952b_2    defaults\n",
      "hdf5                      1.10.6          nompi_h3c11f04_101    conda-forge\n",
      "heapdict                  1.0.1                      py_0    defaults\n",
      "icu                       68.1                 h58526e2_0    conda-forge\n",
      "idna                      2.10               pyh9f0ad1d_0    conda-forge\n",
      "importlib-metadata        3.4.0            py38h578d9bd_0    conda-forge\n",
      "importlib_metadata        3.4.0                hd8ed1ab_0    conda-forge\n",
      "intel-openmp              2021.2.0           h06a4308_610    defaults\n",
      "ipykernel                 5.4.2            py38h81c977d_0    conda-forge\n",
      "ipython                   7.19.0           py38h81c977d_2    conda-forge\n",
      "ipython_genutils          0.2.0                      py_1    conda-forge\n",
      "jasper                    1.900.1              hd497a04_4    defaults\n",
      "jedi                      0.17.2           py38h578d9bd_1    conda-forge\n",
      "jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge\n",
      "jmespath                  0.10.0                     py_0    defaults\n",
      "joblib                    1.0.1              pyhd3eb1b0_0    defaults\n",
      "jpeg                      9d                   h36c2ea0_0    conda-forge\n",
      "json5                     0.9.5              pyh9f0ad1d_0    conda-forge\n",
      "jsonschema                3.2.0                      py_2    conda-forge\n",
      "jupyter-server-proxy      1.6.0                    pypi_0    pypi\n",
      "jupyter_client            6.1.11             pyhd8ed1ab_1    conda-forge\n",
      "jupyter_core              4.7.0            py38h578d9bd_0    conda-forge\n",
      "jupyter_telemetry         0.1.0              pyhd8ed1ab_1    conda-forge\n",
      "jupyterhub                1.2.2                    pypi_0    pypi\n",
      "jupyterlab                2.2.9                      py_0    conda-forge\n",
      "jupyterlab-git            0.23.3                   pypi_0    pypi\n",
      "jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge\n",
      "jupyterlab_server         1.2.0                      py_0    conda-forge\n",
      "keras-preprocessing       1.1.2              pyhd3eb1b0_0    defaults\n",
      "kernel-headers_linux-64   2.6.32              h77966d4_13    conda-forge\n",
      "kiwisolver                1.3.1            py38h2531618_0    defaults\n",
      "krb5                      1.17.2               h926e7f8_0    conda-forge\n",
      "lcms2                     2.12                 h3be6417_0    defaults\n",
      "ld_impl_linux-64          2.35.1               hea4e1c9_1    conda-forge\n",
      "libaec                    1.0.4                he6710b0_1    defaults\n",
      "libcurl                   7.71.1               hcdd3856_8    conda-forge\n",
      "libedit                   3.1.20191231         he28a2e2_2    conda-forge\n",
      "libev                     4.33                 h516909a_1    conda-forge\n",
      "libffi                    3.3                  h58526e2_2    conda-forge\n",
      "libgcc-devel_linux-64     9.3.0               h7864c58_18    conda-forge\n",
      "libgcc-ng                 9.3.0               h2828fa1_18    conda-forge\n",
      "libgfortran-ng            7.3.0                hdf63c60_0    defaults\n",
      "libgomp                   9.3.0               h2828fa1_18    conda-forge\n",
      "libllvm10                 10.0.1               hbcb73fb_5    defaults\n",
      "libnetcdf                 4.7.4           nompi_h56d31a8_107    conda-forge\n",
      "libnghttp2                1.41.0               h8cfc5f6_2    conda-forge\n",
      "libpng                    1.6.37               hbc83047_0    defaults\n",
      "libprotobuf               3.14.0               h8c45485_0    defaults\n",
      "libsodium                 1.0.18               h36c2ea0_1    conda-forge\n",
      "libssh2                   1.9.0                hab1572f_5    conda-forge\n",
      "libstdcxx-devel_linux-64  9.3.0               hb016644_18    conda-forge\n",
      "libstdcxx-ng              9.3.0               h6de172a_18    conda-forge\n",
      "libtiff                   4.1.0                h2733197_1    defaults\n",
      "libuv                     1.40.0               h7f98852_0    conda-forge\n",
      "llvmlite                  0.36.0           py38h612dafd_4    defaults\n",
      "locket                    0.2.1            py38h06a4308_1    defaults\n",
      "lz4-c                     1.9.3                h2531618_0    defaults\n",
      "magics                    1.5.6                    pypi_0    pypi\n",
      "mako                      1.1.4              pyh44b312d_0    conda-forge\n",
      "markdown                  3.3.4            py38h06a4308_0    defaults\n",
      "markupsafe                1.1.1            py38h497a2fe_3    conda-forge\n",
      "matplotlib-base           3.3.4            py38h62a2d02_0    defaults\n",
      "mistune                   0.8.4           py38h497a2fe_1003    conda-forge\n",
      "mkl                       2021.2.0           h06a4308_296    defaults\n",
      "mkl-service               2.3.0            py38h27cfd23_1    defaults\n",
      "mkl_fft                   1.3.0            py38h42c9631_2    defaults\n",
      "mkl_random                1.2.1            py38ha9443f7_2    defaults\n",
      "monotonic                 1.5                        py_0    defaults\n",
      "msgpack-python            1.0.2            py38hff7bd54_1    defaults\n",
      "multidict                 5.1.0            py38h27cfd23_2    defaults\n",
      "mypy-extensions           0.4.3                    pypi_0    pypi\n",
      "nbclient                  0.5.0                    pypi_0    pypi\n",
      "nbconvert                 6.0.7            py38h578d9bd_3    conda-forge\n",
      "nbdime                    2.1.0                    pypi_0    pypi\n",
      "nbformat                  5.1.2              pyhd8ed1ab_1    conda-forge\n",
      "nbresuse                  0.4.0                    pypi_0    pypi\n",
      "ncurses                   6.2                  h58526e2_4    conda-forge\n",
      "nest-asyncio              1.4.3              pyhd8ed1ab_0    conda-forge\n",
      "netcdf4                   1.5.6                    pypi_0    pypi\n",
      "nodejs                    15.3.0               h25f6087_0    conda-forge\n",
      "notebook                  6.2.0            py38h578d9bd_0    conda-forge\n",
      "numba                     0.53.1           py38ha9443f7_0    defaults\n",
      "numcodecs                 0.7.3            py38h2531618_0    defaults\n",
      "numpy                     1.20.2           py38h2d18471_0    defaults\n",
      "numpy-base                1.20.2           py38hfae3a4d_0    defaults\n",
      "oauthlib                  3.0.1                      py_0    conda-forge\n",
      "olefile                   0.46                       py_0    defaults\n",
      "openssl                   1.1.1k               h27cfd23_0    defaults\n",