From 0d362947c4f4d4df64a8b57c2955f2ebc2fe3cb5 Mon Sep 17 00:00:00 2001
From: Aaron Spring <aaron.spring@mpimet.mpg.de>
Date: Wed, 9 Jun 2021 09:52:25 +0000
Subject: [PATCH] fix typo in safeguards
---
CHANGELOG.md | 1 +
notebooks/ML_forecast_template.ipynb | 6 +++---
notebooks/ML_train_and_predict.ipynb | 12 ++++++------
3 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a4b5af9..88f755e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,7 @@
### unreleased
+- Fix typo in safeguards in [ML_forecast_template.ipynb](https://renkulab.io/gitlab/aaron.spring/s2s-ai-challenge-template/-/blob/master/notebooks/ML_forecast_template.ipynb): "We did NOT use `test` explicitly in training or implicitly in incrementally adjusting parameters."" (!8, [Aaron Spring](https://renkulab.io/gitlab/aaron.spring))
- Add notebooks showcasing accessing output of different models from different sources: (!2, [Aaron Spring](https://renkulab.io/gitlab/aaron.spring))
- S2S-Project models:
- from from European Weather Cloud:
diff --git a/notebooks/ML_forecast_template.ipynb b/notebooks/ML_forecast_template.ipynb
index d0c07c0..e123c32 100644
--- a/notebooks/ML_forecast_template.ipynb
+++ b/notebooks/ML_forecast_template.ipynb
@@ -75,12 +75,12 @@
"\n",
"If the organizers suspect overfitting, your contribution can be disqualified.\n",
"\n",
- " - [ ] We didnt use 2020 observations in training (explicit overfitting and cheating)\n",
- " - [ ] We didnt repeatedly verify my model on 2020 observations and incrementally improved my RPSS (implicit overfitting)\n",
+ " - [ ] We did not use 2020 observations in training (explicit overfitting and cheating)\n",
+ " - [ ] We did not repeatedly verify my model on 2020 observations and incrementally improved my RPSS (implicit overfitting)\n",
" - [ ] We provide RPSS scores for the training period with script `skill_by_year`, see in section 6.3 `predict`.\n",
" - [ ] We tried our best to prevent [data leakage](https://en.wikipedia.org/wiki/Leakage_(machine_learning)?wprov=sfti1).\n",
" - [ ] We honor the `train-validate-test` [split principle](https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets). This means that the hindcast data is split into `train` and `validate`, whereas `test` is withheld.\n",
- " - [ ] We did use `test` explicitly in training or implicitly in incrementally adjusting parameters.\n",
+ " - [ ] We did not use `test` explicitly in training or implicitly in incrementally adjusting parameters.\n",
" - [ ] We considered [cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics))."
]
},
diff --git a/notebooks/ML_train_and_predict.ipynb b/notebooks/ML_train_and_predict.ipynb
index 9c23068..8d9a779 100644
--- a/notebooks/ML_train_and_predict.ipynb
+++ b/notebooks/ML_train_and_predict.ipynb
@@ -81,12 +81,12 @@
"\n",
"If the organizers suspect overfitting, your contribution can be disqualified.\n",
"\n",
- " - [x] We didnt use 2020 observations in training (explicit overfitting and cheating)\n",
- " - [x] We didnt repeatedly verify my model on 2020 observations and incrementally improved my RPSS (implicit overfitting)\n",
+ " - [x] We did not use 2020 observations in training (explicit overfitting and cheating)\n",
+ " - [x] We did not repeatedly verify my model on 2020 observations and incrementally improved my RPSS (implicit overfitting)\n",
" - [x] We provide RPSS scores for the training period with script `print_RPS_per_year`, see in section 6.3 `predict`.\n",
" - [x] We tried our best to prevent [data leakage](https://en.wikipedia.org/wiki/Leakage_(machine_learning)?wprov=sfti1).\n",
" - [x] We honor the `train-validate-test` [split principle](https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets). This means that the hindcast data is split into `train` and `validate`, whereas `test` is withheld.\n",
- " - [x] We did use `test` explicitly in training or implicitly in incrementally adjusting parameters.\n",
+ " - [x] We did not use `test` explicitly in training or implicitly in incrementally adjusting parameters.\n",
" - [x] We considered [cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics))."
]
},
@@ -1620,9 +1620,9 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python [conda env:s2s-ai]",
+ "display_name": "Python 3",
"language": "python",
- "name": "conda-env-s2s-ai-py"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
@@ -1634,7 +1634,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.7.10"
+ "version": "3.8.6"
},
"toc-autonumbering": true
},
--
GitLab