Skip to content
Snippets Groups Projects
Commit 9aaef3d8 authored by Aaron Spring's avatar Aaron Spring :baby_symbol:
Browse files

Update README.md

parent 00969ebb
No related branches found
No related tags found
No related merge requests found
Pipeline #198904 passed with stage
in 16 seconds
...@@ -21,7 +21,7 @@ a `Dockerfile`. ...@@ -21,7 +21,7 @@ a `Dockerfile`.
## Join the challenge ## Join the challenge
### 1. The simplest way to join the S2S AI Challenge is forking this renku project. ### 1. The simplest way to join the S2S AI Challenge is forking this renku project.
Ensure you do not fork the gitlab repository, but the renku project. Ensure that you do not fork the gitlab repository, but the renku project.
Fork this template renku project from https://renkulab.io/projects/aaron.spring/s2s-ai-challenge-template/settings. Fork this template renku project from https://renkulab.io/projects/aaron.spring/s2s-ai-challenge-template/settings.
...@@ -29,8 +29,12 @@ Fork this template renku project from https://renkulab.io/projects/aaron.spring/ ...@@ -29,8 +29,12 @@ Fork this template renku project from https://renkulab.io/projects/aaron.spring/
Name your fork `s2s-ai-challenge-$TEAMNAME`. Name your fork `s2s-ai-challenge-$TEAMNAME`.
Your fork will inherit the tags from the template repo. The tag `s2s-ai-challenge` is needed for the `scorer` bot to find your repo.
### 2. Fill our [registration form](https://docs.google.com/forms/d/1KEnATjaLOtV-o4N8PLinPXYnpba7egKsCCH_efriCb4). ### 2. Fill our [registration form](https://docs.google.com/forms/d/1KEnATjaLOtV-o4N8PLinPXYnpba7egKsCCH_efriCb4).
Registrations are not required before October 31st 2021, but highly [appreciated for the flow of information](https://renkulab.io/gitlab/aaron.spring/s2s-ai-challenge/-/issues/4).
### 3. Make the project private ### 3. Make the project private
Now check out the gitlab repository by clicking on "View in gitlab". Now check out the gitlab repository by clicking on "View in gitlab".
...@@ -43,22 +47,16 @@ Now other people cannot steal your idea/code. ...@@ -43,22 +47,16 @@ Now other people cannot steal your idea/code.
Now please modify the README in your fork with team details and a Now please modify the README in your fork with team details and a
description of your method. description of your method.
### 4. Add the `scorer` user to your repo with Reporter permissions Please use different branches if you try out different methods. The scorer finds branches from all branches.
The scorer is not yet ready, but will follow this [verification notebook](https://renkulab.io/gitlab/aaron.spring/s2s-ai-competition-bootstrap/-/blob/master/notebooks/verification_RPSS.ipynb).
### 5. Add a gitlab variable with key `COMPETITION` and name `S2S-AI`
In the gitlab repository, under "Settings" -> "CI/CD" -> "Variables", add the
`COMPETITION` key with value `S2S-AI`, so the `scorer` bot knows where to search
for submissions.
todo: probably not needed instead ensure that project/repo has a certain `tag` ### 4. Add the `scorer` user to your repo with Reporter permissions
The scorer is not yet ready, but will follow this [verification notebook](https://renkulab.io/gitlab/aaron.spring/s2s-ai-challenge-template/-/blob/master/notebooks/verification_RPSS.ipynb).
<img src="docs/screenshots/gitlab_variables.png" width="300"> Todo: How to add scorer to repo
<img src="docs/screenshots/gitlab_add_variable.png" width="300">
## Contribute ## Make Predictions
### 6. Start jupyter on renku or locally ### 5. Start jupyter on renku or locally
The simplest way to contribute is right from the Renku platform - The simplest way to contribute is right from the Renku platform -
just click on the `Environments` tab in your renku project and start a new session. just click on the `Environments` tab in your renku project and start a new session.
This will start an interactive environment right in your browser. This will start an interactive environment right in your browser.
...@@ -73,7 +71,7 @@ renku project URLs - use `renku clone` to clone the project on whichever machine ...@@ -73,7 +71,7 @@ renku project URLs - use `renku clone` to clone the project on whichever machine
Install [renku first with `pipx`](https://renku-python.readthedocs.io/en/latest/installation.html), Install [renku first with `pipx`](https://renku-python.readthedocs.io/en/latest/installation.html),
and then `renku clone https://renkulab.io/gitlab/$YOURNAME/s2s-ai-challenge-$GROUPNAME.git` and then `renku clone https://renkulab.io/gitlab/$YOURNAME/s2s-ai-challenge-$GROUPNAME.git`
### 7. Train your Machine Learning model ### 6. Train your Machine Learning model
Get training data via Get training data via
- [climetlab](https://github.com/ecmwf-lab/climetlab-s2s-ai-challenge) - [climetlab](https://github.com/ecmwf-lab/climetlab-s2s-ai-challenge)
...@@ -83,31 +81,34 @@ Get corresponding observations/ground truth: ...@@ -83,31 +81,34 @@ Get corresponding observations/ground truth:
- [climetlab](https://github.com/ecmwf-lab/climetlab-s2s-ai-challenge) - [climetlab](https://github.com/ecmwf-lab/climetlab-s2s-ai-challenge)
- IRIDL: [temperature](http://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.temperature/.daily/) and accumulated [precipitation](http://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.UNIFIED_PRCP/.GAUGE_BASED/.GLOBAL/.v1p0/.extREALTIME/.rain) - IRIDL: [temperature](http://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.temperature/.daily/) and accumulated [precipitation](http://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP/.CPC/.UNIFIED_PRCP/.GAUGE_BASED/.GLOBAL/.v1p0/.extREALTIME/.rain)
### 8. Let the Machine Learning model perform subseasonal 2020 predictions ### 7. Let the Machine Learning model perform subseasonal 2020 predictions
and save them as `netcdf` files. and save them as `netcdf` files.
The submissions have to placed in the `submissions` folder with filename `submission_your_choice.nc`, The submissions have to placed in the `submissions` folder with filename `submission_your_choice.nc`,
see [example](https://renkulab.io/gitlab/aaron.spring/s2s-ai-competition-bootstrap/-/blob/master/submissions/submission_rb_prediction_2020.nc). see [example](https://renkulab.io/gitlab/aaron.spring/s2s-ai-competition-bootstrap/-/blob/master/submissions/submission_rb_prediction_2020.nc).
### 9. `git commit` training pipeline and netcdf submission ### 8. `git commit` training pipeline and netcdf submission
For later verification of the organizers, reproducibility and scoring of submissions, For later verification of the organizers and reviewers, reproducibility and scoring of submissions,
the training notebook/pipeline and submission file ML_prediction.nc with `git lfs`. the training notebook/pipeline and submission file `ML_prediction_2020.nc` must be committed with `git lfs`:
After commiting, `git tag submission-method_name-number`
```bash ```bash
git lfs track "*.nc" # once, already done in template # run your training and create file ../submissions/ML_prediction_2020.nc
git add submissions/ML_prediction_2020.nc git lfs track "*.nc" # do once, already done in template
git commit -m "commit submission for my_method" git add ../submissions/ML_prediction_2020.nc
git tag "submission-my_method-0.0.1" # if this is to be checked by scorer git commit -m "commit submission for my_method_name" # whatever message you want
git tag "submission-my_method_name-0.0.1" # if this is to be checked by scorer, only the last submitted==tagged version will be considered
git push --tags git push --tags
``` ```
### 10. RPSS scoring by `scorer` bot ### 9. RPSS scoring by `scorer` bot
The `scorer` will fetch your tagged submissions, score them with RPSS against recalibrated ECMWF real-time forecasts. The `scorer` will fetch your tagged submissions, score them with RPSS against recalibrated ECMWF real-time forecasts.
Your score will be added to the private [leaderboard](https://s2s-ai-challenge.github.io/#leaderboard), which will be made public in early November 2021. Your score will be added to the private leaderboard, which will be made public in early November 2021.
The `scorer` is not active for the competition yet. The `scorer` is not active for the competition yet.
## More information ## More information
in the [wiki](https://renkulab.io/gitlab/aaron.spring/s2s-ai-challenge-template/-/wikis/Home)
- in the [`s2s-ai-challenge` wiki](https://renkulab.io/gitlab/aaron.spring/s2s-ai-challenge/-/wikis/Home)
- all different resources for this [competition](https://renkulab.io/gitlab/aaron.spring/s2s-ai-challenge/-/wikis/Flow-of-information:-Where-do-I-find-what%3F)
## Changing interactive environment dependencies ## Changing interactive environment dependencies
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment