Skip to content
Snippets Groups Projects
Commit 7e083f82 authored by Chandrasekhar Ramakrishnan's avatar Chandrasekhar Ramakrishnan Committed by renku 0.9.1
Browse files

renku update --with-siblings

parent 5a221f71
No related branches found
No related tags found
No related merge requests found
Expand all Hide whitespace changes
Inline Side-by-side
.renku/workflow/e382da7b1798416889d1c7a15752c848.cwl 0 → 100644
+ 127
− 0
View file @ 7e083f82
class: Workflow
cwlVersion: v1.0
hints: []
inputs:
input_1:
default: ts_folder
streamable: false
type: string
input_10:
default:
class: Directory
listing: []
path: ../../data/covid-19_jhu-csse
streamable: false
type: Directory
input_11:
default: rates_folder
streamable: false
type: string
input_12:
default:
class: Directory
listing: []
path: ../../data/covid-19_rates
streamable: false
type: Directory
input_13:
default: geodata_path
streamable: false
type: string
input_14:
default:
class: File
path: ../../data/geodata/geo_data.csv
streamable: false
type: File
input_15:
default:
class: File
path: ../../notebooks/Dashboard.ipynb
streamable: false
type: File
input_16:
default: runs/Dashboard.run.ipynb
streamable: false
type: string
input_2:
default:
class: Directory
listing: []
path: ../../data/covid-19_jhu-csse
streamable: false
type: Directory
input_3:
default: worldmap_path
streamable: false
type: string
input_4:
default:
class: File
path: ../../data/worldmap/country_centroids.csv
streamable: false
type: File
input_5:
default: out_folder
streamable: false
type: string
input_6:
default: data/geodata
streamable: false
type: string
input_7:
default:
class: File
path: ../../notebooks/CompileGeoData.ipynb
streamable: false
type: File
input_8:
default: runs/CompileGeoData.run.ipynb
streamable: false
type: string
input_9:
default: ts_folder
streamable: false
type: string
outputs:
output_0:
outputSource: step_1/output_0
streamable: false
type: File
output_1:
outputSource: step_2/output_0
streamable: false
type: File
output_2:
outputSource: step_1/output_1
streamable: false
type: Directory
requirements: []
steps:
step_1:
in:
input_1: input_1
input_2: input_2
input_3: input_3
input_4: input_4
input_5: input_5
input_6: input_6
input_7: input_7
input_8: input_8
out:
- output_0
- output_1
run: 73781f74e51d4e54bc522007a2030ec2_papermill.cwl
step_2:
in:
input_1: input_9
input_2: input_10
input_3: input_11
input_4: input_12
input_5: input_13
input_6: input_14
input_7: input_15
input_8: input_16
out:
- output_0
run: 4cc7ffe9d5a045efb048ef2222a40ffa_papermill.cwl
data/geodata/geo_data.csv 0 → 100644 LFS
+ 3
− 0
View file @ 7e083f82
source diff could not be displayed: it is stored in LFS. Options to address this: view the blob.
runs/CompileGeoData.run.ipynb
+ 78
− 108
View file @ 7e083f82
%% Cell type:markdown id: tags:
# Extract the Geographic Info
Use the Harvard [country_centroids.csv](https://worldmap.harvard.edu/data/geonode:country_centroids_az8) data to extract the geographic info we need for the visualizations.
%% Cell type:code id: tags:
``` python
import pandas as pd
import os
```
%% Cell type:code id: tags:
``` python
ts_folder = "../data/covid-19_jhu-csse/"
worldmap_path = "../data/worldmap/country_centroids.csv"
out_folder = None
PAPERMILL_OUTPUT_PATH = None
```
%% Cell type:markdown id: tags:parameters
## Read in JHU CSSE data
%% Cell type:code id: tags:injected-parameters
``` python
# Parameters
PAPERMILL_INPUT_PATH = "/tmp/9d0h4tlx/notebooks/CompileGeoData.ipynb"
PAPERMILL_INPUT_PATH = "/tmp/dixae90v/notebooks/CompileGeoData.ipynb"
PAPERMILL_OUTPUT_PATH = "runs/CompileGeoData.run.ipynb"
ts_folder = "/tmp/9d0h4tlx/data/covid-19_jhu-csse"
worldmap_path = "/tmp/9d0h4tlx/data/worldmap/country_centroids.csv"
ts_folder = "/tmp/dixae90v/data/covid-19_jhu-csse"
worldmap_path = "/tmp/dixae90v/data/worldmap/country_centroids.csv"
out_folder = "data/geodata"
```
%% Cell type:code id: tags:
``` python
def read_jhu_covid_region_df(name):
filename = os.path.join(ts_folder, f"time_series_19-covid-{name}.csv")
df = pd.read_csv(filename)
df = df.set_index(['Country/Region', 'Province/State', 'Lat', 'Long'])
df.columns = pd.to_datetime(df.columns)
region_df = df.groupby(level='Country/Region').sum()
return region_df
```
%% Cell type:code id: tags:
``` python
confirmed_df = read_jhu_covid_region_df("Confirmed")
```
%% Cell type:markdown id: tags:
# Read in Harvard country centroids
%% Cell type:code id: tags:
``` python
country_centroids_df = pd.read_csv(worldmap_path)
country_centroids_df = country_centroids_df[['name', 'name_long', 'region_un', 'subregion', 'region_wb', 'pop_est', 'gdp_md_est', 'income_grp', 'Longitude', 'Latitude']]
country_centroids_df['name_jhu'] = country_centroids_df['name_long']
```
%% Cell type:code id: tags:
``` python
country_centroids_df.columns
```
%% Output
Index(['name', 'name_long', 'region_un', 'subregion', 'region_wb', 'pop_est',
'gdp_md_est', 'income_grp', 'Longitude', 'Latitude', 'name_jhu'],
dtype='object')
%% Cell type:markdown id: tags:
Fix names that differ between JHU CSSE and Harvard data
%% Cell type:code id: tags:
``` python
region_hd_jhu_map = {
'Brunei Darussalam': 'Brunei',
"Côte d'Ivoire": "Cote d'Ivoire",
'Czech Republic': 'Czechia',
'Hong Kong': 'Hong Kong SAR',
'Republic of Korea': 'Korea, South',
'Macao': 'Macao SAR',
'Russian Federation': 'Russia',
'Taiwan': 'Taiwan*',
'United States': 'US'
}
country_centroids_df['name_jhu'] = country_centroids_df['name_jhu'].replace(region_hd_jhu_map)
```
%% Cell type:code id: tags:
``` python
# Use this to find the name in the series
# country_centroids_df[country_centroids_df['name'].str.contains('Macao')]
```
%% Cell type:markdown id: tags:
There are some regions that we cannot resolve, but we will just ignore these.
%% Cell type:code id: tags:
``` python
confirmed_df.loc[
(confirmed_df.index.isin(country_centroids_df['name_jhu']) == False)
].iloc[:,-2:]
```
%% Output
2020-03-15 2020-03-16
2020-03-16 2020-03-17
Country/Region
Congo (Brazzaville) 1 1
Congo (Kinshasa) 2 2
Cruise Ship 696 696
Eswatini 1 1
French Guiana 0 11
Guadeloupe 3 6
Holy See 1 1
Martinique 9 15
Mayotte 0 1
North Macedonia 14 18
Republic of the Congo 0 1
Reunion 7 9
The Bahamas 0 1
occupied Palestinian territory 0 0
Congo (Brazzaville) 1 1
Congo (Kinshasa) 2 3
Cruise Ship 696 696
Eswatini 1 1
Holy See 1 1
Martinique 15 16
North Macedonia 18 26
Republic of the Congo 1 1
The Bahamas 1 1
%% Cell type:markdown id: tags:
# Save the result
%% Cell type:code id: tags:
``` python
if PAPERMILL_OUTPUT_PATH:
out_path = os.path.join(out_folder, f"geo_data.csv")
country_centroids_df.to_csv(out_path)
```
......
runs/Dashboard.run.ipynb
+ 85
− 85
View file @ 7e083f82
This diff is collapsed.
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

This GitLab is a part of renkulab.io