Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • jovicic/covid-19-public-data
  • sofiane.sarni/covid-19-public-data
  • mojulian/covid-19-public-data
  • paul.ronga/covid-19-public-data
  • covid-19/covid-19-public-data
  • orourke/covid-19-public-data
  • marius.mos/covid-19-public-data
  • gava/covid-19-public-data
  • oleg/covid-19-public-data
  • lorenzo.cavazzi.tech/covid-19-public-data
  • dev/covid-19-public-data
  • cp/covid-19-public-data
  • cchoirat/covid-19-public-data
  • nathanael.perraudin/covid-19-public-data
  • robert/covid-19-public-data
  • edana.beauvais/covid-19-public-data
  • emma.jablonski/covid-19-public-data
  • natasa.tagasovska/covid-19-public-data
  • florianvanhorenbeke/covid-19-public-data
  • rok.roskar/covid-19-public-data
  • joze.roskar/covid-19-public-data
  • kumarsum1984/covid-19-public-data
  • mark.kochanek/covid-19-public-data
  • ableuler/covid-19-public-data
  • plamennavenkova/covid-19-public-data
  • hannah.lantermann/covid-19-public-data
  • rok.roskar/covid-19-public-data-new-fork
  • dandrea.cordoba/covid-19-public-data
  • alfredo.chavarria/covid-19-public-data
  • alfredo.chavarria/covid-19-public-data-alfredo
30 results
Show changes
Commits on Source (7)
Expand all Hide whitespace changes
Inline Side-by-side
notebooks/examples/covidtracking-archon.ipynb 0 → 100644
View file @ f8ea3b03
This diff is collapsed.
notebooks/process/download-covidtracking-data.ipynb
View file @ f8ea3b03
%% Cell type:code id: tags:
``` python
import requests
import os
import pandas as pd
```
%% Cell type:code id: tags:parameters
``` python
out_folder = "../data/covidtracking/"
PAPERMILL_OUTPUT_PATH = None
```
%% Cell type:markdown id: tags:
# Download state metadata
Download a dataset of URLs for data for each US state and several territories. See [Google Doc](https://docs.google.com/spreadsheets/d/18oVRrHj3c183mHmq3m89_163yuYltLNlOmPerQ18E8w/htmlview?sle=true).
%% Cell type:code id: tags:
``` python
url = 'http://covidtracking.com/api/states/info'
r = requests.get(url, allow_redirects=True)
states_metadata_json = r.content
```
%% Cell type:code id: tags:
``` python
# save the result
if PAPERMILL_OUTPUT_PATH:
out_path = os.path.join(out_folder, 'states-metadata.json')
with open(out_path, 'wb') as f:
f.write(states_metadata_json)
```
%% Cell type:code id: tags:
``` python
metadata_df = pd.read_json(states_metadata_json)
print(len(metadata_df), "states and territories have metadata")
metadata_df.head(2)
```
%% Output
56 states and territories have metadata
state covid19SiteOld \
0 AK http://dhss.alaska.gov/dph/Epi/id/Pages/COVID-...
1 AL http://www.alabamapublichealth.gov/infectiousd...
covid19Site \
0 http://dhss.alaska.gov/dph/Epi/id/Pages/COVID-...
1 https://alpublichealth.maps.arcgis.com/apps/op...
covid19SiteSecondary twitter \
0 http://dhss.alaska.gov/dph/Epi/id/Pages/COVID-... @Alaska_DHSS
1 None @alpublichealth
pui pum notes name
0 All data False Unclear if their reported number means "person... Alaska
1 No data False Last negative count from 3/16. Last update tim... Alabama
%% Cell type:markdown id: tags:
# Download daily state data
%% Cell type:code id: tags:
``` python
url = 'https://covidtracking.com/api/states/daily'
r = requests.get(url, allow_redirects=True)
states_daily_json = r.content
```
%% Cell type:code id: tags:
``` python
# save the result
if PAPERMILL_OUTPUT_PATH:
out_path = os.path.join(out_folder, 'states-daily.json')
with open(out_path, 'wb') as f:
f.write(states_daily_json)
```
%% Cell type:code id: tags:
``` python
data_df = pd.read_json(states_daily_json)
print(len(data_df), "data points")
data_df.head(2)
data_df.head(10)
```
%% Output
813 data points
date state positive negative pending death total \
0 20200320 AK 12 686.0 NaN NaN 698
1 20200320 AL 81 28.0 NaN 0.0 109
2 20200320 AR 96 351.0 203.0 NaN 650
3 20200320 AS 0 NaN NaN 0.0 0
4 20200320 AZ 65 211.0 101.0 0.0 377
5 20200320 CA 1063 10424.0 NaN 20.0 11487
6 20200320 CO 277 2675.0 NaN 3.0 2952
7 20200320 CT 194 604.0 NaN 3.0 798
8 20200320 DC 71 501.0 1.0 1.0 573
9 20200320 DE 38 36.0 NaN 0.0 74
dateChecked
0 2020-03-20T20:00:00Z
1 2020-03-20T20:00:00Z
2 2020-03-20T20:00:00Z
3 2020-03-20T20:00:00Z
4 2020-03-20T20:00:00Z
5 2020-03-20T20:00:00Z
6 2020-03-20T20:00:00Z
7 2020-03-20T20:00:00Z
8 2020-03-20T20:00:00Z
9 2020-03-20T20:00:00Z
%% Cell type:code id: tags:
``` python
```
......
runs/download-covidtracking-data.runs.ipynb
View file @ f8ea3b03
%% Cell type:code id: tags:
``` python
import requests
import os
import pandas as pd
```
%% Cell type:code id: tags:parameters
``` python
out_folder = "../data/covidtracking/"
PAPERMILL_OUTPUT_PATH = None
```
%% Cell type:code id: tags:injected-parameters
``` python
# Parameters
PAPERMILL_INPUT_PATH = "/tmp/3mmsluiy/notebooks/process/download-covidtracking-data.ipynb"
PAPERMILL_OUTPUT_PATH = "runs/download-covidtracking-data.runs.ipynb"
out_folder = "data/covidtracking"
```
%% Cell type:markdown id: tags:
# Download state metadata
Download a dataset of URLs for data for each US state and several territories. See [Google Doc](https://docs.google.com/spreadsheets/d/18oVRrHj3c183mHmq3m89_163yuYltLNlOmPerQ18E8w/htmlview?sle=true).
%% Cell type:code id: tags:
``` python
url = 'http://covidtracking.com/api/states/info'
r = requests.get(url, allow_redirects=True)
states_metadata_json = r.content
```
%% Cell type:code id: tags:
``` python
# save the result
if PAPERMILL_OUTPUT_PATH:
out_path = os.path.join(out_folder, 'states-metadata.json')
with open(out_path, 'wb') as f:
f.write(states_metadata_json)
```
%% Output
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-5-3cd4d3ace430> in <module>
2 if PAPERMILL_OUTPUT_PATH:
3 out_path = os.path.join(out_folder, 'states-metadata.json')
----> 4 with open(out_path, 'wb') as f:
5 f.write(states_metadata_json)
FileNotFoundError: [Errno 2] No such file or directory: 'data/covidtracking/states-metadata.json'
%% Cell type:code id: tags:
``` python
metadata_df = pd.read_json(states_metadata_json)
print(len(metadata_df), "states and territories have metadata")
metadata_df.head(2)
```
%% Output
56 states and territories have metadata
state covid19SiteOld \
0 AK http://dhss.alaska.gov/dph/Epi/id/Pages/COVID-...
1 AL http://www.alabamapublichealth.gov/infectiousd...
covid19Site \
0 http://dhss.alaska.gov/dph/Epi/id/Pages/COVID-...
1 https://alpublichealth.maps.arcgis.com/apps/op...
covid19SiteSecondary twitter \
0 http://dhss.alaska.gov/dph/Epi/id/Pages/COVID-... @Alaska_DHSS
1 None @alpublichealth
pui pum notes name
0 All data False Unclear if their reported number means "person... Alaska
1 No data False Last negative count from 3/16. Last update tim... Alabama
%% Cell type:markdown id: tags:
# Download daily state data
%% Cell type:code id: tags:
``` python
url = 'https://covidtracking.com/api/states/daily'
r = requests.get(url, allow_redirects=True)
states_daily_json = r.content
```
%% Cell type:code id: tags:
``` python
# save the result
if PAPERMILL_OUTPUT_PATH:
out_path = os.path.join(out_folder, 'states-daily.json')
with open(out_path, 'wb') as f:
f.write(states_daily_json)
```
%% Output
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-8-473e5b840935> in <module>
2 if PAPERMILL_OUTPUT_PATH:
3 out_path = os.path.join(out_folder, 'states-daily.json')
----> 4 with open(out_path, 'wb') as f:
5 f.write(states_daily_json)
FileNotFoundError: [Errno 2] No such file or directory: 'data/covidtracking/states-daily.json'
%% Cell type:code id: tags:
``` python
data_df = pd.read_json(states_daily_json)
print(len(data_df), "data points")
data_df.head(2)
```
%% Output
813 data points
date state positive negative pending death total \
0 20200320 AK 12 686.0 NaN NaN 698
1 20200320 AL 81 28.0 NaN 0.0 109
dateChecked
0 2020-03-20T20:00:00Z
1 2020-03-20T20:00:00Z
%% Cell type:code id: tags:
``` python
```
......

This GitLab is a part of renkulab.io