Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • learn-renku/teaching-on-renku/autograde
  • fotis.georgatos/autograde
2 results
Show changes
Commits on Source (40)
Expand all Hide whitespace changes
Inline Side-by-side
Showing
with 167 additions and 609 deletions
.gitattributes 0 → 100644
View file @ 94881047
grading/results_[Bar,Foo]_598b9a96.zip filter=lfs diff=lfs merge=lfs -text
Dockerfile
View file @ 94881047
# For finding latest versions of the base image see
# https://github.com/SwissDataScienceCenter/renkulab-docker
ARG RENKU_BASE_IMAGE=renku/renkulab-py:3.9-0.10.1
ARG RENKU_BASE_IMAGE=renku/renkulab-vnc:0.10.1
FROM ${RENKU_BASE_IMAGE}
# Uncomment and adapt if code is to be included in the image
......@@ -47,4 +47,4 @@ RUN if [ -n "$RENKU_VERSION" ] ; then \
fi \
fi
########################################################
\ No newline at end of file
########################################################
README.md
View file @ 94881047
# Jupyter-autograde
# autograde
## Introduction
`autograde` is a toolbox for testing Jupyter notebooks. Autograde executes notebooks with consecutive unit testing of the final notebook state and allows refining results (e.g. grading plots by hand) in the audit mode. Autograde summarizes results in human (html) and machine-readable (JSON) formats.
https://github.com/cssh-rwth/autograde
This is a Renku project - basically a git repository with some
bells and whistles. You'll find we have already created some
useful things like `data` and `notebooks` directories and
a `Dockerfile`.
## Demo
## Working with the project
We have included `autograde` to this envirnment by adding a line `jupyter-autograde==0.2.13` to `requirements.txt` file.
Let's evaluate how it automates grading using the example files provided by `autograde` in the `demo/` subdirectory.
The simplest way to start your project is right from the Renku
platform - just click on the `Environments` tab and start a new session.
This will start an interactive environment right in your browser.
### Grade
To grade `demo/notebook.ipynb` we run `autograde test` against test scenarios specified in `demo/test.py` using context files stored in `demo/context`
``` bash
autograde test demo/test.py demo/notebook.ipynb --target /grading --context demo/context
```
To work with the project anywhere outside the Renku platform,
click the `Settings` tab where you will find the
git repo URLs - use `git` to clone the project on whichever machine you want.
- `demo/test.py` contains test cases we want to apply
- `demo/notebook.ipynb` is the a notebook to be tested (here you may also specify a directory to be recursively searched for notebooks)
- The optional flag `--target` tells autograde where to store results, /tmp in our case, and the current working directory by default.
- The optional flag `--context` specifies a directory that is mounted into the sandbox and may contain arbitrary files or subdirectories. This is useful when the notebook expects some external files to be present such as data sets.
### Changing interactive environment dependencies
This command generates a compressed archive named `results_[Lastname1,Lastname2,...]_XXXXXXXX.zip` (in this case `results_[Bar,Foo]_598b9a96.zip`) with the following contents:
Initially we install a very minimal set of packages to keep the images small.
However, you can add python and conda packages in `requirements.txt` and
`environment.yml` to your heart's content. If you need more fine-grained
control over your environment, please see [the documentation](https://renku.readthedocs.io/en/latest/user/advanced_interfaces.html#dockerfile-modifications).
- `artifacts/`: directory with all files that where created or modified by the tested notebook as well as rendered matplotlib plots.
- `code.py`: code extracted from the notebook including stdout/stderr as comments
- `notebook.ipynb`: an identical copy of the tested notebook
- `restults.json`: test results
## Project configuration
### Inspect and edit automatic grades
Let's use `autograde audit` to inspect and edit the grades generated by automatic testing located in subfolder `grading/`.
```bash
autograde audit grading/
```
To open an interactive audit interface triggered by this function we first need to switch to virtual desktop by clicking the VNC icon:
![image](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/autograde/raw/master/images/toVNC.png)
Now we can access the interactive interface running at http://127.0.0.1:5000/
![image](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/autograde/raw/master/images/audit_interface_480.gif)
**Preview** results of automtic grading.
![image](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/autograde/raw/master/images/preview_results_480.gif)
Project options can be found in `.renku/renku.ini`. In this
project there is currently only one option, which specifies
the default type of environment to open, in this case `/lab` for
JupyterLab. You may also choose `/tree` to get to the "classic" Jupyter
interface.
**Edit** and **comment** the results of automatic grading.
![image](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/autograde/raw/master/images/edit_results_480.gif)
## Moving forward
### Generate Reports
Let's generate a human readable HTML report for grading results using `autograde report`, the command below generates a report inside the results archive `grading/results_\[Bar\,Foo\]_598b9a96.zip`.
```bash
autograde report grading/
```
Let's uncompress the results
```bash
pushd grading/
unzip results_\[Bar\,Foo\]_598b9a96.zip
popd
```
Here is how the generated `grading/report.html` looks like:
![image](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/autograde/raw/master/images/report_html.gif)
You can display the full contents of this report by opening it either from interactive session or locally.
Let's summarise results and generate inside subfolder `grading/` the follwoing 2 files:
- `summary.csv` with aggregated results
- `summary.html` with human readable summary report
```bash
autograde summary grading/
```
Here is how the generated summary `grading/summary.html` looks like:
![image](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/autograde/raw/master/images/summary_html.gif)
Once you feel at home with your project, we recommend that you replace
this README file with your own project documentation! Happy data wrangling!
\ No newline at end of file
code.py deleted 100644 → 0
View file @ 802419b8
#!/usr/bin/env python3
# ------------------------------ CODE CELL SETUP -------------------------------
__IB_FLAG__ = True
__IMPORT_FILTER__ = globals().get('IMPORT_FILTER', None)
__PLOTS__ = []
__LABEL__ = None
if __IMPORT_FILTER__ is not None:
regex, blacklist = __IMPORT_FILTER__
print(f'set import filter: regex=r"{regex}", blacklist={blacklist}')
try:
# If matplotlib is available in the test environment, it is set to headless mode
# and all plots are dumped to disk rather than being displayed.
import matplotlib as _mpl
_mpl.use('Agg')
from functools import wraps
from pathlib import Path
import matplotlib.pyplot as _plt
from autograde.util import snake_case
__show = _plt.show
__save = _plt.savefig
@wraps(__save)
def _save(*args, **kwargs):
__save(*args, **kwargs)
_plt.close()
@wraps(__show)
def _show(*_, **__):
if _plt.gcf().get_axes():
root = Path('figures')
root.mkdir(exist_ok=True)
path = root / snake_case(f'fig_cell_{__LABEL__}_{len(__PLOTS__) + 1}')
__PLOTS__.append(path)
print(f'save figure at {path}')
_save(path)
_plt.savefig = _save
_plt.show = _show
except ImportError:
pass
auto_save_figure = globals().get('_show', lambda *args, **kwargs: None)
# EXECUTED IN 0.477s
# STDOUT
# set import filter: regex=r"re.compile('autograde')", blacklist=True
# -------------------------------- CODE CELL 1 ---------------------------------
team_members = [
{
'first_name': 'Alice',
'last_name': 'Foo',
'student_id': 12345
},
{
'first_name': 'Bob',
'last_name': 'Bar',
'student_id': 54321
}
]
# EXECUTED IN 0.00111s
# ----------------------------- CODE CELL 1 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.00652s
# -------------------------------- CODE CELL 2 ---------------------------------
import sys
import time
import matplotlib.pyplot as plt
import pandas as pd
# EXECUTED IN 1.4s
# ----------------------------- CODE CELL 2 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.0012s
# -------------------------------- CODE CELL 3 ---------------------------------
SOME_CONSTANT = 42
# EXECUTED IN 0.00096s
# ----------------------------- CODE CELL 3 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.001s
# -------------------------------- CODE CELL 4 ---------------------------------
df = pd.DataFrame([[1, 1], [2, 4], [3, 9], [4, 16], [5, 25]])
df.plot()
# EXECUTED IN 0.626s
# ----------------------------- CODE CELL 4 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.563s
# STDOUT
# save figure at figures/fig_cell_4_clean_1
# -------------------------------- CODE CELL 5 ---------------------------------
from IPython.display import display
display(df)
# EXECUTED IN 0.1s
# STDOUT
# 0 1
# 0 1 1
# 1 2 4
# 2 3 9
# 3 4 16
# 4 5 25
# ----------------------------- CODE CELL 5 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.00114s
# -------------------------------- CODE CELL 6 ---------------------------------
with open('foo.txt', mode='rt') as f:
data = f.read()
print(data[:len(data) // 2])
print(data[len(data) // 2:], file=sys.stderr)
# EXECUTED IN 0.00136s
# STDOUT
# this file gets
# STDERR
# read by notebook
# ----------------------------- CODE CELL 6 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.000894s
# -------------------------------- CODE CELL 7 ---------------------------------
with open('bar.txt', mode='a') as f:
f.write('>> modification by notebook <<')
# EXECUTED IN 0.000959s
# ----------------------------- CODE CELL 7 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.000892s
# -------------------------------- CODE CELL 8 ---------------------------------
with open('fnord.txt', mode='wt') as f:
f.write('let\'s leave some artifacts in the system')
# EXECUTED IN 0.00115s
# ----------------------------- CODE CELL 8 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.000714s
# -------------------------------- CODE CELL 9 ---------------------------------
for e in [2, 3]:
plt.plot(*zip(*((v, v ** e) for v in range(-10, 10))))
plt.show()
# EXECUTED IN 1.65s
# STDOUT
# save figure at figures/fig_cell_9_1
# save figure at figures/fig_cell_9_2
# ----------------------------- CODE CELL 9 CLEAN ------------------------------
auto_save_figure()
# EXECUTED IN 0.00474s
# -------------------------------- CODE CELL 10 --------------------------------
plt.plot(*zip(*((v, v ** 5 + (v - 5) ** 3 - (3 * v) ** 2) for v in range(-20, 20))))
plt.savefig('plot.png')
# EXECUTED IN 0.857s
# ----------------------------- CODE CELL 10 CLEAN -----------------------------
auto_save_figure()
# EXECUTED IN 0.00621s
# -------------------------------- CODE CELL 11 --------------------------------
def square(x: float) -> float:
return x ** 2
def cube(x: float) -> float:
return x ** 3
abs_cube = cube
def fail():
raise ValueError('no chance, this function crashes so badly')
def sleep(x):
time.sleep(x)
# EXECUTED IN 0.00129s
# ----------------------------- CODE CELL 11 CLEAN -----------------------------
auto_save_figure()
# EXECUTED IN 0.00104s
# -------------------------------- CODE CELL 12 --------------------------------
sleep(.1)
# EXECUTED IN 0.101s
# ----------------------------- CODE CELL 12 CLEAN -----------------------------
auto_save_figure()
# EXECUTED IN 0.00157s
# -------------------------------- CODE CELL 13 --------------------------------
import autograde
print(autograde.__version__)
# EXECUTED IN 0.00146s
# STDERR
# Traceback (most recent call last):
# File "/opt/conda/lib/python3.9/site-packages/autograde/notebook_executor.py", line 133, in exec_notebook
# shadow_stack.enter_context(shadow_exec(code_cell, state))
# File "/opt/conda/lib/python3.9/contextlib.py", line 429, in enter_context
# result = _cm_type.__enter__(cm)
# File "/opt/conda/lib/python3.9/contextlib.py", line 117, in __enter__
# return next(self.gen)
# File "/opt/conda/lib/python3.9/site-packages/autograde/notebook_executor.py", line 61, in shadow_exec
# exec(compiled_source, *args, **kwargs)
# File "/tmp/tmpiq3u368r", line 1, in <module>
# File "/opt/conda/lib/python3.9/unittest/mock.py", line 1093, in __call__
# return self._mock_call(*args, **kwargs)
# File "/opt/conda/lib/python3.9/unittest/mock.py", line 1097, in _mock_call
# return self._execute_mock_call(*args, **kwargs)
# File "/opt/conda/lib/python3.9/unittest/mock.py", line 1158, in _execute_mock_call
# result = effect(*args, **kwargs)
# File "/opt/conda/lib/python3.9/site-packages/autograde/util.py", line 277, in guard
# raise ImportError(f'usage of "\'{target}\'" is not permitted')
# ImportError: usage of "'autograde'" is not permitted
# ----------------------------- CODE CELL 13 CLEAN -----------------------------
auto_save_figure()
# EXECUTED IN 0.000893s
# -------------------------------- CODE CELL 14 --------------------------------
def illegal_import():
exec('import autograde as ag')
# EXECUTED IN 0.000776s
# ----------------------------- CODE CELL 14 CLEAN -----------------------------
auto_save_figure()
# EXECUTED IN 0.000721s
# -------------------------------- CODE CELL 15 --------------------------------
assert False, 'this cell will crash under any circumstances ...'
# EXECUTED IN 0.000835s
# STDERR
# Traceback (most recent call last):
# File "/opt/conda/lib/python3.9/site-packages/autograde/notebook_executor.py", line 133, in exec_notebook
# shadow_stack.enter_context(shadow_exec(code_cell, state))
# File "/opt/conda/lib/python3.9/contextlib.py", line 429, in enter_context
# result = _cm_type.__enter__(cm)
# File "/opt/conda/lib/python3.9/contextlib.py", line 117, in __enter__
# return next(self.gen)
# File "/opt/conda/lib/python3.9/site-packages/autograde/notebook_executor.py", line 61, in shadow_exec
# exec(compiled_source, *args, **kwargs)
# File "/tmp/tmpwr9vhbgf", line 1, in <module>
# AssertionError: this cell will crash under any circumstances ...
# ----------------------------- CODE CELL 15 CLEAN -----------------------------
auto_save_figure()
# EXECUTED IN 0.000885s
# -------------------------------- CODE CELL 16 --------------------------------
print('... but this cell is not affected :^)')
# EXECUTED IN 0.00107s
# STDOUT
# ... but this cell is not affected :^)
# ----------------------------- CODE CELL 16 CLEAN -----------------------------
auto_save_figure()
# EXECUTED IN 0.000984s
# ----------------------------- CODE CELL TEARDOWN -----------------------------
__IA_FLAG__ = True
# EXECUTED IN 0.000385s
data/.gitkeep deleted 100644 → 0
View file @ 802419b8
grading/raw.csv 0 → 100644
View file @ 94881047
student_id,last_name,first_name,notebook_id,test_id,score,max_score,archive
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,239eadbc01bcd3892b5c01a86b007dca86eeb0b2daa51ad5a580a1ddc079e040,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,f03691ea9f44d6018fd43ef2a106e527843f5984f9eb44c49aeecb48d1266454,2.5,2.5,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,35797fb23f082ed01f8a12050bfd0343e28ceca91cd00ef3be38a144563ec0a2,2.0,3.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,7e0e6ff3545a0510d5ab9a1d7ac6ab2c14609d3843d57cc04638f321423ac3c5,1.0,2.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,90f0231eef0abe9f9b28e3be1315d27a1ef89297c6009d92d7b1f24a165f4673,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,85f2e61341ca4bfbdda86b328c3704ff3983d49c786b51b176dd558979e59122,0.0,1.5,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,95cf4e22dcc910d0384c63a600fe79d886e3f23f4c6bb6eb8fdc616180793d9d,-0.5,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,511d0ac725daadf807d17cfbe7fa0da1a4a446ad85f737f5ca94fb51bc393c4b,0.5,0.5,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,53bca0ce4cd4c8c8c4e1792086e8bd79bc049eb7f61e660bebf485dd8668e84c,0.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,6d1546e07fd32d5d0eab09ea89abc6b5de7f3660236b5486ca83d7c0d44364ee,0.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,ce7ad25b6af8ed05fc10dd726b893fd12abe5408a20d5e2b5aaa36977bbb8e18,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,81b637d8fcd2c6da6359e6963113a1170de795e4b725b84d1e0b4cfd9ec58ce9,,4.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,a0ec927b1044a5e945fbd9cf4370b4be1bc1ebc93bca51bc37f02a8814196bde,,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,a03b221c6c6eae7122ca51695d456d5222e524889136394944b2f9763b483615,0.0,2.5,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,f81f712816b4a576cd12ecf67c0e85267a22109f5ffef8c50576ac85ccf4e048,,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,affcf4c8d293c17fcb845deb7f4bb1aff6697426d75bf28a2bf7403067646673,0.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,d376512f3901b0a8347d0221155af8c527795b23b9b884d8d25484149ea0da47,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,a814e91177667827d91ab4d1c8abe0e3a11471d8299afb274eafdc5b99355309,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,239eadbc01bcd3892b5c01a86b007dca86eeb0b2daa51ad5a580a1ddc079e040,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,f03691ea9f44d6018fd43ef2a106e527843f5984f9eb44c49aeecb48d1266454,2.5,2.5,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,35797fb23f082ed01f8a12050bfd0343e28ceca91cd00ef3be38a144563ec0a2,2.0,3.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,7e0e6ff3545a0510d5ab9a1d7ac6ab2c14609d3843d57cc04638f321423ac3c5,1.0,2.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,90f0231eef0abe9f9b28e3be1315d27a1ef89297c6009d92d7b1f24a165f4673,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,85f2e61341ca4bfbdda86b328c3704ff3983d49c786b51b176dd558979e59122,0.0,1.5,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,95cf4e22dcc910d0384c63a600fe79d886e3f23f4c6bb6eb8fdc616180793d9d,-0.5,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,511d0ac725daadf807d17cfbe7fa0da1a4a446ad85f737f5ca94fb51bc393c4b,0.5,0.5,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,53bca0ce4cd4c8c8c4e1792086e8bd79bc049eb7f61e660bebf485dd8668e84c,0.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,6d1546e07fd32d5d0eab09ea89abc6b5de7f3660236b5486ca83d7c0d44364ee,0.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,ce7ad25b6af8ed05fc10dd726b893fd12abe5408a20d5e2b5aaa36977bbb8e18,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,81b637d8fcd2c6da6359e6963113a1170de795e4b725b84d1e0b4cfd9ec58ce9,,4.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,a0ec927b1044a5e945fbd9cf4370b4be1bc1ebc93bca51bc37f02a8814196bde,,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,a03b221c6c6eae7122ca51695d456d5222e524889136394944b2f9763b483615,0.0,2.5,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,f81f712816b4a576cd12ecf67c0e85267a22109f5ffef8c50576ac85ccf4e048,,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,affcf4c8d293c17fcb845deb7f4bb1aff6697426d75bf28a2bf7403067646673,0.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,d376512f3901b0a8347d0221155af8c527795b23b9b884d8d25484149ea0da47,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,a814e91177667827d91ab4d1c8abe0e3a11471d8299afb274eafdc5b99355309,1.0,1.0,"results_[Bar,Foo]_598b9a96.zip"
grading/report.html 0 → 100644
View file @ 94881047
This diff is collapsed.
grading/results_[Bar,Foo]_598b9a96.zip
View file @ 94881047
No preview for this file type
grading/summary.csv 0 → 100644
View file @ 94881047
student_id,last_name,first_name,notebook_id,score,max_score,archive,duplicate
54321,Bar,Bob,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,10.5,27.0,"results_[Bar,Foo]_598b9a96.zip",False
12345,Foo,Alice,598b9a96d5c04cf88f5b26771572a62544e80369535855044fb4583c3bb78878,10.5,27.0,"results_[Bar,Foo]_598b9a96.zip",False
grading/summary.html 0 → 100644
View file @ 94881047
This diff is collapsed.
images/audit_interface_480.gif 0 → 100644
View file @ 94881047
images/audit_interface_480.gif

1.16 MiB

images/edit_results_480.gif 0 → 100644
View file @ 94881047
images/edit_results_480.gif

3.42 MiB

images/preview_results_480.gif 0 → 100644
View file @ 94881047
images/preview_results_480.gif

2.41 MiB

images/report.png 0 → 100644
View file @ 94881047
images/report.png

242 KiB

images/report_html.gif 0 → 100644
View file @ 94881047
images/report_html.gif

1.12 MiB

images/summary.png 0 → 100644
View file @ 94881047
images/summary.png

103 KiB

images/summary_html.gif 0 → 100644
View file @ 94881047
images/summary_html.gif

738 KiB

images/toVNC.png 0 → 100644
View file @ 94881047
images/toVNC.png

18.8 KiB

notebook.ipynb deleted 100644 → 0
View file @ 802419b8
%% Cell type:markdown id: tags:
# Awesome Class - Assignment X
## Formalities
**Submit in a group of 2-3 people until xx.xx.xxxx 23.59CET. The deadline is strict!**
## Evaluation and Grading
**General advice for programming exercises at _Awesome Institution_**
Evaluation of your submission is done semi-automatically.
Think of it as this notebook being executed once.
Afterwards, some test functions are appended to this file and executed respectively.
Therefore:
- Submit valid _Python3_ code only!
- Use **external** libraries if and only if specified by task.
Using the standard library is fine unless you're told otherwise.
- Ensure your definitions (functions, classes, methods, variables) follow the specification.
The signature of a function/method/class usually can be inferred from task description, code skeletons and test cases.
- Ensure your code does not rely on the current notebook or system state!
- Use `Kernel --> Restart & Run All` to see if you are using any definitions, variables etc. that are not in scope anymore.
- Double check if your code relies on presence of files or directories other than those mentioned in given tasks.
- Tests run under Linux, hence don't use Windows style paths (`some\path`, `C:\another\path`).
`pathlib` provides neat abstractions for that, use it!
Also, use paths only that are relative to and within your working directory (OK: `some/path`, `./some/path`; NOT OK: `/home/alice/python`, `../../python`).
- Keep your code [idempotent](https://en.wikipedia.org/wiki/Idempotence)!
Running your code or parts of it multiple times must not yield different results.
Minimize usage of global variables.
- Ensure your code/notebook terminates in reasonable time.
We enforce timeouts during notebook/test execution.
Exceeding those results in an error and potentially no points for that sub-task.
- Textual answers must always be backed by code and may not refer to results that are not part of your submission.
**There's a story behind each of these points! Don't expect us to fix your code!**
%% Cell type:markdown id: tags:
Credentials of all team members (you may add or remove items from the list)
%% Cell type:code id: tags:
``` python
team_members = [
{
'first_name': 'Alice',
'last_name': 'Foo',
'student_id': 12345
},
{
'first_name': 'Bob',
'last_name': 'Bar',
'student_id': 54321
}
]
```
%% Cell type:code id: tags:
``` python
import sys
import time
import matplotlib.pyplot as plt
import pandas as pd
```
%% Cell type:code id: tags:
``` python
SOME_CONSTANT = 42
```
%% Cell type:markdown id: tags:
Let's do some plotting...
%% Cell type:code id: tags:
``` python
df = pd.DataFrame([[1, 1], [2, 4], [3, 9], [4, 16], [5, 25]])
df.plot()
```
%% Cell type:markdown id: tags:
... or render a table
%% Cell type:code id: tags:
``` python
from IPython.display import display
display(df)
```
%% Cell type:markdown id: tags:
Now, read some data from the file system
%% Cell type:code id: tags:
``` python
with open('foo.txt', mode='rt') as f:
data = f.read()
print(data[:len(data) // 2])
print(data[len(data) // 2:], file=sys.stderr)
```
%% Cell type:markdown id: tags:
... or modify an existing file
%% Cell type:code id: tags:
``` python
with open('bar.txt', mode='a') as f:
f.write('>> modification by notebook <<')
```
%% Cell type:markdown id: tags:
... or create a new file
%% Cell type:code id: tags:
``` python
with open('fnord.txt', mode='wt') as f:
f.write('let\'s leave some artifacts in the system')
```
%% Cell type:markdown id: tags:
Explicit calls to `plt.show` will be re-directed to `auto_save_figure`, storing the plot in a file
%% Cell type:code id: tags:
``` python
for e in [2, 3]:
plt.plot(*zip(*((v, v ** e) for v in range(-10, 10))))
plt.show()
```
%% Cell type:markdown id: tags:
However, we can also save a figure explicitly!
%% Cell type:code id: tags:
``` python
plt.plot(*zip(*((v, v ** 5 + (v - 5) ** 3 - (3 * v) ** 2) for v in range(-20, 20))))
plt.savefig('plot.png')
```
%% Cell type:markdown id: tags:
Here are some more functions we can write tests for
%% Cell type:code id: tags:
``` python
def square(x: float) -> float:
return x ** 2
def cube(x: float) -> float:
return x ** 3
abs_cube = cube
def fail():
raise ValueError('no chance, this function crashes so badly')
def sleep(x):
time.sleep(x)
```
%% Cell type:markdown id: tags:
Tests may define timeouts for cell and test execution.
Hence, the following code may rise a `TimeoutError`!
%% Cell type:code id: tags:
``` python
sleep(.1)
```
%% Cell type:markdown id: tags:
Importing _autograde_ may fail due to import policy, see `NotebookTest.set_import_filter`
%% Cell type:code id: tags:
``` python
import autograde
print(autograde.__version__)
```
%% Cell type:markdown id: tags:
Importing _autograde_ is an issue as it could be used to monkey-patch the library, e.g.
```Python
import autograde
def mock(self, *_, **__):
return self.score, '🖕'
autograde.notebook_test.UnitTest.__call__ = mock
```
%% Cell type:markdown id: tags:
Import filters also work at test time and for various ways of importing
%% Cell type:code id: tags:
``` python
def illegal_import():
exec('import autograde as ag')
```
%% Cell type:markdown id: tags:
Unlike normal Python scripts, notebook execution continues even if a cell crashes!
%% Cell type:code id: tags:
``` python
assert False, 'this cell will crash under any circumstances ...'
```
%% Cell type:code id: tags:
``` python
print('... but this cell is not affected :^)')
```
%% Cell type:markdown id: tags:
Tests may access contents of markdown comments.
This is useful when students are asked to elaborate on their solutions.
%% Cell type:markdown id: tags:
**Q1:** _How many roads must a man walk down before you call him a man?_
%% Cell type:markdown id: tags:
**A1:** The answer, my friend, is blowin' in the wind. The answer is blowin' in the wind.
%% Cell type:markdown id: tags:
**Q2:** _What's the answer to life, universe and everything?_
%% Cell type:markdown id: tags:
**A2:** 42 (forty-two)
notebooks/.gitkeep deleted 100644 → 0
View file @ 802419b8

This GitLab is a part of renkulab.io