Skip to content

Demo pipeline reporting screenshots

Below are screeshots from the mnist-digits-demo-pipeline reporting website.

This is a static website (hosted as a Github Page) and it is updated whenever the demo pipeline has executed.

The demo pipeline is configured to run daily and for all PRs to the main repo.

Thus, this demoes how a sequence of notebooks could be scheduled to run daily with a reporting website.

Main page: List all pipeline runs

list-pipeline-runs.png

View a particular pipeline run

Selecting a pipeline run shows all tasks that run as part of the pipeline:

list-pipeline-runs.png

View a task run

In this demo pipeline, each task in the DAG is defined by a Jupyter notebook.

Inspecting one executed task shows:

  • any parameters passed into the task;
  • any images, files or metrics that have been logged during the task execution;
  • the evaluated notebook for the task as below (unless task was pure Python function).

list-pipeline-runs.png

Technically, the site is built with a modified version of MLFlow that can be hosted as a static website. Currently, data shown in the web UI (eg. logged metrics) are compiled into the front end, and larger files (eg. images) are served as static files using Github Pages.

With the current setup, pipeline runs are persisted on Github for 20 days.