Table of Contents

Airflow

About

Airflow is a data platform to programmatically author, schedule, and monitor workflows written in Python.

Airflow workflow is a list of tasks that are dependent on execution not on data. The dependency graph is an execution one.

The idea behind workflow is that when workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.

Docker

curl -LO https://raw.githubusercontent.com/bitnami/bitnami-docker-airflow/master/docker-compose.yml
docker-compose up

See: https://github.com/bitnami/bitnami-docker-airflow

Documentation / Reference