Job orchestration
Apache Airflow
Dagster
Mage
Prefect
| Attribute | Apache Airflow | Dagster | Mage | Prefect |
|---|---|---|---|---|
| Name | Apache Airflow | Dagster | Mage | Prefect |
| Description | Apache Airflow is a platform to programmatically author, schedule, and monitor workflows. | A data orchestrator for machine learning, analytics, and ETL. | A lightweight and flexible job orchestration tool. | A modern data workflow management system. |
| License | Apache license 2.0 | Apache license 2.0 | Apache license 2.0 | MIT |
| Source code | https://github.com/apache/airflow | https://github.com/dagster-io/dagster | https://github.com/mage-ai/mage-ai | https://github.com/PrefectHQ/prefect |
| Website | https://airflow.apache.org/ | https://dagster.io/ | https://www.mage.ai/ | https://www.prefect.io/ |
| Year created | 2014 | 2019 | 2019 | 2018 |
| Company | Airbnb, Apache | Dagster | Mage | Prefect |
| Language support | python | python | python | python |
| Use cases | Workflow scheduling | Data Orchestration, Machine Learning Pipelines | Task Automation, Workflow Management | Data Workflow Management, ETL |
| Has cron schedule support | ||||
| Has event based trigger support | ||||
| Has api support | ||||
| Has access controls | ||||
| Has workflow versioning | ||||
| Has workflow configuration support | ||||
| Has audit logs | ||||
| Has cost tracking | ||||
| Has data source connection support |