Example-Airflow-DAGs

KO | EN

gitlite — search

#python #hacktoberfest #react #ai #golang #machine-learning #deep-learning #linux #macos #cli #vue #pytorch #ios #openai

Example-Airflow-DAGs

(★ 351)

Example DAGs using hooks and operators from Airflow Plugins

파일 탐색기

최종 버전 다운로드 (.zip)

- - mongo_to_redshift.py
- facebook_ads_to_redshift.py
- github_to_redshift.py
- google_analytics_to_redshift.py
- hubspot_to_redshift.py
- imap_to_redshift.py
- marketo_to_redshift.py
- salesforce_to_redshift.py
- sftp_to_mongo.py
- dbt_example.py
- dummy_sensor_example.py
- dynamic_dag_example.py
- imap_example.py
- mailgun_validation_example.py
- selenium_example.py
- singer_example.py
- dynamic_connection_creation.py
- rate_limit_reset.py
.gitignore
_config.yml
LICENSE
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Example DAGs

This repository contains example DAGs that can be used "out-of-the-box" using
operators found in the Airflow Plugins organization. These DAGs have a range
of use cases and vary from moving data (see [ETL](https://github.com/airflow-plugins/example_dags/tree/master/etl))
to background system automation that can give your Airflow "super-powers".

## Getting Started
The example DAGs found here can be split into three main categories:

### ETL
These DAGs focus on pulling data from various systems and putting them into
Amazon Redshift, with S3 as a staging store. These represent the simplest
implementation of an "ETL" workflow and can either be used "out-of-the-box"
or extended to add additional custom logic.

### PoC (Proof of Concept)
These DAGs demonstrate simple implementations of custom operators and Airflow
setups. They are typically not "copy-and-paste" DAGs but rather walk through
how something would work.

### System
These DAGs are used on the system administration level and can be thought of
as "meta-DAGs" that maintain various states and configurations within Airflow
itself. In some cases, these DAGs are used in concert with other custom
operators, such as the `rate_limit_reset` DAG.

## Contributions
Contributions of your own DAGs are very welcome. Please see some of the example
DAGs for a sense of general formatting guidelines.

## License
Apache 2.0

Example-Airflow-DAGs

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Example-Airflow-DAGs

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README