airflow-operator - GitLite

KO | EN

gitlite — search

#python #typescript #golang #go #machine-learning #linux #docker #macos #windows #awesome #kubernetes #vue #awesome-list #security

airflow-operator

(★ 297)

Kubernetes custom controller and CRDs to managing Airflow

파일 탐색기

최종 버전 다운로드 (.zip)

- - main.go
- - airflow_v1alpha1_airflowbase.yaml
  - airflow_v1alpha1_airflowcluster.yaml
- - kustomization.yaml
  - manager_image_patch.yaml
- - rbac_role.yaml
  - rbac_role_binding.yaml
- - airflow_v1alpha1_airflowbase.yaml
  - airflow_v1alpha1_airflowcluster.yaml
- airflow-base.png
- airflow-cluster.png
- airflow-multi-node.png
- airflow-pod.png
- airflow-region-spread.png
- airflow-zone-spread.png
- api.md
- design.md
- development.md
- quickstart.md
- userguide.md
- appcrd.yaml
- boilerplate.go.txt
- - addtoscheme_airflow_v1alpha1.go
  - apis.go
- - add_airflowbase.go
  - add_airflowcluster.go
  - controller.go
- - webhook.go
- airflow-configmap.yaml
- base-application.yaml
- cluster-application.yaml
- flower-sts.yaml
- headlesssvc.yaml
- mysql-sts.yaml
- nfs-sts.yaml
- pdb.yaml
- postgres-sts.yaml
- redis-sts.yaml
- rolebinding.yaml
- scheduler-sts.yaml
- secret.yaml
- serviceaccount.yaml
- sqlproxy-sts.yaml
- storage.yaml
- svc.yaml
- ui-sts.yaml
- worker-sts.yaml
- - base_test.go
  - cluster_test.go
  - gcp_test.go
.gitignore
cloudbuild.yaml
Dockerfile
Gopkg.lock
Gopkg.toml
LICENSE
Makefile
README.md

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/GoogleCloudPlatform/airflow-operator

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd airflow-operator

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Docker

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Docker Desktop 컨테이너를 빌드하고 실행하려면 필요합니다. 설치 후 실행해서 백그라운드에 켜두세요.

docker build -t airflow-operator .

Dockerfile을 기반으로 실행 가능한 이미지를 빌드합니다.

docker run -p 8080:80 airflow-operator

빌드된 이미지를 실제 컨테이너로 실행합니다.

✅ 터미널에 docker compose ps 를 입력해 컨테이너들이 Up 상태인지 확인하세요. README에 포트 번호가 적혀있다면 브라우저에서 http://localhost:포트번호 로 접속해보세요.

3. Make

보통

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Make Linux/macOS는 보통 기본 설치되어 있습니다. Windows는 별도 설치(예: MSYS2, WSL)가 필요합니다.

make

생성된 빌드 설정을 바탕으로 실제 컴파일을 진행해 실행 파일을 만듭니다.

✅ 에러 없이 끝나면 성공입니다. 생성된 실행 파일을 직접 실행해보세요.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

[![Go Report Card](https://goreportcard.com/badge/github.com/GoogleCloudPlatform/airflow-operator)](https://goreportcard.com/report/github.com/GoogleCloudPlatform/airflow-operator)

**This is not an officially supported Google product.**

## Community

* Join our [Slack channel](https://kubernetes.slack.com/messages/CC1UAMYSV).

## Project Status

*Alpha*

The Airflow Operator is still under active development and has not been extensively tested in production environment. Backward compatibility of the APIs is not guaranteed for alpha releases.

## Prerequisites
* Version >= 1.9 of Kubernetes.
* Uses 1.9 of Airflow (1.10.1+ for k8s executor)
* Uses 4.0.x of Redis (for celery operator)
* Uses 5.7 of MySQL

## Get Started

[One Click Deployment](https://console.cloud.google.com/marketplace/details/google/airflow-operator) from Google Cloud Marketplace to your [GKE cluster](https://cloud.google.com/kubernetes-engine/)

Get started quickly with the Airflow Operator using the [Quick Start Guide](https://github.com/GoogleCloudPlatform/airflow-operator/blob/master/docs/quickstart.md)

For more information check the [Design](https://github.com/GoogleCloudPlatform/airflow-operator/blob/master/docs/design.md) and detailed [User Guide](https://github.com/GoogleCloudPlatform/airflow-operator/blob/master/docs/userguide.md)

## Airflow Operator Overview
Airflow Operator is a custom [Kubernetes operator](https://coreos.com/blog/introducing-operators.html) that makes it easy to deploy and manage [Apache Airflow](https://airflow.apache.org/) on Kubernetes. Apache Airflow is a platform to programmatically author, schedule and monitor workflows. Using the Airflow Operator, an Airflow cluster is split into 2 parts represented by the `AirflowBase` and `AirflowCluster` custom resources.
The Airflow Operator performs these jobs:
* Creates and manages the necessary Kubernetes resources for an Airflow deployment.
* Updates the corresponding Kubernetes resources when the `AirflowBase` or `AirflowCluster` specification changes.
* Restores managed Kubernetes resources that are deleted.
* Supports creation of Airflow schedulers with different Executors
* Supports sharing of the `AirflowBase` across mulitple `AirflowClusters`

Checkout out the [Design](https://github.com/GoogleCloudPlatform/airflow-operator/blob/master/docs/design.md)

![Airflow Cluster](https://raw.githubusercontent.com/GoogleCloudPlatform/airflow-operator/master/docs/airflow-cluster.png)

## Development

Refer to the [Design](https://github.com/GoogleCloudPlatform/airflow-operator/blob/master/docs/design.md) and [Development Guide](https://github.com/GoogleCloudPlatform/airflow-operator/blob/master/docs/development.md).

## Managed Airflow solution

[Google Cloud Composer](https://cloud.google.com/composer/) is a fully managed workflow orchestration service targeting customers that need a workflow manager in the cloud.