abctl

KO | EN

gitlite — search

#hacktoberfest #react #typescript #llm #golang #android #rust #linux #nodejs #java #windows #security #ios #open-source

(★ 90)

Airbyte's CLI for managing local Airbyte installations

File Explorer

Download Latest Version (.zip)

- - airbox-release.yml
  - go.yml
  - pr.yml
  - release.yml
- CODEOWNERS
- - main.go
  - README.md
- minio-to-local-storage-migration.md
- arch.png
- - error.go
  - error_test.go
- - auth.go
  - auth_test.go
  - config.go
  - config_store.go
  - config_store_test.go
  - config_test.go
  - config_validations.go
  - config_validations_test.go
  - credentials.go
  - credentials_test.go
  - env.go
  - env_test.go
  - factories.go
  - factories_test.go
- - api.go
  - api_test.go
  - log_scanner.go
  - log_scanner_test.go
- - client.go
  - client_test.go
  - dataplanes.go
  - dataplanes_test.go
  - organizations.go
  - organizations_test.go
  - regions.go
  - regions_test.go
- - auth.go
  - auth_test.go
  - flow.go
  - flow_test.go
  - mocks_creds_test.go
  - oauth.go
  - oauth_test.go
  - oidc.go
  - oidc_test.go
  - success.html
- - build.go
  - build_test.go
- - cmd.go
  - output.go
  - output_test.go
- - const.go
  - set.go
- - docker.go
  - docker_test.go
  - secret.go
  - secret_test.go
- - airbyte_values.go
  - airbyte_values_test.go
  - chart.go
  - chart_test.go
  - dataplane.go
  - dataplane_test.go
  - factory.go
  - helm.go
  - images.go
  - images_test.go
  - locate.go
  - meta.go
  - meta_test.go
  - nginx_values.go
- - client.go
  - client_test.go
  - default.go
- - client.go
  - client_test.go
  - cluster.go
  - cluster_test.go
  - factory.go
  - ingress.go
  - ingress_test.go
  - load_images.go
  - log.go
  - log_test.go
  - namespace.go
  - provider.go
  - provider_test.go
  - volumes.go
  - volumes_test.go
- - maps.go
  - maps_test.go
- - docker.go
  - docker_test.go
- - paths.go
  - paths_test.go
- - version.go
  - version_test.go
- - install.go
  - install_test.go
  - manager.go
  - manager_test.go
  - mock_test.go
  - status.go
  - uninstall.go
- - client.go
  - client_test.go
  - config.go
  - config_test.go
  - dnt.go
  - dnt_test.go
  - mock.go
  - noop.go
  - noop_test.go
  - segment.go
  - segment_test.go
- - trace.go
- - filterable_select.go
  - filterable_select_test.go
  - json.go
  - json_test.go
  - select.go
  - select_test.go
  - spinner.go
  - spinner_test.go
  - textinput.go
  - textinput_test.go
  - ui.go
  - ui_test.go
  - yaml.go
  - yaml_test.go
- - update.go
  - update_test.go
- - url.go
  - url_test.go
.gitignore
.goreleaser.yaml
abctl_install.sh
go.mod
go.sum
LICENSE
main.go
Makefile
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<img alt="abctl logo" src="https://avatars.githubusercontent.com/u/59758427?size=64" height="100%" align="left" />
<h1 align="left">abctl</h1>
Airbyte's command line tool for local Airbyte deployments.

> [!NOTE]
> This README focuses on the `abctl` tool itself, not the Airbyte platform.
> 
> See Airbyte's [Quickstart](https://docs.airbyte.com/platform/using-airbyte/getting-started/oss-quickstart) to walk through setting up Airbyte for the first time.

---

- [Quickstart](#quickstart)
- [Overview](#overview)
- [Commands](#commands)
- [Contributing](#contributing)

# Quickstart

> [!IMPORTANT]
> Authentication credentials are randomly generated as part of the installation process.
>
> After installation is complete, to find your authentication credentials run `abctl local credentials`.

1. Install `Docker`
   - [Linux](https://docs.docker.com/desktop/install/linux-install/)
   - [Mac](https://docs.docker.com/desktop/install/mac-install/)
   - [Windows](https://docs.docker.com/desktop/install/windows-install/)
   
2. Install `abctl`
   - Via [brew](https://brew.sh/)
     ```
     brew tap airbytehq/tap
     brew install abctl
     ``` 
   - Via [go install](https://go.dev/ref/mod#go-install)
     ```
     go install github.com/airbytehq/abctl@latest
     ```
   - Via [Github ](https://github.com/airbytehq/abctl/releases/latest)
3. Install `Airbyte`
   ```
   # install Airbyte 
   abctl local install
   # fetch the login credentials
   abctl local credentials
   ```
> [!NOTE]
> Depending on your internet speed, `abctl local install` may take up to 30 minutes.
> 
> By default `abctl local install` configures Airbyte to accessible by all inbound requests on port `8000`.
> This typically includes access via the host's ip-address and `localhost`.
>
> If port `8000` is not available. or another port is preferred, `--port [PORT]` can be specified.

4. Login to `Airbyte`

If `abctl local install` completed successfully, it should open a browser to http://localhost:8000
   (or to the `--host` and `--port` overrides if specified).  If this is the first time Airbyte has been
   installed you will be asked to provide an email and organization name.  To retrieve your password
   to login, run `abctl local credentials`.

# Overview

![arch.png](https://raw.githubusercontent.com/airbytehq/abctl/main/imgs/arch.png)

As mentioned in the [quickstart](#quickstart), the only prerequisite `abctl` has is that [Docker](https://www.docker.com/) 
must be installed, running, and accessible. However, the Airbyte platform requires a [Kubernetes](https://kubernetes.io/) 
cluster, which `abctl` creates by utilizing [kind](https://kind.sigs.k8s.io/) (kind runs a Kubernetes cluster within a 
Docker container). With kind's Kubernetes cluster, `abctl` uses [helm](https://helm.sh/) to install (or update to) the 
latest (by default) Airbyte helm chart and the latest [NGINX Ingress Controller](https://docs.nginx.com/nginx-ingress-controller/) 
helm chart.

# Commands

All commands and sub-commands support the following optional global flags:

| Short | Long | Description |
|-------|-----------|---------------------------------------------------------------------------------|
| -h | --help | Displays the help information, description the available options. |
| -v | --verbose | Enables verbose (debug) output. Useful when debugging unexpected behavior. |

All commands support the following environment variables:

| Name         | Description                                     |
|--------------|-------------------------------------------------|
| DO_NOT_TRACK | Set to any value to disable telemetry tracking. |

The following commands are supported:
- [local](#local)
- [version](#version)

## local

```abctl local --help```

The local sub-commands are focused on managing the local Airbyte installation.
The following sub-commands are available:
- [credentials](#credentials)
- [deployments](#deployments)
- [install](#install)
- [status](#status)
- [uninstall](#uninstall)
   
### credentials

```abctl local credentials```

Displays the credentials required to login to the local Airbyte installation.

> [!NOTE]
> When `abctl local install` is first executed, random `password`, `client-id`, and `client-secret`
> are generated.

Returns ths `email`, `password`, `client-id`, and `client-secret` credentials.
The `email` and  `password` are required to login to Airbyte.
The `client-id` and `client-secret` are necessary to create an [`Access Token` for interacting with the Airbyte API](https://reference.airbyte.com/reference/createaccesstoken).

For example:
```
$ abctl local credentials
{
  "password": "[RANDOM PASSWORD]",
  "client-id": "[RANDOM CLIENT-ID]",
  "client-secret": "[RANDOM CLIENT-SECRET]"
}
```

`credentials` supports the following optional flags

| Name       | Default | Description                               |
|------------|---------|-------------------------------------------|
| --email    | ""      | Changes the authentication email address. |
| --password | ""      | Changes the authentication password.      |

### deployments

```abctl local deployments```

Display kubernetes deployment information and allows for restarting a kubernetes deployment.

`deployments` supports the following optional flags

| Name      | Default | Description                       |
|-----------|---------|-----------------------------------|
| --restart | ""      | Restarts the provided deployment. |

### install

```abctl local install```

Installs a local Airbyte instance or updates an existing installation which was initially installed by `abctl`.

> [!NOTE]
> Depending on your internet speed, `abctl local install` may take up to 30 minutes.

`install` supports the following optional flags:

> [!NOTE]
> An `-` in the default column indicates no value can be provided.  
> 
> These flags behave as a switch, enabled if provided, disabled if not.

| Name | Default | Description |
|---------------------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| --chart | "" | Path to chart. |
| --chart-version | latest | Which Airbyte helm-chart version to install. | 
| --docker-email | "" | Docker email address to authenticate against `--docker-server`. Can also be specified by the environment-variable `ABCTL_LOCAL_INSTALL_DOCKER_EMAIL`. |
| --docker-password | "" | Docker password to authenticate against `--docker-server`. Can also be specified by the environment-variable `ABCTL_LOCAL_INSTALL_DOCKER_PASSWORD`. |
| --docker-server | "" | Docker server to authenticate against. Can also be specified by the environment-variable `ABCTL_LOCAL_INSTALL_DOCKER_SERVER`. |
| --docker-username | "" | Docker username to authenticate against `--docker-server`. Can also be specified by the environment-variable `ABCTL_LOCAL_INSTALL_DOCKER_USERNAME`. |
| --insecure-cookies | - | Disables secure cookie requirements. Only set if using `--host` with an insecure (non `https`) connection. |
| --low-resource-mode | false | Run Airbyte in low resource mode. |
| --host | "" | FQDN where the Airbyte installation will be accessed. Default is to allow for all incoming traffic on port `--port`. Set this if the Airbyte installation needs a more restricted host configuration. |
| --no-browser | - | Disables launching the browser when installation completes. Useful to set in situations where no browser is available. |
| --port | 8000 | Port where the Airbyte installation will be accessed. Set this if port 8000 is already in use or if a different port is preferred. |
| --secret | "" | **Can be set multiple times**. Creates a kubernetes secret based on the contents of the file provided. Useful when used in conjunction with `--values` for customizing installation. |
| --values | "" | Helm values file to further customize the Airbyte installation. |
| --volume | "" | **Can be set multiple times**. Mounts additional volumes in the kubernetes cluster. Must be in the format of `<HOST_PATH>:<GUEST_PATH>`. |

#### Low Resource Mode

The `--low-resource-mode` flag optimizes Airbyte for environments with limited CPU and memory resources. When enabled, this mode makes the following changes:

**Resource Requests:**
- Sets all job resource requests (CPU and memory) to `0`, allowing Kubernetes to schedule jobs without minimum resource guarantees
- Applies to all job types: `check`, `discover`, `spec`, and `sidecar` containers

**Feature Adjustments:**
- Disables the Connector Builder Server to reduce overall resource consumption

This mode is ideal for:
- Development and testing environments with limited resources
- Machines with constrained CPU or memory
- Situations where multiple services need to share limited system resources

> [!NOTE]
> While low resource mode reduces resource requirements, it may impact performance for data-intensive operations. It is not recommended for production workloads with large data volumes.

Example usage:
```
abctl local install --low-resource-mode
```

#### Deploying a Specific Airbyte Version

By default, `abctl local install` installs the latest version of Airbyte. To deploy a specific version, create a custom Helm values file and pass it using the `--values` flag.

1. Create an `override_values.yaml` file with the following content (replacing `1.2.0` with your desired version):

```yaml
   global:
     # Docker image config that will apply to all images.
     image:
       # Image tag to use for airbyte images.
       # Does not include non-airbyte images such as temporal, minio, etc.
       tag: "1.2.0"
   ```

2. Install Airbyte using the custom values file:

```
   abctl local install --values override_values.yaml
   ```

> [!NOTE]
> The `--values` flag can be used to customize any Airbyte Helm chart values, not just the image tag.
> See the [Airbyte Helm chart](https://github.com/airbytehq/airbyte-platform/tree/main/charts/airbyte) for all available configuration options.

### status

```abctl local status```

If an Airbyte installation exists, returns information regarding that installation.

For example:
```
$ abctl local status
Existing cluster 'airbyte-abctl' found
Found helm chart 'airbyte-abctl'
  Status: deployed
  Chart Version: 0.422.2
  App Version: 0.63.15
Found helm chart 'ingress-nginx'
  Status: deployed
  Chart Version: 4.11.1
  App Version: 1.11.1
Airbyte should be accessible via http://localhost:8000
```

### uninstall

```abctl local uninstall```

Uninstalls a local Airbyte instance.

> [!NOTE]
> The data associated with the installed Airbyte instance will not be removed.
> 
> This is done to allow Airbyte to be reinstalled at a later date with all the data preserved.

`uninstall` supports the following optional flags:

> [!NOTE]
> An `-` in the default column indicates no value can be provided.
>
> These flags behave as a switch, enabled if provided, disabled if not.

| Name | Default | Description |
|-------------|---------|--------------------------------------------------------------------------------|
| --persisted | - | Will remove all data for the Airbyte installation. This cannot be undone. |

## images

```abctl images```

Manage images used by Airbyte and abctl.

The following sub-commands are available:
- [manifest](#manifest)

### manifest

```abctl images manifest```

Display a manifest of images used by Airbyte and abctl.

| Name                | Default | Description  |
|---------------------|---------|--------------|
| --chart             | ""      | Path to chart. |
| --chart-version     | latest  | Which Airbyte helm-chart version to install.    |
| --values            | ""      | Helm values file to further customize the Airbyte installation. |

## version

```abctl version```

Displays version information about the `abctl` tool.

For example:
```
$ abctl version
version: v0.19.0
```

# Contributing

## Report an Issue
If you have found a problem with `abctl`, please open a [Github Issue](https://github.com/airbytehq/airbyte/issues/new/choose) and use the `🐛 [abctl] Report an issue with the abctl tool` template.

## Build

Install `go`
- Via [brew](https://brew.sh/)
   ```
   brew install go
   ``` 
- Via [go.dev](https://go.dev/doc/install)

This repository utilises a [Makefile](Makefile), wrapping the traditional `go` commands
used for formatting, vetting, building, and testing `go` code.

The following `make` commands are supported:

| name         | description                                                                         |
|--------------|-------------------------------------------------------------------------------------|
| `make build` | Builds the `abctl` binary, placing it in the `build` directory.                     |
| `make clean` | Removes the `build` directory.                                                      |
| `make fmt`   | [Formats the code](https://pkg.go.dev/cmd/go#hdr-Gofmt__reformat__package_sources). |
| `make test`  | Runs all the tests.                                                                 |
| `make vet`   | Runs the [vet](https://pkg.go.dev/cmd/vet) command.                                 |

abctl

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

abctl

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README