cluster-api-k3s

KO | EN

gitlite — search

#python #hacktoberfest #typescript #llm #deep-learning #linux #docker #nodejs #macos #java #kubernetes #vue #awesome-list #mcp

cluster-api-k3s

(★ 186)

Cluster API k3s

File Explorer

Download Latest Version (.zip)

- devcontainer.json
- Dockerfile
- - e2e-full.yaml
  - e2e-pr.yaml
  - pr.yaml
  - release.yaml
- dependabot.yml
- - kthreesconfig_controller.go
  - kthreesconfig_controller_test.go
  - suite_test.go
- main.go
- metadata.yaml
- PROJECT
- - const.go
  - kthreescontrolplane_controller.go
  - machine_controller.go
  - remediation.go
  - scale.go
  - suite_test.go
- main.go
- metadata.yaml
- PROJECT
- tilt-setup.md
- boilerplate.go.txt
- go_install.sh
- kind-setup.sh
- version.sh
- - cloudinit.go
  - controlplane_init.go
  - controlplane_init_test.go
  - controlplane_join.go
  - k3s.sh
  - worker_join.go
  - worker_join_test.go
- - version.go
- - errors.go
- - doc.go
  - etcd-proxy.go
  - etcd-proxy.yaml
  - etcd.go
  - etcd_test.go
- - config.go
  - control_plane.go
  - etcd_client_generator.go
  - management_cluster.go
  - workload_cluster.go
  - workload_cluster_coredns.go
  - workload_cluster_etcd.go
  - workload_cluster_test.go
- - kubeconfig.go
- - control_plane_init_mutex.go
- - machine_filters.go
  - machine_filters_test.go
- - addr.go
  - conn.go
  - dial.go
  - proxy.go
- - certificates.go
  - const.go
  - secret.go
- - token.go
  - token_test.go
- - aws-ccm.yaml
  - aws-csi.yaml
  - aws-setup.sh
  - k3s-cluster.yaml
  - k3s-template.yaml
  - resource-set.yaml
- - azure-setup.sh
  - k3s-template.yaml
- - cluster-template-quickstart.yaml
- - cluster-template-k3s.yaml
  - nutanix-setup.sh
- - k3s-template.yaml
  - setup.sh
- - cluster-template-k3s.yaml
  - README.md
  - setup.sh
- - cluster-template-k3s.yaml
  - setup.sh
- clusterctl.yaml
- - cluster_upgrade.go
  - cluster_upgrade_test.go
  - clusterctl_upgrade_test.go
  - common.go
  - create_test.go
  - e2e_suite_test.go
  - helpers.go
  - inplace_rollout_test.go
  - kcp_remediation_test.go
  - md_remediation_test.go
  - node_scale_test.go
  - README.md
.envrc_sample
.gitignore
.golangci.yml
devbox.json
devbox.lock
Dockerfile
go.mod
go.sum
LICENSE
Makefile
metadata.yaml
README.md
tilt-provider.yaml

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Cluster API Provider k3s

Cluster API Provider k3s provides the following [Cluster API](https://github.com/kubernetes-sigs/cluster-api) (CAPI) providers:

- **Cluster API Bootstrap Provider k3s (CABP3)** is responsible for generating the instructions (and encoding them as cloud-init) to turn a Machine into a Kubernetes Node; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters.
- **Cluster API ControlPlane Provider k3s (CACP3)** is responsible for managing the lifecycle of control plane machines for k3s; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters.

## Getting Started

**Warning**: Project and documentation are in an early stage, there is an assumption that a user of this provider is already familiar with Cluster API. Please consider contributing.

### Prerequisites

Check out the general [Cluster API Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) page to see the prerequisites for Cluster API.

Three main pieces are:

1. Management cluster. In the `samples/azure/azure-setup.sh` script, [k3d](https://k3d.io/) is used, but feel free to use [kind](https://kind.sigs.k8s.io/) as well .
2. clusterctl. Please check out [Cluster API Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) for instructions.
3. Infrastructure specific prerequisites:
   - For more Azure information go to [CAPZ Getting Started](https://capz.sigs.k8s.io/topics/getting-started.html)
   - For more AWS information go to [CAPA Getting Started](https://cluster-api-aws.sigs.k8s.io/)
   - For more Nutanix information go to [CAPX Getting Started](https://opendocs.nutanix.com/capx/latest/getting_started/)
   - For more OpenStack information go to [CAPO Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html)
   - For more Vsphere information go to [CAPV Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html)

In this getting started guide we'll be using Docker as the infrastructure provider (CAPD).

### Create a management cluster

1. Ensure kind is installed ([instructions](https://kind.sigs.k8s.io/docs/user/quick-start/#installation))
2. Create a kind configuration to expose the local docker socket:

```bash
cat > kind-cluster-with-extramounts.yaml <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: capi-quickstart
nodes:
- role: control-plane
  extraMounts:
    - hostPath: /var/run/docker.sock
      containerPath: /var/run/docker.sock
EOF
```

3. Run the following command to create the cluster:

```bash
kind create cluster --config kind-cluster-with-extramounts.yaml
```

4. Check that you can connect to the cluster

```bash
kubectl get pods -A
```

### Install the providers

1. Create a file called `clusterctl.yaml` in the `$HOME/.config/cluster-api` directory with the following content:

```yaml
# NOTE: the following will be changed to use 'latest' in the future
providers:
  - name: "k3s"
    url: "https://github.com/k3s-io/cluster-api-k3s/releases/v0.2.1/bootstrap-components.yaml"
    type: "BootstrapProvider"
  - name: "k3s"
    url: "https://github.com/k3s-io/cluster-api-k3s/releases/v0.2.1/control-plane-components.yaml"
    type: "ControlPlaneProvider"
```

> This configuration tells clusterctl where to look for the provider manifests. You could run `clusterctl config -h` to check default clusterctl configuration file path.

2. Install the providers:

```bash
clusterctl init --bootstrap k3s --control-plane k3s --infrastructure docker
```

3. Wait for the pods to start

### Create a workload cluster

There are a number of different cluster templates in the [samples](./samples/) directory.

> Note: there is an issue with CAPD, it would be better you could do this setup beforehand. [Cluster API with Docker - &#34;too many open files&#34;.](https://cluster-api.sigs.k8s.io/user/troubleshooting.html?highlight=too%20many#cluster-api-with-docker----too-many-open-files)

1. Run the following command to generate your cluster definition:

```bash
export KIND_IMAGE_VERSION=v1.30.0
clusterctl generate cluster --from samples/docker/cluster-template-quickstart.yaml test1 --kubernetes-version v1.30.2+k3s2 --worker-machine-count 2 --control-plane-machine-count 1 > cluster.yaml
```

> NOTE: the kubernetes version specified with the k3s suffix `+k3s2`.

2. Check the contents of the generated cluster definition in **cluster.yaml**
3. Ensure the definition is valid by doing a dry run:

```bash
kubectl apply -f cluster.yaml --dry-run=server
```

4. When you are happy apply the definition:

```bash
kubectl apply -f cluster.yaml
```

### Check the workload cluster

- Check the state of the CAPI machines:

```bash
kubectl get machine
```

- Get the kubeconfig for the cluster:

```bash
clusterctl get kubeconfig test1 > workload-kubeconfig.yaml
```

> Note: if you are using Docker Desktop, you need to fix the kubeconfig by running:

```bash
# Point the kubeconfig to the exposed port of the load balancer, rather than the inaccessible container IP.
sed -i -e "s/server:.*/server: https:\/\/$(docker port test1-lb 6443/tcp | sed "s/0.0.0.0/127.0.0.1/")/g" ./workload-kubeconfig.yaml
```

- Connect to the child cluster

```bash
kubectl --kubeconfig workload-kubeconfig.yaml get pods -A
```

### Deleting the workload cluster

When deleting a cluster created via CAPI you must delete the top level **Cluster** resource. DO NOT delete using the original file.

For the quick start:

```bash
kubectl delete cluster test1
```

### Additional Samples

Cluster API k3s has been tested on AWS, Azure, AzureStackHCI, Nutanix, OpenStack, Docker and Vsphere environments.

- To try out the Azure flow, fork the repo and look at `samples/azure/azure-setup.sh`.
- To try out the AWS flow, fork the repo and look at `samples/aws/aws-setup.sh`.
- To try out the Nutanix flow, fork the repo and look at `samples/nutanix/nutanix-setup.sh`.
- To try out the OpenStack flow, fork the repo and look at `samples/openstack/setup.sh`.
- To try out the Vsphere flow, fork the repo and look at `samples/vsphere-capv/setup.sh`.

## Developer Setup

You could also build and install CABP3 and CACP3 from src:

```sh
# Build image with `dev` tag
make BOOTSTRAP_IMG_TAG=dev docker-build-bootstrap
make CONTROLPLANE_IMG_TAG=dev docker-build-controlplane

# Push image to your registry
export REGISTRY="localhost:5001"  # Set this to your local/remote registry
docker tag ghcr.io/k3s-io/cluster-api-k3s/controlplane-controller:dev ${REGISTRY}/controlplane-controller:dev
docker tag ghcr.io/k3s-io/cluster-api-k3s/bootstrap-controller:dev ${REGISTRY}/bootstrap-controller:dev
docker push ${REGISTRY}/controlplane-controller:dev
docker push ${REGISTRY}/bootstrap-controller:dev

# Install CAPI k3s to management cluster
make install-controlplane # install CRDs
make install-bootstrap
make CONTROLPLANE_IMG=${REGISTRY}/controlplane-controller CONTROLPLANE_IMG_TAG=dev deploy-controlplane  # deploy the component
make BOOTSTRAP_IMG=${REGISTRY}/bootstrap-controller BOOTSTRAP_IMG_TAG=dev deploy-bootstrap
```

For easy development, please refer to [tilt-setup.md](docs/tilt-setup.md).

## Roadmap

- Support for External Databases
- Fix Token Logic
- Clean up Control Plane Provider Code
- Post an issue!

cluster-api-k3s

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

cluster-api-k3s

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README