freenas-iscsi-provisioner

KO | EN

gitlite — search

#php #android #react #javascript #docker #python #midi #plugin #firebase #machine-learning #csharp #dotnet

freenas-iscsi-provisioner

★ 29 Open GitHub ↗

FreeNAS iscsi provisioner

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Deprecation Notice

Non-csi drivers are no longer receiving updates from the Kubernetes team.
As such this driver has been deprecated and replaced with a newer `csi` based
implementation - https://github.com/democratic-csi/democratic-csi

The new implementation works with all the fancy `csi` features such as snapshots,
resizing, etc. Enjoy!

Note that a conversion script has been created to facilitate migrating volumes
created using this project to `democratic-csi`. Please visit the project for
more details.

# What is freenas-provisioner

FreeNAS-iscsi-provisioner is a Kubernetes external provisioner.
When a `PersisitentVolumeClaim` appears on a Kube cluster, the provisioner will
make the corresponding calls to the configured FreeNAS API to create an iscsi
target/lun usable by the claim. When the claim or the persistent volume is
deleted, the provisioner deletes the previously created resources.

See this for more info on external provisioner:
https://github.com/kubernetes-incubator/external-storage

Unless you have a very specific use-case for iscsi/block devices, it is
recommended to use the NFS variant of this project available here:
https://github.com/nmaupu/freenas-provisioner

# Usage

The scope of the provisioner allows for a single instance to service multiple
classes (and/or FreeNAS servers). The provisioner itself can be deployed into
the cluster or ran out of cluster, for example, directly on a FreeNAS server.

Each `StorageClass` should have a corresponding `Secret` created which contains
the credentials and host information used to communicate with with FreeNAS API.
In essence each `Secret` corresponds to a FreeNAS server.

The `Secret` namespace and name may be customized using the appropriate
`StorageClass` `parameters`. By default `kube-system` and `freenas-iscsi` are
used. While multiple `StorageClass` resources may point to the same server
and hence same `Secret`, it is recommended to create a new `Secret` for each
`StorageClass` resource.

It is **highly** recommended to read `deploy/class.yaml` to review available
`parameters` and gain a better understanding of functionality and behavior.

## FreeNAS Setup

You must manually create a dataset. You may simply use a pool as the parent
dataset but it's recommended to create a dedicated dataset.

Additionally, you need to enable the iscsi service with it's corresponding
resources such as portal, initiator, and group.

## Provision the provisioner

Run it on the cluster:

```
kubectl apply -f deploy/rbac.yaml -f deploy/deployment.yaml
```

Alternatively, for advanced use-cases you may run the provisioner out of cluster
including directly on the FreeNAS server if desired. Running out of cluster is
not currently recommended.

```
./bin/freenas-iscsi-provisioner-freebsd --kubeconfig=/path/to/kubeconfig.yaml
```

## Create `StorageClass` and `Secret`

All the necessary resources are available in the `deploy` folder. At a minimum
`secret.yaml` must be modified (remember to `base64` the values) to reflect the
server details. You may also want to read `class.yaml` to review available
`parameters` of the storage class. For instance to set the `datasetParentName`.

```
kubectl apply -f deploy/secret.yaml -f deploy/class.yaml
```

## Example usage

Next, create a `PersistentVolumeClaim` using the storage class
(`deploy/test-claim.yaml`):

```
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: freenas-test-iscsi-pvc
spec:
  storageClassName: freenas-iscsi
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Mi
```

Use that claim on a testing pod (`deploy/test-pod.yaml`):

```
---
kind: Pod
apiVersion: v1
metadata:
  name: freenas-test-iscsi-pod
spec:
  containers:
  - name: freenas-test-isci-pod
    image: gcr.io/google_containers/busybox:1.24
    command:
      - "/bin/sh"
      - "-c"
      - "--"
    args: [ "date >> /mnt/file.log && while true; do sleep 30; done;" ]
    volumeMounts:
      - name: freenas-test-volume
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: freenas-test-volume
      persistentVolumeClaim:
        claimName: freenas-test-iscsi-pvc
```

The underlying zvol, target, extent, etc should be quickly appearing on the
FreeNAS side. In case of issue, follow the provisioner's logs using:

```
kubectl -n kube-system logs -f freenas-iscsi-provisioner-<id>
```

## CHAP settings

You should create a secret which holds CHAP authentication credentials based on `deploy/freenas-iscsi-chap.yaml`.
- If you have authentication enabled for the portal (discovery) then set `discovery*` parameters in the secret, and in StorageClass you should set `targetDiscoveryCHAPAuth` to `true`.
- If you want authentication for the targets, then set `node*` parameters in the secret, and in StorageClass you should set `targetGroupAuthtype` and `targetGroupAuthgroup` accordingly, and also set `targetSessionCHAPAuth` to `true`.

# Performance

100 10MiB PVCs
Creating took ~10 minutes

Deleting took ~6 minutes

# Testing

Choas testing has been performed to ensure the various actions are idempotent.

# Development

```
make vendor && make
```

Binary is located into `bin/freenas-iscis-provisioner`. It is compiled to be
run on `linux-amd64` by default, but you may run the following for different
builds:

```
make vendor && make darwin
# OR
make vendor && make freebsd
```

To run locally with an appropriate `$KUBECONFIG` you may run:

```
./local-start.sh
```

To format code before committing:

```
make fmt
```

- https://github.com/kubernetes-incubator/external-storage/blob/v4.0.0/lib/controller/controller.go
- https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner/blob/master/controller/controller.go
- https://gianarb.it/blog/kubernetes-shared-informer
- https://engineering.bitnami.com/articles/a-deep-dive-into-kubernetes-controllers.html

## Docs

- https://github.com/kubernetes/community/tree/master/contributors/design-proposals/storage
- https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/volume-provisioning.md
- https://kubernetes.io/docs/concepts/storage/storage-classes/
- https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#-strong-api-overview-strong-
- https://docs.openshift.org/latest/install_config/persistent_storage/persistent_storage_iscsi.html
- http://api.freenas.org
- https://doc.freenas.org/11/sharing.html#block-iscsi
- https://github.com/kubernetes-incubator/external-storage/blob/master/iscsi/targetd/provisioner/iscsi-provisioner.go
- https://github.com/dghubble/sling

## TODO

- volume resizing - https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/grow-volume-size.md
- volume snapshots - https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/volume-snapshotting.md
- mount options - https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/mount-options.md
- ~~CHAP~~
- fsType
- properly handle `zvol` API differences with `volsize` getting sent as string and returned as int
- loop GetBy<foo> requests that require `limit` param
- ~~recursive zvol delete in v1 api~~

## Notes

To sniff API traffic between host and server:

```
sudo tcpdump -i any -A -s 0 'host <server ip> and tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
```

freenas-iscsi-provisioner

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README