k8s-flexvolume-lvm

KO | EN

gitlite — search

#javascript #python #react #ai #go #golang #android #deep-learning #linux #macos #kubernetes #vue #awesome-list #mcp

k8s-flexvolume-lvm

(★ 19)

Scripts to manage a shared LVM volume between multiple Docker/Kubernetes nodes

k8s-flexvolume-lvm Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Introduction

This set of script is intended to be used in Kubernetes to manage a shared volume between nodes without (with a single-instance POD with persistent data like MySQL) the need of an external provider like Ceph. These scripts are based on:
- Kubernetes flexVolume volume adapter that supports a plugin architecture based on a shell script. The standard LVM adapter is extended to manage simple lock (see https://github.com/kubernetes/kubernetes/tree/master/examples/volumes/flexvolume)
- lvmlockd and sanlock that is used to manage the shared LVM volume without the complexity of a cluster (pacemaker and clvmd)
- a shared volume provided by a physically shared storage or a shared VMDK file in VMware (my preferred solutions)

The script uses the tag feature of the LVM2 to lock the volume between multiple Kubernetes workers nodes so only one node can activate and mount the volume and other nodes needs to wait a timeout in case of node failure; the lvmlockd/sanlock stack avoids race condition on volume metadata area.

# Installation

If you want use a shared volume provided by VMware you can follow this guide:
https://communities.vmware.com/blogs/Abhilash_hb/2013/08/25/clustering-using-sharing-of-vmdks-between-virtual-machines

Once configured/installed the shared volume, you need to set-up a cluster-based LVM volume so you DON'T NEED to create the phisical volume but you need to use vgcreate's feature that creates a shared volume group with sanlock support:

CentOS

# All nodes
    yum install lvm2-lockd sanlock jq
    systemctl enable wdmd
    systemctl enable sanlock
    systemctl enable lvm2-lvmlockd
    systemctl start wdmd
    systemctl start sanlock
    systemctl start lvm2-lvmlockd

# Edit and customize lvmlocal.conf with host_id and vg list
    cat <<\EOF | tee -a /etc/lvm/lvmlocal.conf
    global {
        use_lvmlockd = 1
    }
    activation {
        auto_activation_volume_list = [ "put_here_all_vg_except_the_shared_one" ]
    }
    local {
        host_id = 1..2000
    }
    EOF

# Single node
    vgcreate --shared sharevg /dev/sdx
    lvcreate -L10G -n sharedlv /dev/sharevg
    mkfs.ext4 /dev/sharevg/sharedlv
    vgchange --lock-start

# All nodes
    vgchange --lock-start

Once cloned the repo you can:

mkdir -p /usr/libexec/kubernetes/kubelet-plugins/volume/exec/angeloxx~lvm
    cp lvm /usr/libexec/kubernetes/kubelet-plugins/volume/exec/angeloxx~lvm/
    cp lvmflexvolumed /usr/local/bin/
    cp lvmflexvolumed.service /etc/systemd/system
    chmod +x /usr/local/bin/lvmflexvolumed
    chmod +x /usr/libexec/kubernetes/kubelet-plugins/volume/exec/angeloxx~lvm/lvm
    systemctl daemon-reload
    systemctl start lvmflexvolumed

# Usage

In a Kubernetes Pod configuration you can use the lvmflexvolumed plugin to attach a previously created volume. The lvm plugin will:
- check if the volume exists
- check if is locked by another node and if the lock is fresh (use the sysconfig/default environment file to set LOCK_TIMEOUT timeout in seconds)
- remove the lock if is old, return an error if the volume is in use on another node or attach the volume
- mount the volume and start the container

The Pod sample configuration is:

apiVersion: v1
    kind: Pod
    metadata:
    name: nginx
    namespace: default
    spec:
    containers:
    - name: nginx
        image: nginx
        volumeMounts:
        - name: test
        mountPath: /data
        ports:
        - containerPort: 80
    volumes:
    - name: test
        flexVolume:
        driver: "angeloxx/lvm"
        fsType: "ext4"
        options:
            volumeID: "sharelv"
            volumegroup: "sharevg"

The lvmflexvolumed script daemon will refresh the lock every LOCK_SECONDS.

k8s-flexvolume-lvm

k8s-flexvolume-lvm Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

k8s-flexvolume-lvm

k8s-flexvolume-lvm Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README