I have an EKS cluster with multiple deployments (microservices). I would like all of them to write logs to same folder in an EFS mount even between restarts/scaling etc. Currently it creates a folder with the persistent volume id, which breaks our requirement. Is it possible to always mount the same folder even when the persistent volume is recreated. Can it always to point to the same folder in EFS?
It currently creates folders like this:
"/logs/pvc-2exxxcxs4-0xx7-4e11-813a-65xxxxxxxx/"
Instead, I would like it to be just "/logs" or a fixed path without any dependency on pvc id/name.
Below are the current yamls:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
labels:
app: my-app
spec:
replicas: 1
selector:
matchLabels:
app: my-app
template:
metadata:
labels:
app: my-app
spec:
containers:
........
volumeMounts:
- name: dev-logs-efs
mountPath: /logs
volumes:
- name: dev-logs-efs
persistentVolumeClaim:
claimName: dev-logs-efs-pvc
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: dev-logs-efs-sc
provisioner: efs.csi.aws.com
parameters:
provisioningMode: efs-ap
fileSystemId: fs-xxxxxxxxxxx
basePath: "/logs"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: dev-logs-efs-pvc
spec:
accessModes:
- ReadWriteMany
storageClassName: dev-logs-efs-sc
resources:
requests:
storage: 5Gi
Related
I am deploying a jenkins on one master one node Kubernetes cluster, iam getting error when i try to do Dynamic Volume Provisioning. not sure what went wrong. please help.
my storageclass file
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
reclaimPolicy: Retain
mountOptions:
- debug
volumeBindingMode: Immediate
my PVC file
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins-pvc
labels:
type: amazonEBS
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 40Gi
storageClassName: standard
volumeMode: Filesystem
Deployment file
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
spec:
selector:
matchLabels:
app: jcasc
replicas: 1
template:
metadata:
labels:
app: jcasc
spec:
volumes:
- name: jenkins-pvc
persistentVolumeClaim:
claimName: jenkins-pvc
containers:
- name: jenkins
image: jenkins:latest
ports:
- containerPort: 8080
volumeMounts:
- name: jenkins-pvc
mountPath: "/var/jenkins_home"
Find this troubleshooting doc for Fixing Pod Has Unbound Immediate PersistentVolumeClaims Error and also
For dynamic provisioning you can see this doc.
For pv doc
After kubectl apply -f pvc.yaml the below yaml file, I can able to find the mount path /var/local/pvctest inside the container that has been created. But, the host path /var/local/pvctest in the worker node is not created.
I'm new to PV & PVC with EKS and any help to fix this issue is much appreciated!
kind: Deployment
apiVersion: apps/v1
metadata:
name: pvctest
labels:
alias: pvctest
spec:
selector:
matchLabels:
alias: pvctest
replicas: 1
template:
metadata:
labels:
alias: pvctest
spec:
containers:
- name: pvctest
image: neo4j
ports:
- containerPort: 7474
- containerPort: 7687
volumeMounts:
- name: testpv
mountPath: /var/local/pvctest
volumes:
- name: testpv
persistentVolumeClaim:
claimName: pvctest-claim
---
kind: PersistentVolume
apiVersion: v1
metadata:
name: pvtest
labels:
type: local
spec:
persistentVolumeReclaimPolicy: Retain
capacity:
storage: 3Gi
accessModes:
- ReadWriteOnce
hostPath:
path: /var/local/pvctest
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvctest-claim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
PersistentVolume with hostPath requires the directory on the host to be pre-created. If you want the directory to be created automatically for you:
...
containers:
- name: pvctest
image: neo4j
...
volumeMounts:
- name: testpv
mountPath: /var/local/pvctest
volumes:
- name: testpv
hostPath:
path: /data
type: DirectoryOrCreate
PV/PVC is actually optonal for hostPath.
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/2e47e8b4-4755-46d6-9bc4-461ea02a6cb9/volumes/kubernetes.io~aws-ebs/pv --scope -- mount -o bind /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-2a/vol-011d7bb42da888b82 /var/lib/kubelet/pods/2e47e8b4-4755-46d6-9bc4-461ea02a6cb9/volumes/kubernetes.io~aws-ebs/pv
Output: Running scope as unit run-20000.scope.
mount: /var/lib/kubelet/pods/2e47e8b4-4755-46d6-9bc4-461ea02a6cb9/volumes/kubernetes.io~aws-ebs/pv: special device /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-2a/vol-011d7bb42da888b82 does not exist.
Warning FailedAttachVolume 7s (x6 over 23s) attachdetach-controller AttachVolume.NewAttacher failed for volume "pv" : Failed to get AWS Cloud Provider. GetCloudProvider returned <nil> instead
Warning FailedMount 7s kubelet, ip-172-31-3-191.us-east-2.compute.internal MountVolume.SetUp failed for volume "pv" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/2e47e8b4-4755-46d6-9bc4-461ea02a6cb9/volumes/kubernetes.io~aws-ebs/pv --scope -- mount -o bind /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-2a/vol-011d7bb42da888b82 /var/lib/kubelet/pods/2e47e8b4-4755-46d6-9bc4-461ea02a6cb9/volumes/kubernetes.io~aws-ebs/pv
Output: Running scope as unit run-20058.scope.
mount: /var/lib/kubelet/pods/2e47e8b4-4755-46d6-9bc4-461ea02a6cb9/volumes/kubernetes.io~aws-ebs/pv: special device /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-2a/vol-011d7bb42da888b82 does not exist.
I have Kubernetes cluster running in same availability zone where EBS volumes is available
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gp2-retain
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
reclaimPolicy: Retain
mountOptions:
- debug
volumeBindingMode: Immediate
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
app: asvignesh
name: _PVC_
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: gp2-retain
volumeMode: Filesystem
volumeName: _PV_
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: _PV_
spec:
accessModes:
- ReadWriteOnce
awsElasticBlockStore:
fsType: xfs
volumeID: aws://us-east-1a/vol-xxxxxxxxx
capacity:
storage: 10Gi
persistentVolumeReclaimPolicy: Retain
storageClassName: gp2-retain
volumeMode: Filesystem
---
apiVersion: v1
kind: Service
metadata:
name: mysql
labels:
app: asvignesh
spec:
ports:
- port: 3306
targetPort: 3306
selector:
app: asvignesh
tier: mysql
clusterIP: None
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
labels:
app: asvignesh
spec:
selector:
matchLabels:
app: asvignesh
tier: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: asvignesh
tier: mysql
spec:
containers:
- image: mysql:5.6
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: password
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: _PVC_
Are you running a cluster on managed K8s or bare metal?
Because
On the Kubernetes side of the house, you’ll need to make sure that the
--cloud-provider=aws command-line flag is present for the API server, controller manager, and every Kubelet in the cluster.
Document to refer : https://blog.scottlowe.org/2018/09/28/setting-up-the-kubernetes-aws-cloud-provider/
Example YAML
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
reclaimPolicy: Retain
mountOptions:
- debug
Ref : https://faun.pub/mysql-pod-with-persistent-ebs-volume-in-eks-150af369ff94
I have applied the following pvc yaml.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ebs-claim
spec:
accessModes:
- ReadWriteOnce
storageClassName: ebs-sc
resources:
requests:
storage: 4Gi
I now want my statefulset to use the PVC I have created. Instead, it is creating new PVC of a different storageclass.
apiVersion: v1
kind: statefulset
metadata:
name: example
spec:
# Name for the service object created by the operator
serviceName: mongodb-service
selector: {}
# Specifies a size for the data volume different from the default 10Gi
volumeClaimTemplates:
- metadata:
name: ebs-claim
template:
spec:
nodeSelector:
eks.amazonaws.com/nodegroup: managed-ng-private-1
How can I get my statefulset to use existing PVCs instead of creating new ones?
Specify it like normal in the volumes section of the pod spec template. But you won't get the special behavior of creating a new PVC for each replica since that requires creating new ones.
The stateful set definition is missing the volume. Give the below yaml a try and check.
apiVersion: v1
kind: statefulset
metadata:
name: example
spec:
# Name for the service object created by the operator
serviceName: mongodb-service
selector: {}
# Specifies a size for the data volume different from the default 10Gi
volumes:
- name: ebs-vol
persistentVolumeClaim:
claimName: PersistentVolumeClaim
volumeClaimTemplates:
- metadata:
name: ebs-claim
template:
spec:
nodeSelector:
eks.amazonaws.com/nodegroup: managed-ng-private-1
I using k8s/client-go library to control and develop my application (https://github.com/kubernetes/client-go).
I have an issue when use sub-path of persistent volume claim.
Example, I'm having two pod and mount data of each container to 2 subpath ORG1/DIR1 and ORG2/DIR2 on persistent volume claim (efs file), detail in blow:
apiVersion: v1
kind: Pod
metadata:
name: my-lamp-site
spec:
containers:
- name: mysql
image: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: "rootpasswd"
volumeMounts:
- mountPath: /var/lib/mysql
name: site-data
subPath: ORG1/DIR1
- name: php
image: php:7.0-apache
volumeMounts:
- mountPath: /var/www/html
name: site-data
subPath: ORG1/DIR2
volumes:
- name: site-data
persistentVolumeClaim:
claimName: hpc-vinhha-test
And when I call to delete this pod, currently k8s only delete pod, and core lib not delete data of pod on persistent volume claim. So, data of PVC will become garbage and become bigger and bigger.
I want to delele all data in sub path ORG1/DIR1 and ORG1/DIR2 when pods deleted.
This is file yaml of pvc:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"efs-claim","namespace":"default"},"spec":{"accessModes":["ReadWriteMany"],"resources":{"requests":{"storage":"5Gi"}},"storageClassName":"efs-sc"}}
pv.kubernetes.io/bind-completed: "yes"
pv.kubernetes.io/bound-by-controller: "yes"
creationTimestamp: "2020-07-10T04:02:51Z"
finalizers:
- kubernetes.io/pvc-protection
name: efs-claim
namespace: default
resourceVersion: "887409"
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/efs-claim
uid: ab66c2f7-744c-4d6f-a508-2bc90f0b1897
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
storageClassName: efs-sc
volumeMode: Filesystem
volumeName: efs-pv-shared
status:
accessModes:
- ReadWriteMany
capacity:
storage: 5Gi
phase: Bound
So, can you help me with this problem. Because I'm a newbie in k8s and aws-efs. So, I don't have much experience about it :(
Thank so much.