How to display constant values using custom-columns format of kubectl? - kubectl

I have multiple clusters and I want to check which ingresses do not specify explicit certificate. Right now I use the following command:
~$ k config get-contexts -o name | grep -E 'app(5|3)41.+-admin' | xargs -n1 -I {} kubectl --context {} get ingress -A -o 'custom-columns=NS:{.metadata.namespace},NAME:{.metadata.name},CERT:{.spec.tls.*.secretName}' | grep '<none>'
argocd argo-cd-argocd-server <none>
argocd argo-cd-argocd-server <none>
reference-app reference-app-netcore-ingress <none>
argocd argo-cd-argocd-server <none>
argocd argo-cd-argocd-server <none>
test-ingress my-nginx <none>
~$
I want to improve the output by including the context name, but I can't figure out how to modify the custom-columns format to do that.

The below command would Not yield the exact desired output, but it will be close. using jsonpath, it's possible:
kubectl config get-contexts -o name | xargs -n1 -I {} kubectl get ingress -A -o jsonpath="{range .items[*]}{} {.metadata.namespace} {.metadata.name} {.spec.tls.*.secretName}{'\n'}{end}" --context {}
If the exact output is needed, then the kubectl output needs to be looped in the bash loop. Example:
kubectl config get-contexts -o name | while read context; do k get ingress -A -o 'custom-columns=NS:{.metadata.namespace},NAME:{.metadata.name},CERT:{.spec.tls.*.secretName}' --context "$context" |awk -vcon="$context" 'NR==1{$0=$0FS"CONTEXT"}NR>1{$0=$0 FS con}1'; done |column -t
NS NAME CERT CONTEXT
default tls-example-ingress testsecret-tls kubernetes-admin-istio-demo.local#istio-demo.local
default tls-example-ingress1 testsecret-tls kubernetes-admin-istio-demo.local#istio-demo.local
default tls-example-ingress2 <none> kubernetes-admin-istio-demo.local#istio-demo.local
To perform post-processing around the header and context, the awk command was used. Here is some details about it:
Command:
awk -vcon="$context" 'NR==1{$0=$0FS"CONTEXT"}NR>1{$0=$0 FS con}1'; done |column -t
-vcon="$context": This is to create a variable called con inside awk to store the value of bash variable($context).
NR==1: Here NR is the record number(in this case line number) and $0 means record/line.
NR==1{$0=$0FS"CONTEXT"}: This means, on the 1st line, reset the line to itself followed by FS(default is space) followed by a string "CONTEXT".
Similarly, NR>1{$0=$0 FS con} means, from the 2nd line onwards, append the line with FS followed by con.
1 in the end is the tell awk to do the print.

Related

Error of the Zabbiks item when monitoring the ssl aws ALB

It is necessary with the help of Zabbiks to enable monitoring of SSL count licenses for ALB in aws.
Zabbiks can't recognize the item
UserParameter = ssl.count, aws elbv2 describe-listener-certificates --listener-arn --profile ******** arn: aws: elasticloadbalancing: eu-central - ******* ****** 1dcfc52e | jq '.Certificates | . []. CertificateArn '| wc -l
zabbix writes
Value of type "string" is not suitable for value type "Numeric (unsigned)".Value "The config profile (****) could not be found0"
If I change the Type of information to text, then everything works except for the triggers, which should return how many SSL licenses are in the ALB,
output from zabbix_agentd -t ssl.count command ssl.count [t | 26] everything works from the console, all credential are fine.
ABL in zabbix declared in macros {$ HOST.NAME} main-devs - *******. Eu-central-1.elb.amazonaws.com
Tell me who came across, what could it be?
The problems is that you are not redirecting the standard error, and your output is not a number. Since it's not a number, is not valid in a Numeric item.
Example:
$ ls -l temp.sh nonexisting.sh
ls: cannot access nonexisting.sh: No such file or directory
-rw-r--r-- 1 myuser domain users 14918 Oct 15 2019 temp.sh
$ ls -l temp.sh nonexisting.sh | wc -l
ls: cannot access nonexisting.sh: No such file or directory
1
$ ls -l temp.sh nonexisting.sh 2>/dev/null | wc -l
1
With your code:
UserParameter = ssl.count, aws elbv2 describe-listener-certificates --listener-arn --profile ******** arn: aws: elasticloadbalancing: eu-central - ******* ****** 1dcfc52e 2>/dev/null | jq '.Certificates | . []. CertificateArn '| wc -l

How to get all pods without jobs

Is it possible to retrieve all pods without taking jobs?
kubectl get pods
pod1 1/1 Running 1 28d
pod2 1/1 Running 1 28d
pods3 0/1 Completed 0 30m
pod4 0/1 Completed 0 30m
I don't want to see jobs, but only the other pod.
I don't want to fetch them basing on "Running State" because I would like to verify if all deployment I am trying to install are "deployed".
Basing on that I wanted to use the following command, but it is fetching also jobs I am trying to exclude:
kubectl wait --for=condition=Ready pods --all --timeout=600s
Add a special label (e.g. kind=pod) to your job pods. Then use kubectl get pods -l kind!=pod.
If using a bit of scripting is OK...this one-liner should return the names of all "non-Jobs" pods in all namespaces:
for p in `kubectl get pods --all-namespaces -o=jsonpath="{range .items[*]}{.metadata.name}{';'}{.metadata.ownerReferences[?(#.kind != 'Job')].name}{'\n'}{end}"`; do v_owner_name=$(echo $p | cut -d';' -f2); if [ ! -z "$v_owner_name" ]; then v_pod_name=$(echo $p | cut -d';' -f1); echo $v_pod_name; fi; done
Using the above as a foundation, the following aims to return all "non-Jobs" pods in Ready status:
for p in `kubectl get pods --all-namespaces -o=jsonpath="{range .items[*]}{.metadata.name}{';'}{'Ready='}{.status.conditions[?(#.type == 'Ready')].status}{';'}{.metadata.ownerReferences[?(#.kind != 'Job')].name}{'\n'}{end}"`; do v_owner_name=$(echo $p | cut -d';' -f3); if [ ! -z "$v_owner_name" ]; then v_pod_name=$(echo $p | cut -d';' -f1,2); echo $v_pod_name; fi; done
This doc explains (arguably - to some degree) the JSONPath support in kubectl.
If your question is -
I would like to verify if all deployment I am trying to install are
"deployed"
Then this is not the right way of checking Pods status in Kubernetes. Please check the replicas and readyReplicas for your deployment.
kubectl get deployment <deployment-Name> -ojson | jq -r '.status | { desired: .replicas, ready: .readyReplicas }'
Output:-
{
"desired": 1,
"ready": 1
}
Here I am using jq (It's very handy) utility to parse the stuff

Jenkinsfile to automatically deploy to EKS

How do I pass my aws credentials when I am running a Jenkinsjob
taking this as an example https://github.com/PaulMaddox/amazon-eks-kubectl
$ docker run -v ~/.aws:/home/kubectl/.aws -e CLUSTER=demo maddox/kubectl get services
The above works on my laptop , but I want to pass aws credentials on the file.I have aws configured in my Jenkins-->credentials .I also have a bitbucket repo which contains a Jenkinsfile and a yam file for "service" and "deployment"
the way I do it now is run the kubectl create -f filename.yaml and it deploys to eks .. just want to do the same thing but automate it with a Jenkinsfile , suggestions on how to do it either with kubectl or with helm
In your Jenkinsfile you should include similar section:
stage('Deploy on Dev') {
node('master'){
withEnv(["KUBECONFIG=${JENKINS_HOME}/.kube/dev-config","IMAGE=${ACCOUNT}.dkr.ecr.us-east-1.amazonaws.com/${ECR_REPO_NAME}:${IMAGETAG}"]){
sh "sed -i 's|IMAGE|${IMAGE}|g' k8s/deployment.yaml"
sh "sed -i 's|ACCOUNT|${ACCOUNT}|g' k8s/service.yaml"
sh "sed -i 's|ENVIRONMENT|dev|g' k8s/*.yaml"
sh "sed -i 's|BUILD_NUMBER|01|g' k8s/*.yaml"
sh "kubectl apply -f k8s"
DEPLOYMENT = sh (
script: 'cat k8s/deployment.yaml | yq -r .metadata.name',
returnStdout: true
).trim()
echo "Creating k8s resources..."
sleep 180
DESIRED= sh (
script: "kubectl get deployment/$DEPLOYMENT | awk '{print \$2}' | grep -v DESIRED",
returnStdout: true
).trim()
CURRENT= sh (
script: "kubectl get deployment/$DEPLOYMENT | awk '{print \$3}' | grep -v CURRENT",
returnStdout: true
).trim()
if (DESIRED.equals(CURRENT)) {
currentBuild.result = "SUCCESS"
return
} else {
error("Deployment Unsuccessful.")
currentBuild.result = "FAILURE"
return
}
}
}
}
}
which will be responsible for automating deployment proccess.
I hope it helps.

Ansible - Replace " and \r\ in a variable

I want to replace the " and \r\ from a variable content using Ansible.
I have the following data in a variable result thatI register the output to the variable from the previous task
curl -s -H \"Authorization: JWT eyJ4NWMiOlsiTUlJQytqQ0NBHuHO96csEQ\r\" https://hub.docker.com/v2/repositories/talasecurityinc/?page_size=10000 | jq -r '.results|.[]|.name'
In the above content I want to replace the \ and \r\ with null.
I have tried the below way but it doesn't work for me.
- set_fact: final_out="{{result | replace('\', "") | replace('\r\', '')}}"
The expected output is
curl -s -H "Authorization: JWT eyJ4NWMiOlsiTUlJQytqQ0NBHuHO96csEQ" https://hub.docker.com/v2/repositories/talasecurityinc/?page_size=10000 | jq -r '.results|.[]|.name'
The example playbook snippet would be helpful for me since I am new to ansible.
Escaping Hell....
I was not able to use replace, probably because I didn't try hard/smart enough. Meanwhile, in your specific case, you can achieve the expected result with a single regex_replace filter call so it was easier (and it worked right away :)).
I used yaml folded blocks (>) with white space control (-) to minimize the escape hassle. If you don't know what that is, have a look at a yaml doc (learn yaml in y minutes is my favorite one)
Note that the remaining backslashes in the last result below are added by ansible to escape the double quotes in the output.
---
- name: Escape chars
hosts: localhost
gather_facts: false
vars:
test: >-
curl -s -H \"Authorization: JWT eyJ4NWMiOlsiTUlJQytqQ0NBHuHO96csEQ\r\"
https://hub.docker.com/v2/repositories/talasecurityinc/?page_size=10000
| jq -r '.results|.[]|.name'
tasks:
- name: Show the untouched var
debug:
var: test
- name: Escape the var as intended
debug:
msg: >-
{{ test | regex_replace('\\r?\\?', '') }}
which results in
PLAY [Escape chars] ********************************************************************
TASK [Show the untouched var] **********************************************************
ok: [localhost] => {
"test": "curl -s -H \\\"Authorization: JWT eyJ4NWMiOlsiTUlJQytqQ0NBHuHO96csEQ\\r\\\" https://hub.docker.com/v2/repositories/talasecurityinc/?page_size=10000 | jq -r '.results|.[]|.name'"
}
TASK [Escape the var as intended] ******************************************************
ok: [localhost] => {
"msg": "curl -s -H \"Authorization: JWT eyJ4NWMiOlsiTUlJQytqQ0NBHuHO96csEQ\" https://hub.docker.com/v2/repositories/talasecurityinc/?page_size=10000 | jq -r '.results|.[]|.name'"
}
PLAY RECAP *****************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0

How to collect a grep and use it in a aws configset

In my aws Cloud Formation cfn configset I have a command to set an environment key to the name of the user group apache belongs to as it might be apache or www-data depending on the distro.
Something like this:
Metadata:
AWS::CloudFormation::Init:
configSets:
joomla:
- "set_permissions"
- "and_some_more..."
configure_cfn:
files:
/etc/cfn/hooks.d/cfn-auto-reloader.conf:
content: !Sub |
[cfn-auto-reloader-hook]
triggers=post.update
path=Resources.EC2.Metadata.AWS::CloudFormation::Init
action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource EC2 --configsets joomla --region ${AWS::Region}
mode: "000400"
owner: root
group: root
.....
set_permissions:
commands:
01_01_get_WebServerGroup:
env:
#webserver group might be apache or www-data depending on the distro
WebServerGp:
command: "ps -ef | egrep '(httpd|apache2|apache)' | grep -v `whoami` | grep -v root | head -n1 | awk '{print $1}'"
However, when I launch this stack, the configsets process halts at this point and I get an an error in the cfn_init.log that looks like this:
File
"/usr/lib/python2.7/dist-packages/cfnbootstrap/command_tool.py", line
80, in apply
raise ToolError(u"%s does not specify the 'command' attribute, which is required" % name) ToolError: 01_01_get_WebServerGroup does
not specify the 'command' attribute, which is required
Is this the preferred method to catch and use a grep result in a configset command? Is there a better way? What can I do to address the error thrown in the cfn_init.log?
OK, I guess I can create parameter and mapping elements to capture the distro type on launch and then set the webserver group accordingly but I am really trying to understand how to set the env: key to a response from the cli.
The problem of your code is this line WebServerGp.
Line command is must be on the same level of env, under the commands name, in your case is 01_01_get_WebServerGroup. So, it has to be like this:
commands:
01_01_get_WebServerGroup:
env: ..
command: ..
If you want to use the result of grep, you can put them on variable and use it later.
You can specify more than one command under that command line using \n for executing the command.
Please check this code below.
command: "result=(ps ef | grep ...)\n echo $result\n ..."
If you have really long command, you can use the Fn::Join to be the value of command.