I am using kubectl in a script to get the current GKE cluster name like so:
CURRENT_CLUSTER=$(kubectl run curl --rm --restart=Never -it --image=appropriate/curl -- -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/attributes/cluster-name)
Unfortunately kubect prints pod "curl" deleted to standard output, so the result is this:
my-cluster-us-west1pod "curl" deleted
How can I stop kubectl from printing this string?
If there were a space between the cluster name and the unwanted text then you could use awk to take the first word of the output:
CURRENT_CLUSTER=$(kubectl run curl --rm --restart=Never -it --image=appropriate/curl -- -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/attributes/cluster-name | awk '{print $1;}')
But there isn't a space so it has to be removed differently, such as using sed to remove 'pod ' and anything after it:
CURRENT_CLUSTER=$(kubectl run curl --rm --restart=Never -it --image=appropriate/curl -- -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/attributes/cluster-name | sed "s/pod .*//g")
Related
I have an AWS EMR running Jupyterhub version 0.8.1+ that I want to check if there are any active notebooks that are running any code.
I've tried the below commands but they don't seem to output what I'm looking for here since the users server is always running and notebooks can be running without any code being executed.
# only lists running servers and jovyan is always running.
sudo docker exec jupyterhub jupyter notebook list
# No useful information outputted
curl -k -i -H "Accept: application/json" "https://localhost:9443/api/sessions"
# always lists processes regardless of running notebooks
ps aux | grep ipykernel
# The last_activity only updates when a user creates a new file or folder in the ui.
curl -k https://localhost:9443/hub/api/users/$user -H "Authorization: token $admin_token" | jq -r .last_activity
curl -k https://localhost:9443/hub/api/users -H "Authorization: token $admin_token" | jq -r .last_activity
Im following this AWS blog to check if the entire EMR is idle before terminating the cluster but they never seemed to have fully implemented the jupyter checks.
https://aws.amazon.com/blogs/big-data/optimize-amazon-emr-costs-with-idle-checks-and-automatic-resource-termination-using-advanced-amazon-cloudwatch-metrics-and-aws-lambda/
Most of the files referenced can be found in Github https://github.com/septian-putra/emr-monitoring
To see if a notebook is "idle" for "busy" you can run curl -ks https://localhost:9443/user/jovyan/api/kernels -H "Authorization: token ${admin_token}"
With this command all you need to do is put it in a simple if statement with a grep -q in order to get a true false idle value.
if [ $(curl -ks https://localhost:9443/user/jovyan/api/kernels -H "Authorization: token ${admin_token}" | grep -q "busy") ]; then
JUPYTER_BUSY_NOTEBOOKS=1
else
JUPYTER_BUSY_NOTEBOOKS=0
fi
(curl -ks For a silent output and to ignore ssl. jovyan being my admin user)
Documentation
https://jupyter-kernel-gateway.readthedocs.io/en/latest/websocket-mode.html#http-resources
/api/sessions might also be useful to look at.
I'm using the wso2 stream integrator in order to run my siddhi application.
The operating environment is windows 10 pro.
I'd like to ask you for your help about way of using Siddhi's REST API.
Open a command prompt as the administrator and go to \wso2si-1.1.0\bin. Then, execute the command server.bat --run.
Deploy any siddhi files in wso2si-1.1.0\wso2\server\deployment\siddhi-files, and it's working correctly.
Open a new command prompt and execute the command curl -X GET "http://localhost:9090/siddhi-apps" -H "accept: application/json" -u "admin:admin" -k.
The command succeeds and a list of applications is displayed.
Next, I defined the ABC as named-window on FlowApp and executed the command curl -X POST "https://localhost:9443/query" -H "content-type: application/json" -u "admin:admin" -d "{"appName" : "FlowApp", "query" : "from ABC select *" }" -k
The above query fails and "Problem accessing: /query. Reason: Not Found" is displayed on the command prompt.
Why can't I find "/query"?
By default query API is available in a different port. Try port number 7443
curl -X POST "https://localhost:7443/query" -H "content-type: application/json" -u "admin:admin" -d "{"appName" : "FlowApp", "query" : "from ABC select *" }" -k
I've created a simple AD on AWS and I'm trying to connect to it using the Administrator credentials set up while creating the simple AD. I'm running the ldapsearch command from another EC2 instance in the same subnet. However I"m running into an authentication error and I'm pretty sure it's not the password, as I've tried changing it multiple time with no luck.
Below is the ldapsearch command I'm using.
$ldapsearch -x -v -h "10.*.*.112" -b "dc=corp-testing,dc=example,dc=com" –D "Administrator#corp-testing.example.com" -W sAMAccountName=Administrator
Below is the output:
ldap_initialize( ldap://10.*.*.112 )
Enter LDAP Password:
ldap_bind: Invalid credentials (49)
additional info: 80090308: LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db1
Would someone be able to point out the issue on this?
I ran into the same issue and I have found the solution, the username needs to be prefixed with the Directory NetBIOS name (this is available from the Directory details page), then login with:
ldapsearch -x \
-h 10.*.*.112 \
-b "cn=Users,dc=corp-testing,dc=example,dc=com" \
–D "${NetBIOSNAME}\\Administrator" \
-W sAMAccountName=Administrator
Obviously, change ${NetBIOSNAME} to the appropriate value.
Okay I figured it out, however I don't know the WHY. Try changing your search to:
ldapsearch -x -v -H "ldap://10.*.*.112:389/" -b "dc=corp-testing,dc=example,dc=com" –D "cn=Administrator,dc=corp-testing,dc=example,dc=com" -W sAMAccountName=Administrator
I tried this several times without the trailing / on the URI but it didn't work.
Problem was that the Administrator is inside of the Users node, so -D should include cn=Users also:
ldapsearch -x -v -h "10.*.*.112" -b "dc=corp-testing,dc=example,dc=com" –D "cn=Administrator,cn=Users,dc=corp-testing,dc=example,dc=com" -W sAMAccountName=Administrator
I have followed the instructions from https://github.com/spring-cloud-services-samples/cook and managed to install and run Config Server in PCF environment (SERVICE: Config Server, PLAN: standard).
I'm now trying to hit /encrypt endpoint of the p-config-server service, in order to encrypt new value. I'm following the instructions at http://docs.run.pivotal.io/spring-cloud-services/config-server/configuring-with-git.html#encryption-and-encrypted-values:
TOKEN=$(curl -k ACCESS_TOKEN_URI -u CLIENT_ID:CLIENT_SECRET -d
grant_type=client_credentials | jq -r .access_token); curl -k -H
"Authorization: bearer $TOKEN" -H "Accept: application/json"
URI/encrypt -d 'VALUE'
...but I always get:
{
"error": "access_denied",
"error_description": "Access is denied"
}
On the other side, if I try to get standard endpoint, to get config for an app, I'm able to retrieve JSON containing app properties. E.g.
TOKEN=$(curl -k ACCESS_TOKEN_URI -u CLIENT_ID:CLIENT_SECRET -d
grant_type=client_credentials | jq -r .access_token); curl -k -H
"Authorization: bearer $TOKEN" -H "Accept: application/json"
URI/my-app/default
... gives result like:
{"name":"my-app","profiles":["default"],"label":null,"version":"bb6e64592ced731ebba272430291a595e0f14a77","state":null,"propertySources":[{"name":"https://github.com/some-user/config/my-app.yml","source":{"my-property.name":"Test123"}}]}
Any idea why I can not access /encrypt endpoint?
Thank you.
Btw, I'm executing the command in CentOS Linux release 7.4.1708, with installed jq (command-line JSON processor).
I've got the answer from CloundFoundry IT support. In my CF environment, "encrypt" endpoint should have a trailing slash (/). So it should be ...URI/encrypt/ -d 'VALUE'. Maybe it helps someone.
One more hint I've got: There is a CF CLI plugin for the Spring-Cloud-Services which you could use for convenience.
https://github.com/pivotal-cf/spring-cloud-services-cli-plugin
cf install-plugin -r CF-Community "Spring Cloud Services"
cf help config-server-encrypt-value
Hi Actually you need to hit cf env command first and take note of configuration values from that which for sample looks like below:
{
"VCAP_SERVICES": {
"p-config-server": [
{
"credentials": {
"access_token_uri": "https://p-spring-cloud-services.uaa.cf.wise.com/oauth/token",
"client_id": "p-config-server-876cd13b-1564-4a9a-9d44-c7c8a6257b73",
"client_secret": "rU7dMUw6bQjR",
"uri": "https://config-86b38ce0-eed8-4c01-adb4-1a651a6178e2.apps.wise.com"
},
[...]
and then use those values in your curl bash script. for example
TOKEN=$(curl -k https://config-86b38ce0-eed8-4c01-adb4-1a651a6178e2.apps.wise.com -u p-config-server-876cd13b-1564-4a9a-9d44-c7c8a6257b73:rU7dMUw6bQjR -d
grant_type=client_credentials | jq -r .access_token); curl -k -H
"Authorization: bearer $TOKEN" -H "Accept: application/json"
URI/ENDPOINT | jq
Basically following values are required:
ACCESS_TOKEN_URI with the value of credentials.access_token_uri
CLIENT_ID with the value of credentials.client_id
CLIENT_SECRET with the value of credentials.client_secret
URI with the value of credentials.uri
Replace ENDPOINT with the relevant endpoint:
application/profile to retrieve configuration from a Config Server service instance
eureka/apps to retrieve the registry from a Service Registry service instance
Then I think you will no more get access denied response.
In my settings file i am getting env variables like this
'NAME': os.environ['PG_DBNAME'], # Database
I am setting in docker file like this
-e PG_DBNAME= "mapp"
Now
The web app work fine
If i log into shell via docker exec ... bash then env variables are also set
But if i log in via ipaddress and port number from ssh client then i am able to login but env variables are not set
As commented in issue 2569:
This is expected. SSH wipes out the environment as part of the login process.
One way to work around it is to dump the environment variables in /etc/environment (e.g. env | grep _ >> /etc/environment) before starting Supervisor.
Further "login processes" should source this file, and tada! There is your environment.
That env | grep _ >> /etc/environment could be part of a default run script associated (through ENTRYPOINT or CMD) to your image.
Daniel A.A. Pelsmaeker suggests jenkinsci/docker-ssh-agent issue 33 for an approach that selects and sets all environment variables excluding a specific denylist:
For my own uses I changed that line to the following:
env | egrep -v "^(HOME=|USER=|MAIL=|LC_ALL=|LS_COLORS=|LANG=|HOSTNAME=|PWD=|TERM=|SHLVL=|LANGUAGE=|_=)" >> /etc/environment
This takes all environment variables, except those listed, and appends then to /etc/environment, overriding any previously defined there.
I also had the exact same problem. I found the example on docs.docker.com appending variables by echo'ing to /etc/profile not the nicest way to do that. So here is my solution:
Dockerbuild:
I execute the docker build by the following command which also fetches the http_proxy, https_proxy and no_proxy variables from the
current shell session. The variables are passed as agruments with the --build-arg option.
[root#localhost dock-centOS]# docker build
--build-arg http_proxy="{{ lookup('env', 'http_proxy')}}"
--build-arg https_proxy="{{ lookup('env', 'https_proxy')}}"
--build-arg no_proxy="{{ lookup('env', 'no_proxy')}}"
-t my_pv_repo:centOS-with-sshd .
Dockerfile:
I use the following dockerfile snippet for setting the enviroment variables for all users. The ARG command is used instead of
ENV because i don't want docker to persist my variables in the image. The ARG variable is only available during the docker build.
The RUN command creates a bash script which is placed in the /etc/profile.d directory. During start-up of the container
/etc/profile script is run and sources all readable files in the /etc/profile.d directory.
FROM centos:7.3.1611
ARG http_proxy=$http_proxy
ARG https_proxy=$https_proxy
ARG no_proxy=$no_proxy
ARG JAVA_HOME=/usr/lib/jvm/jdk1.6.0_45
ARG DOMAIN_HOME=/home/oracle/w001/D1/app/user_projects/domains/fancy_app_domain
ARG PATH=$PATH:/usr/lib/jvm/jdk1.6.0_45/bin
ARG XAUTHORITY=~/.Xauthority
RUN shebang='#!/usr/bin/env bash'; \
env_vars="export http_proxy=${http_proxy} https_proxy=${https_proxy} no_proxy=${no_proxy}"; \
env_vars+=' JAVA_HOME=/usr/lib/jvm/jdk1.6.0_45 DOMAIN_HOME=/home/oracle/w001/D1/app/user_projects/domains/fancy_app_domain'; \
env_vars+=" PATH=${PATH}:/usr/lib/jvm/jdk1.6.0_45/bin XAUTHORITY=${XAUTHORITY}"; \
echo $shebang$'\n'$env_vars > /etc/profile.d/env_vars.sh
Test result: Well lets hit the cli to check if our environment variables are available during a ssh session.
[root#localhost vagrant]# docker exec -u root -it centOS-container bash
[root#33e7efab489c /]#
[root#33e7efab489c /]#
[root#33e7efab489c /]# cat /etc/profile.d/env_vars.sh
#!/usr/bin/env bash
export http_proxy=http://10.0.2.2:3128 https_proxy=http://10.0.2.2:3128 no_proxy=localhost,127.0.0.1 JAVA_HOME=/usr/lib/jvm/jdk1.6.0_45 DOMAIN_HOME=/home/oracle/w001/D1/app/user_projects/domains/fancy_app_domain PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/jdk1.6.0_45/bin XAUTHORITY=~/.Xauthority
[root#33e7efab489c /]#
[root#33e7efab489c /]#
[root#33e7efab489c /]# printenv
HOSTNAME=33e7efab489c
TERM=xterm
http_proxy=http://10.0.2.2:3128
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/jdk1.6.0_45/bin
DOMAIN_HOME=/home/oracle/w001/D1/app/user_projects/domains/fancy_app_domain
PWD=/
JAVA_HOME=/usr/lib/jvm/jdk1.6.0_45
LANG=en_US.UTF-8
https_proxy=http://10.0.2.2:3128
SHLVL=1
HOME=/root
no_proxy=localhost,127.0.0.1
XAUTHORITY=/root/.Xauthority
_=/usr/bin/printenv
[root#33e7efab489c /]#
[root#33e7efab489c /]#
[root#33e7efab489c /]# exit
[root#localhost vagrant]# exit
[vagrant#localhost ~]$ logout
Connection to 127.0.0.1 closed.
me#my-mac$ ssh -X root#localhost -p 7022 -o UserKnownHostsFile=/dev/null -o IdentityFile=/development/workspace/supercalifragilisticexpialidocious-app/.vagrant/machines/default/virtualbox/private_key
The authenticity of host '[localhost]:7022 ([127.0.0.1]:7022)' can't be established.
ECDSA key fingerprint is SHA256:dTd/vsmPTbrA3kPeIfArZMFEgfdlgjGHwMgE3Z5BgBc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[localhost]:7022' (ECDSA) to the list of known hosts.
/usr/bin/xauth: file /root/.Xauthority does not exist
[root#33e7efab489c ~]# su - oracle
bash-4.2$
bash-4.2$
bash-4.2$ printenv
HOSTNAME=33e7efab489c
SHELL=/bin/bash
TERM=xterm-256color
HISTSIZE=1000
http_proxy=http://10.0.2.2:3128
USER=oracle
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
MAIL=/var/spool/mail/oracle
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/jdk1.6.0_45/bin
DOMAIN_HOME=/home/oracle/w001/D1/app/user_projects/domains/fancy_app_domain
PWD=/home/oracle
JAVA_HOME=/usr/lib/jvm/jdk1.6.0_45
LANG=en_US.UTF-8
https_proxy=http://10.0.2.2:3128
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/oracle
no_proxy=localhost,127.0.0.1
LOGNAME=oracle
XAUTHORITY=/home/oracle/.Xauthority
_=/usr/bin/printenv