pilot discovery has exposed http service, but there is no documents on it. How can I do query like listing all registered services through pilot api?
I just found the related document and code:
Debug Interface:
https://github.com/istio/istio/tree/master/pilot/pkg/proxy/envoy/v2
PILOT=istio-pilot.istio-system:9093
# What is sent to envoy
# Listeners and routes
curl $PILOT/debug/adsz
# Endpoints
curl $PILOT/debug/edsz
# Clusters
curl $PILOT/debug/cdsz
# General metrics
curl $PILOT/metrics
# All services/external services from all registries
curl $PILOT/debug/registryz
# All endpoints
curl $PILOT/debug/endpointz[?brief=1]
# All configs.
curl $PILOT/debug/configz
List all endpoints:
/v1/registration
https://github.com/istio/istio/blob/master/pilot/pkg/proxy/envoy/discovery.go#L141
pprof
/debug/pprof/
healthz
/ready
mux.HandleFunc("/debug/edsz", s.edsz)
mux.HandleFunc("/debug/adsz", s.adsz)
mux.HandleFunc("/debug/cdsz", cdsz)
mux.HandleFunc("/debug/syncz", Syncz)
mux.HandleFunc("/debug/registryz", s.registryz)
mux.HandleFunc("/debug/endpointz", s.endpointz)
mux.HandleFunc("/debug/endpointShardz", s.endpointShardz)
mux.HandleFunc("/debug/workloadz", s.workloadz)
mux.HandleFunc("/debug/configz", s.configz)
mux.HandleFunc("/debug/authenticationz", s.authenticationz)
mux.HandleFunc("/debug/config_dump", s.ConfigDump)
mux.HandleFunc("/debug/push_status", s.PushStatusHandler)
i used it like this and it worked . tested on istio 1.7
connect to istiod:
kubectl exec -ti < istiod-pod-name > -c discovery -n istio-system -- /bin/bash
run pilot-discovery command:
pilot-discovery request get metrics | grep citadel
Related
I have a problem, I can't generate the certificates in AWS EC2
Linux AWS
I trying execute this command in SSH - docker run --rm -p 3000:3000 -p 3001:3001 -e ASPNETCORE_HTTPS_PORT=https://+:3001 -e ASPNETCORE_ENVIRONMENT="Development" -e ASPNETCORE_URLS=https://+:3001 $MY ECR CONTAINER HERE$
i try too docker run --rm -p 3000:3000 -p 3001:3001 -e ASPNETCORE_HTTPS_PORT=https://+:3001 -e ASPNETCORE_ENVIRONMENT="Development" -e ASPNETCORE_URLS=https://+:3001 -v ASPNETCORE_Kestrel__Certificates__Default__Password=$MY PW$* -v ASPNETCORE_Kestrel__Certificates__Default__Path=%USERPROFILE%/aspnet/https/aspnetapp.pfx $MY CONTAINER$
Error on SSH
My Dockerfile
My Launch Settings
DOTNET INFO ON LINUX AWS
AWS CERTIFICATE MANAGER
it works perfectly on HTTP 80 but to unable HTTPS 443, a docker need a certificate.
what do i need to do to generate this certificate in aws linux?
Edit*
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[6 0]
Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may n ot be persisted outside of the container. Protected data will be unavailable whe n container is destroyed.
warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
No XML encryptor configured. Key {f37427eb-3dc8-4d33-9177-92caadc2c880} ma y be persisted to storage in unencrypted form.
After a lot of searching find the following answers and my project is on LIVE.
1º I edited my program.cs so that it uses HTTPS Redirection and HSTS and configured the Forward Headers
Follow the codes.
`builder.Services.Configure(options =>
{
options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
});`
app.UseSwagger();
app.UseSwaggerUI(c =>
{
c.SwaggerEndpoint("/swagger/v1/swagger.json",
"Api Documentation for MyLandingApp");
});
app.UseHsts();
app.UseHttpsRedirection();
app.UseCors("MyLandingAppPolicy");
app.UseForwardedHeaders();
app.Use(async (context, next) =>
{
if (context.Request.IsHttps || context.Request.Headers["X-Forwarded-Proto"] == Uri.UriSchemeHttps)
{
await next();
}
else
{
string queryString = context.Request.QueryString.HasValue ? context.Request.QueryString.Value : string.Empty;
var https = "https://" + context.Request.Host + context.Request.Path + queryString;
context.Response.Redirect(https);
}
});
app.UseAuthentication();
app.UseAuthorization();
2º I added some stuff in my Appsettings.Json
"https_port": 3001,
3ºI changed my DockerFile to create a self certificate and enable HTTPS on docker run
.
Docker File
4ª I changed the docker container execution string, removed the HTTP port that I wouldn't use anyway, I'll explain later.
docker run --rm -p 3001:3001 -e ASPNETCORE_HTTPS_PORT=https://+:3001 -e ASPNETCORE_ENVIRONMENT="Production" -e ASPNETCORE_URLS=https://+:3001 $MY CONTAINER IN ESR$
5º I configured the loudbalancer like this:
HTTP80 - Loud Balancer http80
HTTPS443 - Loud bALANCER https443
Só que tem o macete...
you need to create the target group pointing to the main server, then you will take the private IP and create a new target group
Target Group
With this you will have done the redirection and CERTIFICATE configuration for your API.
Remembering that in Listener https 443 you need a valid certificate.
I am trying to list the list of services that are present in istio internal service registry. However, failed to do that, more specifically details added by service entry. Anyone knows the istioctl command to list the services ?
Ref- https://istio.io/docs/reference/commands/istioctl/
There is detailed documentation page about commands that does that.
The proxy-status command allows you to get an overview of your mesh. If you suspect one of your sidecars isn’t receiving configuration or is out of sync then proxy-status will tell you this.
$ istioctl proxy-status
PROXY CDS LDS EDS RDS PILOT VERSION
details-v1-6dcc6fbb9d-wsjz4.default SYNCED SYNCED SYNCED SYNCED istio-pilot-75bdf98789-tfdvh 1.1.2
istio-egressgateway-c49694485-l9d5l.istio-system SYNCED SYNCED SYNCED NOT SENT istio-pilot-75bdf98789-tfdvh 1.1.2
istio-ingress-6458b8c98f-7ks48.istio-system SYNCED SYNCED SYNCED NOT SENT istio-pilot-75bdf98789-n2kqh 1.1.2
istio-ingressgateway-7d6874b48f-qxhn5.istio-system SYNCED SYNCED SYNCED SYNCED istio-pilot-75bdf98789-n2kqh 1.1.2
productpage-v1-6c886ff494-hm7zk.default SYNCED SYNCED SYNCED STALE istio-pilot-75bdf98789-n2kqh 1.1.2
ratings-v1-5d9ff497bb-gslng.default SYNCED SYNCED SYNCED SYNCED istio-pilot-75bdf98789-n2kqh 1.1.2
reviews-v1-55d4c455db-zjj2m.default SYNCED SYNCED SYNCED SYNCED istio-pilot-75bdf98789-n2kqh 1.1.2
reviews-v2-686bbb668-99j76.default SYNCED SYNCED SYNCED SYNCED istio-pilot-75bdf98789-tfdvh 1.1.2
reviews-v3-7b9b5fdfd6-4r52s.default SYNCED SYNCED SYNCED SYNCED istio-pilot-75bdf98789-n2kqh 1.1.2
If a proxy is missing from this list it means that it is not currently connected to a Istiod instance so will not be receiving any configuration.
SYNCED means that Envoy has acknowledged the last configuration Istiod has sent to it.
NOT SENT means that Istiod hasn’t sent anything to Envoy. This usually is because Istiod has nothing to send.
STALE means that Istiod has sent an update to Envoy but has not received an acknowledgement. This usually indicates a networking issue between Envoy and Istiod or a bug with Istio itself.
There is also similar command istioctl proxy-config it can be used to see how a given Envoy instance is configured. More details are also on the same documentation page.
Hope it helps.
UPDATE:
In case of ServiceEntry on pilot there is a debug endpoint:
localhost:8080/debug/registryz
For more information how to use it take a look at this documentation page.
example:
kubectl exec $POD -c istio-proxy -- curl 'localhost:8080/debug/registryz' > service_registry
Hope it helps.
I didn't find the way to get the content of Istio registry in the convenient form using istioctl, but it's achievable using slightly different approach.
Piotr Malec was right (please consider to upvote his answer too ), Pilot's debug interface can provide the list you've mentioned in the question. And the request to Istio Pilot can be sent using any Pod's envoy sidecar. Both ports are valid for the request: 15014 and 8080, and give the same result.
For this example I'm using Pilot's own envoy sidecar. I'm also using jq on my machine to get services' names from the JSON response.
Imagine we have three ServiceEntry in the cluster:
$ kubectl get serviceentry
NAME HOSTS LOCATION RESOLUTION AGE
httpbin [httpbin.org] DNS 18d
httpbin-com-ext [httpbin.org] MESH_EXTERNAL DNS 22d
wikipedia-org-se [wikipedia.org *.wikipedia.org] MESH_EXTERNAL NONE 22d
The following sequence of commands gives us the list of services including the services defined by ServiceEntry objects:
# create a shortcut to Istio Pilot pod
$ PILOT=$(kubectl get -A pods --selector=istio=pilot -o jsonpath='{.items[*].metadata.name}')
# Create shortcut to Istio Pilot namespace
$ PILOT_NS=$(kubectl get -A pods --selector=istio=pilot -o jsonpath='{.items[*].metadata.namespace}')
# Get the list of services in Istio mesh and filter their names using jq
$ kubectl exec $PILOT -n $PILOT_NS -c discovery -- curl -s 'localhost:15014/debug/registryz' | jq '.[].hostname'
# or
$ kubectl exec $PILOT -n $PILOT_NS -c istio-proxy -- curl -s 'localhost:15014/debug/registryz' | jq '.[].hostname'
The output:
( Note the last several entries.)
"ingress-nginx.ingress-nginx.svc.cluster.local"
"istio-citadel.istio-system.svc.cluster.local"
"istio-egressgateway.istio-system.svc.cluster.local"
"istio-galley.istio-system.svc.cluster.local"
"istio-ingressgateway.istio-system.svc.cluster.local"
"istio-pilot.istio-system.svc.cluster.local"
"istio-policy.istio-system.svc.cluster.local"
"istio-sidecar-injector.istio-system.svc.cluster.local"
"istio-telemetry.istio-system.svc.cluster.local"
"kube-dns.kube-system.svc.cluster.local"
"kubernetes.default.svc.cluster.local"
"kubernetes.test.svc.cluster.local"
"metrics-server.metrics.svc.cluster.local"
"productpage.default.svc.cluster.local"
"prometheus.istio-system.svc.cluster.local"
"ratings.default.svc.cluster.local"
"reviews.default.svc.cluster.local"
"httpbin.org"
"httpbin.org"
"wikipedia.org"
"*.wikipedia.org"
You can find more information about Debug Handlers on Istio github page:
Istio - Debug interface
I need to configure a NIFI flow that invokes a webservice. The curl command works fine from the cmd line of my nifi host
curl -U <user>:<password> -x <ip>:8080 -H 'SOAPACTION:"http://method"'-X POST -H 'Content-type: text/xml' -d #req.xml https://url -o result2.xml
Im trying to configure NIFI InvokeHTTP processor to make the call from within a nifi flow with following settings.
HTTP Method:POST
Remote URL: https://url
Proxy Host: <ip>
Proxy Port: 8080
Proxy Username: <user>
Proxy Password: <password>
SOAPAction: method (Dynamic Properties)
But it keeps running in the Failure queue.
how do i configure my InvokeHTTP according to the curl command which works ok?
We updated the version of the InvokeHTTP component from 1.3.0 to the newest
When i try to run a local ESP then i get this error.
ERROR:Fetching service config failed(status code 403, reason Forbidden, url ***)
I have a new created service account this account works fine with gcloud cli.
System: OSX Sierra with Docker for MAC
this is the command that i use to start the container:
docker run -d --name="esp" --net="host" -v ~/Downloads:/esp gcr.io/endpoints-release/endpoints-runtime:1.0 -s 2017-02-07r5 -v echo.endpoints.****.cloud.goog -p 8082 -a localhost:9000 -k /esp/serviceaccount.json
UPDATE:
I have found the error i have set for the service name the verision and for the version the servicename.
Now i get no error but it not works, this is the console output from the container. From my view is all fine but it not works, i can't call the proxy with localhost:8082/***
INFO:Constructing an access token with scope https://www.googleapis.com/auth/service.management.readonly
INFO:Service account email: aplha-api#****.iam.gserviceaccount.com
INFO:Refreshing access_token
INFO:Fetching the service configuration from the service management service
nginx: [warn] Using trusted CA certificates file: /etc/nginx/trusted-ca-certificates.crt
This is the used correct command:
docker run -d --name="esp-user-api" --net="host" -v ~/Downloads:/esp gcr.io/endpoints-release/endpoints-runtime:1.0 -s echo.endpoints.***.cloud.goog -v 2017-02-07r5 -p 8082 -a localhost:9000 -k /esp/serviceaccount.json
Aron, I assume:
(1) you are following this user guide: https://cloud.google.com/endpoints/docs/running-esp-localdev
(2) And you do have a backend running on localhost:9000
Have you issued a curl request as suggested in that user guide to localhost:8082/***? does curl command get stuck or returns any error message?
If you don't have a local backend running yet, I would recommend you to follow the user guide above to run a local backend. Note this guide will instruct you to run it at port 8080, so you'll need to change your docker run command from "-a localhost:9000" to "-a localhost:8080" as well.
Also, please note this user guide is for linux env. We haven't tried this set up in a Mac env yet. We do notice some user gets this working on Windows docker with extra work, where he sets backend to "IP of docker NIC". Note "-a" is short for "--backend".
see https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/google-cloud-endpoints/4sRaSkigPiU/KY8g46NSBgAJ
I am setting up a Kubernetes deployment using auto-scaling groups and Terraform. The kube master node is behind an ELB to get some reliability in case of something going wrong. The ELB has the health check set to tcp 6443, and tcp listeners for 8080, 6443, and 9898. All of the instances and the load balancer belong to a security group that allows all traffic between members of the group, plus public traffic from the NAT Gateway address. I created my AMI using the following script (from the getting started guide)...
# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
# cat <<EOF > /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
# apt-get update
# # Install docker if you don't have it already.
# apt-get install -y docker.io
# apt-get install -y kubelet kubeadm kubectl kubernetes-cni
I use the following user data scripts...
kube master
#!/bin/bash
rm -rf /etc/kubernetes/*
rm -rf /var/lib/kubelet/*
kubeadm init \
--external-etcd-endpoints=http://${etcd_elb}:2379 \
--token=${token} \
--use-kubernetes-version=${k8s_version} \
--api-external-dns-names=kmaster.${master_elb_dns} \
--cloud-provider=aws
until kubectl cluster-info
do
sleep 1
done
kubectl apply -f https://git.io/weave-kube
kube node
#!/bin/bash
rm -rf /etc/kubernetes/*
rm -rf /var/lib/kubelet/*
until kubeadm join --token=${token} kmaster.${master_elb_dns}
do
sleep 1
done
Everything seems to work properly. The master comes up and responds to kubectl commands, with pods for discovery, dns, weave, controller-manager, api-server, and scheduler. kubeadm has the following output on the node...
Running pre-flight checks
<util/tokens> validating provided token
<node/discovery> created cluster info discovery client, requesting info from "http://kmaster.jenkins.learnvest.net:9898/cluster-info/v1/?token-id=eb31c0"
node/discovery> failed to request cluster info, will try again: [Get http://kmaster.jenkins.learnvest.net:9898/cluster-info/v1/?token-id=eb31c0: EOF]
<node/discovery> cluster info object received, verifying signature using given token
<node/discovery> cluster info signature and contents are valid, will use API endpoints [https://10.253.129.106:6443]
<node/bootstrap> trying to connect to endpoint https://10.253.129.106:6443
<node/bootstrap> detected server version v1.4.4
<node/bootstrap> successfully established connection with endpoint https://10.253.129.106:6443
<node/csr> created API client to obtain unique certificate for this node, generating keys and certificate signing request
<node/csr> received signed certificate from the API server:
Issuer: CN=kubernetes | Subject: CN=system:node:ip-10-253-130-44 | CA: false
Not before: 2016-10-27 18:46:00 +0000 UTC Not After: 2017-10-27 18:46:00 +0000 UTC
<node/csr> generating kubelet configuration
<util/kubeconfig> created "/etc/kubernetes/kubelet.conf"
Node join complete:
* Certificate signing request sent to master and response
received.
* Kubelet informed of new secure connection details.
Run 'kubectl get nodes' on the master to see this machine join.
Unfortunately, running kubectl get nodes on the master only returns itself as a node. The only interesting thing I see in /var/log/syslog is
Oct 27 21:19:28 ip-10-252-39-25 kubelet[19972]: E1027 21:19:28.198736 19972 eviction_manager.go:162] eviction manager: unexpected err: failed GetNode: node 'ip-10-253-130-44' not found
Oct 27 21:19:31 ip-10-252-39-25 kubelet[19972]: E1027 21:19:31.778521 19972 kubelet_node_status.go:301] Error updating node status, will retry: error getting node "ip-10-253-130-44": nodes "ip-10-253-130-44" not found
I am really not sure where to look...
The Hostnames of the two machines (master and the node) should be different. You can check them by running cat /etc/hostname. If they do happen to be the same, edit that file to make them different and then do a sudo reboot to apply the changes. Otherwise kubeadm will not be able to differentiate between the two machines and it will show as a single one in kubectl get nodes.
Yes , I faced the same problem.
I resolved by:
killall kubelet
run the kubectl join command again
and start the kubelet service