I am trying to develop spring cloud microservice. I developed a sample demo of Spring Cloud project by using Zuul proxy, Eureka server and Hystrix. I added my developed service as a client of Eureka server and applied the routing. All are working well. Now I need to deploy in my AWS Ec2 machine. In my local I added the default zone URL in application.properties file like the following,
eureka.client.serviceUrl.defaultZone=http://localhost:8071/eureka/
When I am moving to my Ec2 machine or by sing AWS ECS, how I can modify this IP address belongs to cloud for proper configuration? I also using localhost:8090 and 8091 like these ports for Zuul and Turbine dashboard project etc. So how I need to change this URL when I am deploying to cloud?
We use domains. So you would point an A-record of api.yourdomain.com at the IP address or load balancer alias that is supporting your services.
Why? When we decided to change infrastructure we are able to change a DNS entry rather than modify all of our microservices' configurations. We recently moved from Eureka/Zuul to AWS's ALB. Using domains allowed us to run both environments in parallel and cutover with no down time. In the event there was a failure in the new environment, the old one was still running and we could cut back with a simple A-record change.
In your application.yml file you can configure different profiles so that you can test locally and then in ECS you can define the profile to use when creating the task definition.
First here is an example of how you can configure your application.yml file to be able to run on different profiles:
############# for running locally ################
server:
port: 1234
logging:
file: logs/example.log
level:
com.example: INFO
endpoints:
health:
sensitive: true
spring:
datasource:
url: jdbc:mysql://example.us-east-1.rds.amazonaws.com/example_db?noAccessToProcedureBodies=true
username: example
password: example
driver-class-name: com.mysql.jdbc.Driver
security:
oauth2:
client:
clientId: example
clientSecret: examplesecret
scope: webapp
accessTokenUri: http://localhost:9999/uaa/oauth/token
userAuthorizationUri: http://localhost:9999/uaa/oauth/authorize
resource:
userInfoUri: http://localhost:9999/uaa/user
########## For deployment in Docker containers/ECS ########
spring:
profiles: prod
datasource:
url: jdbc:mysql://example.rds.amazonaws.com/example_db?noAccessToProcedureBodies=true
username: example
password: example
driver-class-name: com.mysql.jdbc.Driver
prodnetwork:
ipAddress: api.yourdomain.com
security:
oauth2:
client:
clientId: exampleid
clientSecret: examplesecret
scope: webapp
accessTokenUri: https://${prodnetwork.ipAddress}/v1/uaa/oauth/token
userAuthorizationUri: https://${prodnetwork.ipAddress}/v1/uaa/oauth/authorize
resource:
userInfoUri: https://${prodnetwork.ipAddress}/v1/uaa/user
Second: Setting up ECS to use your Prod profile:
When you build your docker container, tag it with your new profile's name, in this case "prod"
Third: Create a task definition and define your Docker tag in the repo URL and your new profile in your container run command:
Now when you work on your application on your local machine, you can run it with "localhost" and when you deploy it to ECS you can define your new domain/ip to be used in the run command in your container definition.
Related
My applications uses HTTPS to run all services using docker-compose.
The application runs without any issues and we are trying to setup a HTTPS Load Balancer for all the services.
We created a Load Balancer using this Documentation.
We added three backend services and have set Host and path rules for all backend services.
But when trying to view below HTTPS URL's
https://Loadbalancer-ip:/strapi
https://Loadbalancer-ip:/auth
https://Loadbalancer-ip:/images/1.
I am getting the 404 page. But it works alone for All unmatched (default) alone.
I want to help you to fix your current limitation.
A URL redirect redirects your domain's visitors from one URL to another.
Before deploying a URL map, make sure you validate the URL map configuration to ensure that the map is routing requests to the appropriate backends as intended. You can do this by adding tests to the URL map configuration.
Use the gcloud compute url-maps validate command to validate URL map configuration.
gcloud compute url-maps validate --source PATH_TO_URL_MAP_CONFIG_FILE
PATH_TO_URL_MAP_CONFIG_FILE: Replace with a path to the file that contains the URL map configuration for validation.
Validating changes to an existing load balancer's URL map
If you have an existing load balancer that needs changes to the URL map, you can test those configuration changes before making them live.
Export the load balancer's existing URL map to a YAML file.
gcloud compute url-maps export URL_MAP_NAME \
--destination PATH_TO_URL_MAP_CONFIG_FILE \
--global
Edit the YAML file with new configuration. For example, if you want to edit an external HTTP(S) load balancer and send all requests with the path /video to a new backend service called video-backend-service, you can add tests to the URL map configuration as follows:
Existing URL map configuration with a single default web-backend-service:
kind: compute#urlMap
name: URL_MAP_NAME
defaultService: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendService/web-backend-service
Edited URL map configuration with added path matcher and tests for both the default web-backend-service and the new video-backend-service backend service:
kind: compute#urlMap
name: URL_MAP_NAME
defaultService: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendService/web-backend-service
hostRules:
- hosts:
- '*'
pathMatcher: pathmap
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendService/web-backend-service
name: pathmap
pathRules:
- paths:
- /video
- /video/*
service: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendService/video-backend-service
tests:
- description: Test routing to existing web service
host: foobar
path: /
service: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendService/web-backend-service
- description: Test routing to new video service
host: foobar
path: /video
service: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendService/video-backend-service
Validate the new configuration.
gcloud compute url-maps validate --source PATH_TO_URL_MAP_CONFIG_FILE
If all tests pass successfully, you should see a success message such as:
Successfully validated [https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/urlMaps/URL_MAP_CONFIG_FILE_NAME
If the tests fail, an error message appears. Make the required fixes to the URL map config file and try validating again.
Error: Invalid value for field 'urlMap.tests': ''.
Test failure: Expect URL 'HOST/PATH' to map to service 'EXPECTED_BACKEND_SERVICE', but actually mapped to 'ACTUAL_BACKEND_SERVICE'.
Once you know that the new configuration works and does not impact your existing setup, you can import it into the URL map. Note that this step will also deploy the url map with the new configuration.
gcloud compute url-maps import URL_MAP_NAME \
--source PATH_TO_URL_MAP_CONFIG_FILE \
--global
Important: If you originally set up your load balancer in the Cloud Console, the URL map name is the same as your load balancer's name.
Have fun!
Would you mean the page with status code 404, Or just can not access to your page?
Please make sure you have specified the right ip AND port of backend services.
Do you want to map Loadbalancer-ip:/strapi to service-ip:/strapi or service-ip:?
Initially, I've deployed my frontend web application and all the backend APIS in AWS ECS, each of the backend APIs has a Route53 record, and the frontend is connected to these APIs in the .env file. Now, I would like to migrate from ECS to EKS and I am trying to deploy all these application in a Minikube local cluster. I would like to keep my .env in my frontend application unchanged(using the same URLs for all the environment variables), the application should first look for the backend API inside the local cluster through service discovery, if the backend API doesn't exist in the cluster, it should connect to the the external service, which is the API deployed in the ECS. In short, first local(Minikube cluster)then external(AWS). How to implement this in Kubernetes?
http:// backendapi.learning.com --> backend API deployed in the pod --> if not presented --> backend API deployed in the ECS
.env
BACKEND_API_URL = http://backendapi.learning.com
one of the example in the code in which the frontend is calling the backend API
export const ping = async _ => {
const res = await fetch(`${process.env.BACKEND_API_URL}/ping`);
const json = await res.json();
return json;
}
Assuming that your setup is:
Basing on microservices architecture.
Applications deployed in Kubernetes cluster (frontend and backend) are Dockerized
Applications are capable to be running on top of Kubernetes.
etc.
You can configure your Kubernetes cluster (minikube instance) to relay your request to different locations by using Services.
Service
In Kubernetes terminology "Service" is an abstract way to expose an application running on a set of Pods as a network service.
Some of the types of Services are following:
ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.
NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting <NodeIP>:<NodePort>.
LoadBalancer: Exposes the Service externally using a cloud provider's load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.
ExternalName: Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up.
https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
You can use Headless Service with selectors and dnsConfig (in Deployment manifest) to achieve the setup referenced in your question.
Let me explain more:
Example
Let's assume that you have a backend:
nginx-one - located inside and outside
Your frontend manifest in most basic form should look following:
deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend
spec:
selector:
matchLabels:
app: frontend
replicas: 1
template:
metadata:
labels:
app: frontend
spec:
containers:
- name: ubuntu
image: ubuntu
command:
- sleep
- "infinity"
dnsConfig: # <--- IMPORTANT
searches:
- DOMAIN.NAME
Taking specific look on:
dnsConfig: # <--- IMPORTANT
searches:
- DOMAIN.NAME
Dissecting above part:
dnsConfig - the dnsConfig field is optional and it can work with any dnsPolicy settings. However, when a Pod's dnsPolicy is set to "None", the dnsConfig field has to be specified.
searches: a list of DNS search domains for hostname lookup in the Pod. This property is optional. When specified, the provided list will be merged into the base search domain names generated from the chosen DNS policy. Duplicate domain names are removed. Kubernetes allows for at most 6 search domains.
As for the Services for your backends.
service.yaml:
apiVersion: v1
kind: Service
metadata:
name: nginx-one
spec:
clusterIP: None # <-- IMPORTANT
selector:
app: nginx-one
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
Above Service will tell your frontend that one of your backends (nginx) is available through a Headless service (why it's Headless will come in hand later!). By default you could communicate with it by:
service-name (nginx-one)
service-name.namespace.svc.cluster.local (nginx-one.default.svc.cluster.local) - only locally
Connecting to your backend
Assuming that you are sending the request using curl (for simplicity) from frontend to backend you will have a specific order when it comes to the DNS resolution:
check the DNS record inside the cluster
check the DNS record specified in dnsConfig
The specifics of connecting to your backend will be following:
If the Pod with your backend is available in the cluster, the DNS resolution will point to the Pod's IP (not ClusterIP)
If the Pod backend is not available in the cluster due to various reasons, the DNS resolution will first check the internal records and then opt to use DOMAIN.NAME in the dnsConfig (outside of minikube).
If there is no Service associated with specific backend (nginx-one), the DNS resolution will use the DOMAIN.NAME in the dnsConfig searching for it outside of the cluster.
A side note!
The Headless Service with selector comes into play here as its intention is to point directly to the Pod's IP and not the ClusterIP (which exists as long as Service exists). If you used a "normal" Service you would always try to communicate with the ClusterIP even if there is no Pods available matching the selector. By using a headless one, if there is no Pod, the DNS resolution would look further down the line (external sources).
Additional resources:
Minikube.sigs.k8s.io: Docs: Start
Aws.amazon.com: Blogs: Compute: Enabling dns resolution for amazon eks cluster endpoints
EDIT:
You could also take a look on alternative options:
Alernative option 1:
Use rewrite rule plugin in CoreDNS to rewrite DNS queries for backendapi.learning.com to backendapi.default.svc.cluster.local
Alernative option 2:
Add hostAliases to the Frontend Pod
You can also use Configmaps to re-use .env files.
I'm trying to automate the deployment of a system using deployment manager. In essence, it's comprised of:
One compute instance running a proxy server
A second compute instance running the app itself (private IP only)
A CloudSQL instance hosting the database (MySQL)
In the existing environments they have, the database is configured with a private IP address, and private service access in the network so that the compute instance can acccess the DB by its private IP.
I've managed to get the 2 instances running, and the CloudSQL instance, but I"m struggling to get the private IP set up on the SQL instance. I've got the following:
- name: database
type: sqladmin.v1beta4.instance
properties:
backendType: SECOND_GEN
instanceType: CLOUD_SQL_INSTANCE
region: {{ properties["region"] }}
databaseVersion: {{ properties["dbType"] }}
settings:
tier: db-n1-standard-1
dataDiskSizeGb: 10
dataDiskType: PD_SSD
storageAutoResize: true
replicationType: SYNCHRONOUS
locationPreference:
zone: {{ properties['zone']}}
ipConfiguration:
privateNetwork: {{ properties["network"] }}
However, when I try to build this, I receive the error:
Failed to create subnetwork. Please create Service Networking
connection with service 'servicenetworking.googleapis.com' from
consumer project '' network '' again
I've tried to dig through the documentation to find how to create this connection using Deployment Manager, but I'm at a loss! I got as far as creating a private address range for peering:
- name: google-managed-services-<network_name>
type: compute.beta.globalAddress
properties:
network: $(ref.<network_name>.selfLink)
purpose: VPC_PEERING
addressType: INTERNAL
prefixLength: 16
and this appears to create the reservation for private service links correctly, but I can't find the final piece of the puzzle, the actual peer connection to Google's network. The documentation suggests the CLI call I need is:
> gcloud services vpc-peerings connect
--service=servicenetworking.googleapis.com
--ranges=[RESERVED_RANGE_NAME]
--network=[VPC_NETWORK]
--project=[PROJECT_ID]
but as far as I can tell, Deployment Manager doesn't support this API.
Has anyone had success with automating this sort of setup before? Pointers to relevant documentation that I might have missed are of course welcome!
The servicenetworking.googleapis.com is not currently supported by Deployment Manager nor is it a supported GCP-type so this can't be done through DM for now. I recommend creating a feature request for it since it's a relatively new API.
below config works for me, after setting https://cloud.google.com/sql/docs/mysql/configure-private-ip#configure-access
ipConfiguration:
privateNetwork: "internal"
ipv4Enabled: false
authorizedNetworks: null
We have set up OpenShift Origin on AWS using this handy guide. Our eventual
hope is to have some pods running REST or similar services that we can access
for development purposes. Thus, we don't need DNS or anything like that at this
point, just a public IP with open ports that points to one of our running pods.
Our first proof of concept is trying to get a jenkins (or even just httpd!) pod
that's running inside OpenShift to be exposed via an allocated Elastic IP.
I'm not a network engineer by any stretch, but I was able to successuflly get
an Elastic IP connected to one of my OpenShift "worker" instances, which I
tested by sshing to the public IP allocated to the Elastic IP. At this point
we're struggling to figure out how to make a pod visible that allocated Elastic IP,
owever. We've tried a kubernetes LoadBalancer service, a kubernetes Ingress,
and configuring an AWS Network Load Balancer, all without being able to
successfully connect to 18.2XX.YYY.ZZZ:8080 (my public IP).
The most promising success was using oc port-forward seemed to get at least part way
through, but frustratingly hangs without returning:
$ oc port-forward --loglevel=7 jenkins-2-c1hq2 8080 -n my-project
I0222 19:20:47.708145 73184 loader.go:354] Config loaded from file /home/username/.kube/config
I0222 19:20:47.708979 73184 round_trippers.go:383] GET https://ec2-18-2AA-BBB-CCC.us-east-2.compute.amazonaws.com:8443/api/v1/namespaces/my-project/pods/jenkins-2-c1hq2
....
I0222 19:20:47.758306 73184 round_trippers.go:390] Request Headers:
I0222 19:20:47.758311 73184 round_trippers.go:393] X-Stream-Protocol-Version: portforward.k8s.io
I0222 19:20:47.758316 73184 round_trippers.go:393] User-Agent: oc/v1.6.1+5115d708d7 (linux/amd64) kubernetes/fff65cf
I0222 19:20:47.758321 73184 round_trippers.go:393] Authorization: Bearer Pqg7xP_sawaeqB2ub17MyuWyFnwdFZC5Ny1f122iKh8
I0222 19:20:47.800941 73184 round_trippers.go:408] Response Status: 101 Switching Protocols in 42 milliseconds
I0222 19:20:47.800963 73184 round_trippers.go:408] Response Status: 101 Switching Protocols in 42 milliseconds
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
( oc port-forward hangs at this point and never returns)
We've found a lot of information about how to get this working under GKE, but
nothing that's really helpful for getting this working for OpenShift Origin on
AWS. Any ideas?
Update:
So we realized that sysdig.com's blog post on deploying OpenShift Origin on AWS was missing some key AWS setup information, so based on OpenShift Origin's Configuring AWS page, we set the following env variables and re-ran the ansible playbook:
$ export AWS_ACCESS_KEY_ID='AKIASTUFF'
$ export AWS_SECRET_ACCESS_KEY='STUFF'
$ export ec2_vpc_subnet='my_vpc_subnet'
$ ansible-playbook -c paramiko -i hosts openshift-ansible/playbooks/byo/config.yml --key-file ~/.ssh/my-aws-stack
I think this gets us closer, but creating a load-balancer service now gives us an always-pending IP:
$ oc get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jenkins-lb 172.30.XX.YYY <pending> 8080:31338/TCP 12h
The section on AWS Applying Configuration Changes seems to imply I need to use AWS Instance IDs rather than hostnames to identify my nodes, but I tried this and OpenShift Origin fails to start if I use that method. Still at a loss.
It may not satisfy the "Elastic IP" part but how about using AWS cloud provider ELB to expose the IP/port to the pod via a service to the pod with LoadBalancer option?
Make sure to configure the AWS cloud provider for the cluster (References)
Create a svc to the pod(s) with type LoadBalancer.
For instance to expose a Dashboard via AWS ELB.
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
type: LoadBalancer <-----
ports:
- port: 443
targetPort: 8443
selector:
k8s-app: kubernetes-dashboard
Then the svc will be exposed as an ELB and the pod can be accessed via the ELB public DNS name a53e5811bf08011e7bae306bb783bb15-953748093.us-west-1.elb.amazonaws.com.
$ kubectl (oc) get svc kubernetes-dashboard -n kube-system -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes-dashboard LoadBalancer 10.100.96.203 a53e5811bf08011e7bae306bb783bb15-953748093.us-west-1.elb.amazonaws.com 443:31636/TCP 16m k8s-app=kubernetes-dashboard
References
K8S AWS Cloud Provider Notes
Reference Architecture OpenShift Container Platform on Amazon Web Services
DEPLOYING OPENSHIFT CONTAINER PLATFORM 3.5 ON AMAZON WEB SERVICES
Configuring for AWS
Check this guide out: https://github.com/dwmkerr/terraform-aws-openshift
It's got some significant advantages vs. the one you referring to in your post. Additionally, it has a clear terraform spec that you can modify and reset to using an Elastic IP (haven't tried myself but should work).
Another way to "lock" your access to the installation is to re-code the assignment of the Public URL to the master instance in the terraform script, e.g., to a domain that you own (the default script sets it to an external IP-based value with "xip.io" added - works great for testing), then set up a basic ALB that forwards https 443 and 8443 to the master instance that the install creates (you can do it manually after the install is completed, also need a second dummy Subnet; dummy-up the healthcheck as well) and link the ALB to your domain via Route53. You can even use free Route53 wildcard certs with this approach.
I have setup a kubernetes(1.9) cluster on two ec-2 servers(ubuntu 16.04) and have installed a dashboard, the cluster is working fine and i get output when i do curl localhost:8001 on the master machine, but im not able to access the ui for the kubernetes dashboard on my laptops browser with masternode_public_ip:8001, master-machine-output
this is what my security group looks like security group which contains my machine ip.
Both the master and slave node are in ready state.
I know there are a lot of other ways to deploy an application on kubernetes cluster, however i want to explore this particular option for POC purpose.
I need to access the dashboard of the kubernetes UI and the nginx application which is deployed on this cluster.
So, my question: is it something else i need to add in my security group
or its because i need to do some more things on my master machine?
Also, it would be great if someone could throw some light on private and public IP and which one could be used to access the application and how does these are related
Here is the screenshot of deployment details describe deployment [2b][2c]4
This is an extensive topic ranging from Kubernetes Services (NodePort or LoadBalancer for this case) to Ingress Controllers and such. But there is a simple, quick and clean way to access your dashboard without all that.
Use either kubectl proxy or kubectl port-forward to access dashboard via embeded Kube apiserver proxy or directly forward from localhost to POD it self.
Found out the answer
Sorry for the delayed reply
I was trying to access the web application through its container's port but in kubernetes there is a concept of NodePort. so, if your container is running at port 8080 it will redirect it to a port between somewhere 30001 to 35000
all you need to do is add details to your deployment file
and expose the service
apiVersion: v1
kind: Service
metadata:
name: hello-svc
labels:
app: hello-world
spec:
type: NodePort
ports:
- port: 8080
nodePort: 30001