How to add a /api/v1/ prefix to uri in spring cloud gateway? - regex

I have two services and a gateway working with eureka:
user-service running on 8081
health-service running on 8082
So far I tried this:
routes:
- id: user-service-route
uri: http://localhost:8081/user-service
predicates:
- Path=/user-service/**
filters:
- RewritePath=(?<serviceName>.*), /api/v1/{serviceName}
But it doesn't work.

try
filters:
- RewritePath=(?<serviceName>.*), /api/v1/$\{serviceName}

Prefix location for all microservices if you use load balancer:
server:
port: 8765
management:
endpoints:
web:
exposure:
include: gateway
eureka:
client:
service-url:
defaultZone: http://localhost:8761/eureka
spring:
application:
name: gateway
cloud:
gateway:
discovery:
locator:
enabled: true
lower-case-service-id: true
default-filters:
- RewritePath=/api/v1/.*?/(?<remaining>.*), /$\{remaining}
routes:
- predicates:
- Path=/api/v1/profile/**
uri: lb://profile
- predicates:
- Path=/api/v1/dictionary/**
uri: lb://dictionary

Related

SNI based routing in envoy

I am working on a use case to do SNI based routing in envoy without doing TLS termination
Scenario:
Customer makes a HTTPS request for domain name dd-server_name-zion-zorfy.com
I need to extract the domain name using SNI and forward the request to the upstream with DNS name wd-server_name-zion-zorfy.com
The server_name is dynamic in nature and can have different values. Basically, I need to replace dd prefix in the host name with wd and route the request to this destination without doing the TLS termination in envoy.
I have the below envoy configuration and when I make a HTTPS request, I am getting connection timeout error
static_resources:
listeners:
- name: listener_0
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 9002
listener_filters:
- name: envoy.filters.listener.tls_inspector
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector
filter_chains:
- filters:
- name: envoy.filters.network.sni_dynamic_forward_proxy
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.sni_dynamic_forward_proxy.v3.FilterConfig
port_value: 443
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: V4_ONLY
- name: envoy.filters.http.lua
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
inline_code: |
function envoy_on_request(request_handle)
host = string.match(request_handle:headers():get("host"), "dd%-(.*)")
target = "wd-" .. host
REQUESTED_SERVER_NAME = target
end
- name: envoy.tcp_proxy
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
stat_prefix: tcp
cluster: dynamic_forward_proxy_cluster
tunneling_config:
hostname: "%REQUESTED_SERVER_NAME%:443"
clusters:
- name: dynamic_forward_proxy_cluster
lb_policy: CLUSTER_PROVIDED
cluster_type:
name: envoy.clusters.dynamic_forward_proxy
typed_config:
"#type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: V4_ONLY
Can someone help me what changes I need to make in the envoy configuration to get this working. Is this even doable?

client disconnected before any response GCLB

We deployed our site in front GCLB.
LB -> Cloud run -> APP ENGINE API
Cloud run is hosting a react site and App Engine golang API.
After 12 hours we started to saw decline in the amount of clicks via google analytics but traffic was pretty much the same.
Our assumption is that "lost" traffic somehow, I can see in logs 2 main issue.
404 with address of old site components.
client disconnected before any response error.
I can understand the 404 error its cache request that looking for old site components.
But i don`t understand client disconnected error and if its related to our "lost" traffic.
Any suggestion how to analyze our "lost" traffic?
UPDATE:
I found some correlation to the client client disconnected error.
The requestUrl contains images resources for exemple
images/zoom.png?v1.0
Back end service name is empty backend_service_name: ""
not sure how it can be empty, I mapped all the resources and host
LOG
{
"insertId": "cs2fmdg2eo8nba",
"jsonPayload": {
"cacheId": "FRA-1209ea83",
"#type": "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry",
"statusDetails": "client_disconnected_before_any_response"
},
"httpRequest": {
"requestMethod": "GET",
"requestUrl": "https://travelpricedrops.com/images/aero.png?v1.0",
"requestSize": "78",
"userAgent": "Mozilla/5.0 (iPhone; CPU iPhone OS 14_8 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Mobile/15E148 Safari/604.1",
"remoteIp": "109.104.52.1",
"referer": "https://travelpricedrops.com/passthru?tab=front&vert=flights&origin-iata=LEJ&destination-iata=JFK&departure-time=2021-12-26T11%3A00%3A00Z&cabin-class=economy&num-adults=1&num-youth=0&rental-duration=6&dta=48&return-time=2022-01-01T11%3A00%3A00Z&f=cf&fuid=1102&b=k&buid=1043",
"cacheLookup": true,
"latency": "0.071958s"
},
"resource": {
"type": "http_load_balancer",
"labels": {
"zone": "global",
"backend_service_name": "",
"forwarding_rule_name": "tpd-int-https-ipv4",
"target_proxy_name": "int-tpd-target-proxy-2",
"url_map_name": "int-tpd",
"project_id": "tpdrops"
}
},
"timestamp": "2021-11-09T06:13:55.121455Z",
"severity": "INFO",
"logName": "projects/tpdrops/logs/requests",
"trace": "projects/tpdrops/traces/13821ba38ae9e3191381f3f64b0a7b1a",
"receiveTimestamp": "2021-11-09T06:13:55.343086132Z",
"spanId": "a5ae86336a24bc32"
}
Config
**gcloud compute forwarding-rules describe tpd-int-https-ipv4**
IPAddress: 34.149.93.11
IPProtocol: TCP
creationTimestamp: '2021-08-30T11:49:06.047-07:00'
description: ''
fingerprint: CIAg3TcEb9Y=
id: '1815919129513727693'
kind: compute#forwardingRule
labelFingerprint: 42WmSpB8rSM=
loadBalancingScheme: EXTERNAL
name: tpd-int-https-ipv4
networkTier: PREMIUM
portRange: 443-443
selfLink: https://www.googleapis.com/compute/v1/projects/tpdrops/global/forwardingRules/tpd-int-https-ipv4
target: https://www.googleapis.com/compute/v1/projects/tpdrops/global/targetHttpsProxies/int-tpd-target-proxy-2
**gcloud compute backend-services describe tpd-prod-back**
affinityCookieTtlSec: 0
backends:
- balancingMode: UTILIZATION
capacityScaler: 0.0
group: https://www.googleapis.com/compute/v1/projects/tpdrops/regions/us-central1/networkEndpointGroups/tpd-front
cdnPolicy:
cacheKeyPolicy:
includeHost: true
includeProtocol: true
includeQueryString: true
cacheMode: CACHE_ALL_STATIC
clientTtl: 3600
defaultTtl: 3600
maxTtl: 86400
negativeCaching: false
requestCoalescing: true
serveWhileStale: 86400
signedUrlCacheMaxAgeSec: '0'
connectionDraining:
drainingTimeoutSec: 0
creationTimestamp: '2021-10-25T04:09:29.908-07:00'
description: ''
enableCDN: true
fingerprint: 5FNZk6GXJTw=
iap:
enabled: false
id: '6357784085114072710'
kind: compute#backendService
loadBalancingScheme: EXTERNAL
logConfig:
enable: true
sampleRate: 1.0
name: tpd-prod-back
port: 80
portName: http
protocol: HTTP
selfLink: https://www.googleapis.com/compute/v1/projects/tpdrops/global/backendServices/tpd-prod-back
sessionAffinity: NONE
timeoutSec: 30
**gcloud compute url-maps describe int-tpd**
creationTimestamp: '2021-08-29T06:08:35.918-07:00'
defaultService: https://www.googleapis.com/compute/v1/projects/tpdrops/global/backendServices/tpd-prod-back
fingerprint: trtG9xBMlvE=
hostRules:
- hosts:
- acpt.travelpricedrops.com
pathMatcher: path-matcher-2
- hosts:
- int.travelpricedrops.com
pathMatcher: path-matcher-1
- hosts:
- api.acpt.travelpricedrops.com
pathMatcher: path-matcher-3
- hosts:
- api.int.travelpricedrops.com
pathMatcher: path-matcher-4
- hosts:
- api.travelpricedrops.com
pathMatcher: path-matcher-5
- hosts:
- travelpricedrops.com
pathMatcher: path-matcher-6
id: '6018005644614187068'
kind: compute#urlMap
name: int-tpd
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/tpdrops/global/backendServices/tpd-acpt-back
name: path-matcher-2
- defaultService: https://www.googleapis.com/compute/v1/projects/tpdrops/global/backendServices/tpd-int-http
name: path-matcher-1
- defaultService: https://www.googleapis.com/compute/v1/projects/tpdrops/global/backendServices/tpd-api-acpt
name: path-matcher-3
- defaultService: https://www.googleapis.com/compute/v1/projects/tpdrops/global/backendServices/tpd-api-int
name: path-matcher-4
- defaultService: https://www.googleapis.com/compute/v1/projects/tpdrops/global/backendServices/tpd-api
name: path-matcher-5
- defaultService: https://www.googleapis.com/compute/v1/projects/tpdrops/global/backendServices/tpd-prod-back
name: path-matcher-6
selfLink: https://www.googleapis.com/compute/v1/projects/tpdrops/global/urlMaps/int-tpd
**gcloud compute target-http-proxies describe int-tpd-target-proxy-2**
ERROR: (gcloud.compute.target-http-proxies.describe) Could not fetch resource:
- The resource 'projects/tpdrops/global/targetHttpProxies/int-tpd-target-proxy-2' was not found
Your load balancer's configuration looks ok; you have a https-ssl-secured frontend on port 443 pointing to a http backend on port 80 which means that SSL is resolved at the load balancer and sent in plain http to your backend.
Error you're getting means (as per documentation) that the client disconnected before load balancer could reply:
client_disconnected_before_any_response - The connection to the client was broken before the load balancer sent any response.
Now to answer your questions.
Since the images are served directly by your app (I didn't see any host-path rules saying otherwise) make sure that application can serve images in time. Set your application response timeout to 10 seconds or more and this should solve the issue. Have a look at this discussion which may be quite usefull for you.
1.1 - there's also a configurable request timeout for Cloud Run services - you can check it by running gcloud run services describe SERVICE_NAME
The backend_service_name: "" string you mentioned may be empty - nothing to worry about - this is an expected behavior.
Additionally have a look at the Backend service timeout Timeouts and retries in external load balancing which may also put some light onto your case.
Lastly - have a look at How to debug failed requests with client_disconnected_before_any_response.

Nginx Ingress rules not find a match

I'm struggling on an Ingress configuration in yaml because the pattern matching seems not to work.
I would like the frontend-lb ClusterIP Service for the frontend deployment to respond to any of these:
https://example.com
https://example.com/home
https://example.com/login
... any other without /api/
And the backend-lb ClusterIP Service for the backend deployment to respond to any of these:
https://example.com/api/...
The yaml for the ingress rules is the following:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- secretName: example-com-tls
hosts:
- example.com
rules:
- host: example.com
http:
paths:
- path: /api
backend:
serviceName: gateway-lb
servicePort: 80
- path: /
backend:
serviceName: frontend-lb
servicePort: 80
The result is that every backend url is recognized as a frontend url and gives back 404 Not Found.
I've tried many other regexp and also I've tried to exclude /api in the frontend path (!?api).* but with no success.
UPDATE:
What I see with the logs it's like in the backend the url path gets blanked because if I call:
https://example.com/api/javalin-api-gateway/login
I get this error:
Not found. Request is below context-path (context-path:
'/javalin-api-gateway')
While when I call the frontend with a specific url path:
https://example.com/home
The /home controller is effectively called (it doesn't get blanked).
If I call the backend service directly (if the service is a LoadBalancer) with the same url:
http://192.168.64.17:31186/javalin-api-gateway/login
I get the right response, signal that the backend part is working properly.
How is possible that only the backend service doesn't receive the complete path?

SSL issues with jinja HTTPS load balancer - Google Cloud

I have been able to deploy this load balancer as a HTTP load balancer, and am now trying to deploy this as a HTTPS load balancer. I am attempting this with .JUNJA and .YAML on Google Cloud.
I keep getting the error: Invalid value for field 'resource.sslCertificates[0]': 'www.example.com'. The\\ URL is malformed.
resources:
- name: web-lb-hc #Create a health check for the backend
type: compute.v1.httpsHealthCheck
properties:
port: 443
requestPath: /
- name: web-url-map #Required to map url to backend
type: compute.v1.urlMap
properties:
defaultService: $(ref.backend.selfLink)
- name: backend
type: compute.v1.backendService #Deployment of backend for VM's
properties:
port: 443
portName: https
protocol: HTTPS #Defeined HTTP port for communication with backends
backends:
- name: backend
balancingMode: UTILIZATION
capacityScaler: 1.0
group: $(ref.web-ins-group.selfLink)
maxUtilization: 0.8
connectionDraining:
drainingTimeoutSec: 300
healthChecks:
- $(ref.web-lb-hc.selfLink)
- name: web-http-proxy
type: compute.v1.targetHttpsProxy
properties:
urlMap: $(ref.web-url-map.selfLink)
sslCertificates: ["www.example.com"]
- name: web-ipaddress
type: compute.v1.globalAddress
- name: web-http-forwardingrule #Creation of forwarding rule
type: compute.v1.globalForwardingRule
properties:
target: $(ref.web-http-proxy.selfLink)
IPAddress: $(ref.web-ipaddress.address)
IPProtocol: TCP #Chosen protocol
portRange: 443-443
Any help with this would be much appriciated!
sslCertificates is an array of urls.
To determine the url for an SSL certificate follow these steps.
Get a list of SSL certificates:
gcloud compute ssl-certificates list
Get the details for the SSL certificate:
gcloud compute ssl-certificates describe NAME
In the returned data near the bottom look for selfLink. This is the value you want to use. This value will look like this:
https://www.googleapis.com/compute/v1/projects/development/global/sslCertificates/production-lb

express gateway jwt issuing not working

I'm kinda of new in EG. I have followed the documentation about issuing a jwt token instead of opaque but still receiving an opaque access token. Not sure what I'm missing to change.
This is my system.config file
db:
redis:
host: localhost
port: 6379
namespace: EG
crypto:
cipherKey: sensitiveKey
algorithm: aes256
saltRounds: 10
session:
secret: keyboard cat
resave: false
saveUninitialized: false
accessTokens:
timeToExpiry: 7200000
tokenType: 'jwt'
issuer: 'express-gateway'
audience: 'something'
subject: 'test'
secretOrPrivateKey: 'ssssst'
refreshTokens:
timeToExpiry: 7200000
authorizationCodes:
timeToExpiry: 300000
Already added a user and app with their own credentials (oauth2, basic-auth, jwt) without changing anything on the models.
this is my gateway.config file
http:
port: 8080
admin:
port: 9876
hostname: localhost
apiEndpoints:
api:
host: localhost
paths: '/api/*'
serviceEndpoints:
httpbin:
url: 'https://httpbin.org'
policies:
- jwt
- oauth2
- proxy
- rate-limit
pipelines:
default:
apiEndpoints:
- api
policies:
- oauth2:
action:
jwt:
issuer: express-gateway
audience: something
subject: test
secretOrPublicKey: ssssst
checkCredentialExistence: false
- proxy:
- action:
serviceEndpoint: httpbin
changeOrigin: true
the request for login is like this and secret is the keySecret generated with jwt credential of the app.
http://localhost:8080/oauth2/authorize?response_type=token&client_id=ae921ba9-7b4b-4c53-aaba-354bd6398e52&redirect_uri=http://localhost:3002/explorer&client_secret=0qOpBZkwO2ayQ8dO18yRuh
and the token response looks like this.
14070f7c4ffc49efb1fc1709cc4a7267|90c6efd2cf8342859756d8e3705417a3
Thanks for anyone who can help me.
The configuration looks correct. I've also personally tried the configuration you provided and I got back a JWT
Maybe you can try to set up a test project on Glitch.me so it's going to be easier to replicate locally.