GCP CDN - server requests return a 404 - google-cloud-platform

I followed the guide to set up an external global, app engine based, load balancer. I linked it to Google's CDN by ticking the little box in the LB configuration settings.
Now, when I load my domain name, it says CANNOT GET /. The request returns a 404, along with some C S P: The page’s settings blocked the loading of a resource at inline (“default-src”). error messages.
It was working well before adding the CDN. So, I'm assuming my app server configuration is fine.
In the Load Balancer details, there is a little chart under the Monitoring section, with how traffic flows.
It shows traffic coming from the 3 global regions, going to the frontend of the LB, then to / (unknown) and / (unmatched) as URL Rule, then to the backend service I defined, and finally to a backend instance labelled NO_BACKEND_SELECTED.
I'm guessing the issue comes either from the URL Rule or Backend Instance, but there is little in the doc. to troubleshoot.
I followed the doc. to setup the LB. Settings are pretty simple using App Engine, so there is little room for wrong doing. But I may have missed something still.
In the 'create serverless NEG', I did select App Engine, and default as the service name (although i'm not sure what default actually means).
Any idea what's missing ?
EDIT :
So, in the load balancing menu, I go to the 'Backends' section at the top, and select my backend. Here I have the list of 'General properties' of my backend. Except, under 'Backends', it says the following : Backends contain instance groups of VMs or network endpoint groups. This backend service has no backends yet edit
From there, I can click the edit link, which redirects me to the 'Backend service edit' menu. I DO have a backend selected in there. I did create a serverless NEG using App Engine.
So, what's missing ? Is there anything wrong with Google's serverless backend ?

I wanna help you with the issue that you are facing.
If the responses from your external backend are not cached by Cloud CDN
Ensure that:
-You have enabled Cloud CDN on the backend service containing the NEG that points to your external backend by setting enableCDN to true. (DONE as per your description).
-Responses served by your external backend meet Cloud CDN caching requirements. For example, you are sending Cache-Control: public, max-age=3600 response headers from the origin.
The current implementation of Cloud CDN stores responses in cache if all of the following are true.
Attribute:
Served by
Requirement:
Backend service, backend bucket, or an external backend with Cloud CDN enabled
Attribute:
In response to
Requirement:
GET request
Attribute:
Status code
Requirement:
200, 203, 204, 206, 300, 301, 302, 307, 308, 404, 405, 410, 421, 451, or 501.
Attribute:
Freshness
Requirement:
The response has a Cache-Control header with a max-age or s-maxage directive, or an Expires header with a timestamp in the future.
For cacheable responses without an age (for example, with no-cache), the public directive must be explicitly provided.
With the CACHE_ALL_STATIC cache mode, if no freshness directives are present, a successful response with static content type is still eligible for caching.
With the FORCE_CACHE_ALL cache mode, any successful response is eligible for caching.
If negative caching is enabled and the status code matches one for which negative caching specifies a TTL, the response is eligible for caching, even without explicit freshness directives.
Attribute:
Content
Requirement:
Contains a valid Content-Length, Content-Range, or Transfer-Encoding: chunked header.
For example, a Content-Length header that correctly matches the size of the response.
Attribute:
Size
Requirement:
Less than or equal to the maximum size.
For responses with sizes between 10 MB and 5 TB, see the additional cacheability constraints described in byte range requests.
Please validate the URL Mapping too:
This is an example as reference adapt this according your project.
Create a YAML file /tmp/http-lb.yaml, making sure to substitute PROJECT_ID with your project ID.
When a user requests path /*, the path gets rewritten in the backend to the actual location of the content, which is /love-to-fetch/*.
defaultService: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendBuckets/cats
hostRules:
- hosts:
- '*'
pathMatcher: path-matcher-1
name: http-lb
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendBuckets/cats
name: path-matcher-1
pathRules:
- paths:
- /*
routeAction:
urlRewrite:
pathPrefixRewrite: /love-to-fetch/
service: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendBuckets/dogs
tests:
- description: Test routing to backend bucket, dogs
host: example.com
path: /love-to-fetch/test
service: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendBuckets/dogs
Validate the URL map.
gcloud compute url-maps validate --source /tmp/http-lb.yaml
If the tests pass and the command outputs a success message, save the changes to the URL map.
Update the URL map.
gcloud compute url-maps import http-lb \
--source /tmp/http-lb.yaml \
--global
Using URL maps

Related

Access-Control-Allow-Origin wildcard subdomains in Google Cloud Load Balancing service

I'm using Google Cloud Load Balancing service, and want to enable CORS for all subdomains.
For example, I want to be able to run an XHR request from
https://sub.mywebsite.example to https://www.mywebsite.example
Typically, I will do the below, but it does not work:
As mentioned by # derpirscher you must either specify * as Allow-Origin header or the exact protocol://host:port.
In your use case the response to the CORS request is missing the required Access-Control-Allow-Origin header, which is used to determine whether or not the resource can be accessed by content operating within the current origin.
You can also configure a site to allow any site to access it by using the * wildcard. You should only use this for public APIs. Private APIs should never use *, and should instead have a specific domain or domains set. In addition, the wildcard only works for requests made with the crossorigin attribute set to anonymous, and it prevents sending credentials like cookies in requests.
Access-Control-Allow-Origin: *
Ensure that the request has an Origin header and that the header value matches at least one of the Origins values in the CORS configuration. Note that the scheme, host, and port of the values must match exactly. Some examples of acceptable matches are as follows:
http://origin.example.com matches http://origin.example.com:80 (because 80 is the default HTTP port), but does not match https://origin.example.com, http://origin.example.com:8080, http://origin.example.com:5151, or http://sub.origin.example.com.
https://example.com:443 matches https://example.com but not http://example.com or http://example.com:443.
http://localhost:8080 only matches exactly http://localhost:8080 , not http://localhost:5555 or http://localhost.example.com:8080 .

How to use CloudFront as reverse proxy to my App Runner service?

I have an App Runner service running a NodeJS/Express server which is a REST API. App Runner has given me the following end point -
https://example_server_random_id.awsapprunner.com
The frontend is developed using React and deployed in S3 as a static website. I have created a CloudFront distribution for serving the static content from S3 and have a default behavior (*). CloudFront has given me the following URL -
https://cloud_front_ranom_id.cloudfront.net
So I have updated the backend code to support CORS and added the CloudFront URL as allowed origin. From the React app, I am using the App Runner URL as my base URL for all the API calls. This setup works perfectly.
Now, I wanted to get rid of CORS related setup by using the CloudFront distribution as a reverser proxy to my App Runner service. To do that, I have done following -
I have added a custom origin that points to my App Runner service endpoint - https://example_server_random_id.awsapprunner.com
I have created a new behavior (/api*) that has precedence 0, so it should be handling all the requests that have "/api" in its path instead of the default behavior (*). This behavior uses the custom origin that I have created in the previous step.
I have allowed all the HTTP request (GET, POST, OPTION, PUT etc.) for this new behavior.
For the new behavior, I have used "CachingDisabled" as the Cache Policy and "All Viewer" as the Origin Request Policy.
I have replaced the base URL and use /api as the new base URL instead of the App Runner endpoint.
I did not remove the CORS related settings from the backend code (access-control-allow-origin header value is the CloudFront endpoint and access-control-allow-credential header value is true).
This setup is not working as expected. I can still access the static contents but for all the requests to the backend (with /api base URL) give me the following errors -
Cannot POST /api/users/sign-in Status Code: 404
Cannot GET /api/users Status Code: 404
etc.
Can you please let me know if there is any way to debug this and what could be the problem in the setup?
I am writing down how I fixed this in case someone faced the same issue.
The following article helped me to pinpoint the issue - https://advancedweb.hu/how-to-debug-cloudfront-origin-requests/
Basically, the problem was the cache policy I was using. I used the following webhook tester to find out the header values, query parameter and cookies and modified the cache policy to use the legacy cache settings (Appropriate header, cookies and query parameters)-
https://webhook.site/

GCP Load Balancer redirect while keeping query parameters

In my GCP project am a doing a Host and path rules redirect on one of my load balancers. I am trying to get the redirect to pass the query parameters, but it is not.
Essentially trying to do:
subdomain.domain.com?foo=bar -> www.domain.com/path?foo=bar.
The redirect works to the path, but it does not keep the query params. From what I can see here, it should work.
https://cloud.google.com/load-balancing/docs/https/traffic-management#redirects
Redirect to a different URL formed by modifying the host, path, or both the host and path portion of the URL, and either stripping out or retaining any query parameters.
In Google Cloud, the stripQuery field can also be used with a Load Balancer, specifically the GCP HTTP(S) Load Balancer. This feature allows you to configure whether the query string present in the incoming request's URL will be included or removed when the request is forwarded to a backend service.
When stripQuery is set to false, the query string will be included in
the path when forwarding traffic to the backend service.
i.e: if a request is made to "example.com/path?key=value", and stripQuery is set to false, the request will be forwarded to the backend service as if it were made to "example.com/path?key=value".
By disabling the stripQuery feature in Google Cloud Load balancer we can forward the query params to the new host we have redirected to.
Here is a configuration YAML we can use to redirect the site to another site preserving the query parameters.
defaultService: projects/example/global/backendServices/example-test
name: example-matcher1
routeRules:
- urlRedirect:
stripQuery: false
hostRedirect: ie.example.com
redirectResponseCode: MOVED_PERMANENTLY_DEFAULT
If you are using DefaultRedirect in Advanced host and path rule:
You should uncheck the Strip query in Google Cloud Load Balancer Edit Page for Advanced Route Rules Routing Rules .
Reference:
enter image description here

why does api gateway with http api using api mapping fail with 404?

I have a custom domain name in AWS API gateway. I am using the same domain for 2 separate API stages. One API stage is REST API, and the other HTTP API.
When I test out my setup, everything works for the REST API. However, the mapping path for HTTP API is not working and I get status-code = 404 Not Found, with 0kb body.
references used:
https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-mappings.html
https://docs.aws.amazon.com/apigateway/latest/developerguide/rest-api-mappings.html
From my testing in postman, i get the following result when calling the custom domain.
1. {{api.gateway.custom.domain.url}}/foobar - works
2. {{api.gateway.custom.domain.url}}/this-no-work/foobar - 404
3. {{api.gateway.custom.domain.url}}/this-works/foobar - works
does anyone know why (2) gives 404? api-gateway REST API with mapping works while api-gateway HTTP API will return 404 with 0kb body. Is there something I am missing?
note: the (none) path mapping has been added for a sanity check, and I was able to get the expected response.
I had the same problem.
The problem is in the configuration of routers of your application.
When you configure one API mapping in AWS API Gateway, the configured path (this-no-work) is passed to your web application as a prefix of routes.
SO ... if you have one route like this:
/api/foobar
you need configure one more route with the prefix point to the same action:
/this-no-work/api/foobar
A good one is to make one global configuration to your web app.
You can note it enabling the cloudwatch logs of your API Gateway stage and looking into the cloudwatch logs the path property passed from API Gateway to the Web application.
Thanks to answer from #gean-ribeiro, I was able to figure why my HTTP API was returning 404 Not Found.
Turns out it was not an issue with HTTP API. The error 404 Not Found with 0kb body was coming from the API integration sitting behind APIG. Specifically, it was a load balancer using rules based on HTTP path pattern.
By default, any unmatched path pattern will return 404 with text/plain body. this-no-work was a new HTTP API I added, and it did not have the necessary listener rules.
Once I added a new listener rule for HTTP Path pattern is /this-no-work/*, it worked as expected..
when default endpoin is created in my case it uses this pattern
https://{api_id}.execute-api.{region}.amazonaws.com/
an answer might be: Disable the default endpoint for an HTTP API
more details here
https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-disable-default-endpoint.html

Cookies from backend service not visible in Postman using zuul routing

I have a zuul routing app deployed on the cloud. Below in my application.yml
---
spring:
profiles: default
zuul:
routes:
cloud:
path: /cardsvcs/acs/**
sensitiveHeaders:
url: https://vst0.mapi.checkFin.com/
stripPrefix: false
ribbon:
eureka:
enabled: false
Routing works perfectly fine and I also get the response headers back from the back end service. The requestContext originResponseHeaders also have all the cookies as part of setCookie header. But these cookies are not visible on the Postman response after routing.
Do we need to gets the cookies from this header in the filter and map them again?
Please try the below.
zuul.sensitiveHeaders: Authorization
The sentive-headers that are configured in zuul are not passed to downstream servers. And below three headers are defaults.
zuul.sensitiveHeaders: Cookie,Set-Cookie,Authorization
To pass cookie related headers to your downstream server, you need to redefine above propeprties.
Please refer to the section 'Cookies and Sensitive Headers' in the the linked doc : http://cloud.spring.io/spring-cloud-static/Dalston.RELEASE/