I have an App Runner service running a NodeJS/Express server which is a REST API. App Runner has given me the following end point -
https://example_server_random_id.awsapprunner.com
The frontend is developed using React and deployed in S3 as a static website. I have created a CloudFront distribution for serving the static content from S3 and have a default behavior (*). CloudFront has given me the following URL -
https://cloud_front_ranom_id.cloudfront.net
So I have updated the backend code to support CORS and added the CloudFront URL as allowed origin. From the React app, I am using the App Runner URL as my base URL for all the API calls. This setup works perfectly.
Now, I wanted to get rid of CORS related setup by using the CloudFront distribution as a reverser proxy to my App Runner service. To do that, I have done following -
I have added a custom origin that points to my App Runner service endpoint - https://example_server_random_id.awsapprunner.com
I have created a new behavior (/api*) that has precedence 0, so it should be handling all the requests that have "/api" in its path instead of the default behavior (*). This behavior uses the custom origin that I have created in the previous step.
I have allowed all the HTTP request (GET, POST, OPTION, PUT etc.) for this new behavior.
For the new behavior, I have used "CachingDisabled" as the Cache Policy and "All Viewer" as the Origin Request Policy.
I have replaced the base URL and use /api as the new base URL instead of the App Runner endpoint.
I did not remove the CORS related settings from the backend code (access-control-allow-origin header value is the CloudFront endpoint and access-control-allow-credential header value is true).
This setup is not working as expected. I can still access the static contents but for all the requests to the backend (with /api base URL) give me the following errors -
Cannot POST /api/users/sign-in Status Code: 404
Cannot GET /api/users Status Code: 404
etc.
Can you please let me know if there is any way to debug this and what could be the problem in the setup?
I am writing down how I fixed this in case someone faced the same issue.
The following article helped me to pinpoint the issue - https://advancedweb.hu/how-to-debug-cloudfront-origin-requests/
Basically, the problem was the cache policy I was using. I used the following webhook tester to find out the header values, query parameter and cookies and modified the cache policy to use the legacy cache settings (Appropriate header, cookies and query parameters)-
https://webhook.site/
Related
I have a Lambda function (a packaged next.js app) which I'm trying to access via CloudFront. The web app works unless I try to hit the homepage.
When I hit /search or /video/{videoId} the page loads just fine.
When I try to hit the homepage, I get the following error page:
502 ERROR
The request could not be satisfied.
The Lambda function returned invalid JSON: The JSON output is not parsable. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
Generated by cloudfront (CloudFront)
Request ID: {id}
Why would just the homepage be invalid JSON? Where can I see this JSON to determine what is wrong? I created a mock Cloudfront request test in the Lambda function and it just returns successfully.
The problem was due to the 1 MB size limit of CloudFront Lambda#Edge responses. I didn't realize that Next.js's serverside rendering was creating a large <script id="__NEXT_DATA__"> tag on my homepage with all the fetched info from my API duplicated multiple times over. This resulted in my app's homepage being >2 MB.
I refactored my app to only send one network request, and made sure that data is only put into the __NEXT_DATA__ tag once. The app now works.
I have the current configuration in AWS:
S3 Bucket set to static website for my frontend
ELB which has an EC2 instance for my backend
Cloudfront distribution with two origins, two behaviors, and two custom error pages
Origins:
S3 Bucket (static website)
dev-api.mydomain.com (ALIAS record mentioned below that points to ELB)
Behaviors:
"/api/*" routes to dev-api.mydomain.com
"*" routes to S3
Error pages (this part is the most important to my question):
403 returns 200 response with index.html (which comes from S3)
404 returns 200 response with index.html (which comes from S3)
Route53:
registered domain
dev.mydomain.com points to cloudfront distribution
dev-api.mydomain.com points to ELB
How do I distinguish between the 403 and 404 errors that come back from my S3 and backend server?
Note: I am using react router to handle all routing in the static website so every request to S3 would return a 404 from S3 unless the user specifically asked for the /index.html resource.
Scenario 1: When a user goes to dev.mydomain.com/logi (mispelled login), I would expect that to hit my S3 bucket and return with a 404 which would then fetch the index.html file in the bucket. Cloudfront would return the index.html file with a 200 OK Status and my application would handle the invalid route by displaying a 404 page.
Scenario 2: When a user goes to dev.mydomain.com/login, it would hit my S3 bucket, return with a 404 and return the index.html file. Once I land on the login page, for this examples sake, I fire off a network request to dev.mydomain.com/api/non_existant_route. I would expect that to hit my API server and return with a 404 which my frontend would then handle by displaying an error message about the API request failing.
However what is happening in Scenario 2 is that a 404 is returned from my backend server to Cloudfront, which then returns the index.html file (from S3) as the response with a 200 status code.
So, my question is - Is it possible to configure Cloudfront to return different error pages based on the origin that is sending back the error code? If not, how can I accomplish returning the correct response for the backend server?
Would it require me to re-architect my AWS solution?
Another way is to:
turn on Static Web Hosting on S3
replace the S3 bucket name in CloudFront origin with the S3 Static Website URL.
remove Error Pages from CloudFront.
The error pages will be handled by the respective origins instead of CloudFront.
I was able to solve the 403 and 404 issues by removing Custom Error responses from cloudfront.
I created a Lambda#Edge function on the default behavior (*) which routes to my S3 bucket. On Origin Request, the function modifies requests to pages like /login to instead return /index.html from the bucket.
In my case, I also have two origins and both are one page app (OPA) so both contains index.js
React App (Responsible for my main application) hosted through S3 static website. (CSR build)
Webflow App (Static website for product info and blog, etc..)
Our react app has many different behaviours based on routes (application need), so I configured Webflow as default behaviour.
First, I added custom error handling for 404 with redirect page path to /index.js and HTTP Status 200, as many app routes are not directly present in S3 Website and result into 404 (This is suggested by many tech blog and works very well in case of single origin)
But as soon as I added Webflow website origin as my Default(*) behaviour, whenever 404 occurs in React app it will get redirected to default behaviours /index.js (Our static product info website)
Finally, I removed all custom error handling and it will start working as expected. All the React routes are redirecting to React behaviours and not matching behaviour routes are going to Default(*) Webflow website.
I try to block the possibility to send a post request to my app via postman or insomnia. I want to limit the source of a request to one domain www.sample.com. I did add this domain to the ALLOWED_HOSTS and CORS_ORIGIN_WHITELIST, and nothing more and still I can send a request and save the data in my django app.
How can I limit the origin of a request to the one domain and block all others or return "Unauth msg"?
Stack: DJANGO REST Framework
I am able to add a check:
if not request.headers.__contains__("Origin") or request.headers["Origin"] not in HOSTS:
But I can fill the free field with key Origin and domain as a value in the postman app
You cannot prevent a HTTP client from setting a header. If your API is available on Internet it is possible to send you any header. CORS (Cross Origin Resource Sharing) rules are only applied in Web browsers.
Following this article, I'm trying to serve my website's static content from multiple regions.
The lambda function in that article is trying to modify the property of an object within this path:
event.Records[0].cf.request.origin.s3
This is in the case that my lambda function is not receiving such a property. Instead, I'm getting:
event.Records[0].cf.request.origin.custom
Apparently, this means that I'm receiving a CustomOriginConfig while the article is expecting an S3OriginConfig. I'm not sure what these two mean but the UI depicted in the article for the "Edit Origin" page is totally different from mine.
The article shows this:
And I've got this:
Can someone please help me find why I'm receiving a CustomOriginConfig instead of an S3OriginConfig?
CloudFront only considers the origin to be an S3 Origin if the Origin Domain Name is the REST endpoint for the bucket -- e.g. ${bucketname}.s3.amazonaws.com. This is the configuration that supports authentication of requests on the back-side of CloudFront using an Origin Access Identity.
If you are using S3's web site hosting features (index and error documents, and/or redirects) then you use the web site hosting endpoint for the bucket, e.g. ${bucketname}.s3-website.${region}.amazonaws.com. CloudFront actually treats this configuration as a Custom Origin, the same as if you are using any (non-S3) web service as the origin server. Origin Access Identity and S3 website endpoints are not compatible with each other.
The console options change depending on whether the console sees that you're creating an S3 or Custom Origin (based on the hostname).
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/DownloadDistS3AndCustomOrigins.html
I am using AWS codestar to deploy by react application using serverless nodejs template. This is the url that is given by codestar after successfully completion of all the stages https://xxxxx.execute-api.us-east-1.amazonaws.com/Prod . This url displayed all the components in my app correctly. In navbar of my app i have items like this a ,b,c. where clicking on each one of them will redirect to a new component.(i.e.https://xxxxx.execute-api.us-east-1.amazonaws.com/a,https://xxxxx.execute-api.us-east-1.amazonaws.com/b etc. But when i refresh the page which is having a url like this https://xxxxx.execute-api.us-east-1.amazonaws.com/b i am getting a error like {"message":"Forbidden"} and in my console it is showing like this favicon.ico:1 GET https://xxxx.execute-api.us-east-1.amazonaws.com/favicon.ico 403
It seems the chrome is fetching the favicon based on the https link, which fails because there is no such favicon at the location. I tried to remove favicon.ico link in index.html but even then the chrome is using the same url to fetch the favicon which eventually fails. I followed max number of suggestions in SO to acheive this but no luck. Is there any way to say api-gateway to exclude these favicon get requests and display my app rather than showing message forbidden.
And i am pretty sure that i had enabled logs for both the agi-gateway and lambda where i didnt find any forbidden errors(i.e.403) which is weird because i can see those 403 errors in my console.
Thanks
Any help is highly appreciated.
The https://xxxxx.execute-api.us-east-1.amazonaws.com/Prod url provided by API Gateway is the base url for your site, so those paths would have to be /Prod/a instead of /a.
One way to get around that is to register your own domain and connect it to API Gateway via a custom domain. That would allow you to have https://example.com as your base url, and your paths could stay /a, /b, etc.