"Fetch as Google" doesn't load React routes on S3 with CloudFront - amazon-web-services

I have a react-router site on Amazon S3, with 404 redirects set to hit index.html, where they find the code they need and the site does what it should.
In Google Webmaster Tools, "Fetch as Google" for one of the routes gave error status: Not Found.
I set up Cloudfront for that origin with a Custom Error Response (404 to 200, with Response Page Path /index.html).
But "Fetch as Google" for one of the routes still shows error status: Not Found.
In Chrome or with curl, when I load http://[bucketname].s3-website-us-west-2.amazonaws.com/list/listname/username I see a 404 but the page then loads as expected.
But when I load http://[randomchars].cloudfront.net/list/listname/username I see a 200 with no re-direct and the page loads as expected.
I've submitted a sitemap with the relevant URLs I'd like indexed and asked Google to crawl the site. It hasn't been crawled yet, but my best hope right now are a couple of suggestions I've seen that "Fetch as Google" won't follow redirects, while Googlebot will when it actually crawls the site. I hope this is the case, but if I've missed any other obvious thing, I'd love to know. Thanks in advance!

I found setting cloudfront with the response page path as "/" worked for me!
I was stuck with the exact same problem.

Related

link preview not working (shows 301) on shortened bitly links on Facebook

When posting a link on Facebook, after setting the open graph meta tags (e.g. og:title, og:image) it successfully shows a link preview as I intend it to. However after shortening the link with bit.ly, when I post it on Facebook the link preview becomes "301 moved permanently" with no image. I get the same with tinyurl, are there any specific tags I should be adding here? I have tried refreshing with the sharing debugger, re crawling, trying iterations of the url with http and https and it's all the same result with the short url
Answering my own question here for all those in the future who may also have this problem,
I had to actually go into my site settings and turn https redirecting off, and enable both http and https

AWS Amplify Redirects Gatsby 404 to Root URL

I have deployed a multi-page gatsby site both to AWS Amplify and Netlify.
When I go to a page that doesn't exist on my site that was deployed with Netlify, I get the 404 page I have created. But when I go to the same non-existent route on the site with Amplify, it redirects me to the root path.
I have checked the network tab, and I get a 404 status code without any HTTP redirects. So I assume the redirect happens in the JavaScript code?
Does this happen to anyone else?
Here is the default Rewrites and Redirects for my site on Amplify:
The solution was to change the target address from /index.html to /404.html like this:
I exprienced this same issue, but with all pages reloading to the homepage if they weren't ended with a '/'. Removing all the rewrite rules seemed to fix this for me. I wasn't using any of them and they were set as defaults. Such as 'www' rerouting to '/'.

Invoking a Lambda through API-Gateway giving 403 response?

I am using AWS codestar to deploy by react application using serverless nodejs template. This is the url that is given by codestar after successfully completion of all the stages https://xxxxx.execute-api.us-east-1.amazonaws.com/Prod . This url displayed all the components in my app correctly. In navbar of my app i have items like this a ,b,c. where clicking on each one of them will redirect to a new component.(i.e.https://xxxxx.execute-api.us-east-1.amazonaws.com/a,https://xxxxx.execute-api.us-east-1.amazonaws.com/b etc. But when i refresh the page which is having a url like this https://xxxxx.execute-api.us-east-1.amazonaws.com/b i am getting a error like {"message":"Forbidden"} and in my console it is showing like this favicon.ico:1 GET https://xxxx.execute-api.us-east-1.amazonaws.com/favicon.ico 403
It seems the chrome is fetching the favicon based on the https link, which fails because there is no such favicon at the location. I tried to remove favicon.ico link in index.html but even then the chrome is using the same url to fetch the favicon which eventually fails. I followed max number of suggestions in SO to acheive this but no luck. Is there any way to say api-gateway to exclude these favicon get requests and display my app rather than showing message forbidden.
And i am pretty sure that i had enabled logs for both the agi-gateway and lambda where i didnt find any forbidden errors(i.e.403) which is weird because i can see those 403 errors in my console.
Thanks
Any help is highly appreciated.
The https://xxxxx.execute-api.us-east-1.amazonaws.com/Prod url provided by API Gateway is the base url for your site, so those paths would have to be /Prod/a instead of /a.
One way to get around that is to register your own domain and connect it to API Gateway via a custom domain. That would allow you to have https://example.com as your base url, and your paths could stay /a, /b, etc.

Google couldn't crawl your site because we were unable to access your site's robots.txt file

I verified my site using google webmaster. I have made my website in django and I also added robots.txt.
Now google is showing green tick mark I think its good on DNS and Server Connectivity but and red warning mark on robots.txt fetch..
My robots.txt looks like
User-agent: *
Disallow:
Is that google takes time to crawl site ? or I have errors in my robots.txt or its settings.
When I open robots.txt from my site like mysite.com/robots.txt I can see the robots.txt file..
Also when I run robots.txt test in webmaster it gives allowed result..
My site is not even being searched in google..
But why its not crawling my site ?
Google generally caches robots.txt. It tries to recrawls mostly in 24 hrs. If you are sure there is nothing with your robots.txt, you probably have to wait.

Magento - Internal server error

I have a Magento site which has multistore setup.
The problem is when we browse the frontend after a few page visits suddenly the site stops loading. When I look into the server access log, it shows response code 500 for the URL which was accessed last.
I am unable to reproduce the issue because the errorneous URL is not consistent.
If I would delete the cookie "frontend" set by Magento, the site works seamlessly again.
I don't know how I can narrow down the debugging considering the effect of "frontend" cookie. Can anyone please advise.
Thanks in advance.