My AWS web servers are not showing a request for example '42.26.32.120'
Is there a way to see if our ALB has received a request from that?
We are unable to identify the cause of the issue, as the IPs is not in any of the access logs
Therefore, I am trying to find out where the request was lost.
I found some output against there queries
SELECT * FROM alb_logs
WHERE client_ip= '42.26.32.120'
but from the results This seems to indicate that after the new app launch ip haven't been able to connect ?
would that be behavior if clients had written our ALB location statically ?
and If the request just hung in ALB would it log
e.g. if it couldnt find target machine, or if ALB was no longer around
would we have logs in those cases ?
Any Short of help would be apricated.
Related
I'm trying to create a AWS Client VPN endpoint. I followed this AWS tutorial and I always get a timeout error like this:
DNS resolution error: 30 times.
I'm not sure what to do, I saw some videos on this topic and it seems I did everything correctly, does anyone know how to debug this? (or what could be the cause)?
This is really stupid. I tried to check IPs for my endpoint
host *.cvpn-endpoint-XXXX.prod.clientvpn.[region].amazonaws.com
and
host cvpn-endpoint-02aa72c3aa8d442d6.prod.clientvpn.eu-west-1.amazonaws.com
and both failed. As described in this response, you need to add a random subdomain. By adding this on the .ovpn file (on the remote parameter), it works!
I am evaluating Dialogflow ES Trail and created an agent, with fulfillment to explore the features.
For that, I have configured the application service in the Dialogflow console in fulfillment and specified the application endpoint URL for the service that is hosted on our secure network and environment. When a specific intent matches that have the fulfillment enabled it will invoke the service that is configured, but there is a failure "Dialogflow fulfillment error: Webhook call failed. Error: DEADLINE_EXCEEDED." since this request is getting blocked on our firewall.
Please note we are not hosted on the google cloud platform and using other cloud services and also we are using a different firewall that has custom rules.
I'm seeking assistance with whitelisting the IP addresses or DNS from which Google Dialogflow fulfillment is sending the traffic since this seems to be dynamic and changing every time the requests are getting blocked on our firewall.
I went through this documentation and tried allowing the IP Address ranges specified, but the IP addresses from which Google is sending the traffic are different. Also, it seems like this is more specific to Google Cloud Platform
https://cloud.google.com/vpc/docs/access-apis-external-ip#config
Also configuring the dynamic IP addresses ranges from these files goog.json and cloud.json hosted on the internet which keeps on updating daily seems to be difficult to handle in our firewall
https://cloud.google.com/vpc/docs/access-apis-external-ip#ip-addr-defaults
Can anyone please help me with How I can whitelist dialogflow.cloud.google.com traffic to our firewall since their IP Address and DNS is dynamic?
I recommend you to forgive this solution and to accept the traffic! Ok, surprising, let me explain.
If you whitelist the Dialogflow URL or IP, all the users that use Dialogflow will be authorized on your firewall. And because anyone can use Dialogflow, you will open the firewall to everybody.
Thus, don't waste time with that. "Don't trust the network" as Google say, but trust the authentication of the request. You can set, at least a static "API Key" on your webhook calls, it's much better than IP Filtering (even if not so strong, it's still better).
I recommend you to focus on this solution instead.
I´m consistently being charged for a surprisingly high amount of data transfer out (from Amazon to Internet).
I looked into the Usage Reports of the past few months and found out that the Data Transfer Out was coming out of an Application Load Balancer (ALB) between the Internet and multiple nodes of my application (internal IPs).
Also noticed that DataTransfer-Out-Bytes is very close to the DataTransfer-In-Bytes in the same load balancer, which is weird (coincidence?). I was expecting the response to each request to be way smaller than the request itself.
So, I enabled flow logs in the ALB for a few minutes and found out the following:
Requests coming from the Internet (public IPs) in to ALB = ~0.47 GB;
Requests coming from ALB to application servers in the same availability zone = ~0.47 GB - ALB simply passing requests through to application servers, as expected. So, about the same amount of traffic.
Responses from application servers back into the same ALB = ~0.04 GB – As expected, responses generate way less traffic back into ALB. Usually a 1K request gets a simple “HTTP 200 OK” response.
Responses from ALB back to the external IP addresses => ~0.43 GB – this was mind-blowing. I was expecting ~0.04GB, the same amount received from the application servers.
Unfortunately, ALB does not allow me to use packet sniffers (e.g. tcpdump) to see that is actually coming in and out. Is there anything I´m missing? Any help will be much appreciated. Thanks in advance!
Ricardo.
I believe the next step in your investigation would be to enable ALB access logs and see whether you can correlate the "sent_bytes" in the ALB access log to either your Flow log or your bill.
For information on ALB access logs see: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html
There is more than one way to analyze the ALB access logs, but I've always been happy to use Athena, please see: https://aws.amazon.com/premiumsupport/knowledge-center/athena-analyze-access-logs/
I work in the dev-ops team at my company. Recently, we shifted to aws's application load balancer and we are forwarding the request based on a cookie's value. For some reason, the rule isn't working and AWS doesn't support logs to get information on why a rule faied.
There could be 2 reasons for this, that we can think of:
Load balancer isn't able to read the cookie: We don't think this should be the issue as the applications under this load balancer are able to read and also print the cookies.
The load balancer doesn't read subsequent cookies after the first request: We have raised a concern with AWS on this and they are still to get back.
Meanwhile, can anyone point to any possible issues which we might be overlooking?
I use a VPN to access services in an AWS VPC. I also use this VPN as a gateway to my local internet. The strange thing is that when I'm connected to the VPN, I can't browse amazon.com or amazon.co.uk I can get to the home page and it displays correctly, but whatever I try to do, I get an error 503 - Service Unavailable:
"We're sorry
An error occurred when we tried to process your request.
We're working on the problem and expect to resolve it shortly. Please note that if you were trying to place an order, it will not have been processed at this time. Please try again later.
We apologise for the inconvenience."
Again, this is Amazon's retail/shopping website.
It works fine with the VPN disabled.
What can I do to get this fixed?
Thanks!
It appears that amazon.com prevents access to the IP address range used by Amazon EC2 instances. This is possibly done to prevent scraping of information.
I accessed a page via an EC2 instance and noticed this message as a comment in the beginning of the HTML page:
To discuss automated access to Amazon data please contact api-services-support#amazon.com.
For information about migrating to our APIs refer to our Marketplace APIs at https://developer.amazonservices.com/ref=rm_5_sv, or our Product Advertising API at https://affiliate-program.amazon.com/gp/advertising/api/detail/main.html/ref=rm_5_ac for advertising use cases.
In fact, I have seen this behaviour on many websites.
While this does not assist with your use-case of sending traffic via your VPN connection to the Internet, at least it explains why it is occurring.