AWS S3 endpoint issue - amazon-web-services

This is a really strange one as it started throwing errors over night - it's been working fine up until yesterday - this morning it's been playing all day.
I'm using illuminate/filesystem in my project and for the endpoint I was using:
https://s3.eu-west-2.amazonaws.com
This morning we started getting errors saying:
Error executing "ListObjects" on "bucket-01.https://s3.eu-west-2.amazonaws.com"; AWS HTTP error: cURL error 1: Protocol "bucket-01.https" not supported or disabled in libcurl (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
File: .../vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php
Line: 195
Seeing that it tries to prepend bucket name before the protocol of the endpoint I've decided to remove protocol from the endpoint - making it
s3.eu-west-2.amazonaws.com
Now I'm getting error saying
Error executing "ListObjects" on "//bucket-01.s3.eu-west-2.amazonaws.com/bucket-01.s3.eu-west-2.amazonaws.com";
AWS HTTP error: Client error: GET http://bucket-01.s3.eu-west-2.amazonaws.com/bucket-01.s3.eu-west-2.amazonaws.com resulted in a 404 Not Found response
NoSuchKey The specified key does not exist.
As you can see now it appends endpoint after the initial endpoint.
Does anyone know what might have happened?

After hours of searching for the solution I've came across this issue on laravel/framework repository https://github.com/laravel/framework/issues/36694

Related

Mysterious 500 error with AWS Lambda; unable to debug

I have an API that I host using Lambda (nodejs), with API-gateway. I'm using serverless to deploy.
Generally things have been fine, but while I was working on a specific function today, I started to receive HTTP 500 errors when hitting the endpoint. However, while there were still API-Gateway access logs for the end point, there were no Cloudwatch logs for the lambda functions getting hit. I was able to verify that the Authorizer was getting hit successfully, and not returning any issue (if it was, it would have been a 401). After using CLI tools to invoke the function from the command line, the 500 error went away and I was able to successfully hit the endpoints again.
Has anyone ever ran into this before? If I'm missing a debug step, I would really like to know. It was really concerning that my API could be generating 500 errors with no paper trail to help me understand what was happening.
You can check your role and permissions ,this link could help you https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-lambda-stage-variable-500/
Also you can debug further with X-ray : https://docs.aws.amazon.com/lambda/latest/dg/services-xray.html

Media Tailor ad returning 504 error in AWS

I'm using AWS Media Tailor to test an ad inserting demo. The demo page is this one: https://github.com/aws-samples/aws-media-services-simple-vod-workflow/tree/master/12-AdMarkerInsertion.
When I place my manifest into a TheoPlayer I always get an 504 error. My manifes page is: https://ebf348c58b834d189af82777f4f742a6.mediatailor.us-west-2.amazonaws.com/v1/master/3c879a81c14534e13d0b39aac4479d6d57e7c462/MyTestCampaign/llama.m3u8.
I have also tried with: https://ebf348c58b834d189af82777f4f742a6.mediatailor.us-west-2.amazonaws.com/v1/master/3c879a81c14534e13d0b39aac4479d6d57e7c462/MyTestCampaign/llama_with_slates.m3u8.
The specific error is:
{"message":"failed to generate manifest: Unable to obtain template playlist. sessionId:[c915d529-3527-4e37-89e0-087e393e75de]"}
I have read about this error: https://docs.aws.amazon.com/mediatailor/latest/ug/playback-errors-examples.html
But don't know how to fix it.
Maybe I did something wrong or do I need a quote in AWS?
Any idea?
Thanks for the inquiry!
The following example shows the result when a timeout occurs between AWS Elemental MediaTailor and either the ad decision server (ADS) or the origin server.
An HTTP 504 error is known as a Gateway Timeout meaning that a resource was unresponsive and prevented the request from completing successfully. In this case since MediaTailor is returning an HTTP 504 this means that either the ADS or Origin failed to respond within the timeout period.
To troubleshoot this you will need to determine which dependency is failing to respond to MediaTailor and correct it. Typically the issue is the ADS failing to respond to a VAST request performed by MediaTailor which you can confirm by reviewing your CloudWatch logs.
https://docs.aws.amazon.com/mediatailor/latest/ug/monitor-cloudwatch-ads-logs.html
Make sure that your ADS follows the guidelines listed below for integrating with MediaTailor.
https://docs.aws.amazon.com/mediatailor/latest/ug/vast-integration.html

What is causing cfn-signal with waithandle.url to get 403 error

Good day.
My cloudformation stack keeps getting rolled back due to the error for the WaitCondition.
The EC2 instance has Userdata which calls custom bootstrap.sh that uses cfn-signal with the waithandle.url (replaced sensitive info with "masked"):
cfn-signal --success true --http-proxy http://proxyAbc:123 --https-proxy http://proxyAbc:123 --region ap-southeast-2 https://cloudformation-waitcondition-ap-southeast-2.s3-ap-southeast-2.amazonaws.com/arn%3Aaws%3Acloudformation%3Aap-southeast-2%3A747462550105%3Astack/asg-masked-20200508162554-0b080289adf738030/35459000-90f5-11ea-a7af-0a0ad6464e74/WaitHandle?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200508T062906Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86399&X-Amz-Credential=masked&X-Amz-Signature=masked
However it is encountering this error now:
Error signaling CloudFormation: [Errno 403] HTTP Error 403 : AccessDeniedAccess DeniedXYZ...123
There has been no code change except for using a new Centos 7 AMI.
Has anyone encountered this error and managed to resolve it?
Edit:
The userdata has this in the CloudFormation template (which has single-quote surrounding it), where the WAITHANDLE environment variable is used in the cfn-signal command above (the https://cloudformation-waitcondition-ap-southeast-2....):
BASH_SCRIPT[8]=\"export WAITHANDLE='", {"Ref": "WaitHandle"}, "'\"\n
The issue turned out to be due to proxy issue.
The proxy being used works before (for years) but now has been broken.
This is probably not a general answer as the 403 seems to be to broad, though pay attention to the part where it says AccessDenied (without 'Request has Expired') in any case just answering this in case anyone else encounters it.
For example:
Pre-signed URL expired:
Error signaling CloudFormation: [Errno 403] HTTP Error 403 :
AccessDeniedRequest has expired863992020-05-15T05:17:56Z2020-05-18T20:41:19Z[somehashvalue]
Proxy issue:
Error signaling CloudFormation: [Errno 403] HTTP Error 403 : AccessDeniedAccess Denied[somehashvalue]

Route53 Domain Transfer - Registry error - 2400 : Command failed (421 SESSION TIMEOUT)

I am trying to transfer a domain using Route53 and after a few minutes I receive an email with the following error.
Registry error - 2400 : Command failed (421 SESSION TIMEOUT)
Anyone have any ideas what this means or how to get around it?
I have never seen your error. There is a document on transferring domains with error messages. The reason that I am responding is that I have seen domain transfers fail going to Route 53 without every learning why they failed. Maybe this will help you.
NSI Registry Registrar Protocol (RRP)
421 Command failed due to server error. Client should try again A
transient server error has caused RRP command failure. A subsequent
retry may produce successful results.

Issues in performing operations with KIbana dashboard : 501 Not Implemented

I have Kibana and Elasticsearch installed on AWS instance. Elasticsearch seems to work fine. However when I try to perform some action (PUT and DELETE requests) using Kibana dashboard it gives me error 501 (Not Implemented). It works in case of GET requests .I access Kibana from my local system using http://aws-ip:5601/. Below is the error trace that I get
Failed to load resource: the server responded with a status of 501 (Not Implemented)
commons.bundle.js?v=15104:38 Error: Not Implemented
at respond (kibana.bundle.js?v=15104:13)
at checkRespForFailure (kibana.bundle.js?v=15104:13)
at kibana.bundle.js?v=15104:1
at processQueue (commons.bundle.js?v=15104:38)
at commons.bundle.js?v=15104:38
at Scope.$eval (commons.bundle.js?v=15104:39)
at Scope.$digest (commons.bundle.js?v=15104:39)
at Scope.$apply (commons.bundle.js?v=15104:39)
at done (commons.bundle.js?v=15104:37)
at completeRequest (commons.bundle.js?v=15104:37)
(anonymous) # commons.bundle.js?v=15104:38
Also if I copy the request payload from network tab of Chrome developer tools and perform the same request (PUT/DELETE) using CURL from AWS terminal, it works fine.
Please help!!!
Thanks in advance .
I am posting my findings in case somebody else is also stuck in similar type of situation. In my case , corporate proxy was just blocking PUT and DELETE requests resulting in 501 (Not implemented) error.