I have an docker image with an API that publishes provided request body to a SNS topic. Now I am able to perform this operation locally, but when I deploy this image to ECS through ECR, I get following error form AWS Go SDK(v2) after ~1 min of request initiation.
https response error StatusCode: 0, RequestID: , canceled, context canceled
I have verified following things:
ECS service (as well as EC2 as I am using EC2 launch type) has a role that grants full access to SNS
SNS access policy allows all requests from current AWS account.
I am able to publish to same topic after logging into EC2 instance.
API request has 5 min timeout.
I am using default VPC.
For the sake of experiment, I am not using HTTPS and attached security group allows all traffic on port 80. I tried everything but didn't find what's wrong. I would appreciate your great help, thanks.
Related
Trying to turn a containerized Cube.js deployment into a serverless one. I’m getting Continue wait on every API call, and I notice the cubejsProcess lambda is never invoked. I checked SNS reachability by (successfully) publishing a message to it using the AWS SDK in the cubejs lambda. The cubejsProcess does get invoked with this inserted message. What might be preventing cubejs from publishing messages to SNS?
This is similar to this question, however, connectivity to SNS has been verified in this case.
I'd check IAM permissions to see if the cubejs Lambda's role. Does the role allow making requests to SNS?
Another thing that could be causing the issue is if the cubejs Lambda is in a VPC with no Internet access; calling any AWS API endpoint would fail since they all require an Internet connection.
We're working with AWS and wish to log all incoming http requests from our API Gateway to the EC2 box in their entirety (i.e. not truncated at 1024 bytes). As we know the logging for API Gateway is not configurable our next thought was to place a lambda between the api gateway and EC2 which would intercept the requests, log them to S3, and then forward the request to EC2.
My question is two fold:
How does one submit an API Gateway 'event' to an EC2 instance manually? I've gone over the boto3 documentation and there doesn't seem to be a good way to do this out of the box.
Is this a logical way to log requests in their entirety, or are we missing a built in connector between lambda -> ec2 or request logging in general?
def lambda_handler(event, context):
#pass event to the ec2
#log event (done)
#return response
You can enable CloudWatch logging for API gateway. By enabling you can find the logs for your requests in the cloud watch.
Create an IAM role that gives permissions for the API gateway to write logs to cloud watch
set the IAM role to your API
set the desired log levels INFO/ERROR
you can also choose to log the full request/responses
To enable access logs, you also need to create a log group and provide the arn.
The detailed steps can be found in the reference below.
Hope this helps.
Reference:
https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-cloudwatch-logs/
I'm working with an on-premise older version of BitBucket Server v5.15.1 that does not have the Bitbucket Pipelines feature and I need how to get the webhooks to notify AWS Lambda via HTTPS POST via AWS API Gateway after a commit is made to master branch...then Lambda downloads a copy of the repo, zips it up and places it into an S3 bucket...and of course this is where CodePipeline can finally be triggered...But I'm having issues getting this on-premise BitBucket Server located within my AWS account to connect its webhook to Lambda.
I tried following this documentation below and launched the CloudFormation template with all the needed resources but I'm assuming it is for BitBucket Cloud not Bitbucket Server OP.
https://aws.amazon.com/blogs/devops/integrating-git-with-aws-codepipeline/
Anyones help with this would be really appreciated.
I suppose you are following this below blog from AWS :
https://aws.amazon.com/blogs/devops/integrating-codepipeline-with-on-premises-bitbucket-server/
We had also implemented it. If the event is coming to Lambda, then make sure your Lambda is within a VPC and it has correct outbound(read as inbound) rules to connect the Bitbucket server over HTTPS. Also the Bitbucket server accepts the VPC IP range.
We have implemented Dynatrace and we need to add our AWS account to pull CloudWatch logs.
The problem: We have a corporate proxy and firewall which is super locked down and seems to block whichever requests Dynatrace is trying to make to AWS to authenticate with the key and secret.
The infra guys has allowed a bunch of AWS domains per region to read CloudWatch logs, but we still can't authenticate (see image below).
I have set up a Dynatrace security gateway in AWS which our local gateways are able to access. We are successfully getting logs form OneAgents through the SGW. The problem is getting the CloudWatch logs/integration going.
The error in the image below leads me to believe that Dynatrace is not able to communicate to the AWS auth servers at all.
Any advice would be appreciated.
I'm just a dev
We are not allowed to open the amazon domain.
#corporate-devlife
We are working on SNS to HTTPS API integration between 2 teams. So, another team has SNS topic configured in us-east-1 region and our HTTPS API is running in us-west-2 region.
Just wanted to check if we can process SNS messages generated from us-east-1 and a different AWS account in us-west-2 region.
I read somewhere (can't remember the blog post link now), that if SNS messages are generated in us-east-1, they have to be processed in us-east-1 only. If we try to process in us-east-2, message signature verification will fail and throws an invalid TopicArn exception.
Please guide if this is correct.
So, another team has SNS topic configured in us-east-1 region and our HTTPS API is running in us-west-2 region.
When SNS is publishing to an HTTPS endpoint, the endpoint can be anywhere on the Internet. It doesn't even need to be in AWS at all. As long as your HTTPS endpoint is accessible from the Internet and has a valid SSL certificate (matches the hostname, not expired, signed by an accredited certificate authority, not self-signed), that is all SNS will require.
The destination region and destination AWS account lose all meaningfulness in such a setup. There is no constraint, here.
If the topic is in us-east-1 then all requests sent to SNS must be sent to the us-east-1 endpoint (e.g. Subscribe or Publish) but even then, they can be sent from anywhere on the Internet.
SNS can be configured across regions via HTTPS endpoint of lambda by attaching the API Gateway to that lambda and copying the HTTPS URL of same.
After getting the HTTPS URL of the lambda may be whichever region. It may be that you just add in SNS service subscription part in SNS service by creating the subscription in the service.
After that, just publish a message. You will see entries in your Cloudwatch logs if you print an event in lambda. You will see something like "Subscribe URL". Copy that URL and paste it in the SNS service where you will be able to see "Pending on Subscription". This will work with latest amazon SNS service.
I tried and tested it to verify that this works.