Amazon EMR: Terminated with errors Bootstrap failure - amazon-web-services

I created a custom bootstrap action for my Amazon EMR cluster. The cluster fails to launch, and the bootstrap action returns an error:
failed to start. bootstrap action 1 failed with non-zero exit code
I trying to follow the step by step, about the Amazon S3 buckets file. I saw in the log file this message <Message>Access Denied</Message>

Related

Setting up Elastic inference accelerator with aws deep learning AMI

I am trying to launch an instance with AWS deep learning AMI with an elastic inference accelerator. I have set the role for the policy for the IAM to connect as described in the documentation "elastic-inference:Connect". But When i try launch the instance i get the error. Initiating launches failure invalid attachment reservation 0 for accelerator type : eia1.medium
I tried adding further permission. I added the following permissions.
AmazonEC2FullAccess
AmazonEC2ContainerRegistryFullAccess
GlobalAcceleratorFullAccess
trial_policy.json (contains the json which is mentioned in the documentation for adding "elastic-inference:Connect")
AmazonDevOpsGuruFullAccess
But is still get the following error.
Launch Failed Invalid attachment reservation 0 for accelerator type :
eia1.medium.
Verifying entitlement to product Successful
Initiating launches Failure

How do I use AWS credentials with Jenkins to deploy to Elastic Beanstalk?

I have entered AWS credentials in Jenkins at /credentials, however they do not show up in the drop down list for the Post Build steps in the AWS Elastic Beanstalk plugin.
If I click Validate Credentials, I get this strange error.
Failure
com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), com.amazonaws.auth.profile.ProfileCredentialsProvider#5c932b96: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper#32abba7: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]
at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136)
I don't know where it got that IP address. When I search for that IP in the Jenkins directory, I turn up with
-bash-4.2$ grep -r 169.254.169.254 *
plugins/ec2/AMI-Scripts/ubuntu-init.py:conn = httplib.HTTPConnection("169.254.169.254")
The contents of that file is here: https://pastebin.com/3ShanSSw
There are actually 2 different Amazon Elastic Beanstalk plugins.
AWSEB Deployment Plugin, v 0.3.19, Aldrin Leal
AWS Beanstalk Publisher Plugin, v 1.7.4, David Tanner
Neither of them work. Neither will display the credentials in the drop down list. Since updating Jenkins, I am unable to even show "Deploy to Elastic Beanstalk" as a post-build step for the first one (v0.3.19) even though it is the only one installed.
For the 2nd plugin (v1.7.4), I see this screen shot:
When I fill in what I can, and run it, it gives the error
No credentials provided for build!!!
Environment found (environment id='e-yfwqnurxh6', name='appenvironment'). Attempting to update environment to version label 'sprint5-13'
'appenvironment': Attempt 0/5
'appenvironment': Problem:
com.amazonaws.services.elasticbeanstalk.model.AWSElasticBeanstalkException: No Application Version named 'sprint5-13' found. (Service: AWSElasticBeanstalk; Status Code: 400; Error Code: InvalidParameterValue; Request ID: af9eae4f-ad56-426e-8fe4-4ae75548f3b1)
I tried to add an S3 sub-task to the Elastic Beanstalk deployment, but it failed with an exception.
No credentials provided for build!!!
Root File Object is a file. We assume its a zip file, which is okay.
Uploading file awseb-4831053374102655095.zip as s3://appname-sprint5-15.zip
ERROR: Build step failed with exception
com.amazonaws.services.s3.model.AmazonS3Exception: The XML you provided was not well-formed or did not validate against our published schema (Service: Amazon S3; Status Code: 400; Error Code: MalformedXML; Request ID: 7C4734153DB2BC36; S3 Extended Request ID: x7B5HflSeiIw++NGosos08zO5DxP3WIzrUPkZOjjbBv856os69QRBVgic62nW3GpMtBj1IxW7tc=), S3 Extended Request ID: x7B5HflSeiIw++NGosos08zO5DxP3WIzrUPkZOjjbBv856os69QRBVgic62nW3GpMtBj1IxW7tc=
Jenkins is hopelessly out of date and unmaintained. I added the Post Build Task plugin, installed eb tool as jenkins user, ran eb init in the job directory, edited .elasticbeanstalk/config.yml to add the lines
deploy:
artifact: target/AppName-Sprint5-SNAPSHOT-bin.zip
Then entered in the shell command to deploy the build.
/var/lib/jenkins/.local/bin/eb deploy -l sprint5-${BUILD_NUMBER}
For Eleastic beanstalk plugin right place to configure AWS key is Jenkins Master configure
http://{jenkinsURL}/configure

AWS Data Pipeline EmrClusterForLoad Error

I'm try to transfer data between s3 and dynamodb with AWSDataPipeline.
error message below...
Unable to create resource for #EmrClusterForLoad_2017-05-15T18:51:19
due to: The supplied ami version is invalid. (Service:
AmazonElasticMapReduce; Status Code: 400; Error Code:
ValidationException; Request ID: 7ebf0367-399f-11e7-b1d7-29efc4730e41)
but, i cannot solve the problem.
help me
aws datapipeline error
ami 3.9.0 is not supported in all regions.
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-3x.html
Also make sure to select a supported EC2 instance type
https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-emr-supported-instance-types.html
Finally you need to set "Resize Cluster Before Running" as false in the Table Load activity.
I made it run after doing all these changes. Hopefully it will help you too.

AWS SDK Error: Failed to get the Amazon S3 bucket name

I am attempting to setup AWS with an Elastic Beanstalk instance I have previously created. I have entered the various details into the config file, however when I try to use aws.push I get the error message
Updating the AWS Elastic Beanstalk environment x-xxxxxx...
Error: Failed to get the Amazon S3 bucket name
I have checked the credentials in IAM and have full administrator privileges. When I run eb status show green I get the following message:
InvalidClientTokenId. The security token included in the request is invalid.
Run aws configure again, and re-enter your credentials.
It's likely you're using old (disabled or deleted) access/secret keys, or you accidentally swapped the access key with the secret key.
For me is was that my system clock was off by more than 5 minutes. Updating the time fixed the issue.

Amazon Elastic Beanstalk TV instance start fails

I have an identity TVM on Amazon Elastic Beanstalk that when I try to start it gives
014-07-07 15:29:46 UTC+0100 ERROR Stack named 'awseb-e-ybrpewdr7z-stack' aborted operation. Current state: 'CREATE_FAILED' Reason: The following resource(s) failed to create: AWSEBInstanceLaunchWaitCondition. (Service: AmazonCloudFormation; Status Code: 400; Error Code: OperationError; Request ID: null)
Now If I go to logs and click snapshot logs all that happens is it waits for a while showing processing but then I get no logs showing up. Does anyone please have an idea what the problem is so I can see either the logs and/or sort the startup problem?
This was security issue.
Amazon ECB uses a VPC, now that may already be running especially if a RDB has been created first. Now the groups on the EB instance can show acces rights reuired. But the VPC has an underlying security ACL that is not group based. When a RDB is created first that initially created the VPC the VPC instance gets the rights just to access the DB, so will not allow for example HTTP traffic through this cannot get to the EB to set it up.