Trying to create and run an AWS CodePipeline that pulls from Github, builds and deploys to an EC2 instance. The pipeline is as follows:
Source (Github) -> Build (AWS CodeBuild) -> Deploy (AWS CodeDeploy)
The source and build steps both succeed. However, deploy fails consistently giving the following error:
Insufficient permissions
Unable to access the artifact with Amazon S3 object key '[redacted]-2nd-test-pip/BuildArtif/IbiHzen' located in the Amazon S3 artifact bucket 'codepipeline-us-east-1-[redacted]'. The provided role does not have sufficient permissions.
Below is the IAM policy for the CodeBuild service role policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Resource": [
"arn:aws:logs:us-east-1:362490217134:log-group:/aws/codebuild/[Redacted]-Build-Project",
"arn:aws:logs:us-east-1:362490217134:log-group:/aws/codebuild/[Redacted]-Build-Project:*"
],
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
]
},
{
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::codepipeline-us-east-1-*"
],
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectVersion"
]
},
{
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::codepipeline-us-east-1-[Redacted]/*"
],
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectVersion"
]
}
]
}
The CodePipeline service role created by the pipeline wizard has assigned S3 full access:
"Action": [
"elasticbeanstalk:*",
"ec2:*",
"elasticloadbalancing:*",
"autoscaling:*",
"cloudwatch:*",
"s3:*",
"sns:*",
"cloudformation:*",
"rds:*",
"sqs:*",
"ecs:*"
],
"Resource": "*",
"Effect": "Allow"
},
I have confirmed numerous times that the artifact referenced in the pipeline deploy step matches the artifact created by the build step.
If I go and look at path referenced, there is not a directory or zip file (not sure which SHOULD be there, but neither is) with that name. Additionally, a zip file is generated during the build, but it is never named what the deploy step expects.
I've also gone into the build project and attempted builds using other artifact configurations, but they seem to be ignored when running the build through CodePipeline.
Disclaimer: I've seen similar questions posted here and elsewhere on the interwebs, but each of them deal with ECS or another situation that differs from mine. Thank you for your help
The issue was unrelated to roles/policies. As mentioned, the expected zip file did not exist in the S3 bucket. This was due to an invalid artifact files path specified in the buildspec. Once corrected, the zip file is created and the deploy no longer fails on this error. Seems odd to me that CodePipeline would allow the build to report as completed successfully without validating that the files created as the artifact and passed to the deploy step were, in fact, created.
Related
My project uses a library from AWS CodeArtifact. I can fetch the library and build the project in my local and in the github build. When the AWS CodePipeline runs in our Dev environment, CodeArtifact returns a 401 when trying to access the library.
I updated the IAM role running the Pipeline so that it has these actions:
{
"Effect": "Allow",
"Action": [
"codeartifact:GetAuthorizationToken",
"codeartifact:GetRepositoryEndpoint",
"codeartifact:ReadFromRepository"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "sts:GetServiceBearerToken",
"Resource": "*",
"Condition": {
"StringEquals": {
"sts:AWSServiceName": "codeartifact.amazonaws.com"
}
}
}
I updated the Repository Policy to include this IAM role. The Repository Policy has these actions:
"codeartifact:DescribePackageVersion",
"codeartifact:DescribeRepository",
"codeartifact:GetPackageVersionReadme",
"codeartifact:GetPackageVersionAssets",
"codeartifact:GetRepositoryEndpoint",
"codeartifact:ListPackageVersionAssets",
"codeartifact:ListPackageVersionDependencies",
"codeartifact:ListPackageVersions",
"codeartifact:ListPackages",
"codeartifact:PublishPackageVersion",
"codeartifact:PutPackageMetadata",
"codeartifact:ReadFromRepository",
"codeartifact:GetAuthorizationToken"
My build.gradle is using the repository as described here and works in my local this way - https://medium.com/#chauyan/use-aws-codeartifact-in-your-project-7bf5d3e0d3dc
I'm guessing it has to do with the way the library was published, maybe it doesn't have the right permissions, but I don't see that the permissions are changeable here. Why am I getting a 401 from CodeArtifact when trying to build the project in CodePipeline?
You have to publish to maven central
I have my build artifacts in an S3 bucket in my dev account. I have an EC2 instance (target of deployment) and CodeDeploy setup in the prod account. Both instance profile role and CodeDeployRole have access to the bucket in dev account. From instance I can manually copy the files.
However deployment fails during download bundle phase with access denied.
Please help.
Bucket policy in dev account:
"Principal": {
"AWS": [
"arn:aws:iam:: 222222222222:role/CodeDeployRole",
"arn:aws:iam:: 222222222222:role/instanceprofilerole"
]
},
"Action": [
"s3:PutObject",
"s3:ListBucket",
"s3:GetObject",
"s3:Get*",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::mybucketname",
"arn:aws:s3:::mybucketname/*"```
Using .Net Core, visual studio 2017 and AWS Toolkit for Visual 2017, I created a basic web api, the api works as designed.
However when it comes to publishing/deploying it, the first time works perfectly when the Stack doesnt exist, creates everything its suppose to. When I make a change and need to re-deployed/publish, it comes back with the following error.
Error creating CloudFormation change set: Stack [TestStack] already exists and cannot be created again with the changeSet [Lambda-Tools-636366731897711782].
Just above the error message is this
Found existing stack: False
Im wondering if there is something not quite right with it detecting if the Stack exists.
Im just wondering if Im missing something, or if this is actually be design, as for me to republish it I have to log into my AWS Console and go into the cloud formation section and delete the existing Stack.
Publish Dialog
Project Structure
After a bit of digging, and general trial and error. I believe this is actually to do with permissions of the user performing the publish. (The user in AWS)
I changed an inline policy to
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"cloudformation:*"
],
"Resource": [
"*"
]
}
]
}
Where cloudformation:* used to be several lines for individual permissions.
This now successfully publishes over an existing Stack, however visual studio doesnt like it and crashes. (Although the update does go through to AWS)
AWS' Serverless Application Model is … very new still. And for lack of any documentation about what IAM permission one needs to deploy an App with their CLI, I've worked out this policy that seems to work, and only grants the least needed permissions for the task.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"lambda:UpdateFunctionCode",
"s3:PutObject",
"cloudformation:DescribeStackEvents",
"cloudformation:UpdateStack",
"cloudformation:DescribeStackResource",
"cloudformation:CreateChangeSet",
"cloudformation:DescribeChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:GetTemplateSummary",
"cloudformation:ListChangeSets",
"cloudformation:DescribeStacks"
],
"Resource": [
"arn:aws:lambda:*:123456789012:function:*-SAM-*",
"arn:aws:cloudformation:*:123456789012:stack/<STACK NAME OR GLOB>/*",
"arn:aws:cloudformation:<AWS_REGION>:aws:transform/Serverless-2016-10-31",
"arn:aws:s3:::aws-sam-cli-managed-default-samclisourcebucket-*/*"
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucketMultipartUploads"
],
"Resource": "arn:aws:s3:::aws-sam-cli-managed-default-samclisourcebucket-*"
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": "cloudformation:ValidateTemplate",
"Resource": "*"
}
]
}
Note:
Replace <STACK NAME OR GLOB> with something that best suits your needs, like:
* If you don't care which CloudFormation Stack this grants access to
*-SAM-* If you name your SAM CloudFormation apps with some consistency
Replace <AWS_REGION> with the region you're operating in.
The arn:aws:s3:::aws-sam-cli-managed-default-samclisourcebucket-* is the standard bucket naming that SAM CLI uses for creating a bucket for deploying CloudFormation Templates or Change Sets. You could alter this to explicitly be the name of the bucket SAM created for you.
OK Here is the scenario.
I have a Jenkins Slave in AWS and I've attached to it a Role that allows it to create EC2 resources. I found the role via the Packer github issue list. Here is the role
I have my Packer project attempting to build on the slave. When the build starts it fails with:
[1;31mBuild 'amazon-ebs' errored: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors[0m
==> Some builds didn't complete successfully and had errors:
--> amazon-ebs: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
If I run aws configure and put in real credentials this obviously works, but I'm trying to avoid that. I have verified that the instance has the proper role attached. I have also verified that I can properly switch into this role via the command line.
What I seem to be missing is that with the role associated with the instance and packer specifying the role with: 'iam_instance_profile' why this continues to fail.
Any thoughts?
So after a lot of help from Castaglia I was able to get this to work. There seemed to be something wrong with the Role I had created. I deleted it and recreated it with the same name and same policy attached. After that it worked fine.
To note, I believe the Packer instructions have an error. They list the following as all that is needed for the role:
{
"Statement": [{
"Effect": "Allow",
"Action" : [
"ec2:AttachVolume",
"ec2:CreateVolume",
"ec2:DeleteVolume",
"ec2:CreateKeypair",
"ec2:DeleteKeypair",
"ec2:DescribeSubnets",
"ec2:CreateSecurityGroup",
"ec2:DeleteSecurityGroup",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateImage",
"ec2:CopyImage",
"ec2:RunInstances",
"ec2:TerminateInstances",
"ec2:StopInstances",
"ec2:DescribeVolumes",
"ec2:DetachVolume",
"ec2:DescribeInstances",
"ec2:CreateSnapshot",
"ec2:DeleteSnapshot",
"ec2:DescribeSnapshots",
"ec2:DescribeImages",
"ec2:RegisterImage",
"ec2:CreateTags",
"ec2:ModifyImageAttribute"
],
"Resource" : "*"
}]
}
But I believe you need one more piece:
{
"Sid": "PackerIAMPassRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": [
"*"
]
}
Doing this allowed me to assume the role and build what I needed.
I have server S3 buckets belonging to different clients. I am using AWS SDK for PHP in my application to upload photos to the S3 bucket. I am using the AWS SDK for Laravel 4 to be exact but I don't think the issue is with this specific implementation.
The problem is unless I give the AWS user my server is using the FullS3Access it will not upload photos to the bucket. It will say Access Denied! I have tried first with only giving full access to the bucket in question, then I realized I should add the ability to list all buckets because that is probably what the SDK tries to do to confirm the credentials but still no luck.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::clientbucket"
]
}
]
}
It is a big security concern for me that this application has access to all S3 buckets to work.
Jeremy is right, it's permissions-related and not specific to the SDK, so far as I can see here. You should certainly be able to scope your IAM policy down to just what you need here -- we limit access to buckets by varying degrees often, and it's just an issue of getting the policy right.
You may want to try using the AWS Policy Simulator from within your account. (That link will take you to an overview, the simulator itself is here.) The policy generator is also helpful a lot of the time.
As for the specific policy above, I think you can drop the second statement and merge with the last one (the one that is scoped to your specific bucket) may benefit from some * statements since that may be what's causing the issue:
"Action": [
"s3:Delete*",
"s3:Get*",
"s3:List*",
"s3:Put*"
]
That basically gives super powers to this account, but only for the one bucket.
I would also recommend creating an IAM server role if you're using a dedicated instance for this application/client. That will make things even easier in the future.