I am trying to build a pipeline in gitlab which is using their provided gl-cloudformation Template to deploy Infrastructure to aws:
https://gitlab.com/gitlab-org/cloud-deploy/-/blob/master/aws/src/bin/gl-cloudformation
I am running into a problem with creating IAM roles since cloudformation is needing extra confirmation to deploy stacks which create IAM resources. Normally i would just run aws create-stack --capabilities CAPABILITY_NAMED_IAM but since i am useing their template i can't.
Anyone got any experience with running gitlab and cloudformation?
This is not possible with that image. You must either use different CI Image or do not add custom name for the IAM resoucre (let the AWS generate name).
Related
If a Codebuild project runs on a custom image that has awscli preinstalled, but not configured for that AWS account, would it be still possible to run aws * in that project's buildspec without updating its AWS credentials there first?
In other words, are these credentials made available by Codebuild (e.g. via providing this information in automatically picked up environment variables) , or if I am using a custom image, it is up to me to take care of that explicitly, and aws * is only expected to work in buildspec out of the box without additional efforts on Codebuild managed images?
(I mean configuration/credentials for the account and role the Codebuild project in question operates in)
When you attach an IAM service role with your AWS Codebuild project, you don't need to configure AWS cli. IAM service role is part of environment configuration and this role will be assumed whenever you try to access resources in AWS. This goes same for your custom image for AWS Codebuild as well.
I'm trying to get this repo going here - https://github.com/mydatastack/google-analytics-to-s3.
A link is provided to launch the AWS CloudFormation stack but it is no longer working as the S3 bucket containing the template is no longer active.
I have 2 questions about getting data pipeline running:
My first question would be what is 631216aef6ab2824fc63572d1d3d5e6c.template and can I create it through the 3 .yml files in the CloudFormation folder?
I've tried to create a template through CloudFormation designer , collector-ga.yml but it fails. I think its because the Resources within the yml aren't available when creating a template just from collector-ga. I've also tried uploading the repo to s3 and creating a template from there but that was also unsuccessful.
How can I launch the stack from the repo? I've found very little information online so an explanation or a pointer to some relevant resources would be appreciated.
This repository doesn't use the "standard" CloudFormation resources, but it uses AWS SAM. You'll have to install the SAM CLI tool and use that to deploy the CloudFormation stack. If you run sam deploy --guided it will help you with the setup of the necessary S3 bucket etc on your AWS account. SAM will upload the necessary files, resolve the internal local links between the templates by updating them with the S3 URLs and construct a packaged.yml template which it will use to deploy the stack.
Also, check out the AWS SAM user guide for more information.
AWS CDK stacks target an account or region based on an evironment, details here. Here is an example of an app that deploys one stack into multiple target accounts:
const envEU = { account: '2383838383', region: 'eu-west-1' };
const envUSA = { account: '8373873873', region: 'us-west-2' };
new MyFirstStack(app, 'first-stack-eu', { env: envEU });
new MyFirstStack(app, 'first-stack-us', { env: envUSA });
My question is how to deploy these 2 stacks - is it possible to deploy them as a single operation? If so, what credentials are used and what roles are required on the 2 accounts?
Ideally, I'd like to be able to do a single command to deploy all stacks across all accounts:
cdk deploy ...
Or is the deployment only possible via 2 steps?
cdk deploy first-stack-eu --profile=profile_for_account_2383838383
cdk deploy first-stack-us --profile=profile_for_account_8373873873
I ended up using the cdk-assume-role-credential-plugin to perform the task. The description of that plugin states:
This plugin allows the CDK CLI to automatically obtain AWS credentials
from a stack's target AWS account. This means that you can run a
single command (i.e. cdk synth) with a set of AWS credentials, and the
CLI will determine the target AWS account for each stack and
automatically obtain temporary credentials for the target AWS account
by assuming a role in the account.
I wrote up a detailed tutorial on how to use this plugin to perform AWS cross-account deployments using CDK here: https://johntipper.org/aws-cdk-cross-account-deployments-with-cdk-pipelines-and-cdk-assume-role-credential-plugin/
In cloudformation you can use Stack Sets for multi-account and multi-region deployments.
However, this is not yet supported in CDK according to the GitHub issue:
Support for CloudFormation StackSets #66
As of v2 of CDK this is available by default:
Now by default when you bootstrap an AWS account it will create a set of IAM roles for you, which the CDK will assume when performing actions in that account.
If you have multiple stacks in your app you have to pass every stack into the cdk deploy command e.g. cdk deploy WmStackRouteCertStack004BE231 WmStackUploadStackF8C20A98
I don't know of a way to deploy all stacks in an app, I don't like this behavior and it's the reason I try to avoid creating multiple stacks
The SFProSetup stack was deleted by mistake. This deleted an IAM role needed for serverless deploy
So I redeployed the template under a new name but the following error message still occurs when deploying my service.
Error: {"errorMessage":"Your AWS credential for deployment profile default is configured to use the IAM role arn:aws:iam::730609332320:role/SFProRole, but access to that role was denied by IAM. - Please contact support and provide this identifier to reference this issue - NVB2DG9Q46SQ"}
I have one aws profile and have been deploying successfully before this.
I redeployed the template with the following command
aws cloudformation deploy --template-file ~/Desktop/serverless_setup.yml --stack-name SFProSetup --region us-east-1 --profile default --parameter-overrides OrgId=xyz123 ProfileName=default --capabilities CAPABILITY_NAMED_IAM
A series of steps led me to find that the stack must be deployed in us-east-1 because that's the only region the Reporter component can talk to. --capabilities CAPABILITY_NAMED_IAM was necessary because the template creates an AWS role with a capability. And finally, I noticed a parameter for OrgId was used in the original stack creation, so I added it to the paramter overrides in the command. I'm guessing this is my id in serverless OrgId=xyz123
I'm creating a new user pool in AWS Cognito. As you might know, CF support is missing for a lot of the features in Cognito, so I´ve resorted to using the CLI for Cognito. But I still want to use CloudFormation for other resources like API Gateway that will need to reference the new user pool.
Is there any way I can create parameters with the CLI that I can use in CloudFormation?
Yes, if you have Parameters in your template, then you can use the CloudFormation Deploy command to do exactly this.
For example, you can call aws cloudformation deploy --template-file <file_path> --stack-name <stack_name> --parameter-overrides ParameterKey1=ParameterValue1 ParameterKey2=ParameterValue2 ... where <file_path> is the path to your CloudFormation Template, and <stack_name> is the name of your CloudFormation Stack. If this stack doesn't exist yet, Deploy will create it, but if it does exist, Deploy will update it.