I am new to DevOps and creating a sample CI/CD pipeline in AWS. Once CI is successful, the code should be moved to S3 bucket. I have to write an appspec.yml file to deploy artifacts from S3 to IIS.
Here I have few queries:
(1) Once CI is successful, are the files moved to S3 bucket as .zip?
(2) Where should I keep the appspec.yml?
(3) What should the appspec.yml code look like so that CodeDeploy reads it and deploys the artifacts to IIS?
If you are using Codedeploy, then you should zip the file locally, upload to S3 and then register the revision against your deployment group.
The appspec.yml is always found in the root folder of the deploy package.
Here is a sample appspec.yml
version: 0.0
os: windows
files:
- source: \index.html
destination: C:\inetpub\wwwroot
This will deploy index.html to the default document root.
Your deployment package / zip file will contain appspec.yml and index.html in the root folder.
Related
I have created the below-mentioned pipeline in AWS.
CodeCommit -> CodeBuild -> CodeDeploy
I am using ng serve command to build the Angular JS source code and it creates a folder named dist and moves all the build content.
When deploying the build result to an Amazon S3 bucket, is it possible to move only the content inside the dist folder?
At the moment, all the source files are being transferred to the Amazon S3 bucket with the dist folder.
Other than writing a Lambda function to achieve this, is there a shortcut to achieve this within the pipeline?
I wonder why you have used CodeDeploy for S3 copying. You can achieve the same using CodeBuild itself.
CodeCommit => CodePipeline => CodeBuild => S3
After creating the build (dist), add commands for S3 sync in your buildspec.yml file
Sample few lines of buildspec.yml for your use case.
- ng serve
- ls -ltr
- cd dist
- aws s3 sync . s3://YOUR_BUCKET --acl bucket-owner-full-control
I am using a AWS CodePipeline which fetches the source from my git repository, uses CodeBuild buildSpec to build and save output artifacts to S3 bucket, which ultimately gets deployed to Elastic BeanStalk (NodeJS Environment).
Everything works fine but I require the pipeline to copy 1 particular file from one of my AWS S3 buckets and add it to the output artifacts before deploying it to the EB
Can it be done using the buildSpec?
artifacts:
files:
- '**/*'
# - How to add a file from S3 to the artifacts?
My recommendation is as part of the build or post_build, copy the required file from s3 into your build directory.
build:
commands:
- echo "Build commands"
- aws s3 cp --region=xx-xxxx-x "s3://file/in/s3" "local-file-instance-to-include"
Then you will have the file copied from s3, available for your build, and you can add it to the artifacts output.
Project is built using AWS lambda and some other AWS services. In order to make a build and just to serve a build static files through node express server, it requires the credentials.js file which contains all my AWS credentials of the project.
Now I' making creating an aws codepipeline which pulls the code from GitHub and deploys it on ec2 using elastic beanstalk.
The problem is I don't push my credentials file on Github as it is not secure to do so. And without credentials file the code cannot be deployed. Please suggest me a solution to this problem.
I have already tried placing the file manually in ec2 instance by logging in, but when I make a commit to my repo, the code pipeline executes and it replaces the whole directory of my app in ec2, so the file gets removed.
You can securely store your credentials config file in a private S3 bucket and add commands to your buildspec.yml file (used by the CodeBuild stage of your pipeline) to get that file and put it in the right place for your project.
You will need to give the CodeBuild service role the correct permissions to access the private S3 bucket.
The following is an example of what I mean. Note that I might be storing the config file for multiple environments in S3, so I use an environment variable to specify the exact naming of the file, e.g. my-creds.dev.json or my-creds.prod.json. The filename is then converted to just my-creds.json so you can rely on the same name in your program.
CodeBuild will look for buildspec.yml which defines your build in the root of your project and execute these commands
version: 0.2
phases:
pre_build:
commands:
# Get the creds config file for the correct environment
# and put it in the my projects config directory (or wherever you need it)
- aws s3 cp s3://my-s3-bucket/my-creds.${ENVIRONMENT}.json ./config/my-creds.json
- npm install
build:
commands:
- npm run build:${ENVIRONMENT}
artifacts:
files:
- dist/**/*
The critical line is aws s3 cp s3://my-s3-bucket/my-creds.${ENVIRONMENT}.json ./config/my-creds.json.
The commands are run in the root of your project directory, where the buildspec.yml file is.
The above command is in the pre_build phase so it will run before the commands (npm build) in the build phase are run.
The command copies your credential file from your s3 bucket to the path defined by the last part of the command (in this case ./config/my-creds.json). Again remember this is relative to the root of your project directory, so if your project contains a src folder in the root directory, then your path might be ./src/my-creds.json, or ./src/creds/my-creds.json.
You should not add the credentials file as it is not recommended. You can do the deployment by assigning a role to your code pipeline with permission to access the required resources.
I'm trying to hook my GitHub repo with S3 so every time there's a commit, AWS CodePipeline will deploy the ./<path>/public folder to a specified S3 bucket.
So far in my pipeline, the Source works (hooked to GitHub and picks up new commits) but the Deploy failed because: Action execution failed
BundleType must be either YAML or JSON.
This is how I set them up:
CodePipeline
Action name: Source
Action provider: GitHub
Repository: account/repo
Branch: master
GitHub webhooks
CodeDeploy
Compute type: AWS Lambda
Service role: myRole
Deployment settings: CodeDeployDefault.LambdaAllAtOnce
IAM Role: myRole
AWS Service
Choose the service that will use this role: Lambda / CodeDeploy
Select your use case: CodeDeploy
Policies: AWSCodeDeployRole
I understand that there must be a buildspec.yml file in the root folder. I've tried using a few files I could find but they don't seem to work. What did I do wrong or how should I edit the buildspec file to do what I want?
Update
Thanks to #Milan Cermak. I understand I need to do:
CodePipeline:
Stage 1: Source: hook with GitHub repo. This one is working.
Stage 2: Build: use CodeBuild to grab only the wanted folder using a buildspec.yml file in the root folder of the repo.
Stage 3: Deploy: use
Action Provider: S3
Input Artifacts: OutputArtifacts (result of stage 2).
Bucket: the bucket that hosts the static website.
CodePipeline works. However, the output contains only files (.html) not folders nested inside the public folder.
I've checked this and figured how to remove path of a nested folder with discard-paths: yes but I'm unable to get all the sub-folders inside the ./<path>/public folder. Any suggestion?
CodeBuild use buildspec, but CodeDeploy use appspec.
Is there any appspec file?
You shouldn't use CodeDeploy, as that's a service for automation of deployments of applications, but rather CodeBuild, which executes commands and prepares the deployment artifact for further use in the pipeline.
These commands are in thebuildspec.yml file (typically in the root directory of the repo, but it's configurable). For your use case, it won't be too complicated, as you're not compiling anything or running tests, etc.
Try this as a starting point:
version: 0.2
phases:
build:
commands:
- ls
artifacts:
files:
- public/*
The phases section is required, that's why it's included (at least, thanks to the ls command, you'll see what files are present in the CodeBuild environment), but it's not interesting for your case. What is interesting is the artifacts section. That's where you define what is the output of the CodeBuild phase, i.e. what gets passed further to the next step in the pipeline.
Depending on how you want to have the files structured (for example, do you want to have the public directory also in the artifact or do you only want to have the files themselves, without the parent dir), you might want to use other configuration that's possible in the artifacts section. See the buildspec reference for details.
Remember to use the output artifact of the CodeBuild step as the input artifact of the Deploy to S3 step.
Buildspec is for CodeBuild as t_yamo pointed out.
You are using CodeDeploy which uses an appspec.yml file, which looks something like this for my config.
version: 0.0
os: linux
files:
- source: /
destination: /path/to/destination
hooks:
BeforeInstall:
- location: /UnzipResourceBundle.sh
ApplicationStart:
- location: /RestartServer.sh
timeout: 3600
UnzipResourceBundle.sh is just a bash script which can be used to do any number of things.
#!/bin/bash
// Do something
You can find a sample for the AppSpec.yml file from Amazon Documentation here - https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-example.html#appspec-file-example-lambda
CodePipeline recently announced a deploy to S3 action: https://aws.amazon.com/about-aws/whats-new/2019/01/aws-codepipeline-now-supports-deploying-to-amazon-s3/
i'm learning AWS-based continuous integration and have a basic CodeCommit → CodePipeline → CodeDeploy workflow setup per the tutorial. when i push to CodeCommit, CodePipeline triggers a build in CodeDeploy, however it does not find my appspec.yml manifest because it's not in the root of my repository.
how do i configure CodePipeline/CodeDeploy to look for appspec.yml under the subdirectory inst/aws/ instead of the repository root? i don't want to restructure my repo (which also has non-AWS content) just to accommodate a vendor-specific preference.
By default CodeDeploy agent looks at the root of your bundle for appspec.yml. Unfortunately you can't configure the location of the appspec.
You can do that with the use of AppSpecTemplatePath.
The file name of the AppSpec file stored in the pipeline source file
location, such as your pipeline's CodeCommit repository. The default
file name is appspec.yaml. If your AppSpec file has the same name and
is stored at the root level in your file repository, you do not need
to provide the file name. If the path is not the default, enter the
path and file name.
Source: https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-ECSbluegreen.html#action-reference-ECSbluegreen-type