With CodeBuild I compile my SpringBoot application and put the resulting jar in a folder called deploy/. In my buildspec.yml at the end I provide:
artifacts:
files:
- deploy/*
When I plug this in to CodePipeline, this creates a zip file in S3. But the problem is, that this zip file contains the root folder, i.e. deploy. So the structure of the zip file is:
deploy.zip
- deploy/
- appspec.yml
- app.jar
- ...
This leads to the problem that CodeDeploy cannot find the appspec.yml, becuase it's not looking inside the deploy folder. I've been trying to overcome this providing a zip file as artifact at the end of the build process, but then in the pipeline it just gets zipped again..
Any idea how to solve this would be much appreciated.
There is a new flag you can add in the artifacts section called base-directory to specify a base path for your artifacts package. This will allow you to truncate the deploy/ path out of your artifacts without losing the rest of the structure underneath.
artifacts:
files:
- **/*
base-directory: deploy
http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html
You can add discard-paths: yes
artifacts:
files:
- deploy/*
discard-paths: yes
Keep in mind that it discards ALL paths, including sub-directories
Source: http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html
Related
Here is my current buildspec file:
version: 0.2
phases:
build:
commands:
- pip install mkdocs
- mkdocs build
artifacts:
files:
- site/**/*
I want to get all the files and folders in the site folder.
With this file, the generated artifacts have this structure:
site
├── folder-1
│ └── file-1.html
└── folder-2
├── file-2.html
└── subfolder
└── file-3.html
I want to get rid of the site/ top folder and have all the subfolders and files going up at the "root" of the artifact folder, as it contains a static website I want to upload to an S3 bucket.
Therefore, I can't use the discard-paths argument as everything is going to be flattened, when I want to keep the subfolder hierarchy.
You can use aws s3 sync site/ s3://<bucket_name> in the post build
eg.
version: 0.2
phases:
build:
commands:
- pip install mkdocs
- mkdocs build
post_build:
commands: aws s3 sync site/ s3://<bucket_name>
It's better than pushing the artifact files. I tried it and it just adds the build artifacts into the S3. Using the sync command copies missing or outdated files, you can use the --delete option as well to remove files if there are not present in your new build.
View more info here: https://docs.aws.amazon.com/cli/latest/userguide/cli-services-s3-commands.html#using-s3-commands-managing-objects-sync
p.s. make sure to add a policy for codebuild to allow codebuild to interact with the S3 bucket
It sounds like you are after the base-directory option, which strips leading paths off the artifact files. In your case base-directory: site would seem to be what you want.
If the artifact handing isn't working for you, you can add another build step that zips up the files however you prefer, and then just add that .zip you created as the artifact instead.
This way you will end up with the same .zip output, only you'll have created it just the way you want instead of getting CodeBuild to do it for you.
I have a CodePipeline which has a few stages, both Plan and Check are CodeBuild projects:
Source (Github) > Plan > Check
The files from the github source are used in plan and then at the end of the plan I use output artifact to export all the files in CODEBUILD_SRC_DIR using this:
files:
- '$CODEBUILD_SRC_DIR/**/*'
name: PlanArtifact
In the export s3 bucket the files are outputted in the same pathways so CODEBUILD_SIR_DIR/all-files
I want to export the artifact so the s3 bucket is just immediatelly all-files, i want to omit the CODEBUILD_SRC_DIR as it changes with each codebuild and I just need the files in the .zip to use. I tried playing around with the below code but it doesn't seem to work:
base-directory: '$CODEBUILD_SRC_DIR'
Can anyone help?
If you remove the $CODEBUILD_SRC_DIR variable from the files it should work:
artifacts:
files:
- '**/*'
name: PlanArtifact
CodeBuild will always look for the artifacts in the original build location (ie. $CODEBUILD_SRC_DIR), so there is no need to include it into the path.
base-directory can be used if you only want one or more subdirectories as artifact. If you set it, CodeBuild will go to that directory first (starting from the original build location, so it should be a relative path, too) and then start to resolve the files pattern.
artifacts/files in the buildspec reference
artifacts/base-directory in the buildspec reference
I have angular client and Nodejs server deployed into one elasticBeanstalk.
The structure is that I put angular client files in 'html' folder and the proxy is defined in .ebextensions folder.
-html
-other serverapp folder
-other serverapp folder
-.ebextensions
....
-package.json
-server.js
Everytime when I do a release, I build angular app and put it into html folder in the node app, zip it and upload to elasticBeanstalk.
Now I want to move on to CICD. Basically I want to automate the above step, use two source(angular and node app), do the angular build and put it into html folder of node app and generate only one output artifact.
I've got to the stage where have separate pipeline works for each app. I'm not very familiar with AWS yet, I just have vague idea that I might need to use aws lambda.
Any help would be really appreciated.
The output artifact your CodeBuild job creates can be thought of as a directory location that you ask CodeBuild to zip as artifact. You can use regular UNIX commands to manipulate this directory before the packaging of the artifacts. Following 'buildspec.yml' is an example:
version: 0.2
phases:
build:
commands:
# build commands
#- command
post_build:
commands:
- mkdir /tmp/html
- cp -R ./ /tmp/html
artifacts:
files:
- '**/*'
base-directory: /tmp/html
Buildspec reference: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#build-spec-ref-syntax
I am a .net developer and using a .net core 2.x application to build and upload the release code to s3 bucket. later that code will be used to deploy to ec2 instance.
I am new to CI/CD using aws and in learning phase.
In order to create CI/CD for my sample project, I gone through some aws tutorials and was able to create the following buildspec.yml file. Using that file I am able to run the successful build.
The problem comes in the phase UPLOAD_ARTIFACTS. I am unable to understand how to create a zip file that will be used to upload to the s3 bucket specified in the build project.
My buildspec.yml files contains the following code, Please help me finding what is wrong or what I am missing.
version: 0.2
phases:
build:
commands:
- dotnet restore
- dotnet build
artifacts:
files:
- target/cicdrepo.zip
- .\bin\Debug\netcoreapp2.1\*
I think I have to add post_build and some commands that will generate the zip file. But don't know the commands.
Following is the output image from the build logs.
your file is good all what you need to do is to create a S3 bucket then
you need to configure your CodeBuild to generate zip (or not) your artifacts for you, and to store it to s3.
this is the step you need to configure:
Edit:
if you want all your files to be copied on the root of your Zip file you can use:
artifacts:
files:
- ...
discard-paths: yes
I am attempting to get CodePipeline to fetch my code from GitHub and build it with CodeBuild. The first (Source) step works fine. But the second (Build) step fails during the "UPLOAD_ARTIFACTS" part. Here are the relevant log statements:
[Container] 2017/01/12 17:21:31 Assembling file list
[Container] 2017/01/12 17:21:31 Expanding MyApp
[Container] 2017/01/12 17:21:31 Skipping invalid artifact path MyApp
[Container] 2017/01/12 17:21:31 Phase complete: UPLOAD_ARTIFACTS Success: false
[Container] 2017/01/12 17:21:31 Phase context status code: ARTIFACT_ERROR Message: No matching artifact paths found
[Container] 2017/01/12 17:21:31 Runtime error (No matching artifact paths found)
My app has a buildspec.yml in its root folder. It looks like:
version: 0.1
phases:
build:
commands:
- echo `$BUILD_COMMAND`
artifacts:
discard-paths: yes
files:
- MyApp
It would appear that the "MyApp" in my buildspec.yml should be something different, but I'm pouring through all of the AWS docs to no avail (what else is new?). How can I get it to upload the artifact correctly?
The artifacts should refer to files downloaded from your Source action or generated as part of the Build action in CodePipeline. For example, this is from a buildspec.yml I wrote:
artifacts:
files:
- appspec.yml
- target/SampleMavenTomcatApp.war
- scripts/*
When I see that you used MyApp in your artifacts section, it makes me think you're referring to the OutputArtifacts of the Source action of CodePipeline. Instead, you need to refer to the files it downloads and stores there (i.e. S3) and/or it generates and stores there.
You can find a sample of a CloudFormation template that uses CodePipeline, CodeBuild, CodeDeploy, and CodeCommit here: https://github.com/stelligent/aws-codedeploy-sample-tomcat/blob/master/codebuild-cpl-cd-cc.json The buildspec.yml is in the same forked repo.
Buildspec artifacts are information about where CodeBuild can find the build output and how CodeBuild prepares it for uploading to the Amazon S3 output bucket.
For the error "No matching artifact paths found" Couple of things to check:
Artifacts file(s) specified on buildspec.yml file has correct path and file name.
artifacts:
files:
-'FileNameWithPath'
If you are using .gitignore file, make sure file(s) specified on Artifacts section
is not included in .gitignore file.
Hope this helps.
In my case I received this error because I had changed directory in my build stage (the java project I am building is in a subdirectory) and did not change back to the root. Adding cd ..at the end of the build stage did the trick.
I had the similar issue, and the solution to fix the problem was "packaging directories and files inside the archive with no further root folder creation".
https://docs.aws.amazon.com/codebuild/latest/userguide/sample-war-hw.html
Artifacts are the stuff you want from your build process - whether compiled in some way or just files copied straight from the source. So the build server pulls in the code, compiles it as per your instructions, then copies the specified files out to S3.
In my case using Spring Boot + Gradle, the output jar file (when I gradle bootJar this on my own system) is placed in build/libs/demo1-0.0.1-SNAPSHOT.jar, so I set the following in buildspec.yml:
artifacts:
files:
- build/libs/*.jar
This one file appears for me in S3, optionally in a zip and/or subfolder depending on the options chosen in the rest of the Artifacts section
try using the version 0.2 of the buildspec
here is a typical example for nodejs
version: 0.2
phases:
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- npm install
- npm run build
post_build:
commands:
- echo Build completed on
artifacts:
files:
- appspec.yml
- build/*
If you're like me and ran into this problem whilst using Codebuild within a CodePipeline arrangement.
You need to use the following
- printf '[{"name":"container-name-here","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > $CODEBUILD_SRC_DIR/imagedefinitions.json
There was the same issue as #jd96 wrote. I needed to return to the root directory of the project to export artifact.
build:
commands:
- cd tasks/jobs
- make build
- cd ../..
post_build:
commands:
- printf '[{"name":"%s","imageUri":"%s"}]' $IMAGE_REPO_NAME $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json
artifacts:
files: imagedefinitions.json