I am trying to switch over to using CodeBuild to build my code so I can then easily push it to my EC2 instances instead of manually building and copying.
I can manually run ant on my station and all will build as it should.
I am now trying to use the AWS CodeBuild console to try this.
I zipped up my source code files and put it in an S3 bucket and put its location in the source fields of AWS CodeBuild. I have the build.xml in this same bucket and I also put the build.xml in the base of the codes zip file. In the build commands I put "ant".
I assume that the build.xml needs to go somewhere else?
Do I need more then just "ant" in the build commands? That is all I use when i manually build the project.
From what I have read i should be able to zip up my code , put it in the S3 location and CodeBuild will extract it and build it correct?
Also, under "Environment: How to Build" - what is the "Output files" section for? It's not for the artifacts that are built correct?
Any other tips or tricks? I am very new to all of this so any help is appreciated! I just learned about ant this week. This is building a rather large project with many classes being built - Will this cause an issue? Like I stated earlier - I do have it building file if I run it manually on my system.
Here is the error I get when I build through Code Build:
[Container] 2019/03/21 15:32:27 Entering phase BUILD
[Container] 2019/03/21 15:32:27 Running command ant
Buildfile: build.xml does not exist!
Build failed
I figured out my issue - I zipped the build files from the folder level and not the root level. I re-zipped and it can now see the build.xml.
I built again with these changes and it looks like I am close! It failed for the following -
https://forums.aws.amazon.com/ 2019/03/21 20:57:13 Expanding myapp.jar
https://forums.aws.amazon.com/ 2019/03/21 20:57:13 Skipping invalid artifact path myapp.jar
https://forums.aws.amazon.com/ 2019/03/21 20:57:13 Phase complete: UPLOAD_ARTIFACTS Success: false
https://forums.aws.amazon.com/ 2019/03/21 20:57:13 Phase context status code: CLIENT_ERROR Message: no matching artifact paths found
Isn't myapp.jar what the build is creating?
I am very confused as to what the Artifact/name should be - isn't this what is being created from the build? It is asking for an ARN - how can there be an ARN for it when it is not created?
Also very confused as to what the Environment/Output files is? It is required but I have no idea what should go in this field? It states that output files can not be empty. Does this mean it wants all the class files that are being built? If so then this build is creating over 30 class files in multiple locations - that is a ton to list.
Thanks
Ernie
I have it working! I will post my findings for others going that might be struggling -
So I figured out that the "Outputs" means what are all the files and/or directories that you want to go into your final artifact after all is built.
I have two directories that I want in the final jar artifact. One is WebContent and the other is build. They both have multiple sub-directories. I put "WebContent/*,build/*" in the output files field. It gave me a jar artifact but when I open the jar it did not have any sub-directories. In order to get it to include all sub-directories I had to make the output files field with "WebContent/**/*,build/**/*". All sub-directories are now in the zip and it appears as if the build was successful.
Hopefully this can help others out.
Now on to creating a script for this and also getting this to work from GitLab.
Related
The CodeDeploy agent did not find an AppSpec file within the unpacked revision directory at revision-relative path "appspec.yml".
The revision was unpacked to directory "/opt/codedeploy-agent/deployment-root/0bb5a5aa-5894-4575-a69c-a7a4e79b4cdf/d-HQ5GBC7SW/deployment-archive"
The AppSpec file was expected but not found at path "/opt/codedeploy-agent/deployment-root/0bb5a5aa-5894-4575-a69c-a7a4e79b4cdf/d-HQ5GBC7SW/deployment-archive/appspec.yml".
I had the same problem and the other answer helped me reach the right conclusion. In my situation, I had the appspec.yml file in my git repo, but I forgot to add it to the artifact files section. As a result, the appspec.yml wasn't included in the zip and so the deployment step couldn't find it.
In your buildspec.yml, add:
artifacts:
files:
- appspec.yml
- ... other files to include in your build ...
I had some other errors in my deployment configuration too. Looking at the bottom of the log file helped discover them:
less /var/log/aws/codedeploy-agent/codedeploy-agent.log
At one point, my EC2 instance also hung when trying to run a deployment and stopping and restarting the codedeploy agent didn't help. I had to completely restart the EC2 instance.
These docs where helpful: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html
Are you putting the appspec.yml file at root level of your folder bundle? If yes, how are you creating the bundle? If you are just creating a .zip then you need to make sure you are adding the files to the zip instead of a folder.
Adding onto Aura's answer which solved it for me. Thank you again Aura! For mac users:
To create a zip with all of the files, open up the folder you want to zip in finder, highlight all of files (use command A), right click and then select "Compress XYZ items".
The mistake you might be making is right clicking on the folder itself and clicking "Compress FolderName".
The problem with compressing the entire folder is that the "unpacked directory" ends up being a directory itself. Hence, when the CodeDeploy agent goes looking for the appspec.yml file, it will see something like this "FolderName/" rather than the contents of the FolderName (which should include an appspec.yml).
Hope this extra detail helps.
#It seems to have a successful deployment at some point of time.
It's looking for the most recent deployment and if it's not available above error will occur.
Go into /opt/codedeploy-agent/deployment-root/deployment-instructions/ and delete all the files in there.
#Then it won't look for this last deploy.
Not the most useful answer. But I had this problem as well as codebuild could not find the scripts defined in appspec.yml I spent a whole day and then at the end just started rebooted the ec2 and it was able to find the scripts. 😖
Issue can be fixed by:
Edit your aws codepipeline.
Edit the codedeploy configuration.
select "SourceArtifact" in input artifacts. (This will deploy all the source code to your desired server and location along with appspec.yml file.)
Using GoCD, how can I define a stage's entire workspace (as a single artifact) for the next stage? This would highly simplify my setup, in which the second stage needs to fetch many different artifacts from the previous one.
I have tried the following artifact declarations:
Artifact source = .
This causes an error already during the upload in the first stage:
[go] The rule [.] cannot match any resource under [pipelines/mypipeline]
[go] [go] Uploading finished. Failed to upload [.]
Artifact source = *
This does not cause errors, but causes a separate upload for each directory in the root folder, instead of a single artifact of the entire workspace. As a result, I still need to fetch multiple concrete artifacts, instead of one big workspace artifact.
[go] Uploading artifacts from /var/lib/go-agent/pipelines/mypipeline/.svn to [defaultRoot]
[go] Uploading artifacts from /var/lib/go-agent/pipelines/mypipeline/cruise-output to [defaultRoot]
[go] Uploading artifacts from /var/lib/go-agent/pipelines/mypipeline/<dir1> to [defaultRoot]
[go] Uploading artifacts from /var/lib/go-agent/pipelines/mypipeline/<...> to [defaultRoot]
[go] Uploading artifacts from /var/lib/go-agent/pipelines/mypipeline/<dirN> to [defaultRoot]
I could probably zip everything myself with another task and define that as an artifact, but with GoCD already zipping and unzipping on its onw, I thought there must be a simpler solution to my problem.
I am not an expert, just a newbie with GoCD, but having gone through the same pain of trial and error, I'll share the syntax I found to work from my experience. For build artifacts in an upstream pipeline:
Source: #{BUILD_DIR}/*.whl - it seems wild card syntax works when defining build artifacts; NOTE: the values between the curly braces {XXX_XX} are pipeline parameters.
Destination: #{ARTIFACT_DIR} - I prefer to put all my artifacts in a separate folder as it is easier to fetch. GoCD will zip and transfer the folder and unzip it in the downstream pipeline.
In downstream pipeline or another stage in the same pipeline when retrieving artifacts using "Fetch Artifact" task:
Source: #{ARTIFACT_DIR} - I use only the directory name to fetch the entire directory
Destination ./ - the fetched directory will be extracted in current folder in this case
NOTE: The above syntax is for fetching directory of artifacts only where the Source is a file(Not a directory) checkbox in the "Fetch Artifact Task" is not selected.
I was not able to fetch artifact files when I tried parametrizing a file name with pipeline parameters or environment variables (for example my file artifact contained a dynamically generated version) and I did not find anything in the GoCD documentation or examples suggesting it is possible. The docs have example of downloading a file name that is a constant and does not change.
Please share examples of parametrizing file artifacts if possible or suggested workarounds if possible at all.
So I am attempting to setup CodeDeploy for my application and I keep getting an error during the BeforeInstall part of the deployment. Below is the error.
Error Code UnknownError
Script Name
Message No such file or directory - /opt/codedeploy-agent/deployment-root/06100f1b-5495-42d9-bd01-f33d59fb5deb/d-NL5K1THE8/deployment-archive/appspec.yml
Log Tail
I assumed this meant the YAML file was in the wrong place. However it is in the root directory of my revision. I have tried using a simple AppSpec file like so instead of a more complex one.
## YAML Template.
---
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/www
More or less since this is a first deployment I want it to add all files in the revision to the public directory on the web server.
I am tearing my hair out over this and I feel it is a simple issue. I have the IAM policies and roles correct and I have CodeDeploy setup and running on my instance I am trying to deploy to.
It seems to think you had a successful deploy at some point.
Go into /opt/codedeploy-agent/deployment-root/deployment-instructions/ and delete all the files in there. Then it won't look for this last deploy.
I just had this SAME problem and I figured it out! Make sure your AppSpec file has the right EXTENSION! I was using yaml and not yml, now everything works perfectly.
I made it work like this:
I had a couple of failed deployments for various reasons.
The thing is that CD keeps in the EC2 instance and in the path /opt/codedeploy-agent/deployment-root/​ a folder named by the ID of the failed deployment [a very long alphanumeric sting] .
Delete this folder and create a new deployment [from the aws UI console] and redeploy the application. This way the appspec.yml file that is in the wrong place will be deleted.
It should now succeed.
Extra Notice:
CD does not rewrite files [that have not been created by it's specific deployment]
CodeDeploy does not deploy in a folder that there is already code[files] as it does not want to interfere with different CD deployments and/or other CI/CD tools [like Jenkins].
It only deploys in a path that has already deploy code with the specific deployment.
You can empty the folder where your deployment want to happen and redeploy your code via CD.
When you login to the host, do you see the appspec.yml file in the directory there? If not are you positive it has been checked in with the rest of your deployed code?
Just encountered this issue too. In my case, the revision zip file extracts into a directory when deployed. Because of that /opt/codedeploy-agent/deployment-root/xxx/xxx/deployment-archive contains the parent directory of my revision files (instead of the actual revision files).
The key is to compress your revision without the parent directory. In mac terminal,
cd your-app-directory-containing-appspec
zip -r app.zip .
When I build my cloudbees project it gives the following error:
ERROR: No such file /scratch/jenkins/workspace/glassfish-myruby/pom.xml
Where my project name is glassfish-myruby.
Go to the job you created and click on the "Workspace" link on the left. This will show you how your project was checked out. The pom may be in a different place than you expect.
I ran into a similar problem. My project was building fine, then suddenly stopped with the same error:
Parsing POMs
ERROR: No such file /scratch/jenkins/workspace/MyMoney/pom.xml
Perhaps you need to specify the correct POM file path in the project configuration?
[cloudbees-deployer] Skipping deployment as build result is FAILURE
Finished: FAILURE
I realized the only thing I had changed was adding a GitHub page (as described in http://pages.github.com/)
For reasons I don't understand, this was causing my project to check out from the gh-pages branch instead of master, as reflected in the console output:
Commencing build of Revision 88d8d7ee822e02aa2cdae11e07265eb7a8403fed (origin/gh-pages)
This was also reflected in the jenkins workspace (it contained the GitHub pages files, rather than my project files).
I'm sure there is a way to get this to work AND use GitHub pages, but for me the GitHub page was just an experiment, so I deleted the branch using:
git push origin --delete gh-pages
I then also did another commit to make sure all was OK and rebuilt.
The project is now checking out from master again and building OK.
I'm using Jenkins to build a maven 2 project. As part of the build a couple of jar files get generated in the target directory. I would like Jenkins to archive/copy a specific jar from the target location to a custom folder.
How can I achieve this ? I've tried using the 'Archive the artifacts' post build option but it does not allow me to select the file under target. I get a error message saying such a location does not exist.
I'm new to Jenkins so any help is appreciated.
Thanks.
Sirius
You may have your file specification or the base directory for the artifacts wrong. From the help text:
Can use wildcards like 'module/dist/*/.zip'. See the #includes of Ant fileset for the exact format. The base directory is the workspace.
So you'll need to figure out where your target directory is relative to the workspace directory.
The archive feature copies/saves your build artifacts out of the workspace into the build's individual directory. You cannot specify where it puts them. That said, I would probably leave archiving turned on if you'll ever need to refer back to a previous version.
You can use a script build step to create the dir if it does not exist and perform the copy.
But you have not said why you want to move the artifacts around. If it is to make them available to other projects, you should look instead at the Copy Artifact build step.