Which ways are there to get a gitlab pipeline triggered when a new artifact becomes available in a certain artifactory location? E.g. like Jenkins Artifactory Trigger: https://www.jfrog.com/confluence/display/JFROG/Jenkins+Artifactory+Plug-in or TeamCity: https://www.jfrog.com/confluence/display/JFROG/TeamCity+Artifactory+Plug-in#TeamCityArtifactoryPlug-in-TriggeringBuildsinReactiontoChangesinArtifactory.
There is no Artifactory GitLab CI plugin, like you listed for Jenkins and TeamCity, but you can use JFrog CLI to integrate GitLab CI and Artifactory.
Check out this blog:
https://jfrog.com/blog/gitlab-and-artifactory-on-your-mark-get-set-build/
Related
I'm trying to figure out a way to come up with a CI/CD pipeline for CloudFormation. We use Cloudformation Console directly to deploy our infrastructure and app to the cloud.
Does anyone have any examples of how they have created a CI/CD pipeline using Jenkins or other types of CI tools to do some type of linting, CI, version control, and artifact deployment to Artifactory (or similar toolset)? I'd like to execute a pipeline once a new version of the cloud formation templates is uploaded to Artifactory.
You can always use CodePipeline.
see docs:
CodePipeline
CI:
I am using GitHub, so before i can merge a pull request, my code must pass 3 tests.
Those tests are 3 Codebuilds containers that run tests.
CD:
After my code merged it invoke a CodePipeline that use mainly CodeDeploy and CodeBuild.
About your goal:
I'd like to execute a pipeline once a new version of the cloud formation templates is uploaded to Artifactory.
I don't really think you need a pipeline for this.
Let assume your artifacts uploaded to s3 bucket called artifact-bucket.
You can create a CloudWatch rule that will execute StepFunctions state machine when file added to
artifact-bucket.
see docs:
Trigger StepFunctions from S3
You can easily deploy stack with StepFunctions.
I want to implement CI/CD in AWS CodeCommit.
I know its possible manually to kickstart the process once the code reached CodeCommit. But I am using Azure DevOps Repo as my source code repo and want to automate the process.
The deployement is done using AWS SAM. I am looking for a method like; when I push a code to Azure Repo , it should reach the AWS CodeCommit and do the CI/CD without any further manual intervention.
Is there any way to do that?
Azure repos and CodeCommit are compliant with the git standard. The git standard allows you to specify multiple remotes. This is useful for if you were maintaining a mirror or, as in your use case, you need to do something in different environments.
You can read about setting multiple remotes here (provided by github; even though you’re not using github the process and commands should be the same).
Once you have your multiple remotes setup, you can configure your CI/CD pipeline to kick off its process to deploy your SAM template based on your push; when you push your code changes they will be sent to both your Azure repo, and your CodeCommit repo, and your CI/CD pipeline that is monitoring your CodeCommit repo will see the change and kickoff its execution.
Its worth pointing out that you’ll need to properly setup and configure your CI/CD pipeline. AWS provides a number of services to support this including AWS CodePipeline, AWS CodeBuild, and AWS CodeDeploy.
I have jenkins installed on an AWS EC2 Instance. My end state is whenever I commit cloudformation templates to my bitbucket repo, jenkins will automatically create/update/delete cf stack.
My thoughts on it was via aws cf cli commands in the jenkinsfile after installing aws cli on the server. Is there a better way of approaching this? I am new to devops
You could try AWS Cloudformation Plugin, but it's up for adoption and wasn't updated in 3 years.
I would say your approach with using the AWS cf cli commands looks safer.
I would say using CLI commands in your Jenkins pipelines is a good practice.
I am a fan of setting up Jenkins pipelines using the S3 artifact manager so your pipeline artifacts like CF templates are automatically available from S3. From there just execute the CloudFormation stack in a Jenkins task.
If your hosting Jenkins in AWS it's also nice to just add an IAM role to the instance to control what API actions Jenkins is allowed to run and use a plugin like CloudBees AWS CLI for your pipeline tasks.
Is there a way to integrate SonarQube with AWS CodeBuild?
Since I cannot install any plugins on AWS CodeBuild, I would like some advice on how to achieve this.
------EDIT--------
I think I can use https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner
I recently integrated SonarQube with Codebuild -
https://github.com/aws-samples/aws-codebuild-samples/blob/master/buildspecs/sonarqube.yml
This buildspec file gets you started. Dont forget to copy your sonar scanner properties to /sonar-scanner-3.0.3.778-linux/bin/
We're building an application for which we are using GitLab repository. Manual deployment of code to the test server which is Amazon AWS EC2 instance is tedious, I'm planning to automate deployment process, such that when we commit code, it should reflect in the test instance.
from my knowledge we can use AWS code-deploy service to fetch the code from GitHub. But code deploy service does not support GitLab repository . Is there a way to automate the code deployment process to AWS Ec2 instance through GitLab. or Is there a shell scripting possibility to achieve this? Kindly educate me.
One way you could achieve this with AWS CodeDeploy is by using the S3 option in conjunction with Gitlab-CI: http://docs.aws.amazon.com/codepipeline/latest/userguide/getting-started-w.html
Depending on how your project is setup, you may have the possibility to generate a distribution Zip (Gradle offers this through the application plugin). You may need to generate your "distribution" file manually if your project does not offer such a capability.
Gitlab does not offer a direct S3 integration, however through the gitlab-ci.yml you would be able to download it into the container and run the necessary upload commands to put the generated zip file on the S3 container as per the AWS instructions to trigger the deployment.
Here is an example of what your brefore-script could look like in the gitlab-ci.yml file:
before_script:
- apt-get update --quiet --yes
- apt-get --quiet install --yes python
- pip install -U pip
- pip install awscli
The AWS tutorial on how to use CodeDeploy with S3 is very detailed, so I will skip attempting to reproduce the contents here.
In regards to the actual deployment commands and actions that you are currently performing manually, AWS CodeDeploy provides the capability to run certain actions through scripts defined in the app-spec file depending on event hooks for the application:
http://docs.aws.amazon.com/codedeploy/latest/userguide/writing-app-spec.html
http://docs.aws.amazon.com/codedeploy/latest/userguide/app-spec-ref.html
http://docs.aws.amazon.com/codedeploy/latest/userguide/app-spec-ref-hooks.html
I hope this helps.
This is one of my old post. But I happened to find an answer for this. Although my question is specific to work with code deploy I would say there is no such need to use any aws requirements using gitlab.
We don't require Code Deploy at all. There is no need to use any external CI server like the team city or the jenkins to perform the CI from the GitLab anymore.
We need to add the .gitlab-ci.yml file in the source directory of the branch and write an .yml script in it. There are pipelines in the GitLab that will perform the CI/CD automatically.
The pipelines of the GitLab CI/CD looks more similar to the working functionality of Jenkins Server. using the YML script we can perform SSH on the EC2 instance and place the files in it.
An example of how to write the gitlab .yml file to ssh to ec2 instance is here https://docs.gitlab.com/ee/ci/yaml/README.html