Currently our company is creating individual software for B2B customers.
Some applications can be used for multiple customers.
Usually we can host the application in the cloud and deploy everything with Docker.
Running a GitLab pipeline and deploying etc. is fine for that.
Now we got some customers who rely on an external installation.
Since some of them still use Windows Server (2008 tho), I can not install a proper Docker environment on there and we need to install an Apache Tomcat and run the application inside the tomcat.
Question: How to deal with that? I would need a pipeline to create a docker image and a war file.
Simply create two completely independent pipelines?
Handle everything in a single pipeline?
Our current gitlab-ci.yml file for the .war
image: maven:latest
variables:
MAVEN_CLI_OPTS: "-s settings.xml -q -B"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
cache:
paths:
- .m2/repository/
- target/
stages:
- build
- test
- deploy
build:
stage: build
script:
- mvn $MAVEN_CLI_OPTS compile
test:
stage: test
script:
- mvn $MAVEN_CLI_OPTS test
install:
stage: deploy
script:
- mvn $MAVEN_CLI_OPTS install
artifacts:
name: "datahub-$CI_COMMIT_REF_SLUG"
paths:
- target/*.war
Using to separate delivery pipeline is preferable: you are dealing with two very installation processes, and you need to be sure which one is running for a given client.
Having two separate GitLab pipeline allows for said client to chose the right one.
Related
I have a django application running locally and i've set up the project on CircleCi with python and postgres images.
If I understand correctly what is happening, CircleCi would use the images to build a container to test my application with code database.
Then I'm using the job heroku/deploy-via-git to deploy it to Heroku when the tests are passed.
Now I think Heroku is using some images too to run the application.
I would like to get the image used by heroku to run my site locally on another machine.
So pull the image then push it to Docker Hub and finally download it back to my computer to only have to use a docker compose up.
Here is my CircleCi configuration's file
version: 2.1
docker-auth: &docker-auth
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_PASSWORD
orbs:
python: circleci/python#1.5.0
heroku: circleci/heroku#0.0.10
jobs:
build-and-test:
docker:
- image: cimg/python:3.10.2
- image: cimg/postgres:14.1
environment:
POSTGRES_USER: theophile
steps:
- checkout
- run:
command: pip install -r requirements.txt
name: Install Deps
- run:
name: Run MIGRATE
command: python manage.py migrate
- run:
name: Run loaddata from Json
command: python manage.py loaddata datadump.json
- run:
name: Run tests
command: pytest
workflows:
heroku_deploy:
jobs:
- build-and-test
- heroku/deploy-via-git:
requires:
- build-and-test
I don't know if it is possible, if not, what should be the best way to proceed ? (I assume that there is a lot of possibilites)
I was considering to build an image from my local directory with docker compose up then use this image direclty on CircleCi, then i would be able to use this image an on other computer. But building images into images with CircleCi seems really messy and I'm not sure how I should proceed.
I've tried to pull images from Heroku but it seems I can only pull the code or get/modify the database but I can't get the image builds itself.
I hope this question is relevant and clear, as the CircleCi and Heroku documentation seems not clear and it's my first post on stackoverflow !
Thanks in advance
Heroku's platform is proprietary, so we can't be sure how it works internally.
We know that their stacks are based on Ubuntu LTS releases, and we know that they use open-source buildpacks to compile application slugs from source code, but details about the underlying infrastructure are murky. They certainly don't provide base images like heroku/python:3.11.0 for you to download.
If you want to use the same image locally, on CircleCI, and Heroku, a better option would be to start deploying with Heroku's Container Registry instead of Git. This allows you to build an image locally, push it into the container registry, and release it as the next version of your application.
I suggest you read the entire documentation page linked above, but the short version is:
Log into the container registry using the Heroku CLI:
heroku container:login
Assuming you already have a Dockerfile for your application, build and push an image:
heroku container:push web
In this case we are building from Dockerfile and pushing the resulting image to be used as a web process.
Release your application:
heroku container:release web
That's a basic Docker deployment from your local machine, and even if that's not your final plan I suggest you start by getting that working.
From there, you have options. One option would be to move this flow to CircleCI—continue to build images there, but have CircleCI push the resulting container to Heroku's Container Registry.
Another option might be as you suggest in your question: to build images locally and use them with both CircleCI and Heroku.
I am trying to create a CI/CD pipeline to build war file and deploy it to Tomcat Container from GitLab.
I am using a Maven image to build the project. Once the war file is created, I would like to copy it to some folder so that from there it can be copied to the tomcat server, in a container, webapps directory.
My current approach and goal is to use a Dockerfile in my project. From a Tomcat image, run Tomcat using the project war. I tried using ADD in the Dockerfile but the directory paths where the war resides, $CI_PROJECT_DIR, is not where the Dockerfile is looking.
The following is the ".gitlab-ci.yml" file.
stages:
# Build project
- build
- package
# Build and deploy mailService
#- deploy
variables:
# Variables that can be used throughout the pipeline are defined here.
# Maven variables
MAVEN_CLI_OPTS: '-s /appdir/.m2/settings.xml'
MAVEN_PATH: '/appdir/opt/apache-maven-3.8.4/bin'
IMAGE_PATH: 'gitlab-registry.gs.mil/gteam-development/docker'
project-build:
image: ${IMAGE_PATH}/maven
services:
- tomcat:latest
stage: build
script:
- ${MAVEN_PATH}/mvn ${MAVEN_CLI_OPTS} clean package
- ls -als
- ls -als target
build docker image:
stage: package
image: docker
services:
- docker:dind
script:
- echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER $CI_REGISTRY --password-stdin
- docker build -t $CI_REGISTRY_IMAGE .
- docker push $CI_REGISTRY_IMAGE
tags:
- dind
The following is the "Dockerfile" I am using to run Tomcat using the image. I need to copy my war file to the Tomcat webapps folder and build another image.
FROM tomcat:latest
LABEL maintainer=”Jacquelyne Wilson”
# ADD $CI_PROJECT/target/geoint-rfi-data-api.war /usr/local/tomcat/webapps/
EXPOSE 8080
CMD [“catalina.sh”, “run”]
NOTE: These images I have in our Gitlab Container Registry. Hope this is enough information.
This is my first experience creating Gitlab CI pipeline. My apologies if my terms and approach is not ideal.
You can provide the WAR file built in your project-build job as an artifact, so it is available in the following job. By the way, you should probably not use spaces in the job name.
This could look like the following:
project-build:
image: ${IMAGE_PATH}/maven
services:
- tomcat:latest
stage: build
artifacts:
paths:
- /path/to/your/war
script:
- ${MAVEN_PATH}/mvn ${MAVEN_CLI_OPTS} clean package
- ls -als
- ls -als target
And in the docker build job, you can then copy the WAR file from the artifact to the docker build context path so it can be ADDed to your image.
Hope this helps :)
Best regards
Andreas
amplify auto-ignores aws-exports.js in .gitignore possibly simply because it may change frequently and is fully generated - however maybe there are also security concerns?
For this project my github project is private so that is not a concern, but I am wondering for future projects that could be public.
The reason I ask is because if I want to run my app setup/build/test through github workflows then I need this file for the build to complete properly on github machines?
Also I appear to need it for my amplify CI hosting to work on amplify console (I have connected my amplify console build->deploy to my github master branch and it all works perfectly but only when aws-exports.js is in source control).
Here is my amplify.yml, I am using reason-react with nextjs, and my amplify console is telling me I have connected to the correct backend:
version: 1
frontend:
phases:
preBuild:
commands:
- yarn install
build:
commands:
- yarn run build
artifacts:
baseDirectory: out
files:
- '**/*'
cache:
paths:
- node_modules/**/*
I'm new to Gitlab (and I only know the basic features of git : pull, push, merge, branch...).
I'm using a local DynamoDB database launched with docker run -p 8000:8000 amazon/dynamodb-local to do unit testing on my Python project. So I have to launch this docker container in the Gitlab CI/CD so that my unit tests work.
I already read the documentation on this subject on the site of gitlab without finding an answer to my problem and I know that I have to modify my gitlab-ci.yml file in order to launch the docker container.
When using Gitlab you can use Docker-in-Docker.
At the top of your .gitlab-ci.yml file
image: docker:stable
services:
- docker:dind
Then in your stage for tests, you can start up the database and use it.
unit_tests:
stage: tests
script:
- export CONTAINER_ID=$(docker run -p 8000:8000 amazon/dynamodb-local)
## You might need to wait a few seconds with `sleep X` for the container to start up.
## Your database is now here docker:8000
## Run your tests here. Database host=docker and port=8000
This is the best way I have found to achieve it and the easiest to understand
What I am trying to do is to enable Continuous delivery from GitLab to my compute engine on Google Cloude. I have Ubuntu 16.04 TSL running over there. I did install all components needed to run my project like: Swift, vapor, nginx.
I have manage to install Gitlab runner as well and created a runner whcihc is accessible from my gitlab repo. Everytime I do push on master the runner triggers. What happen is a failure due to:
could not create leading directories of '/home/gitlab-runner/builds/2bbbbbd/0/Server/Packages/vapor.git': Permission denied
If I change the permissions to chmod -R 777 It will hange on running for build stage visible on gitlab pipeline.
I did something like:
sudo chown -R gitlab-runner:gitlab-runner /home/gitlab-runner/builds
sudo chown -R gitlab-runner:gitlab-runner /home/gitlab-runner/cache
but this haven't help, the error is same Permission denied
Below you have my .gitlab-ci.yml
before_script:
- swift --version
stages:
- build
- deploy
job_build:
stage: build
before_script:
- vapor clean
script:
- vapor build --release
only:
- master
job_run_app:
stage: deploy
script:
- echo "Deploy a API"
- vapor run --name=App --env=production
environment:
name: production
job_run_frontend:
stage: deploy
script:
- echo "Deploy a Frontend"
- vapor run --name=Frontend --env=production
environment:
name: production
But that haven't pass to next stage eg. deploy. I had waited more then 14h for that but with out result.
And... I have few more questions:
Gitlab runner creates builds under location /home/gitlab-runner/builds/ in this location every new job have own folder. for eg. /home/gitlab-runner/builds/2bbbbbd/ in which is my project and the commands are executed. So what happens when the first one is running and I do deploy new version? the ports are blocked by the first instance and so on?
If I want to enable supervisor how do I do that with this when every time I deploy folder is different?
Can anyone explain or show me or point me to tutorial how do Continuous deployment with out docker?
How to start a service using GitLab runner
Thanks to long deep search I finally found an answer! The full article can be found above.
Briefly GitLab CI documentation recommends using dpl for deployment. Gitlab runner run test and process should end. The runner is designed to kill all created processes after finishing each build. The GitLab runner is unable to perform operations outside the catalogue.