What I am trying to do is to enable Continuous delivery from GitLab to my compute engine on Google Cloude. I have Ubuntu 16.04 TSL running over there. I did install all components needed to run my project like: Swift, vapor, nginx.
I have manage to install Gitlab runner as well and created a runner whcihc is accessible from my gitlab repo. Everytime I do push on master the runner triggers. What happen is a failure due to:
could not create leading directories of '/home/gitlab-runner/builds/2bbbbbd/0/Server/Packages/vapor.git': Permission denied
If I change the permissions to chmod -R 777 It will hange on running for build stage visible on gitlab pipeline.
I did something like:
sudo chown -R gitlab-runner:gitlab-runner /home/gitlab-runner/builds
sudo chown -R gitlab-runner:gitlab-runner /home/gitlab-runner/cache
but this haven't help, the error is same Permission denied
Below you have my .gitlab-ci.yml
before_script:
- swift --version
stages:
- build
- deploy
job_build:
stage: build
before_script:
- vapor clean
script:
- vapor build --release
only:
- master
job_run_app:
stage: deploy
script:
- echo "Deploy a API"
- vapor run --name=App --env=production
environment:
name: production
job_run_frontend:
stage: deploy
script:
- echo "Deploy a Frontend"
- vapor run --name=Frontend --env=production
environment:
name: production
But that haven't pass to next stage eg. deploy. I had waited more then 14h for that but with out result.
And... I have few more questions:
Gitlab runner creates builds under location /home/gitlab-runner/builds/ in this location every new job have own folder. for eg. /home/gitlab-runner/builds/2bbbbbd/ in which is my project and the commands are executed. So what happens when the first one is running and I do deploy new version? the ports are blocked by the first instance and so on?
If I want to enable supervisor how do I do that with this when every time I deploy folder is different?
Can anyone explain or show me or point me to tutorial how do Continuous deployment with out docker?
How to start a service using GitLab runner
Thanks to long deep search I finally found an answer! The full article can be found above.
Briefly GitLab CI documentation recommends using dpl for deployment. Gitlab runner run test and process should end. The runner is designed to kill all created processes after finishing each build. The GitLab runner is unable to perform operations outside the catalogue.
Related
I am very confused regarding how to set and access API secrets in a Next.js app within an AWS Amplify project.
The scenario is: I have a private API key that fetches data from an API. Obviously, this is a secret key and I don't want to share it in my github repo or the browser. I create a .env.local file and place my secret there.
API_KEY="qwerty123"
I am able to access this key in my code through using process.env.API_KEY
Here is an example fetch request with that API Key: https://developer.nps.gov/api/v1/parks?${parkCode}&api_key=${process.env.API_KEY}
This works perfectly when I run yarn dev and yarn build -> yarn start
This is the message I get when I run yarn start
next start
ready - started server on 0.0.0.0:3000, url: http://localhost:3000
info - Loaded env from /Users/tmo/Desktop/Code/projects/visit-national-parks/.env.local
The env is loaded and able to be called on my local machine.
However,
When I push this code to github and start the Build process in AWS Amplify, the app builds, but the API fetch calls do not work. I get a ````500 Server Error`````
This is what I have done to try and solve this issue:
Added my API_KEY in the Environment variables tab in Amplify
2. Update my Build settings
frontend:
phases:
preBuild:
commands:
- yarn install
build:
commands:
- API_KEY=${API_KEY} '#Added my API_KEY from the environment variables tab in Amplify`
- yarn run build
I am not sure what else to do. After building the app again, I still get 500 server error
Here is the live amplify app with the server error.
We're working on something similar right now. Our dev designed it so it reads an .env file.
frontend:
phases:
preBuild:
commands:
- yarn install
build:
commands:
- echo API_KEY=$API_KEY >.env
- echo OTHERKEY=$OTHER_KEY >> .env
- yarn run build
We were able to pick it up and pass it to AWS' DynamoDB Client SDK.
Not sure if it's your call or not, but yarn can be fickle in our Amplify projects sometimes, so we usually resort to using npm if it starts acting up.
TL;DR The command npm run build is taking forever to run on the Amazon EC2 [Ubuntu] instance when I tried running it explicitly by making an SSH. Meanwhile, when I try to create a deployment using CodeDeploy, the deployment takes a good 1 hour time and succeeds but the build folder doesn't get populated, hence I am unable to view my website on the public URL of the EC2 instance. Also, the instance reachability check fails every time after I try to run the command explicitly, and then I have to start and then stop the ec2 instance again! Woof!
Hello everyone, I am trying to deploy my MERN Stack application to AWS but I am stuck now!
Current Progress:
Added both Nginx configs.[Attaching image below]
Nginx is running and there is no problem there!
Added build-app.sh in appspec.yml in the root directory. [View code below]
#!/bin/bash
#clear build directory
cd /home/ubuntu/badlav-app/badlav-client
sudo rm -rf build
sudo mkdir build
#client (Generates a new `build` directory)
cd /home/ubuntu/badlav-app/badlav-client
sudo sh set-prod-env-aws.sh
sudo rm -rf node_modules
sudo npm i
sudo npm run build
#server
cd /home/ubuntu/badlav-app/badlav-server
sudo sh set-prod-env.sh
#back to root
cd /home/ubuntu
appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/badlav-app
hooks:
BeforeInstall:
- location: scripts/build-app.sh
runas: root
Using the above appspec.yml file, the deployment using CodeDeploy succeeds but didn't populate the build folder within /home/badlav-app/badlav-client/build.
So I tried to debug on my own and started running the commands one by one by myself after SSH(ing :P) into the EC2 instance. But when I reach npm run build, the instance just hangs forever. After being exhausted, I have no option left, I terminate the task. Now, when I view my instance on the AWS Console, it has gone berserk! The instance reachability check fails! The only way, I get my instance back is by stopping it and starting it again.
Since I am new to CI/CD, please don't judge my appspec.yml. It'd be great if anyone of you could suggest a better way, thanks for that! :)
To sum up, I want to be able to create a deployment using AWS CodeDeploy, but due to this npm run build taking so much time and hanging my server(instance reach check fails!), I am unable to do so. Moreover, I am not even sure whether npm run build is a problem at all!
I would be more than happy to share any further details/screenshots in order to support my question. Please ask over.
Thanks in advance!
/etc/nginx/nginx.conf/etc/nginx/conf.d/default.conf
If you're using EC2 free tier, the chance is that the instance may have low spec and memory (t2.nano has 0.5G and t2.micro has 1G of memory).
Maybe npm run build consumes all of the memory space.
I often face the same problem with my vue project.
Solution: Do NOT use free tier for medium and large projects. Upgrade your plan and use better instances e.g. t2.medium
Currently our company is creating individual software for B2B customers.
Some applications can be used for multiple customers.
Usually we can host the application in the cloud and deploy everything with Docker.
Running a GitLab pipeline and deploying etc. is fine for that.
Now we got some customers who rely on an external installation.
Since some of them still use Windows Server (2008 tho), I can not install a proper Docker environment on there and we need to install an Apache Tomcat and run the application inside the tomcat.
Question: How to deal with that? I would need a pipeline to create a docker image and a war file.
Simply create two completely independent pipelines?
Handle everything in a single pipeline?
Our current gitlab-ci.yml file for the .war
image: maven:latest
variables:
MAVEN_CLI_OPTS: "-s settings.xml -q -B"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
cache:
paths:
- .m2/repository/
- target/
stages:
- build
- test
- deploy
build:
stage: build
script:
- mvn $MAVEN_CLI_OPTS compile
test:
stage: test
script:
- mvn $MAVEN_CLI_OPTS test
install:
stage: deploy
script:
- mvn $MAVEN_CLI_OPTS install
artifacts:
name: "datahub-$CI_COMMIT_REF_SLUG"
paths:
- target/*.war
Using to separate delivery pipeline is preferable: you are dealing with two very installation processes, and you need to be sure which one is running for a given client.
Having two separate GitLab pipeline allows for said client to chose the right one.
I have just started working with AWS. I am trying to deploy a nodejs application using codeship and AWS codedeploy. I am successful in deploying the application from codeship to Ec2 instance. But the problem is that I am not able to run the hooks file in appspec.yml. My appspec.yml is given below:
---
version: 0.0
os: linux
files:
- destination: /home/ec2-user/node-project
source: /
hooks:
ApplicationStart:
- location: bin/app-start.sh
runas: root
timeout: 100
In app-start.sh I have:
#!/bin/bash
npm install
The app-start.sh never works and node-modules are never installed. I have also tried to debug in the logs path(/var/log/aws/codedeploy-agent/codedeploy-agent.log) for code-deploy but there is no error and warning.I have also tried multiple things but nothing is working.
The project is successfully installed in Ec2 instance but appspec.yml never launches app-start.sh. Any help would be appreciated.
The issue is that you're moving the files to /home/ec2-user/node-project, which happens before your app-start.sh gets run at the ApplicationStart lifecycle hook. You need to cd into the right directory before running npm install.
Updated ApplicationStart scripts:
#!/bin/bash
cd /home/ec2-user/node-project
npm install
# You'll need to start your application too.
npm start
As an aside, you may want to use the AfterInstall lifecycle hook to run npm install just for organization purposes, but it will have no functional different.
I try to connect together our cicleCI with browserstack and run our integration_test and unit tests not only with PhantomJS but on real Firefox and Internet Explorer as well, using Browserstack service.
I try to configure browserstack-cli. I can run the test from circleci via tunnel on Browserstack, but never report back to circleci server.
Could you please share your experience if you already played with this stack? Thank you very much!
The solution is to use BrowserStackLocal and browserstack-cli tools together. The 64bit linux version of BrowserStackLocal builds up the tunnel from circleCI server to Browserstack server. After that we can use browserstack-cli to launch browsers and running tests from testem.
Download BrowserStackLocal
and insert in .browserstack folder in your project.
64bit linux version of BrowserStackLocal: http://www.browserstack.com/local-testing (Binnaries)
Create a script,
which will run and create settings for browserstack-cli. You have to setup global variables in circleCI, and you can keep your access details there secretly. Let's call this file runthis.sh and save in .browserstack folder. This script will run your BrowserStackLocal binary as well, so the tunnel will be exist.
#!/bin/bash
echo "{\"username\":\"`echo $BS_USER`\", \"password\":\"`echo $BS_PASSWORD`\", \"privateKey\": \"`echo $BS_KEY`\", \"apiKey\":\"`echo $BS_KEY`\"}" >> ~/.browserstack/browserstack.json
./.browserstack/BrowserStackLocal $BS_KEY &
CircleCI config
(circle.yml) file mainly depend of your project. We have to copy .browserstack folder in home folder, install bower, browserstack-cli and testem.
An example:
machine:
timezone:
Pacific/Auckland
node:
version: v0.10.28
dependencies:
pre:
- mv ./.browserstack ~/
- sh ~/.browserstack/runthis.sh
post:
- bower install
- npm install browserstack-cli -g
- npm install testem -g
test:
override:
- PATH=$PATH:bin grunt integration_tests_cli; testem ci
- PATH=$PATH:bin grunt tests_cli; testem ci
Testem config:
testem.yml - Most of the part is depend of your project. Important in our case is launchers section.
framework: "qunit"
test_page: "tmp/index.html"
src_files:
- "tmp/assets/application.js"
- "tmp/tests.js"
- "tmp/integration_tests.js"
launchers:
bs_chrome:
command: browserstack launch chrome --attach
protocol: browser
timeout: 300
launch_in_ci:
- "PhantomJS"
- "bs_chrome"
launch_in_dev:
- "Chrome"
- "Firefox"
- "PhantomJS"
parallel: 2
So, if you update your project on github, circleci will launch your test and connect to browserstack and gonna use browsers there...