first, I have two AWS Ubuntu servers that I can access by windows cmd using ssh and pem key. However, I have Jenkins installed on one of them.
I have a task which is to use Jenkins Pipeline from the first server to deploy simple files from GitHub repository into the second server. To clarify, the result in server 2 should be the files output from GitHub that is managed by Jenkins in server 1.
In other words:
Server 1: IP = XXX.XXX.XXX.XXX:8080 that contains Jenkins used to manage the output for server 2.
GitHub: NodeJs and HTML file that outputs "Hello World".
Server 2: IP = YYY.YYY.YYY.YYY should output "Hello World" by just using the IP as URL.
So, I,m new to this and looking for step by step explanation. Thanks!
Step 1: Install nginx server 2, config the nginx and know the location where the static files are served.
Step 2: Bring the ssh key to server 1, install nodejs, rsync in server 1 and also create a Jenkins pipeline which can pull the source code from Github, install and build that source code, and rsync to the location where the nginx serves content
I think this is a simple and easy way for you to setup. There are many better ways and you should research how Jenkins and Node work
Related
I followed the setup guide for Cloud9 IDE Workspace setup for Amazon MQ workshop. When I ran ./setup.sh
I got the following output but the setup could not be completed:
Installing jq...
Updating java to 1.8...
Updating maven to 3.6...
There was no error, just the 3 lines and I waited forever.
Apparently, the setup file tried to download maven from http://mirror.cc.columbia.edu/pub/software/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz but the site could not be reached.
The solution was to edit setup.sh file and replace the URL with the direct link from apache website
https://archive.apache.org/dist/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz
I have installed Jenkins and Apache to one instance of CentOS in aws. I have connected Jenkins with github but I am not able to access the application through url as it shows the following error.
You should add your website content to the directory /var/www/html/.
I need to copy files from jenkins directory to the mentioned one can you please help me how to copy app so that I can use it on browser.
I need to deploy my spring-boot application on compute engine in Google cloud platform. I have already created an instance and through SSH Apache and Maven have been installed. Further, war file has been uploaded into the bucket. Anybody can provide me with the remaining commands to deploy the war file on tomcat instance or any other cloud platforms with linux?
Thanks
Deploy in compute engine instance of google not substantially different from AWS, Azure or another linux host provider.
You just need an ssh connection to the remote machine and install the required software to compile, build, zip, deploy, etc
I will list some approaches from basic(manually) to advanced(automated):
#1 Bash scripting
unzip and configure git
unzip and configure java
unzip and configure maven
unzip and configure tomcat (this is not required if spring-boot is used)
configure the linux host to open 8080 port
create a script called /devops/pipeline.sh in your remote cloud linux instance, with the following steps
For war deployment :
# get the source code
cd /tmp/folder/3dac58b7
git clone http://github.com/myrepo.git .
# create war
mvn clean package
# move war to deploy tomcat folder
cp target/my_app.war /my/tomcat/webapps
# stop tomcat
bash /my/tomcat/shutdown.sh
# start tomcat
bash /my/tomcat/startup.sh
Or spring-boot startup
# get the source code
cd /tmp/folder/3dac58b7
git clone http://github.com/myrepo.git .
# create jar
mvn clean package
# kill or stop the application
killall java
# start the application
java $JAVA_OPTS -jar $jar_file_name
After push to git, just connect to you instance using ssh and execute
bash /devops/pipeline.sh
Improvements: Parametrize repository name, branch name, mvn profile, database credentials, create a tmp/uuid folder in every execution, delete the tmp/uuid after deploy,optimize start and stop of application using pid, etc
#2 Docker
Install docker in your remote cloud linux instance
Create a Dockerfile with all the steps for war or springboot (approach #1) and store it close to your source code (I mean in your git repository)
Perform a git push of your changes
Connect to your remote cloud linux instance using ssh:
Build your docker image: docker build ...
Delete previous container and run a new version:
docker rm my_app -f
docker run -d --name my_app -p 8080:8080 my-container-name
In the previous approaches, build operations are performed in the remote server. To do that, several tools are needed in that server. In the following approaches, build is performed in an intermediate server and just deploy is executed in the remote server. This is a a little better
#3 Local Build (an extra instance is required)
In this approach, the build is performed in the developer machine and its uploaded to some kind of repository. I advice you docker instead of just war or jar compilation.
In order to build and upload the docker image, one of these docker registries are required:
Docker simple registry
Amazon Elastic Container Registry (ECR)
Docker hub.
Harbor.
JFrog Container Registry.
Nexus Container Registry.
Portus
Azure Container Registry.
Choose one and install it in a new server. Configure your developer machine and your remote server to point to your new docker registry.
Final steps are:
Perform a docker build in your developer machine after. This will create a new docker image of your java application (tomcat+war or springboot jar)
Upload your local image to your new docker registry with something like:
docker push example.com/test-image
Connect to your remote cloud linux instance using ssh and just download the docker image
docker pull example.com/test-image
In the remote server, just start your new downloaded image with docker run...
#4 Use a continuous integration server (an extra instance is required)
Same to the #3 but not in the developer machine. All the steps are performed in another server called: Continuous integration server.
#4.1 Use a continuous integration server (an extra instance is required)
Install Jenkins or another Continuous integration server in the new instance
Configure plugins and other required things in jenkins in order to enable webhook url : https://jrichardsz.github.io/devops/configure-webhooks-in-github-bitbucket-gitlab
Create a job in jenkins to call the script of approach #1 or execute docker commands of approach #2. If you can, Approach #3 would be perfect.
Configure your SCM (github, bitbucket, gitlab, etc) to point to the webhook url published by Jenkins.
When you are ready to deploy, just push the code to your scm, jenkins will be notified and will execute the previous created job. As you can see, there is no human required to deploy de application in the server(With the exception of developer push)
Note: At this point, you could migrate the scripts of approaches #1 and #2 to :
Jenkins pipeline script
https://gist.github.com/jrichardsz/a62e3790c6db7654808528bd5e5a385f#file-jenkins_scripted_pipeline_java_mvn_basic-js
Jenkins declarative pipeline
https://gist.github.com/jrichardsz/a62e3790c6db7654808528bd5e5a385f#file-jenkins_declarative_pipeline_hello_world-js
These are more advanced and scalable approaches to mapping all the commands and configurations required from the beginning to the deployment.
#5 Advanced (Sysadmin team or extra people and knowledge are required )
More instances and technologies will be required.
Kubernetes
Ansible
High availability / Load balancer
Backups
Configurations Management
And more automations
This will be necessary when more and more web applications, microservices are required in your company/enterprise.
#6 Saas
All the previous approaches could be simplified using WORLD CLASS platforms like:
Jelastic
Heroku
Openshift, etc
I am struggling about how to upload my Cloud Code files that i had on Parse.com to my Parse Server hosted on AWS EB.
So far i have:
Parse Server hosted on AWS EB. To host it on AWS i used the Orange Deploy Button which basically makes all stuff easier for people without having to install the Parse Server locally and upload it later to AWS.
iOS App written in objective C connected to the Parse server and working perfectly
Parse Dashboard locally on my mac connected to the Parse Server on AWS
The only thing that i would need is to upload all my cloud code files to the Parse Server. How could i do this? I have researched a lot over Google, stackoverflow, etc without success. There is some information but its unclear. Thanks in advance.
Finally and thanks to Ran Hassid i now have a Fully functional Parse Server on AWS with Cloud Code. For those who are in the same situation where i was, here is the answer to my question:
Go to this link here and follow all the steps (By the time i asked the question, the information provided by this link of AWS wasn't that clear as it is now. They improved the explanations and the info.)
After you finish all the previous steps from the link. You would have a Parse Server on AWS working.
Now the part of CLOUD CODE. Just create a folder in your MAC or PC wherever you like. Let's say on the desktop and called it Parse Server AWS (You can call it whatever you want)
Install the EB CLI which is the Command line interface to user Terminal (On Mac) or the equivalent on windows to work with the parse server you just set up on AWS (Similar to CloudCode with Parse CLI). The easy way to install it is running this command:
brew install awsebcli
Now open terminal on mac (or the equivalent on windows) and go to the folder that you just created on the step 3.
Run the next command. It will ask you to select the location of your parse server, and then the name.
eb init
Now this command. It will download all the files from AWS of your parse server to this folder you are in.
eb labs download
Finally, you will have a folder called Cloud where you can put all your cloud code files in.
When you finish just run the command:
eb deploy
Now you have your parse server with all your cloud code files working on AWS.
Now any change you need to make to your cloudCode files, just change the local files inside this folder just created on step 3 and run again the command from the step 9. Just exactly as you used to do with Parse Deploy command
Hopefully this information will help many people as it helped to me.
Have a happy coding!
parse-server cloud code is a bit different from Parse.com cloud code. In Parse.com we use the Parse CLI in order to modify and deploy our cloud code (parse deploy ...) in parse-server your cloud code exist under the following path of your parse project ./cloud/main.js* so your cloud code endpoint is the main.js file which by default located under the **cloud folder of your parse project. If you really want you can change this path but to keep it simple use the default location.
Now about deployment. in parse-server you need to redeploy your parse server again when you do some modification to your cloud code. Another option is to edit your cloud code remotely but from my POV its better to redeploy it
We are using jenkins for CI. we get late night builds. Is there any way to automate the build deploy as soon as we get a mail or intimation ? Any suggestions would be appreciated..
One mechanism to deploy off of a build on Jenkins is to use artifacts to place the latest binary in a known location, and then kick off a new job (only on success of the compile/test phase) which uses (private key protected) ssh or scp to copy the artifacts to the test/production machine and then perform the install.
We use a similar mechanism for some automated testing that we do. The tricky part is getting the shell command to handle the ssh keys, so we do the following:
eval `ssh-agent -s`
ssh-add ~/.ssh/your_private_key_here
As long as that private key is on the Jenkins server and the public key is on the server you're trying to push to, you can then use ssh and scp commands in the rest of the script to perform functions on the server in question.
If you prefer to run the process entirely from the target server end, you can create a small script that runs on the server that checks for new files in the artifact directory of your Jenkins server build. Thanks to the latest path, you don't have to know the build number to do this. To find the specific path, you can log in to your Jenkins server (once you've saved at least one artifact), and find the project you are using and look at the Last Successful Artifacts, which will be URLs to the last successful builds of the artifacts. These URLs remain constant and always point at the most recent successful build, so you don't have to worry about them changing unless the project name or server name changes.
NOTE: there are security holes here that you can drive a truck through if you are doing this for anything other than a deployment to test. In the case of the first mechanism, your build server has an ssh key that gives it access (potentially destructive) to the target. In the case of the second mechanism, you are trusting that the Jenkins server will only serve up binaries that are good for you. However, for test environments, push to stage, etc. these techniques will work well.
These are the ways I know:
With a script:
In the Jenkins configurations, you can execute windows/shell commands after the execution of your maven goals. In my case, I have a Glassfish on a Linux, and via ssh I execute the asadmin parameters for the deployment. I have installed an instance in the server, and the process that I follow is: stop instance, undeploy app, deploy app, start instance (commands).
With a Maven Deploy Plugin:
This plugin takes a war/ear file and deploys that to a running remote application server at the end of a build. The list of currently supported containers include:
Tomcat 4.x/5.x/6.x/7.x
JBoss 3.x/4.x
Glassfish 2.x/3.x
https://wiki.jenkins-ci.org/display/JENKINS/Deploy+Plugin
With Cargo:
The Deploy Plugin is based on this. You must edit your pom.xml and execute the goals of deploy with maven.
http://cargo.codehaus.org/
In tomcat, configuration with jenkins and tomcat:
Install and download the jenkins on your server and start the server go to jenkins portal after that create the project using 'New Item' and select the pom.xml and create the maven project.
Now go to your project and click on Configure and select the "Restrict where this project can be run" and add master in your Level Expression.
select the "Source Code Management" clisck on git and configure your git repository and credential and branch name.
Select the "Build" add Root pom : pom.xml and Goals and options : clean install -DskipTests
select the "Post-build Actions" and select the "Deploy war/ear to a container"
WAR/EAR files : target/test.war
Context path : test
Containers select tomcat and add Credentials
Tomcat URL : example : http://localhost:8080/
Update the 'apache-tomcat-8.5.5\webapps\manager\META-INF\context.xlm file. uncomment the Value tag. and restart server
context.xml file
Before :
<Context antiResourceLocking="false" privileged="true">
<Valve className="org.apache.catalina.valves.RemoteAddrValve"
allow="192\.168\.0\.9|127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" />
</Context>
After change :
<Context antiResourceLocking="false" privileged="true" >
</Context>
for auto deployment: go to 'apache-tomcat-8.5.5\conf\context.xml' and add antiResourceLocking="true" in 'Context' tag