I am running this command to install web socket protocols in my AWS Elastic Beanstalk EC2 server
commands:
01_install_websockets:
command: "powershell.exe Install-WindowsFeature -name Web-WebSockets"
ignoreErrors: false
02_install_iis_websockets_feature:
command: "powershell.exe Enable-WindowsOptionalFeature -Online -FeatureName IIS-WebSockets"
ignoreErrors: false
The command above was not executed in my server, running those script in the EC2 manually always works but doing this using .ebextensions does not work.
Here's the structure of my code.
And when this one is published, the .ebextensions is added at the root of the zip file
zip
- .ebextensions
- other files...
Please let me know what's missing here. I don't do any special configurations on the AWS EB.
I am able to run with following steps:
I am using container_commands but, I am confident it will work with commands as well.
---
container_commands:
"2":
command: powershell.exe -ExecutionPolicy Bypass -Command".\\.ebextensions\\IISScripts.ps1"
Zip file looks like below:
zip
- .ebextensions
- 01-windows.config
- IISScript.ps1
- other files...
EC2 instance configuration: Windows server 2019
Related
I have a grails app that I deploy to AWS Elastic Beanstalk through Jenkins. I want to add a splunk forwarder to my project so I can keep track of my logs outside of AWS and set up easy notifications.
The problem is, I have multiple environments of the app running (dev, pre-prod, prod, etc), which is fine because you can just change the environment name for the forwarded and be able to easily sort through that in Splunk.
However, the same .ebextensions file has to be used between all the environments, no I need a way to set the environment name to whatever AWS has the name as. Is there a way I can easily do this that I'm overlooking?
Start of the script:
container_commands:
01install-splunk:
command: /usr/local/bin/install-splunk.sh
02set-splunk-outputs:
command: /usr/local/bin/set_splunk_outputs.sh
env:
SPLUNK_SERVER_HOST: "splunk.host"
03add-inputs-to-splunk:
command: /usr/local/bin/add-inputs-to-splunk.sh
env:
ENVIRONMENT_NAME: "Development"
cwd: /root
ignoreErrors: false
That ENVIRONMENT_NAME variable I'm setting that's passed to the 3rd script is what I want to be able to change based on what environment is being deployed. Can I set this in Jenkins or pull it through AWS somehow?
You can try below steps:
Configure your AWS Elasticbeanstalk environment with the environment variable
ENVIRONMENT_NAME = 'Development' or 'QA' or 'Prod'
please refer aws-official-docs for same.
Then update config as below:
container_commands:
01install-splunk:
command: /usr/local/bin/install-splunk.sh
02set-splunk-outputs:
command: /usr/local/bin/set_splunk_outputs.sh
env:
SPLUNK_SERVER_HOST: "splunk.host"
03add-inputs-to-splunk:
command: /usr/local/bin/add-inputs-to-splunk.sh
env:
ENVIRONMENT_NAME: "$ENVIRONMENT_NAME"
cwd: /root
ignoreErrors: false
Hope this should work for you.
I want to use AWS CodeDeploy to deploy a jar file and then run my java -jar command from there once it is on the EC2. But I've noticed that AWS CodeDeploy only pulls zip, tar and tar.gz` from S3. I'm thinking I will use CLI from my local jenkins to push a .zip file (containing the jar) to S3, then run another CLI command to start AWS CodeDeploy which will pull the .zip from S3.
However I do have a question the details on AWS CodeDeploy:
Can I use the appspec.yml to issue two commands,
1) unzip the .zip from S3 once it is on the EC2
2) Issue the java -jar on a specific location?
Thanks
1) You do not need issue an unzip command, the files section in appspec.yml is used to specify the files in your archive (source) you wish to copy to the file system on ec2 (destination) and will be done by the code deploy agent RE https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-files.html
2) Create a run script to issue java -jar command under the hook ApplicaitonStart
RE https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html
Example appspec.yml on os linux based:
version: 0.0
os: linux
files:
- source: ./
destination: /home/ubuntu/myapp
hooks:
ApplicationStart:
- location: runapp.sh
runas: ubuntu
runapp.sh:
#!/usr/bin/env bash
echo 'Starting my app'
cd '/home/ubuntu/myapp'
java -jar myapp.jar
In this example, you would include runapp.sh in your deployment package
Can you help me find a useful step-by-step guide or a Gist outlining in detail how to configure CircleCI (using 2.0 syntax) to deploy to AWS EC2?
I understand the basic requirements and the moving pieces, but unsure what to put in the .circleci/config.yml file in the deploy step.
So far I got:
A "Hello World" Node.js app which is building successfully in CircleCI (just without the deploy step)
A running EC2 instance (Ubuntu 16.04)
An IAM user with sufficient permissions added to CircleCI for that particular job
Can you help out with the CircleCI deploy step?
Following your repository, you could create a script just like that: deploy.sh
#!/bin/bash
echo "Start deploy"
cd ~/circleci-aws
git pull
npm i
npm run build
pm2 stop build/server
pm2 start build/server
echo "Deploy end"
And in your .circleci/conf.yml you do it:
deploy:
docker:
- image: circleci/node:chakracore-8.11.1
steps:
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package.json" }}
- run:
name: AWS EC2 deploy
command: |
#upload all the code to machine
scp -r -o StrictHostKeyChecking=no ./ ubuntu#13.236.1.107:/home/circleci-aws/
#Run script inside of machine
ssh -o StrictHostKeyChecking=no ubuntu#13.236.1.107 "./deploy.sh"
But this is so ugly, try something like AWS Codedeploy or ecs for using containers.
I am trying to get travis-ci to run a custom deploy script that uses awscli to push a deployment up to my staging server.
In my .travis.yml file I have this:
before_deploy:
- 'curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"'
- 'unzip awscli-bundle.zip'
- './awscli-bundle/install -b ~/bin/aws'
- 'export PATH=~/bin:$PATH'
- 'aws configure'
And I have set up the following environment variables:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION
with their correct values in the travis-ci web interface.
However when the aws configure runs, it stops and waits for user input. How can I tell it to use the environment variables I have defined?
Darbio's solution works fine but it's not taking into consideration that you may end up pushing your AWS credentials in your repository.
That is a bad thing especially if docker is trying to pull a private image from one of your ECR repositories. It would mean that you probably had to store your AWS production credentials in the .travis.yml file and that is far from ideal.
Fortunately Travis gives you the possibility to encrypt environment variables, notification settings, and deploy api keys.
gem install travis
Do a travis login first of all, it will ask you for your github credentials. Once you're logged in get in your project root folder (where your .travis.yml file is) and encrypt your access key id and secret access key.
travis encrypt AWS_ACCESS_KEY_ID="HERE_PUT_YOUR_ACCESS_KEY_ID" --add
travis encrypt AWS_SECRET_ACCESS_KEY="HERE_PUT_YOUR_SECRET_ACCESS_KEY" --add
Thanks to the --add option you'll end up with two new (encrypted) environment variables in your configuration file. Now just open your .travis.yml file and you should see something like this:
env:
global:
- secure: encrypted_stuff
- secure: encrypted_stuff
Now you can make travis run a shell script that creates the ~/.aws/credentials file for you.
ecr_credentials.sh
#!/usr/bin/env bash
mkdir -p ~/.aws
cat > ~/.aws/credentials << EOL
[default]
aws_access_key_id = ${AWS_ACCESS_KEY_ID}
aws_secret_access_key = ${AWS_SECRET_ACCESS_KEY}
EOL
Then you just need to run the ecr_credentials.sh script from your .travis.yml file:
before_install:
- ./ecr_credentials.sh
Done! :-D
Source: Encription keys on Travis CI
You can set these in a couple of ways.
Firstly, by creating a file at ~/.aws/config (or ~/.aws/credentials).
For example:
[default]
aws_access_key_id=foo
aws_secret_access_key=bar
region=us-west-2
Secondly, you can add environment variables for each of your settings.
For example, create the following environment variables:
AWS_DEFAULT_REGION
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
Thirdly, you can pass region in as a command line argument. For example:
aws eb deploy --region us-west-2
You won't need to run aws configure in these cases as the cli is configured.
There is further AWS documentation on this page.
Following the advice from #Darbio, I came up with this solution:
- stage: deploy
name: "Deploy to AWS EKS"
language: minimal
before_install:
# Install kubectl
- curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
- chmod +x ./kubectl
- sudo mv ./kubectl /usr/local/bin/kubectl
# Install AWS CLI
- if ! [ -x "$(command -v aws)" ]; then curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" ; unzip awscliv2.zip ; sudo ./aws/install ; fi
# export environment variables for AWS CLI (using Travis environment variables)
- export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- export AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
# Setup kubectl config to use the desired AWS EKS cluster
- aws eks update-kubeconfig --region ${AWS_DEFAULT_REGION} --name ${AWS_EKS_CLUSTER_NAME}
deploy:
- provider: script
# bash script containing the kubectl commands to setup the cluster
script: bash k8s-config/deployment.sh
on:
branch: master
It is also possible to avoid installing AWS CLI altogether. Then you need to configure kubectl:
kubectl config set-cluster --server= --certificate-authority=
kubectl config set-credentials --client-certificate= --client-key=
kubectl config set-context myContext --cluster= --namespace= --user=
kubectl config use-context myContext
You can find most of the needed values in your users home directory in /.kube/config, after you performed the aws eks update-kubeconfig command on your local machine.
Except for the client certificate and key. I couldn't figure out where to get them from and therefore needed to install AWS CLI in the pipeline as well.
I'm banging my head against a wall trying to both install and then enable a service in elastic beanstalk. What I want to do is:
Install a service in /etc/init.d that points to my python app in /opt/python/current/app/
Have Elastic Beanstalk start and keep-alive the service, as specified in an .ebextensions/myapp.config file.
(Reference: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#customize-containers-format-services)
Here's my .ebextensions/myapp.config file:
container_commands:
01_copy_service:
command: "cp /opt/python/ondeck/app/my_service /etc/init.d/"
02_chmod_service:
command: "chmod +x /etc/init.d/my_service"
services:
sysvinit:
my_service:
enabled: true
ensureRunning: true
files : [/etc/init.d/my_service]
This fails because services are run before container_commands. If I comment out services, deploy, then uncomment services, then deploy again, it will work. But I want to have a single-step deploy, because this will be an auto-scaling node.
Is there a solution? Thanks!
Nate, I have the exact same scenario as you and I solved it this way:
Drop the "services" section and add a "restart" command.
container_commands:
...
03_restart_service:
command: /sbin/service my_service restart
You can cause the service to restart after a command is run by using a commands: key under the services: key. The documentation for the services: key is here:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#customize-containers-format-services
I haven't done it myself, but I want to give you some ideas which should work. It's just the matter of convenience and the workflow.
Since it is not really application file, but rather EC2 file, and unlikely to be changed often, you can do one of the following:
Use files content to create the service init script. You can even have a specific config file just for that script.
Store service init script on S3 and copy the contents with command.
Create dummy service script, replace the contents with the one from deployment with container command and dependency on the above command to the service.
(this one is heavy) Create custom AMI and specify it in Autoscaling configuration.
Hope it helps.