I am setting up a web app through code pipeline. My cloud formation script is creating an ec2 instance. In that ec2 user data, I have written a logic to get a code from the s3 and copy the code in the ec2 and start the server. A web app is in Python Pyramid framework.
code pipeline is connected with GitHub. It creates a zip file and uploads to the s3 bucket. (That is all in a buildspec.yml file)
When I changed the user data script and run code pipeline it works fine.
But When I changed some web app(My code base) file and re-run the code pipeline. That change is not reflected.
This is for ubuntu ec2 instance.
#cloud-boothook
#!/bin/bash -xe
echo "hello "
exec > /etc/setup_log.txt 2> /etc/setup_err.txt
sleep 5s
echo "User_Data starts"
rm -rf /home/ubuntu/c
mkdir /home/ubuntu/c
key=`aws s3 ls s3://bucket-name/pipeline-name/MyApp/ --recursive | sort | tail -n 1 | awk '{print $4}'`
aws s3 cp s3://bucket-name/$key /home/ubuntu/c/
cd /home/ubuntu/c
zipname="$(cut -d'/' -f3 <<<"$key")"
echo $zipname
mv /home/ubuntu/c/$zipname /home/ubuntu/c/c.zip
unzip -o /home/ubuntu/c/c.zip -d /home/ubuntu/c/
echo $?
python3 -m venv venv
venv/bin/pip3 install -e .
rm -rf cc.zip
aws configure set default.region us-east-1
venv/bin/pserve development.ini http_port=5000 &
The expected result is when I run core pipeline, every time user data script will execute.
Give me a suggestion, any other
The User-Data script gets executed exactly once upon instance creation. If you want to periodically synchronize your code changes to the instance you should think about implementing a CronJob in your User-Data script or use a service like AWS CodeDeploy to deploy new versions (this is the preferred approach).
CodePipeline uses a different S3 object for each pipeline execution artifact, so you can't hardcore a reference to it. You could publish the artifact to a fixed location. You might want to consider using CodeDeploy to deploy the latest version of your application.
Related
I am using an ebs app on linux 2 platforms, and I need to clone a directory during deployment to get configfiles for my app.
I did a predeploy hook so that the files are there when the app starts after deployment: /.platform/hooks/predeploy/01_import
After deployment in a predeploy hook, the files are not there.
When I run the exact same script in a postdeploy hook, the files are there.
So the command works, I see the predeploy hook is running (I see the echo text in the log), but the files are not present. Anyone knows why?
#!/bin/bash
mkdir /var/app/current/config
echo Adding github in known hosts
ssh-keyscan -H github.com >> /home/webapp/.ssh/known_hosts
echo Done Adding github in known hosts
echo deleting old flows
echo cloning
git -c core.sshCommand="ssh -i /etc/pki/tls/certs/githubKey" clone -b dev --single-branch <mygithub> /var/app/current/config
echo done cloning
In predeploy stage, the new code is deployed to /var/app/staging, not /var/app/current.
/var/app/current is actually overwritten by staging if the new staging deployment is successful.
So in predeploy, I've cloned to staging instead, and it works.
This is not well documented in AWS docs; this helped me.
I have been playing around with AWS Batch, and I am having some trouble understanding why everything work when I build a docker image from my local windows machine and push it to ECR, while it doesn't work when I do this from a ubuntu EC2 instance.
What I show below is adapted from this tutorial.
The docker file is very simple:
FROM python:3.6.10-alpine
RUN apk add --no-cache --upgrade bash
COPY ./ /usr/local/aws_batch_tutorial
RUN pip3 install -r /usr/local/aws_batch_tutorial/requirements.txt
WORKDIR /usr/local/aws_batch_tutorial
Where the local folder contains the following bash script (run_job.sh):
#!/bin/bash
error_exit () {
echo "${BASENAME} - ${1}" >&2
exit 1
}
################################################################################
###### Convert envinronment variables to command line arguments ########
pat="--([^ ]+).+"
arg_list=""
while IFS= read -r line; do
# Check if line contains a command line argument
if [[ $line =~ $pat ]]; then
E=${BASH_REMATCH[1]}
# Check that a matching environmental variable is declared
if [[ ! ${!E} == "" ]]; then
# Make sure argument isn't already include in argument list
if [[ ! ${arg_list} =~ "--${E}=" ]]; then
# Add to argument list
arg_list="${arg_list} --${E}=${!E}"
fi
fi
fi
done < <(python3 script.py --help)
################################################################################
python3 -u script.py ${arg_list} | tee "${save_name}.txt"
aws s3 cp "./${save_name}.p" "s3://bucket/${save_name}.p" || error_exit "Failed to upload results to s3 bucket."
aws s3 cp "./${save_name}.txt" "s3://bucket/logs/${save_name}.txt" || error_exit "Failed to upload logs to s3 bucket."
It also contains a requirement.txt file with a three packages (awscli,boto3,botocore),
and a dummy python script (script.py) that simply lists the files in a s3 bucket and saves the list in a file that is then uploaded to s3.
Both in my local windows environment and in the EC2 instance I have set up my AWS credentials with aws configure, and in both cases I can successfully build the image, tag it and push it to ECR.
The problem arises when I submit the job on AWS Batch, which should run the ECR container using the command ["./run_job.sh"]:
if AWS Batch uses the ECR image pushed from windows, everything works fine
if it uses the image pushed from ec2 linux, the job fails, and the only info I can get is this:
Status reason: Task failed to start
I was wondering if anyone has any idea of what might be causing the error.
I think I fixed the problem.
The run_job.sh script in the docker image has to have execute permissions to be run by AWS Batch (but I think this is true in general).
For some reason, when the image is built from Windows, the script has this permission, but it doesn't when the image is built from linux (aws ec2 - ubuntu instance).
I fixed the problem by adding the following line in the Dockerfile:
RUN chmod u+x run_job.sh
I am working on Glue in AWS and trying to test and debug in local dev. I follow the instruction here https://aws.amazon.com/blogs/big-data/developing-aws-glue-etl-jobs-locally-using-a-container/ to develop Glue job locally. On that post, they use Glue 1.0 image for testing and it works as it should be. However when I load and try to dev by Glue 3.0 version; I follow the guidance steps but, I can't open Jupyter notebook on :8888 like the post said even every step seems correct.
here my cmd to start a Jupyter notebook on Glue 3.0 container
docker run -itd -p 8888:8888 -p 4040:4040 -v ~/.aws:/root/.aws:ro --name glue3_jupyter amazon/aws-glue-libs:glue_libs_3.0.0_image_01 /home/jupyter/jupyter_start.sh
nothing shows on http://localhost:8888.
still have no idea why! I understand the diff. between versions of Glues just wanna develop and test on the latest version of it. Have anybody got the same issue?
Thanks.
It seems that GLUE 3.0 image has some issues with SSL. A workaround for working locally is to disable SSL (you also have to change the script paths as documentation is not updated).
$ docker run -it -p 8888:8888 -p 4040:4040 -e DISABLE_SSL="true" \
-e AWS_ACCESS_KEY_ID=$(aws --profile default configure get aws_access_key_id) \
-e AWS_SECRET_ACCESS_KEY=$(aws --profile default configure get aws_secret_access_key) \
-e AWS_DEFAULT_REGION=$(aws --profile default configure get region) \
--name glue_jupyter amazon/aws-glue-libs:glue_libs_3.0.0_image_01 \
/home/glue_user/jupyter/jupyter_start.sh
After a few seconds you should have a working jupyter notebook instance running on http://127.0.0.1:8888
I have developed an application in nodejs/vuejs and I want to dockerize the whole project before push it in production.
Knowing that my API is executing an aws command at a specific time, I need to install and configure AWS-CLIv2 in production.
crontab.scheduleJob('30 8,12 * * *', () => {
shelljs.exec("rm -rf src/data/*.csv && aws s3 cp s3://${bucketName}/`aws s3 ls s3://${bucketName} | tail -n 1 | awk '{print $4}'` src/data");
});
For development, I installed (from line command) and configured AWS locally from https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html
Would it be possible to install aws-cliv2 via the API's Dockerfile? What structure should I adopt?
Otherwise offer me your solutions please ?
Thank you
Maybe you can try use the SDK of AWS for interact with the AWS API (in this case the S3 Bucket), but if you prefer use the aws-cli binary you can install the binary like this docker image is made it or make your docker image based in this image.
I have Ansible playbooks ready, they includes several encrypted vars. With normal process, I can feed a vault password file to decrypt them with --vault-password-file ~/.vault_pass.txt and deploy the change to remote EC2 instance. So I needn't expose the password file.
But my request is different here. I need include ansible-playbook change in user-data script when create a new EC2 instance. Ideally I should automatically have all setting ready after the instance is running.
I deploy the instances with Terraform by below simple user-data script:
#!/usr/bin/bash
yum -y update
/usr/local/bin/aws s3 cp s3://<BUCKET>/ansible.tar.gz ansible.tar.gz
gtar zxvf ansible.tar.gz
cd ansible
ansible-playbook -i inventory/ec2.py -c local ROLE.yml
So I have to upload my password file into user-data script as well, if in the playbook, there are some encrypted vars.
Anything I can do to avoid it? Will Ansible Tower help for this request?
I did test with CredStash, but still a chicken and egg issue.
If you want your instances to configure themselves they are going to either need all the credentials or another way to get the credentials, ideally with some form of one time pass.
The best I can think of off the top of my head is to use Hashicorp's Vault to store the credentials (potentially all of our secrets or maybe just the Ansible Vault password that then can be used to un-vault your Ansible variables) and have your deploy process create a one time use token that is injected into the user-data script via Terraform's templating.
To do this you'll probably want to wrap your Terraform apply command with some form of helper script that might look like this (untested):
#!/bin/bash
vault_host="10.0.0.3"
vault_port="8200"
response=`curl \
-X POST \
-H "X-Vault-Token:$VAULT_TOKEN" \
-d '{"num_uses":"1"}' \
http://${vault_host}:${vault_port}/auth/token/create/ansible_vault_read`
vault_token=`echo ${response} | jq '.auth.client_token' --raw-output`
terraform apply \
-var 'vault_host=${vault_host}'
-var 'vault_port=${vault_port}'
-var 'vault_token=${vault_token}'
And then your user data script will want to be templated in Terraform with something like this (also untested):
template.tf:
resource "template_file" "init" {
template = "${file("${path.module}/init.tpl")}"
vars {
vault_host = "${var.vault_host}"
vault_port = "${var.vault_port}"
vault_token = "${var.vault_token}"
}
}
init.tpl:
#!/usr/bin/bash
yum -y update
response=`curl \
-H "X-Vault-Token: ${vault_token}" \
-X GET \
http://${vault_host}:${vault_port}/v1/secret/ansible_vault_pass`
ansible_vault_password=`echo ${response} | jq '.data.ansible_vault_pass' --raw-output`
echo ${ansible_vault_password} > ~/.vault_pass.txt
/usr/local/bin/aws s3 cp s3://<BUCKET>/ansible.tar.gz ansible.tar.gz
gtar zxvf ansible.tar.gz
cd ansible
ansible-playbook -i inventory/ec2.py -c local ROLE.yml --vault-password-file ~/.vault_pass.txt
Alternatively you could simply have the instances call something such as Ansible Tower to trigger the playbook to be run against it. This allows you to keep the secrets on the central box doing the configuration rather than having to distribute them to every instance you are deploying.
With Ansible Tower this is done using callbacks and you will need to set up job templates and then have your user data script curl the Tower to trigger the configuration run. You could change your user data script to something like this instead:
template.tf:
resource "template_file" "init" {
template = "${file("${path.module}/init.tpl")}"
vars {
ansible_tower_host = "${var.ansible_tower_host}"
ansible_host_config_key = "${var.ansible_host_config_key}"
}
}
init.tpl:
#!/usr/bin/bash
curl \
-X POST
--data "host_config_key=${ansible_host_config_key}" \
http://{${ansible_tower_host}/v1/job_templates/1/callback/
The host_config_key may seem to be a secret at first glance but it's a shared key that can be used for multiple hosts to access a job template and Ansible Tower will still only run if the host is either defined in a static inventory for the job template or if you are using dynamic inventories then if the host is found in that lookup.