I want to deploy my application to aws, I am using codeDeploy for this.
following is my appsec.yml file:
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/project
permissions:
- object: /home/ubuntu/project
owner: root
mode: 777
type:
- directory
hooks:
BeforeInstall:
- location: scripts/install_dependencies.sh
timeout: 900
runas: root
AfterInstall:
- location: ./scripts/after-install.sh
timeout: 900
ApplicationStart:
- location: ./scripts/application-start.sh parameter1 parameter2
timeout: 900
ValidateService:
- location: ./scripts/validate-service.sh
timeout: 900
I am not able to pass paramerters to scripts.
Currently this is not possible.
As a workaround, you can design your Hook Scripts to consume System Environment Variables which can be defined on a instance at launch (through user-data) or you can also retrieve the parameters from AWS SSM Parameter Store (specially if they are secrets) using AWS CLI:
https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-cli.html
You can create an if clause using the predefined environment variables. Based on the situation you configure the required values predefined environment variables
Related
My deployment is putting the files on the server but all of the files have a modified time of 0 so Apache isn't hosting the updated files.
I added an AfterInstall script that is supposed to touch every file in the directory but it's not working for some reason. There is no error and if I run the script manually it works fine, just not during the deploy process.
Has anyone else run into this issue? Is there something simple I'm overlooking to make this work?
Touch script
#!/bin/bash
find /var/www/html/docs -type f -exec touch {} +
YML file
version: 0.0
os: linux
files:
- source: /source/
destination: /var/www/html/site/
file_exists_behavior: OVERWRITE
permissions:
- object: /var/www/html/site
pattern: "**"
owner: [redacted]
group: [redacted]
hooks:
AfterInstall:
- location: scripts/after_install
timeout: 10
runas: [redacted]
I am using Elastic Beanstalk to deploy a worker tier environment using SQS.
In my .ebextensions I have the following file:
option_settings:
aws:elasticbeanstalk:sqsd:
WorkerQueueURL:
Ref: WorkerQueue
HttpPath: "/sqs/"
InactivityTimeout: 1650
VisibilityTimeout: 1680
MaxRetries: 1
Resources:
WorkerQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: "tpc-clients-aws-queue"
VisibilityTimeout: 1680
However, this fails with the following error:
"option_settings" in one of the configuration files failed validation. More details to follow.
Invalid option value: 'Ref=WorkerQueue' (Namespace: 'aws:elasticbeanstalk:sqsd', OptionName: 'WorkerQueueURL'): Value does not satisfy regex: '^$|^http(s)?://.+$' [Valid non empty URL starting with http(s)]
It seems that the AWSCloudFormation Ref function cannot be used in the option_settings. Can someone confirm if this is the case?
I have seen some code snippets here on StackOverflow using intrinsic functions in the option_settings, such as in the mount-config.config of this answer and also on this question. So, are these examples using an invalid syntax? Or there are some intrinsic functions or specific resources that can be used on the option_settings?
And lastly, if I cannot use the Ref function, how can I go about this?
Yes, you can reference in .ebextentions, but the syntax is a bit strange. It is shown in the docs here.
You can try something along these lines (note the various quotations marks):
option_settings:
aws:elasticbeanstalk:sqsd:
WorkerQueueURL: '`{"Ref" : "WorkerQueue"}`'
HttpPath: "/sqs/"
InactivityTimeout: 1650
VisibilityTimeout: 1680
MaxRetries: 1
Resources:
WorkerQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: "tpc-clients-aws-queue"
VisibilityTimeout: 1680
You can also use ImportValue, if you export the WorkerQueue in outputs.
Update
To check the value obtained, you can set it as an env variable, and inspect in EB console:
option_settings:
aws:elasticbeanstalk:application:environment:
SQS_NAME: '`{"Ref" : "WorkerQueue"}`'
After digging further in this issue I made some discoveries I would like to share with future readers.
Ref can be used on option_settings
As #Marcin answer states, the Ref intrinsic function can be used in the option_settings. The syntax is different though:
'`{"Ref" : "ResourceName"}`'
Using Ref on aws:elasticbeanstalk:application:environment (environment variable)
An use case of the above is to store the queue URL in an environment variable, as follows:
option_settings:
aws:elasticbeanstalk:application:environment:
QUEUE_URL: '`{"Ref" : "WorkerQueue"}`'
This will let your .sh script access the URL of the queue:
Note that if you check the Elastic Beanstalk console (Environment > Config > Software), you won't see the actual value:
Using Ref on aws:elasticbeanstalk:sqsd:WorkerQueueURL
If you try to use the following setting:
option_settings:
aws:elasticbeanstalk:sqsd:
WorkerQueueURL: '`{"Ref" : "WorkerQueue"}`'
HttpPath: "/sqs/"
It will fail:
Invalid option value: '`{"Ref" : "WorkerQueue"}`' (Namespace: 'aws:elasticbeanstalk:sqsd', OptionName: 'WorkerQueueURL'): Value does not satisfy regex: '^$|^http(s)?://.+$' [Valid non empty URL starting with http(s)]
It seems that this configuration option don't accept a reference.
Instead of creating a new queue and assign it to the sqs daemon, you can just update the queue that Elastic Beanstalk creates:
option_settings:
# SQS daemon will use default queue created by EB (AWSEBWorkerQueue)
aws:elasticbeanstalk:sqsd:
HttpPath: "/sqs/"
Resources:
# Update the queue created by EB
AWSEBWorkerQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: "tpc-clients-aws-queue"
I have been trying to figure out how to configure the docker version of Concourse (https://github.com/concourse/concourse-docker) to use the AWS Secrets Manager and I added the following environment variables into the docker-compose file but from the logs it doesn't look like it ever reaches out to AWS to fetch the creds. Am I missing something or should this automatically happen when adding these environment variables under environment in the docker-compose file? Here are the docs I have been looking at https://concourse-ci.org/aws-asm-credential-manager.html
version: '3'
services:
concourse-db:
image: postgres
environment:
POSTGRES_DB: concourse
POSTGRES_PASSWORD: concourse_pass
POSTGRES_USER: concourse_user
PGDATA: /database
concourse:
image: concourse/concourse
command: quickstart
privileged: true
depends_on: [concourse-db]
ports: ["9090:8080"]
environment:
CONCOURSE_POSTGRES_HOST: concourse-db
CONCOURSE_POSTGRES_USER: concourse_user
CONCOURSE_POSTGRES_PASSWORD: concourse_pass
CONCOURSE_POSTGRES_DATABASE: concourse
CONCOURSE_EXTERNAL_URL: http://XXX.XXX.XXX.XXX:9090
CONCOURSE_ADD_LOCAL_USER: test: test
CONCOURSE_MAIN_TEAM_LOCAL_USER: test
CONCOURSE_WORKER_BAGGAGECLAIM_DRIVER: overlay
CONCOURSE_AWS_SECRETSMANAGER_REGION: us-east-1
CONCOURSE_AWS_SECRETSMANAGER_ACCESS_KEY: <XXXX>
CONCOURSE_AWS_SECRETSMANAGER_SECRET_KEY: <XXXX>
CONCOURSE_AWS_SECRETSMANAGER_TEAM_SECRET_TEMPLATE: /concourse/{{.Secret}}
CONCOURSE_AWS_SECRETSMANAGER_PIPELINE_SECRET_TEMPLATE: /concourse/{{.Secret}}
pipeline.yml example:
jobs:
- name: build-ui
plan:
- get: web-ui
trigger: true
- get: resource-ui
- task: build-task
file: web-ui/ci/build/task.yml
- put: resource-ui
params:
repository: updated-ui
force: true
- task: e2e-task
file: web-ui/ci/e2e/task.yml
params:
UI_USERNAME: ((ui-username))
UI_PASSWORD: ((ui-password))
resources:
- name: cf
type: cf-cli-resource
source:
api: https://api.run.pivotal.io
username: ((cf-username))
password: ((cf-password))
org: Blah
- name: web-ui
type: git
source:
uri: git#github.com:blah/blah.git
branch: master
private_key: ((git-private-key))
When storing parameters for concourse pipelines in AWS Secrets Manager, it must follow this syntax,
/concourse/TEAM_NAME/PIPELINE_NAME/PARAMETER_NAME`
If you have common parameters that are used across the team in multiple pipelines, use this syntax to avoid creating redundant parameters in secrets manager
/concourse/TEAM_NAME/PARAMETER_NAME
The highest level that is supported is concourse team level.
Global parameters are not possible. Thus these variables in your compose environment will not be supported.
CONCOURSE_AWS_SECRETSMANAGER_TEAM_SECRET_TEMPLATE: /concourse/{{.Secret}}
CONCOURSE_AWS_SECRETSMANAGER_PIPELINE_SECRET_TEMPLATE: /concourse/{{.Secret}}
Unless you want to change the prefix /concourse, these parameters shall be left to their defaults.
And, when retrieving these parameters in the pipeline, no changes required in the template. Just pass the PARAMETER_NAME, concourse will handle the lookup in secrets manager as per the team and pipeline name.
...
params:
UI_USERNAME: ((ui-username))
UI_PASSWORD: ((ui-password))
...
i have been use code pipeline and code deploy to deploy my asp.net application using git hub , every time deployment failed with this error message in event log
" Script at specified location: scripts/stop_service run as user root failed with exit code 5 "
i have been installed the Code Deploy Agent on ec2 instance and here is sample from appspec file
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html/
hooks:
BeforeInstall:
- location: scripts/install_dependencies
timeout: 300
runas: root
- location: scripts/start_server
timeout: 300
runas: root
ApplicationStop:
- location: scripts/stop_server
timeout: 300
runas: root
enter image description here
CodeDeploy is trying to run the script 'scripts/stop_server' and failing e.g. because the script is not present at this location.
If the script is at this location, check whats wrong with the execution by inspecting the log at:
/opt/codedeploy-agent/deployment-root/[deployment-group-ID]/[deployment-ID]/logs/scripts.log
Replace '[deployment-group-ID]' and '[deployment-ID]' with actual Ids of your deployment.
I am trying to deploy my app on AWS with CodeDeploy.
Here is my appspec.yml file:
version: 0.0
os: linux
files:
- source: /
destination: /home/ec2-user/todos // <== this dir already exists in my instance
hooks:
ApplicationStop:
...
BeforeInstall:
- location: scripts/prerequisites
timeout: 1200
runas: root
AfterInstall:
...
ApplicationStart:
...
ValidateService:
...
I keep getting the following error:
Error Code: ScriptMissing
Script Name: scripts/prerequisites
Message: Script does not exist at specified location:
/opt/codedeploy-agent/deployment-root/2e557520-7ffe-4881-8c7c-991952c56e05/d-UWR3Z01FE/deployment-archive/scripts/prerequisites
Log Tail: LifecycleEvent - BeforeInstall
My scripts are stored in a file called 'scripts' that is found at the root of my app.
What am I missing? or doing wrong? If anyone could help me in the right direction I would be very grateful!
Ok so it seems that the problem was simply a typo. I forgot to write the file extension to my scripts (.sh)
Like so:
BeforeInstall:
- location: scripts/prerequisites.sh // <--- this fixed it
timeout: 1200
runas: root
Hope it helps anyone.