Why doesn't my custom recipes run on AWS OpsWorks? - amazon-web-services

I've created a GitHub repo for my simple custom recipe:
my-cookbook/
|- recipes/
|- appsetup.rb
I've added the repo to Custom Chef Recipes as https://github.com/my-github-user/my-github-repo.git
I've added my-cookbook::appsetup to the Setup "cycle".
I know it's executed, because it fails to load if I mess up the syntax.
This is my appsetup.rb:
node[:deploy].each do |app_name, deploy|
script "install_composer" do
interpreter "bash"
user "root"
cwd "#{deploy[:deploy_to]}/current"
code "curl -sS https://getcomposer.org/installer | php && php composer.phar install --no-dev"
end
end
When I log into the instance by SSH with the ubuntu user, composer isn't installed.
I've also tried the following to no avail (A nodejs install):
node[:deploy].each do |app_name, deploy|
execute "installing node" do
command "add-apt-repository --yes ppa:chris-lea/node.js && apt-get update && sudo apt-get install python-software-properties python g++ make nodejs"
end
end
Node doesn't get installed, and there are no errors in the log. The only references to the cookbook in the log just says:
[2014-03-31T13:26:04+00:00] INFO: OpsWorks Custom Run List: ["opsworks_initial_setup", "ssh_host_keys", "ssh_users", "mysql::client", "dependencies", "ebs", "opsworks_ganglia::client", "opsworks_stack_state_sync", "mod_php5_apache2", "my-cookbook::appsetup", "deploy::default", "deploy::php", "test_suite", "opsworks_cleanup"]
...
2014-03-31T13:26:04+00:00] INFO: New Run List expands to ["opsworks_initial_setup", "ssh_host_keys", "ssh_users", "mysql::client", "dependencies", "ebs", "opsworks_ganglia::client", "opsworks_stack_state_sync", "mod_php5_apache2", "my-cookbook::appsetup", "deploy::default", "deploy::php", "test_suite", "opsworks_cleanup"]
...
[2014-03-31T13:26:05+00:00] DEBUG: Loading Recipe my-cookbook::appsetup via include_recipe
[2014-03-31T13:26:05+00:00] DEBUG: Found recipe appsetup in cookbook my-cookbook
Am I missing some critical step somewhere? The recipe is clearly recognized and loaded, but doesn't seem to be executed.
(The following are fictitious names: my-github-user, my-github-repo, my-cookbook)

I know you've abandoned the cookbook but I'm almost 100% sure it's because you don't have a metadata.rb file in the root of your cookbook.

Your cookbook name should not contain a dash. I had the same problem, replacing by '_' solved it for me.

If those commands are failing silently, it could be that your use of && is obscuring a failure.
As for add-apt-repository, that is an interactive command. Try using the "--yes" option to answer yes by default, making it no longer interactive.

If you do not execute your command successfully, you will not find the files in the current directory. Check inside the last release folder to see if it had been put there.
It maybe prudent to check if you got the right directory etc setup by changing the CWD to : /tmp

Related

Unable to execute a step on a running EMR

I have an EMR cluster 5.28.1 running in AWS but I forgot to install from python libraries as part of the bootstrap action. Now that the cluster is running, I was simply attempting to add a step via the EMR console. Here are my settings
JAR: s3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar
Main class: None
Arguments: s3://xxxx/install_python_libraries.sh
Unfortunately, I get the following error.
Cannot run program "s3://xxxxx/install_python_libraries.sh" (in directory "."): error=2, No such file or directory
I am not sure what I am doing wrong. The shell script looks like this.
#!/bin/bash -xe
# Non-standard and non-Amazon Machine Image Python modules:
sudo pip-3.6 install boto3
sudo pip-3.6 install xmltodict
I also tried this by simply using 'command-runner.jar' but I get the same error. Can you please help me figure out the problem so I do this via the console? I would like to install the libraries on all nodes - master and core.
Thanks
The issue is the xxx.sh files EOL/carriage return type.
In other words, if it is Windows ("\r\n") then it will not work and return the ./ file not found error.
Convert it to unix type ("\n") using something like notepad++ and it will run fine.
(In notepad++ edit>EOL Conversion>Unix(LF) hit save and try again)

Run make command in Jenkins

I'm trying to build c++ project.
When I run the make command in terminal it works,
but when I do it through Jenkins, it shows me a message that files are missing.
What can be the problem, and how can I solve it?
The Error:
+ make
make -f enclave_lib.mk SGX_DEBUG=1
make[1]: Entering directory '/home/yoni/Documents/private_ledger-tp/CryptoLib'
mt19937ar.c:44:19: fatal error: stdio.h: No such file or directory
From your comments, the problem is that Jenkins is executed as root user, and can not find the lib stdio.h.
To fix this you can have several options:
locate stdio.h
You run this command from your user. It will give you the path to stdio.h. That you can feed in your make
sudo apt-get install build-essential
As a root user, you install build-essential. That should install this missing dependency
execute Jenkins with your privilege, not with root privilege
in your build process, connect to your account (su youruser)
It turned out that in our case it was an issue of environment variables.
What I did to solve it is
Getting the data on environment variables both from Terminal and Jenkins, and write them sorted into 2 files.
Comparing the 2 files with meld
Any variables that seemed relevant that the Terminal environment and the Jenkins didn't I placed into /etc/environment file (Jenkins takes additional environment vars from there)
env | sort > envInTerminal.txt
env | sort > envInJenkins.txt
meld envInTerminal.txt envInJenkins.txt
sudo gedit /etc/environment

How to start odoo server automatically when system is ON

Haii everyone
How to start Odoo server automatically when system is ON.
Normally i searched in google i had found a link " http://www.serpentcs.com/serpentcs-odoo-auto-startup-script-322 "
i follow the each and every step and i started the odoo-server
ps -ax | grep python
5202 ? Sl 0:01 python /home/tejaswini/Odoo_workspace/workspace_8/odoo8/openerp-server --config /etc/odoo-server.conf --logfile /var/log/odoo-server.log
it is showing the server path also
but when i run 0.0.0.0:8069/localhost:8069 in browser it is running
shows This site can’t be reached
please any one help me
Thanks in advance
To start a service automatically when the system turns on, you need to put that service into init script. Try below command
sudo update-rc.d <service_name> defaults
In your case,
sudo update-rc.d odoo-server defaults
Hope it will help you.
For the final step we need to install a script which will be used to start-up and shut down the server automatically and also run the application as the correct user. There is a script you can use in /opt/odoo/debian/init but this will need a few small modifications to work with the system installed the way I have described above. here is the link
Similar to the configuration file, you need to either copy it or paste the contents of this script to a file in /etc/init.d/ and call it odoo-server. Once it is in the right place you will need to make it executable and owned by root:
sudo chmod 755 /etc/init.d/odoo-server
sudo chown root: /etc/init.d/odoo-server
In the configuration file there’s an entry for the server’s log file. We need to create that directory first so that the server has somewhere to log to and also we must make it writeable by the openerp user:
sudo mkdir /var/log/odoo
sudo chown odoo:root /var/log/odoo
reference

WebP support with AWS ElasticBeanstalk

I try to support the use of the webp format with EB, however it's not working as expected...
I created a .config file in .ebextensions with this:
commands:
01-command:
command: wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-0.5.0.tar.gz
02-command:
command: tar xvzf libwebp-0.5.0.tar.gz
03-command:
command: cd libwebp-0.5.0
04-command:
command: ./configure
05-command:
command: make
06-command:
command: sudo make install
But when deploying I got this error:
ERROR: Command failed on instance. Return code: 127 Output: /bin/sh: ./configure: No such file or directory.
Am I doing something wrong?
(environment: 64bit Amazon Linux 2015.09 v2.0.6 running PHP 5.6)
You need to execute the install post deployment. AWS hasn't really documented how to execute commands post deployment, so I'll do so here.
commands:
create_post_dir:
command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
ignoreErrors: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_install_libwebp.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/support/envvars
cd $EB_CONFIG_APP_CURRENT
wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-0.5.0.tar.gz
tar xvzf libwebp-0.5.0.tar.gz
cd libwebp-0.5.0
sudo ./configure
sudo make
sudo make install
As I mentioned, AWS has not really documented that you can actually execute scripts on ElasticBeanstalk post deployment. If you talk a look in the eb-commandprocessor.log file you will see that eb looks for AppDeployPreHook (4 of 6) and AppDeployPostHook (1 of 2). It will look something like this:
[2016-04-13T14:15:22.955Z] DEBUG [8851] : Loaded 6 actions for stage 0.<br>
[2016-04-13T14:15:22.955Z] INFO [8851] : Running 1 of 6 actions: InfraWriteConfig...<br>
[2016-04-13T14:15:22.962Z] INFO [8851] : Running 2 of 6 actions: DownloadSourceBundle...<br>
[2016-04-13T14:15:23.606Z] INFO [8851] : Running 3 of 6 actions: EbExtensionPreBuild...<br>
[2016-04-13T14:15:24.229Z] INFO [8851] : Running 4 of 6 actions: AppDeployPreHook...<br>
[2016-04-13T14:15:28.469Z] INFO [8851] : Running 5 of 6 actions: EbExtensionPostBuild...<br>
[2016-04-13T14:15:28.970Z] INFO [8851] : Running 6 of 6 actions: InfraCleanEbextension...<br>
[2016-04-13T14:15:28.974Z] INFO [8851] : Running stage 1 of command CMD-AppDeploy...<br>
[2016-04-13T14:15:28.974Z] DEBUG [8851] : Loaded 2 actions for stage 1.<br>
[2016-04-13T14:15:28.974Z] INFO [8851] : Running 1 of 2 actions: AppDeployEnactHook...<br>
[2016-04-13T14:15:29.600Z] INFO [8851] : Running 2 of 2 actions: AppDeployPostHook...<br>
[2016-04-13T14:16:42.048Z] INFO [8851] : Running AddonsAfter for command CMD-AppDeploy... <br>
That little "AppDeployPostHook" tells us that it is searching for scripts to run post deployment. You can find the eb deployment scripts in the /opt/elasticbeanstalk directory on the server, and if you ssh in and ls on that directory you'll find hooks, which is what we're looking for, and if you cd hooks you'll find the appdeploy directory, cd appdeploy and then ls and you'll get two directories pre and enact. This seems mundane but is really great, because now we know where eb is looking for scripts it's running. Since the AppDeployPreHook scripts are executing from the "pre" directory we know that we'll need a "post" directory to execute a command post deployment with that AppDeployPostHook that eb is running. Now that we know what to do, we can start writing our commands.
create_post_dir First step is to actually going to create the "post" directory on the server using the mkdir command. mkdir "/opt/elasticbeanstalk/hooks/appdeploy/post" will do that for us, so we'll create that as the command.
files The files config allows us to create a file in a directory via ElasticBeanstalk. Pretty convenient for our purposes! The first line of the files action gives us the name of the file to create. We'll create a shell script to execute out commands, and you can call it whatever you want, but I'd start with 99 and go onwards. We'll call this shell script that we're creating "99_install_libwebp.sh".
File settings The next three lines set the file settings. Make sure root owns them and that there 000755'd.
File Contents This is the content of the file we're creating. Straight forward. Put your shell script in there and you're good to go.
Load environment vars We opted to load the eb environment variables so our script can know where the current version of the app is. It's usually in /var/app/current but it could be elsewhere depending on a variety of factors. We'll use the environment variables to make life a bit easier for us.
Change to our current app directory We're going to cd to our current app directory so we can do what we we're here to do.
Get the package we want use wget to get the libwebp we want
Unpack the package self explanatory
Change to the package directory Now that we've unpacked the package we can change to the package directory.
Do what we need to do We can now run our ./configure, make, and make install.
That's it. You can use the stealthy AppDeployPostHook to run pretty much any post deployment command that you need. Super useful if you need to install packages, restart services, or do anything else post deployment.
I added the code I deployed to Github, for easy reference too. https://github.com/hephalump/testphp
Note: I did this successfully running a slightly different environment. I used ElasticBeanstalk to deploy a new PHP application using the latest environment version which is PHP 5.6 on 64bit Amazon Linux 2016.03 v2.1.0; the environment type that you are using was not available as an option to me... Actually, this was the only version with PHP 5.6 that was available to me so I went with that.

cloudformation composer install

So I am using cloudformation for my AWS setup, I am trying to run composer but for some reason no matter what command I put in my userdata section I always can an error, this is my error:
php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev
[RuntimeException]
The HOME or COMPOSER_HOME environment variable must be set for composer to run correctly
This is my code within the userdata section:
"#composer\n",
"curl -sS https://getcomposer.org/installer | php\n",
"mv composer.phar /usr/local/bin/composer.phar\n",
"#satis\n",
"php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev\n",
Does anyone have any ideas why this might not work and should I should be doing ?
Composer is looking for the location of the .composer directory. Export the HOME or COMPOSER_HOME env variable, e.g. : HOME=/root php /usr/local/bin/composer.phar create-project composer/satis /var/www/satis --stability=dev, it will work fine then.
I had the similar issue with amazon linux ami 2, it was showing in the log All settings correct for using Composer. The HOME or COMPOSER_HOME environment variable must be set for composer to run correctly, but it was not installed at all. Below is the way to fix it. Might be helpful to somebody rather waisting 2,3 hours!
sudo curl -sS https://getcomposer.org/installer | sudo php
mv composer.phar /usr/bin/composer
chmod +x /usr/bin/composer
export COMPOSER_HOME=/root
Agree with Ntwobike's answer.
When launching AWS EC2 instances I was installing composer by running an Ansible playbook during in the user data script run. (The user data script is called by cloud-init during the instance build process).
For some reason at this point in the build the $HOME environment variable is not set. So I needed to add 'export HOME=/root' - e.g.
# These need to be set to enable the composer installer to run. It is probably due to an issue
# with the $HOME variable not yet being set at this point in the instance creation.
export HOME=/root
ansible-playbook --extra-vars "target=localhost" playbooks/debian-9/drush.yml