I am trying to use chef with aws opsworks.
however it doesn't seem to be loading my template.
Here is the part of the recipe concerned which is in /api/recipes/configure.rb
# write out our applications site configurations
template "/etc/httpd/sites-available/#{application}.conf" do
source 'site-config.erb'
owner 'root'
group 'root'
mode 0644
variables(
:docroot => "#{deploy[:deploy_to]}/public"
)
end
Here is the stack trace
[2014-04-15T05:08:03+00:00] INFO: Processing template[/etc/httpd/sites-available/testing_app.conf] action create (api::configure line 33)
================================================================================
Error executing action `create` on resource 'template[/etc/httpd/sites-available/testing_app.conf]'
================================================================================
Chef::Exceptions::FileNotFound
------------------------------
Cookbook 'api' (0.1.0) does not contain a file at any of these locations:
templates/amazon-2013.09/site-config.erb
templates/amazon/site-config.erb
templates/default/site-config.erb
Resource Declaration:
---------------------
# In /var/lib/aws/opsworks/cache/cookbooks/api/recipes/configure.rb
33: template "/etc/httpd/sites-available/#{application}.conf" do
34: source 'site-config.erb'
35: owner 'root'
36: group 'root'
37: mode 0644
38: variables(
39: :docroot => "#{deploy[:deploy_to]}/public"
40: )
41: end
42:
Compiled Resource:
------------------
# Declared in /var/lib/aws/opsworks/cache/cookbooks/api/recipes/configure.rb:33:in `block in from_file'
template("/etc/httpd/sites-available/testing_app.conf") do
provider Chef::Provider::Template
action "create"
retries 0
retry_delay 2
path "/etc/httpd/sites-available/testing_app.conf"
backup 5
atomic_update true
source "site-config.erb"
variables {:docroot=>"/srv/www/testing_app/public"}
cookbook_name "api"
recipe_name "configure"
owner "root"
group "root"
mode 420
end
[2014-04-15T05:08:03+00:00] INFO: Running queued delayed notifications before re-raising exception
[2014-04-15T05:08:03+00:00] ERROR: Running exception handlers
[2014-04-15T05:08:03+00:00] ERROR: Exception handlers complete
[2014-04-15T05:08:03+00:00] FATAL: Stacktrace dumped to /var/lib/aws/opsworks/cache/chef-stacktrace.out
[2014-04-15T05:08:03+00:00] ERROR: template[/etc/httpd/sites-available/testing_app.conf] (api::configure line 33) had an error: Chef::Exceptions::FileNotFound: Cookbook 'api' (0.1.0) does not contain a file at any of these locations:
templates/amazon-2013.09/site-config.erb
templates/amazon/site-config.erb
templates/default/site-config.erb
[2014-04-15T05:08:04+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
And here is my directory structure
You will need to put the template into one of the following directories api/templates/default api/templates/amazon or api/templates/amazon-2013.09. If you want the same template files to work on any platform, then put the templates in the api/templates/default directory.
Check out http://docs.chef.io/templates.html#file-specificity for more details on where you might want to place your templates.
Related
I'm no developer and a newbie with Chef.
I created a simple recipe and tested it on a test Kitchen and it works well but when I upload it to Jenkins it won't pass.
Can you help me point out what I'm missing?
Recipe:
# Recipe:: default
# Download VerneMQ
remote_file "#{Chef::Config[:file_cache_path]}/vernemq_1.4.0-1_amd64.deb" do
source 'https://bintray.com/artifact/download/erlio/vernemq/deb/trusty/vernemq_1.4.0-1_amd64.deb'
end
# Install
dpkg_package 'vernemq_1.4.0-1_amd64.deb' do
source "#{Chef::Config[:file_cache_path]}/vernemq_1.4.0-1_amd64.deb"
action :install
end
# Service Start
service 'vernemq' do
action [:enable, :start]
end
# Configurations
template '/etc/vernemq/vernemq.conf' do
source 'vernemq.conf.erb'
owner 'root'
group 'root'
mode '0644'
end
template '/etc/vernemq/vmq.passwd' do
source 'vmq.passwd.erb'
end
This is my unit test.
Unit Test:
require 'spec_helper'
require 'chefspec'
describe 'vernemq::default' do
let(:chef_run) { ChefSpec::SoloRunner.new(platform: 'ubuntu', version: '14.04').converge(described_recipe) }
it 'downloads the remote_file' do
expect(chef_run).to create_remote_file('#{Chef::Config[:file_cache_path]}/vernemq_1.4.0-1_amd64.deb')
end
it 'installs a dpkg_package' do
expect(chef_run).to install_dpkg_package('vernemq')
end
it 'executes both actions' do
expect(chef_run).to enable_service('vernemq')
expect(chef_run).to start_service('vernemq')
end
it 'creates a file' do
expect(chef_run).to create_directory('/etc/vernemq/vernemq.conf')
end
it 'creates a file' do
expect(chef_run).to create_directory('/etc/vernemq/vmq.passwd')
end
end
errors from Jenkins
01:06:57 tm_vernemq::default
01:06:58 [2018-07-19T01:06:58-07:00] WARN: Chef::Provider::AptRepository already exists! Cannot create deprecation class for LWRP provider apt_repository from cookbook apt
01:06:58 [2018-07-19T01:06:58-07:00] WARN: Property sensitive of resource apt_repository overwrites an existing method. Please use a different property name. This will raise an exception in Chef 13. (CHEF-11)/tmp/d20180719-10648-933ma2/cookbooks/apt/resources/repository.rb:54:in class_from_file'.
01:06:58 Please see https://docs.chef.io/deprecations_property_name_collision.html for further details and information on how to correct this problem. at /opt/chefdk/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/event_dispatch/dispatcher.rb:43:incall'
01:06:58 [2018-07-19T01:06:58-07:00] WARN: AptRepository already exists! Deprecation class overwrites Custom resource apt_repository from cookbook apt
01:06:58 executes both actions
01:06:58 installs a dpkg_package (FAILED - 1)
01:06:58 creates a file (FAILED - 2)
01:06:58 creates a file (FAILED - 3)
01:06:58 downloads the remote_file (FAILED - 4)
01:06:58
01:06:58 Failures:
01:06:58
01:06:58 1) vernemq::default installs a dpkg_package
01:06:58 Failure/Error: expect(chef_run).to install_dpkg_package('vernemq')
01:06:58
01:06:58 expected "dpkg_package[vernemq]" with action :install to be in Chef run. Other dpkg_package resources:
01:06:58
01:06:58 dpkg_package[vernemq_1.4.0-1_amd64.deb]
01:06:58
01:06:58 # ./test/unit/recipes/default_spec.rb:18:in `block (2 levels) in <top (required)>'
01:06:58
01:06:58 2) tm_vernemq::default creates a file
01:06:58 Failure/Error: expect(chef_run).to create_directory('/etc/vernemq/vmq.passwd')
01:06:58
01:06:58 expected "directory[/etc/vernemq/vmq.passwd]" with action :create to be in Chef run. Other directory resources:
01:06:58
You didn't include the full error message so this is mostly a guess but you have dpkg_package 'vernemq_1.4.0-1_amd64.deb' do vs install_dpkg_package('vernemq'). Those need to have the same name in them.
I need to include res-client into my module in order to use RestClient::Resource.
I use the method in the module in a Chef recipe (ruby_block resource).
When my ruby_block resource tries to run the method in the module it output this error:
ERROR: cannot load such file -- rest-client
The following is my module residing in libraries folder:
module SecretServer
def fetch_token
payload = { :username => "***", :password => "***", :grant_type => "****"}
uri = ***
request = RestClient::Resource.new(uri, :verify_ssl => false)
post_request = request.post payload, :content_type => 'application/x-www-form-urlencoded'
token = JSON.parse(post_request)["access_token"]
return token
end
end
require 'rest-client'
Chef::Recipe.include(SecretServer)
Chef::Resource.include(SecretServer)
The following is the resource that calls the function in module:
ruby_block 'parse data' do
block do
res = fetch_token
puts res
end
end
This is just one of the several recipes in my cookbook. This recipe runs after the target node is almost ready and 'bundle install' has been run on the target node.
I also install rest-client in the target node. I tried each of following resource before my ruby_block resource:
chef_gem 'rest-client' do
action :install
end
gem_package 'rest-client' do
action :install
end
My question is how to include 'rest-client' and utilize it in Chef recipes?
Long ago, the HTTP clients lived together in harmony. Then everything changed when the JSON gem attacked. Only the Chef::HTTP was still to be included with Chef as all the other clients were under too much flux to include.
We don't include that gem anymore, so to use it, you would have to install it yourself either via a cookbook gem dependency or a chef_gem resource. But for simple stuff you can just use our Chef::HTTP::SimpleJSON client instead:
Chef::HTTP::SimpleJson.new(uri).post("", args)["access_token"]
Or something like that (specifics depend on if you must use form encoding or if the server also speaks JSON).
I want to use the environment_variables of an App to set some properties, but my instance keep failing at setup, reading the documentation (http://docs.aws.amazon.com/opsworks/latest/userguide/workingcookbook-events.html) I found this:
"Setup includes Deploy; it runs the Deploy recipes after setup is complete.".
This is part of my Deploy script
if node['deploy']['ws']['application'] && node['deploy']['ws']['environment_variables']['is_root_app'] == 'true'
web_app node['deploy']['ws']['application'] do
server_name node['fqdn']
server_aliases node['deploy']['ws']['domains']
template 'web_app.conf.erb'
docroot node.default['apache']['docroot_dir']
ssl_enabled node['deploy']['ws']['ssl_support']
enable true
end
end
This is the Setup log
NoMethodError
-------------
undefined method `[]' for nil:NilClass
Cookbook Trace:
---------------
/var/chef/runs/254b8caa-cd70-4525-a2fd-71044afdf62d/local-mode-cache/cache/cookbooks/WebService/recipes/deploy_app.rb:8:in `from_file'
Relevant File Content:
----------------------
/var/chef/runs/254b8caa-cd70-4525-a2fd-71044afdf62d/local-mode-cache/cache/cookbooks/WebService/recipes/deploy_app.rb:
1: #
2: # Cookbook Name:: WebService
3: # Recipe:: default
4: #
5: # Copyright (c) 2016 The Authors, All Rights Reserved.
6:
7:
8>> if node['deploy']['ws']['application'] && node['deploy']['ws']['environment_variables']['is_root_app'] == 'true'
9: web_app node['deploy']['ws']['application'] do
10: server_name node['fqdn']
11: server_aliases node['deploy']['ws']['domains']
12: template 'web_app.conf.erb'
13: docroot node.default['apache']['docroot_dir']
14: ssl_enabled node['deploy']['ws']['ssl_support']
15: enable true
16: end
17: else
Whats the purpose of running the Deploy recipes after Setup, if I didnt deployed any App yet??? Where the environment_variables can be used if not on the Deploy recipes???
Imagine you have everything set up and you just start a new machine. It will run setup, configure then deploy otherwise you would end up with an instance that's not ready for production.
deployment fails when you're trying to access attributes that aren't set
You can use http://ruby-doc.org/core-1.9.3/Hash.html#method-i-has_key-3F to check this
e.g.
if node[:deploy].has_key?("ws")
end
Same for your env variables
I have inherited a Rails project and I am unable to deploy it using the supplied instructions using Capistrano. I am not familiar with capistrano at all.
Whatever I try I always get the following error message
(Backtrace restricted to imported tasks)
cap aborted!
ArgumentError: wrong number of arguments (1 for 2)
Tasks: TOP => production
(See full trace by running task with --trace)
This includes
$ bundle exec cap -T
$ bundle exec cap production deploy
I am following the guide here https://github.com/capistrano/capistrano
I have starred out important info but here is the a copy of the config/deploy/staging.rb
role :app, %w{deploy#**.**.**.**:****}
role :web, %w{deploy#**.**.**.**:****}
role :db, %w{deploy#**.**.**.**:****}
set :branch, 'develop'
set :deploy_to, '/var/www/soreen.staged.ws'
set :ssh_options, {
keepalive: true,
keepalive_interval: 60 #seconds
}
and here is Capfile
# Load DSL and Setup Up Stages
require 'capistrano/setup'
# Includes default deployment tasks
require 'capistrano/deploy'
require 'capistrano/rbenv'
require 'capistrano/bundler'
require 'capistrano/rails/migrations'
require 'capistrano/rails/assets'
require 'capistrano/rails/collection'
# Loads custom tasks from `lib/capistrano/tasks' if you have any defined.
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }
and here is config/deploy.rb
# config valid only for current version of Capistrano
lock '3.4.0'
set :application, 'Soreen'
set :repo_url => "****************"
set :linked_dirs, fetch(:linked_dirs, []).push('log', 'tmp/pids', 'tmp/cache', 'tmp/sockets', 'vendor/bundle', 'public/system')
set :rbenv_type, :user # or :system, depends on your rbenv setup
set :rbenv_ruby, '2.1.2'
set :rbenv_prefix, "RBENV_ROOT=#{fetch(:rbenv_path)} RBENV_VERSION=# {fetch(:rbenv_ruby)} #{fetch(:rbenv_path)}/bin/rbenv exec"
set :rbenv_map_bins, %w{rake gem bundle ruby rails}
set :rbenv_roles, :all # default value
namespace :deploy do
desc 'Restart application'
task :restart do
on roles(:app), in: :sequence, wait: 5 do
# Your restart mechanism here, for example:
execute :mkdir, '-p', "#{ release_path }/tmp"
execute :touch, release_path.join('tmp/restart.txt')
end
end
after :publishing, :restart
after :restart, :clear_cache do
on roles(:web), in: :groups, limit: 3, wait: 10 do
end
end
end
Please help !
You can get a backtrace for the error using the following:
bundle exec cap --backtrace -T
That will give you more information. Likely this is a syntax error in your configuration.
I have two recipe in my AWS OpsWorks custom cookbook
First one is myserver/recipes/supervisor.rb
#myserver::supervisor
supervisor_service "mylistener" do
command "/usr/bin/php /var/www/listener.php"
autostart true
autorestart true
numprocs 1
process_name "%(program_name)s-%(process_num)s"
end
and the second one is myserver/recipes/update_code.rb
#myserver::update_code
include_recipe "myserver::supervisor"
execute "update_code" do
command %Q {
/usr/local/bin/downloadfile.sh
/usr/local/bin/changelink.sh
}
notifies :restart, "supervisor_service[mylistener]"
end
The supervisor_service is provided from here.
At first I tried executing the first recipe in running instances. The execution succeeded and the supervisor was running smoothly.
The second recipe is basically updating the code run by the supervisor. So I need to restart the supervisor. But the then I got this error everytime I executed the service.
================================================================================
Error executing action `restart` on resource 'supervisor_service[mylistener]'
================================================================================
RuntimeError
------------
Supervisor service mylistener cannot be restarted because it does not exist
Resource Declaration:
---------------------
# In /opt/aws/opsworks/releases/20140116091718_218/site-cookbooks/myserver/recipes/supervisor.rb
9: supervisor_service "mylistener" do
10: command "/usr/bin/php /var/www/listener.php"
11: autostart true
12: autorestart true
13: numprocs 1
14: process_name "%(program_name)s-%(process_num)s"
15: end
What am I doing wrong?
UPDATE
I put include_recipe 'supervisor' in the file myserver/recipes/supervisor.rb.
#myserver::supervisor
include_recipe 'supervisor'
supervisor_service "mylistener" do
command "/usr/bin/php /var/www/listener.php"
autostart true
autorestart true
numprocs 1
process_name "%(program_name)s-%(process_num)s"
end
But when I execute myserver::update_code in my AWS OpsWorks, it still returns error below, but with a new section Compiled Resource.
================================================================================
Error executing action `restart` on resource 'supervisor_service[mylistener]'
================================================================================
RuntimeError
------------
Supervisor service mylistener cannot be restarted because it does not exist
Resource Declaration:
---------------------
# In /opt/aws/opsworks/releases/20140116091718_218/site-cookbooks/myrecipe/recipes/supervisor.rb
11: supervisor_service "mylistener" do
12: command "/usr/bin/php /var/www/listener.php"
13: autostart true
14: autorestart true
15: numprocs 1
16: process_name "%(program_name)s-%(process_num)s"
17: end
Compiled Resource:
------------------
# Declared in /opt/aws/opsworks/releases/20140116091718_218/site-cookbooks/myserver/recipes/supervisor.rb:11:in `from_file'
supervisor_service("mylistener") do
priority 999
numprocs_start 0
autorestart true
retries 0
updated true
stderr_capture_maxbytes "0"
exitcodes [0, 2]
stderr_logfile "/var/log/mylistener.err.log"
cookbook_name :myserver
stdout_capture_maxbytes "0"
autostart true
recipe_name "supervisor"
serverurl "AUTO"
action :enable
stderr_logfile_backups 10
startretries 3
process_name "%(program_name)s-%(process_num)s"
stdout_logfile_backups 10
stopwaitsecs 10
stderr_logfile_maxbytes "50MB"
startsecs 1
numprocs 1
retry_delay 2
stdout_logfile_maxbytes "50MB"
command "/usr/bin/php /var/www/listener.php"
stopsignal :TERM
service_name "mylistener"
stdout_logfile "/var/log/mylistener.out.log"
end
UPDATE
I finally use this manual execution
execute "supervisor-restart-mylistener" do
command "supervisorctl restart mylistener:*"
end
But still trying to figure out why the notifies doesn't work.
I was getting the same error but it was due to a bug in the supervisor cookbook.
https://github.com/poise/supervisor/issues/43
I reverted back to the Supervisor cookbook 0.4.2 and it works fine again.
Unless you're including both of those recipes in the run_list or using include_recipe, this error is correct. If update_code.rb notifies supervisor_service, then the supervisor.rb recipe is a "dependency" of that recipe. You should add the following to the top of your update_code.rb:
include_recipe 'myserver::supervisor'