I have a provisioning setup with vagrant and puppet that works well locally and I'm now tryign to move it to AWS using vagrant-aws.
As I understand it I can make use the AWS user-data field in vagrant as specified to run commands on the first boot of a new vm like so:
aws.user_data = File.read("user_data.txt")
Where user_data.txt contains:
#!/bin/bash
sudo apt-get install -y puppet-common
Then my existing puppet provisioning scripts should be able to run. However this errors out on the vagrant up command with:
[aws] Running provisioner: puppet...
The `puppet` binary appears to not be in the PATH of the guest. This
could be because the PATH is not properly setup or perhaps Puppet is not
installed on this guest. Puppet provisioning can not continue without
Puppet properly installed.
But when I ssh into the machine I see that the user-data did get parsed and puppet is installed successfully. Is the puppet provisioner running before the user-data install puppet maybe? Or is there some better way to install puppet on a vm before trying to provision?
It is broken, but there's a workaround if you're using Ubuntu which is far simpler than building your own AMI.
Add the following line to your config:
aws.user_data = "#cloud-config\nbootcmd:\n - echo 'manual' > /etc/init/ssh.override\npackages:\n - puppet\nruncmd:\n - [ 'rm', '/etc/init/ssh.override' ]\n - [ 'service', 'ssh', 'start' ]\n"
This tells Cloudinit to disable SSH startup early in the boot process and re-enable it once your packages are installed. Now Vagrant can only SSH in to run puppet once the packages are fully installed.
This will probably work for other distros that use Cloudinit aside from Ubuntu, altho it is Upstart specific so the commands may need tweaking.
Well I worked around this by building my own AMI with puppet and other things I need installed, still seems like vagrant-aws is broken or I'm misunderstanding something else here.
Related
I am trying to run an AWS Lambda project locally on Ubuntu. When I run the project with AWS SAM Local it shows me this error: Error: Running AWS SAM projects locally requires Docker. Have you got it installed?
I had trouble installing it on Fedora.
When I followed the Docker postinstall instructions I managed to get past this issue.
https://docs.docker.com/install/linux/linux-postinstall/
I had to:
Delete the ~/.docker directory;
Create the "docker" group;
Add my user to the "docker" group;
Logout and back in again;
Restart the "docker" daemon.
I was then able to run the command:
sam local start-api
If you want to run local sam-cli, you have first install docker from docker official website then run sudo sam local start-api. Note that sudo is necessary for running local developer with needed privileges.
This error mostly arises due to lack of admin privilege to use docker. Just add sudo to your command. This will work.
eg: sudo sam local start-api --region eu-west-3
We are working on Mac and were seeing same message when using an older version of Docker (1.12.6). Have since updated to a newer (but not latest) version 17.12.0-ce-mac49 and it is now fine.
Another cause for this is this recent issue within Docker for Mac.
A quick workaround, as specified in the issue itself, is to run SAM with:
$ DOCKER_HOST=unix://$HOME/.docker/run/docker.sock sam local start-api
You don't need to run SAM as root.
I am using colima for docker on mac with intel chip. and faced this error. was able to resolve it by adding DOCKER_HOST in .zshrc file
vi ~/.zshrc
paste export DOCKER_HOST="unix://$HOME/.colima/docker.sock" in the .zshrc file
escape :wq
I have a working setup of puppet open-source 3.8 with a puppet master and several nodes.
I would like to install puppetDB and a dashboard so I can get a good overview about my nodes.
To not destroy anything from the current setup, I wish to install puppetDB on a separate server. ( Everything is running on AWS EC2 instances.)
I was trying to use the following descriptions and let puppet install puppetDB:
PuppetDB 2.3.8-1.el6 - I believe this is the latest version compatible with puppet 3.8
I've managed to install puppetDB on the DB node, however I can't connect my puppet master to the node.
Based on this documentation:
Connecting Puppet Masters to PuppetDB
I need to install puppetdb-terminus. However I'm using Puppet Open-source, so "sudo puppet resource package puppetdb-terminus ensure=latest" is failing on puppet agent as a dependency.
It's fine, but I have also issues with the alternative solution:
Download the PuppetDB source code, unzip it and navigate into the resulting directory in your terminal.
Run sudo cp -R puppet/lib/puppet/ /usr/lib/ruby/site_ruby/1.8/puppet.
There is no puppet/lib/puppet in neither puppetdb-2.3.8.tar.gz nor puppetdb-3.2.4.tar.gz archives.
As my last hope, I went to github to grab the source:
https://github.com/puppetlabs/puppetlabs-puppetdb/tree/master/lib/puppet
I've copied these files to /usr/lib/ruby/site_ruby/1.8/puppet.
/var/log/messages:Mar 18 13:08:03 ip-10-84-4-172 puppet-master[25616]: Could not configure routes from /etc/puppet/routes.yaml: Could not find terminus puppetdb for indirection facts
At this point I'm completely stuck. How can I verify my puppet-terminus installation? If this way is not good, how can I install it on my puppet master?
(I'm using RHEL6, Puppet Open-source 3.8, I've did all the other changes on puppet master based on the documentation.)
Just for the sake of completeness, here is my puppetDB puppet manifest:
class { 'puppetdb::globals':
version => '2.3.8-1.el6',
}
class { 'puppetdb::database::postgresql':
listen_addresses => $postgres_host,
}
class { 'puppetdb::server':
database_host => $puppetdb_host,
}
I've just migrate my PuppetDB from puppetmaster server to a standalone one. To handle the installation of PuppetDB I've used this module from Puppet Labs.
Its was straightforward. The db migration was done with puppetdb --export from master server and puppetdb --import in the new server. The last thing was change the address for PuppetDB within puppet master config.
[]'s
The puppet repo pages is tricky, there is a separate repo for the Pre4.0 Open Source binaries:
https://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#pre-40-open-source-repositories
After using this repo, there was no more issue during installation.
Just FYI ... context here is AWS Elastic Beanstalk. I'm trying to the install audiowaveform program on 64bit Amazon Linux 2015.03 v1.4.3 (the customer AMI ID is ami-6b50291c). Running this ... 👇
$ sudo yum install git cmake libmad-devel libsndfile-devel gd-devel boost-devel
... successfully installs all packages except libmad-devel and libsndfile-devel. Below is the relevant output ...
Failed to set locale, defaulting to C
Loaded plugins: priorities, update-motd, upgrade-helper
amzn-main/2015.03 | 2.1 kB 00:00
amzn-updates/2015.03 | 2.3 kB 00:00
Package git-2.1.0-1.38.amzn1.x86_64 already installed and latest version
Package cmake-2.8.12-2.20.amzn1.x86_64 already installed and latest version
No package libmad-devel available.
No package libsndfile-devel available.
Package gd-devel-2.0.35-11.10.amzn1.x86_64 already installed and latest version
Package boost-devel-1.53.0-14.21.amzn1.x86_64 already installed and latest version
Nothing to do
That said, this is not a problem with audiowaveform ... all this means is that the repositories enabled for Amazon Linux AMIs do not have libmad-devel and libsndfile-devel by default. I probably have to simply add my own sources I guess.
Also to note is that no yum packages exist for audio waveform so I have to build this manually.
Obtain the source ... 👇
$ git clone https://github.com/bbcrd/audiowaveform.git
$ cd audio waveform
Then build and install ... 👇
$ mkdir build
$ cd build
$ cmake ..
$ make
$ sudo make install
Question 1
On AWS EB ... the EC2 instances are configured to use Amazon sources which don't have the above packages i.e. libmad-devel and libsndfile-devel. What would be the recommended approach to adding these packages so that they are available to yum?
I stress recommended because I feel that changing the sources from Amazon's could not be the best approach. Nor is adding another source that could conflict with Amazon's packages ... etc etc etc ...
Question 2
Assuming I'm able to install libmad-devel and libsndfile-devel. I still have to build this manually since there are no packages of audiowaveform. On AWS EB I could write a script to do this as each instance is being instantiated ... but I feel this isn't ideal, slow and kinda error-prone. Anyone have advice on how I can do this better?
Probably prepare an AMI with this already built that's based off ami-6b50291c. Thoughts?
Core Objective
I don't have to use audiowaveform ... my objective really is to extract the peak points of some audio (MP3). I will set this up as a separate question.
Amazon Elastic Beanstalk tends to be very restricted in terms of what software you can install on it. I solved it by dockerizing my application environment. This is possible now even on Elastic Beanstalk.
Learn more about Elastic Beanstalk's support for Docker ...
AWS Elastic Beanstalk makes it easy for you to deploy and manage
applications in the AWS cloud. After you upload your application,
Elastic Beanstalk will provision, monitor, and scale capacity (Amazon
EC2 instances), while also load balancing incoming requests across all
of the healthy instances.
Docker automates the deployment of applications in the form of
lightweight, portable, self-sufficient containers that can run in a
variety of environments. Containers can be populated from pre-built
Docker images or from a simple recipe known as a Dockerfile.
Docker’s container-based model is very flexible. You can, for example,
build and test a container locally and then upload it to the AWS Cloud
for deployment and scalability. Docker’s automated deployment model
ensures that the runtime environment for your application is always
properly installed and configured, regardless of where you decide to
host the application.
This way ... you can do whatever you want in the container and that container will run on the kernel provided by the Amazon Linux AMI instance (obviously completely isolated).
I'm also somehow having hard time getting yum to find libsndfile on Amazon Linux AMI (RedHat 7.4). Repositories I've added to yum never seem to contain it. (How to add new repos is described here )
Finally I just downloaded and installed the rpms directly:
wget http://ftp.altlinux.org/pub/distributions/ALTLinux/Sisyphus/x86_64/RPMS.classic//libsndfile-1.0.28-alt1.x86_64.rpm
wget http://ftp.altlinux.org/pub/distributions/ALTLinux/Sisyphus/x86_64/RPMS.classic//libsndfile-devel-1.0.28-alt1.x86_64.rpm
sudo yum localinstall libsndfile-devel-1.0.28-alt1.x86_64.rpm
This way I got PySoundfile working finally.
I am trying to deploy a Django application on Google Compute Engine. I'm using a Debian 7 image and want to set up Django with Nginx, Gunicorn, virtualenv, supervisor and PostgreSQL. I have everything running fine on my development machine which is running Ubuntu 14.04 with Django installed and PostgreSQL as the backend.
I'm using the tutorial located at http://datacommunitydc.org/blog/2013/12/a-tutorial-for-deploying-a-django-application-that-uses-numpy-and-scipy-to-google-compute-engine-using-apache2-and-modwsgi/. I'm also using the tutorial located at http://michal.karzynski.pl/blog/2013/06/09/django-nginx-gunicorn-virtualenv-supervisor/ as it's specific to virtualenv and PostgreSQL which I'm using on my development machine. I've setup my GCE instance, instaled and updated aptitude. I've installed PostgreSQL however when I attempt to create a database user and a new database for the app I get an error and nothing is created.
Following the tutorial I've run:
$ sudo su - postgres
postgres#django:~$ createuser -P
Enter name of role to add: hello_django
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n
When it attempts to create the new user role I receive the following error:
createuser: could not connect to database postgres: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
When I run the shell script ls /etc/init.d it says that postgresql is running, but I still can't add the new role. Can someone tell me what I'm doing wrong?
Regards.
I wasn't able to reproduce the issue on my end, but it appears to be an issue with PostgreSQL and its dependencies. You can try removing all installed PostgreSQL components and dependencies and then reinstalling PostgreSQL:
sudo apt-get remove --purge postgresql-9.1*
sudo apt-get install postgresql-9.1
If it's still unable to connect to the database, the issue might be originating from your $PATH, in which case you'll need to point it to /usr/local/bin/psql.
I have just had the same problem.
This is most likely cause the postgres cluster has not been initialised yet.
And the reason that this didn't install automatically is because you have set up the locale of the box yet. This is something that has to be done on Amazon EC2 instances as well.
You need to run:
sudo apt-get install locales
And then:
sudo dpkg-reconfigure locales
I had to choose which locales I wanted to setup, I chose en_AU UTF-8.
After this I rebooted, then I could run this to initialise the cluster:
sudo pg_createcluster 9.1 main --start
This started the service and created the pg_hba.conf files etc.
I faced a similar problem a while back. It can resolved using a few simple steps:
As postgres user run : initdb --locale en_US.UTF-8 -E UTF8 -D 'var/lib/postgres/data'. Note depending on the distro postgres in the command can be pgsql. You can easily check if the directory exists with an ls command.
systemctl start postgresql (if you have systemd) or just a /etc/init.d/postgresql start should do. These commands must be rub as the superuser.
All this is from the ArchWiki.
I'm using Vagrant to deploy chef scripts to an AWS server (and it mostly works awesome). I have set up a local rsync in my Vagrantfile to mirror my dev directory onto the server.
config.vm.synced_folder "../geoevents", "/vagrant/geoevents-repo"
And this syncs find on 'vagrant provision'. I'm wondering if there is an easier way that I can have vagrant only trigger that rsync, or to control how often rsync occurs?
Or, should I not be using rsync, but instead mount a shared file system?
Vagrant CLI now has two new commands, vagrant rsync and vagrant rsync-auto which can do the job.
Command: vagrant rsync
This command forces a re-sync of any rsync synced folders.
Note that if you change any settings within the rsync synced folders such as exclude paths, you will need to vagrant reload before this command will pick up those changes.
https://www.vagrantup.com/docs/cli/rsync.html
https://www.vagrantup.com/docs/cli/rsync-auto.html
https://www.vagrantup.com/docs/cli/non-primary.html
Currently, you can fit your needs with the following plugin:
https://github.com/cromulus/vagrant-rsync
By the way, most of the plugin features will be available in Vagrant 1.5 (currently in development).
The vagrant-rsync is deprecated as of Vagrant 1.5. One solution out there is vagrant-unison. You may also check out this discussion. What should also work is a vagrant reload.