We're using AwWS to deploy ou app. We have installed chromimum and epel packages on our instance to male Puppeteer lib work. However for somexreason that instance is terminated and newcinstance assigned to elasticbeanstalk app, apparently without any package installed
How this could be bypassed.
Is it best to create some predeploy sh script in ebextensions?
Related
this last week I have been trying to upload a flask app using AWS Beanstalk.
The main problem for me was loading a very heavy library as part of the bundle (there is a 500mb limit for uploading the bundle code).
Instead, I tried to use requirements.txt file so it would download the library directly to the server.
Unfortunately, every time I tried to include the library name in the requirements file, it failed to load it (torch library).
on pythonanywhere server there is a console which allows you to access the virtual environment and simply type
pip install torch
which was very useful and comfortable.
I am looking for something similar in AWS beanstalk, so that I could install the library directly instead of relying on the requirements.txt file.
I have been at it for a few days now and can't make any progress.
your help would be much appreciated.
another question,
is it possible to load the venv to Amazon-S3 and then access the folder from the beanstalk environment?
Its not a good practice to "manually" install your dependencies or configure your EB env from inside. This is only useful for testing and debugging purposes. Thus keep that it mind.
To get your venv, you have to ssh to your EB instance using regular ssh or web-based clients available in AWS EC2 console when you locate your EB EC2 instance. Session manager should work out-of-the-box to enable you to login to the instance.
When you login to the instance, then to activate your venv, you do:
# start bash
bash
# source venv
source /var/app/venv/staging-*/bin/activate
I am running into an issue with the install wizard with Jenkins when following a Set Up a Jenkins Build Server tutorial from Amazon.
My EC2 instance is a t2.small. It was a t2.micro until I saw this SO post so I switched it to a t2.small. It doesn't appear to be a memory issue. I am getting an error when creating my initial user or trying to Continue as admin.
When inspecting the element, trying to Save and Finish when creating an initial user, POST http://<domain>:8080/setupWizard/createAdminUser errors out with a ERR_CONNECTION_RESET error. (I don't see anything in /var/log/jenkins/jenkins.log about this failure either)
I am running java 1.8 and I've tried with Jenkins 2.71-1.1 and Jenkins 2.61-1.1
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)
I grepped the error log and only found one log pertaining to errors (but I'm not sure this is related):
Jul 24, 2017 11:09:50 PM hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1 error
INFO: Failed to instantiate optional component hudson.plugins.build_timeout.operations.AbortAndRestartOperation$DescriptorImpl; skipping
I created a CDK to provision a Jenkins service in AWS. have a try
https://github.com/seraphjiang/jenkinscdk
Install the certificates-
sudo apt install ca-certificates
then, Try Updating & Upgrading the packages-
sudo apt upgrade
sudo apt update
Then follow the link to install the Jenkins :)
https://www.digitalocean.com/community/tutorials/how-to-install-jenkins-on-ubuntu-18-04
Just FYI ... context here is AWS Elastic Beanstalk. I'm trying to the install audiowaveform program on 64bit Amazon Linux 2015.03 v1.4.3 (the customer AMI ID is ami-6b50291c). Running this ... 👇
$ sudo yum install git cmake libmad-devel libsndfile-devel gd-devel boost-devel
... successfully installs all packages except libmad-devel and libsndfile-devel. Below is the relevant output ...
Failed to set locale, defaulting to C
Loaded plugins: priorities, update-motd, upgrade-helper
amzn-main/2015.03 | 2.1 kB 00:00
amzn-updates/2015.03 | 2.3 kB 00:00
Package git-2.1.0-1.38.amzn1.x86_64 already installed and latest version
Package cmake-2.8.12-2.20.amzn1.x86_64 already installed and latest version
No package libmad-devel available.
No package libsndfile-devel available.
Package gd-devel-2.0.35-11.10.amzn1.x86_64 already installed and latest version
Package boost-devel-1.53.0-14.21.amzn1.x86_64 already installed and latest version
Nothing to do
That said, this is not a problem with audiowaveform ... all this means is that the repositories enabled for Amazon Linux AMIs do not have libmad-devel and libsndfile-devel by default. I probably have to simply add my own sources I guess.
Also to note is that no yum packages exist for audio waveform so I have to build this manually.
Obtain the source ... 👇
$ git clone https://github.com/bbcrd/audiowaveform.git
$ cd audio waveform
Then build and install ... 👇
$ mkdir build
$ cd build
$ cmake ..
$ make
$ sudo make install
Question 1
On AWS EB ... the EC2 instances are configured to use Amazon sources which don't have the above packages i.e. libmad-devel and libsndfile-devel. What would be the recommended approach to adding these packages so that they are available to yum?
I stress recommended because I feel that changing the sources from Amazon's could not be the best approach. Nor is adding another source that could conflict with Amazon's packages ... etc etc etc ...
Question 2
Assuming I'm able to install libmad-devel and libsndfile-devel. I still have to build this manually since there are no packages of audiowaveform. On AWS EB I could write a script to do this as each instance is being instantiated ... but I feel this isn't ideal, slow and kinda error-prone. Anyone have advice on how I can do this better?
Probably prepare an AMI with this already built that's based off ami-6b50291c. Thoughts?
Core Objective
I don't have to use audiowaveform ... my objective really is to extract the peak points of some audio (MP3). I will set this up as a separate question.
Amazon Elastic Beanstalk tends to be very restricted in terms of what software you can install on it. I solved it by dockerizing my application environment. This is possible now even on Elastic Beanstalk.
Learn more about Elastic Beanstalk's support for Docker ...
AWS Elastic Beanstalk makes it easy for you to deploy and manage
applications in the AWS cloud. After you upload your application,
Elastic Beanstalk will provision, monitor, and scale capacity (Amazon
EC2 instances), while also load balancing incoming requests across all
of the healthy instances.
Docker automates the deployment of applications in the form of
lightweight, portable, self-sufficient containers that can run in a
variety of environments. Containers can be populated from pre-built
Docker images or from a simple recipe known as a Dockerfile.
Docker’s container-based model is very flexible. You can, for example,
build and test a container locally and then upload it to the AWS Cloud
for deployment and scalability. Docker’s automated deployment model
ensures that the runtime environment for your application is always
properly installed and configured, regardless of where you decide to
host the application.
This way ... you can do whatever you want in the container and that container will run on the kernel provided by the Amazon Linux AMI instance (obviously completely isolated).
I'm also somehow having hard time getting yum to find libsndfile on Amazon Linux AMI (RedHat 7.4). Repositories I've added to yum never seem to contain it. (How to add new repos is described here )
Finally I just downloaded and installed the rpms directly:
wget http://ftp.altlinux.org/pub/distributions/ALTLinux/Sisyphus/x86_64/RPMS.classic//libsndfile-1.0.28-alt1.x86_64.rpm
wget http://ftp.altlinux.org/pub/distributions/ALTLinux/Sisyphus/x86_64/RPMS.classic//libsndfile-devel-1.0.28-alt1.x86_64.rpm
sudo yum localinstall libsndfile-devel-1.0.28-alt1.x86_64.rpm
This way I got PySoundfile working finally.
I'm deploying an app using Elastic Beanstalk and part of the app has a grunt task that runs "sass", I have sass being installed but it is being installed locally and thus isn't part of the PATH, so the grunt task fails.
I just attempted adding a command to the beanstalk config that does sudo gem install sass but that fails with Command failed on instance. Return code: 1 Output: sudo: sorry, you must have a tty to run sudo.
What would be the best way to get sass into the PATH? There didn't seem to be an easy way to update the PATH / set the .bashrc with elastic beanstalk
Using ebextensions commands is the way to go. You don't need sudo, as the commands run with the necessary privileges.
Also looks like you are using node solution stack (since you mentioned grunt). There may be multiple versions of ruby on your instance. You want to be sure to use the right gem binary so your dependencies are installed in the right location.
There is a ruby installed in /usr/bin and another one in /opt/elasticbeanstalk/lib. The latter is used by Elastic Beanstalk which is not what you want. You want to run the gem binary under /usr/bin.
I have a provisioning setup with vagrant and puppet that works well locally and I'm now tryign to move it to AWS using vagrant-aws.
As I understand it I can make use the AWS user-data field in vagrant as specified to run commands on the first boot of a new vm like so:
aws.user_data = File.read("user_data.txt")
Where user_data.txt contains:
#!/bin/bash
sudo apt-get install -y puppet-common
Then my existing puppet provisioning scripts should be able to run. However this errors out on the vagrant up command with:
[aws] Running provisioner: puppet...
The `puppet` binary appears to not be in the PATH of the guest. This
could be because the PATH is not properly setup or perhaps Puppet is not
installed on this guest. Puppet provisioning can not continue without
Puppet properly installed.
But when I ssh into the machine I see that the user-data did get parsed and puppet is installed successfully. Is the puppet provisioner running before the user-data install puppet maybe? Or is there some better way to install puppet on a vm before trying to provision?
It is broken, but there's a workaround if you're using Ubuntu which is far simpler than building your own AMI.
Add the following line to your config:
aws.user_data = "#cloud-config\nbootcmd:\n - echo 'manual' > /etc/init/ssh.override\npackages:\n - puppet\nruncmd:\n - [ 'rm', '/etc/init/ssh.override' ]\n - [ 'service', 'ssh', 'start' ]\n"
This tells Cloudinit to disable SSH startup early in the boot process and re-enable it once your packages are installed. Now Vagrant can only SSH in to run puppet once the packages are fully installed.
This will probably work for other distros that use Cloudinit aside from Ubuntu, altho it is Upstart specific so the commands may need tweaking.
Well I worked around this by building my own AMI with puppet and other things I need installed, still seems like vagrant-aws is broken or I'm misunderstanding something else here.