Drupal on Elastic Beanstalk Redirects to Install.php on "eb deploy" - amazon-web-services

I am encountering what I believe to be permission issues when trying to deploy a Drupal application onto Elastic Beanstalk
I followed this tutorial to get Drupal up and running: http://comm-press.de/en/blog/drupal-climbs-aws-elastic-beanstalk
I am using a Postgres database and I am entering the correct credentials when filling out the forms on install.php, without error.
Any subsequent deploys after the initial deploy brings me back to install.php. After entering in my database information, I get this message, telling me Drupal is already installed (which it is).
http://i.imgur.com/N6KDvvo.png
Why does my site get redirected to install.php after 'eb deploy'? What permissions should I set my drupal folder such that settings.php and /sites/default/files is generated?

The install state is controlled by the DB-- if Drupal bootstraps with no DB information, you are routed to the code that asks you for it.
I was able to bypass this part by setting up a AWS RDS DB and connecting all instances to it.
--But, wait, there's more. Now that having all the instances reading from the same DB has squashed most of the concurrency problems between instances*. Go ahead, try and add a photo to your admin profile. I will wait. Yep, most of the time you'll get the wrong instance and the single image on one instance is not shown on all instances.
I am working on solving that problem with a start up & cron job script that loads updates to resources from the AWS S3 service.
Step A load code into S3
Step B set an accessible timestamp for $lastModified to now()
Step 1 wget/curl a timestamp of the last remote modification ($lastModified)
Step 2 compare the local last updated stamp ($lastUpdated) to remote last modified timestamp
Step 3 if ($lastModified == $lastUpdated) {die} else {update incremental changes && set $lastUpdated = $lastModified}
Watch that first incremental update, it's a doozy.
So... additionally I should mention that we are installing completely vanilla drupal when we instantiate an image, as part of the Docker file from drupal the drupal apache image the last thing the Docker file runs is a setup script.
Elastic Beanstalk sets Environment variables-- some of those variables are the amazon access key id and access secret key.
I curl an IP only available inside of Elastic Beanstalk curl -v 169.254.169.254
From that output in the setup script I can tell if I am local AWS EB or in AWS EB. That allows me to conditionally change certain configurations-- like connecting RDS or a local MySQL DB to Drupal.
The setup script uses aws cli to pull from S3 (sync) to add, replace, update everything in the webroot turning the instance into a setup Drupal installation as far as file level assets go.
sed and service reload is done a lot. Elasticache vs local Redis...
last we start the web server in the foreground && tail -f /dev/null so the container doesn't immediately close.
Drupal is just for static assets pages and a header/menu/footer wrapper for our web app (templates are served..., Twig/JS fills the template in with data). Authentication happens via API-- not even using 90% or so of the goodness in Drupal...
Incremental changes are pulled by comparing hash values and acting to run the update process if they are different.

Related

PCF Credhub Service broker - Unable to rotate credentials and demands restart

I created a credhub instance and stored my secrets using below cf command
cf create-service credhub default my-secrets -c "{\"secretMessage\":\"1234567\"}"
My spring boot app picks up the secretMessage value without any issue. And now i executed update-service command and rotated the secretMessage with the below command
cf update-service my-secrets -c "{\"secretMessage\":\"1234567-updated\"}"
But my spring-boot app isn't picking the updated one unless and untill I restart the app in PCF.
Is there any way to pick up the rotated-credentials without restart ?
But my spring-boot app isn't picking the updated one unless and untill I restart the app in PCF.
Is there any way to pick up the rotated-credentials without restart ?
No, that is the expected behavior.
https://docs.pivotal.io/credhub-service-broker/using.html#update
Your service instance is bound to your app. Like all service instances bound to an app, it's providing information through the VCAP_SERVICES environment variable. Environment variables, including VCAP_SERVICES, are only updated when you restart the app.
You might be thinking, wait my VCAP_SERVICES has placeholders from CredHub, shouldn't those automatically update? Unfortunately no. The Diego lifecycle is responsible for looking up CredHub place holders and resolving them. This runs prior to your application starting. It then puts the actual value into VCAP_SERVICES. This is how your app can see the actual value and not the place holders, while simultaneously not having any knowledge about CredHub.

How do you maintain the manual setup of Drupal installation?

I have set up an elastic beanstalk deployment of Drupal to host a Drupal built website.
When I start up my ec2 instance, I go through the installation steps of setting up Drupal.
However, when the instance is Restarted, or Stopped, restarting the instance goes back to this page!
How can I configure the instance so that these installation steps do not need to be repeated even when the instance goes down. This is quite worrying as I am looking to host my website this way.
Any help on this issue would be greatly appreciated!
This step means that drupal can't find database or it can find it but installation is not done yet (required tables are not created). So if you pass installation and you see again this screen it can mean 2 things:
Your database configuration is lost, so drupal can't find DB.
DB configuration file is ok, but DB it self is lost so configuration must be done again.
So first check what is the problem in your case and then solve it. Probably since you are installing in instance, DB configuration is then recorded and then lost when your box expires. If so...find a way to make permanent change do config file (should be /sites/default/settings.php).

AWS Server setup with JIRA, Docker

I have an Amazon EC2 instance that I'd like to use as a development server for client projects as well as run JIRA. I have a domain pointed to the EC2 server IP. I'm new to docker so unsure if my approach is correct.
I'd like to have a JIRA container installed (with another jiradb MYSQL container) running at jira.domain.com as well as the potential to host client staging websites at client.domain.com which point to the client's docker containers.
I've been trying to use This JIRA docker image using the provided command
docker run --detach --publish 8080:8080 cptactionhank/atlassian-jira:latest
but the container always stops running mid setup (set up takes a while in-between steps). When I run the container again it goes back to the start of setup.
Once I have JIRA set up how would I run it under a subdomain? And how could I then have client.domain.com point to a separate docker container?
Thanks in advance!
As you probably know there's two considerations for getting Jira setup, whether as server or container:
You need to enter a license key early in the setup process (and it requires an Internet connection for verification), even if it's an evaluation
By default Jira will use its built-in (H2, IIRC) database, unless you configure an external one
So, in the case of 2) you probably want to make sure you have your external database ready and set up.
See Connecting Jira applications to external databases for preparatory steps for a variety of databases.
You didn't mention at what stage your first setup run fails, however once you've gotten past step 1) or any further successful setup, one of the first things I did, so as not to lose all work I'd done, was to commit the container!
docker commit -a 'My Name' -m 'Jira configured and set up' <container ID> myrepo/myjira:mytag
That way you don't lose all your previous work and you save your container into a new image in one fell swoop.

Bypassing SNS notifications in development for Elastic Transcoder

I have an Elastic Transcoder job that sends out SNS Notifications to an endpoint in my app when the job has finished. Obviously, it is non-trivial to be getting the SNS to be hitting endpoints on my local machine when in the development environment. I can't quite work out how to overcome this.
When a video is uploaded it create a database row with the mezzanine url and some empty columns for transcoded urls. It then creates an Elastic Transcoder job.
When the notification arrives at the endpoint from the job it adds the URLs of the transcoded files to the row of the database in which the url of the mezzanine file is located. I know what these URLs will be but don't want to add them to the database until the job is complete (so I can serve placeholders if the database column is NULL, or if indeed it fails).
It feels wrong to check for the ENV in the controller and just add the URLs to the database immediately if we're in development, so I wondered if there was a better way?
Typically, your development environment should be completely separate from production's moving parts -- including SNS topics. If you duplicate your whole stack into a development tier ("dev db", "dev sqs", "dev rails") and regularly copy data over to it, you can isolate your changes' impact on production data while making assurances about behavior.
You could also then use a simple K/V pair in your Rails config (environments.rb) to tell it which environment to live in.

Deploying WordPress on Elastic Beanstalk?

Suppose I create a site in Wordpress, which is running on Elastic Beanstalk. Now, on the running app I will create posts /pages, upload images, etc. That is, some data, videos, files and records in a database will be added to the running application.
3 questions:
If WordPress is running on Elastic Beanstalk with multiple Amazon EC2 instances actually running my WordPress install, then will those files propagate automatically to all running instances? And will this also happen, if a new EC2 instance is fired up - for example, to handle increased load?
From what I see in AWS console, I can deploy different versions of an app-- but as per scenario above, if I deploy a new version, wont I lose all the files uploaded directly into running app (i.e. files and database records)? How do I keep those and at the same time deploy a new version of my app?
The WordPress team keeps issuing upgrades. Can I directly upgrade my running WordPress install, through the web interface? Or do I have to first upgrade my local version of WordPress, and then upload the new version of the app to Beanstalk? If I have to upgrade my local version and then upload, then again I am back to point 1, i.e. changes made by users directly to the older version of running app. How do I preserve those changes?
I've been working on this as well, and have learned a couple of things that are relevant here -- your question about uploads in particular has been on my mind:
(1) The best way to handle uploads, it seems to me, is to either go the NFS/NAS route like you suggest, but one better than that is to use an Amazon S3 plugin for WordPress, so that any uploads automatically copy up to S3 and the URLs in your WordPress media library reflect the FQDN of your bucket and not your specific site. That way you could have one or ten WP nodes in your Beanstalk and media/images are independent of any one of those servers.
(2) You should absolutely be using RDS here. Few things are easier to work with and as stress-free as a Multi-AZ, reserved MySQL RDS instance. Either that or your own EC2 running MySQL that is independent of the Beanstalk, but why run that when RDS is so much easier?
(3) Yes you definitely have to commit changes to your Git repository or local file first (new plugins, changes to themes, WP upgrades) and then upload/install as a revision to the Beanstalk code. Otherwise, all the changes you make via the web interface to one node will never be in the new load for a new node -- in fact you'll have an upgraded database but an older set of code in the Beanstalk application, so it's likely to create errors of some kind or another.
I took an AWS architecture course, and their advice for EC2 and the Beanstalk is to start to think about server instances as very disposable -- so you should try to think about easy ways for your boxes to provision themselves in the bootstrapping process and to take over work for one another without any precious resources on just one box. So losing an instance should never be a big deal. (This is definitely not how we thought in the world of physical servers, where we got everything tweaked 'just so'.)
Good luck!
Well, I'm no expert, but since no one has answered, I'll give it my best shot.
You are absolutely right--kind of. While each EC2 instance does have some local storage, it is destroyed and reset with each new instance. Because of this, Amazon has things like Elastic Block Storage and S3 for persistent files. I don't know how one would configure WP to utilize this, but that will likely be the solution.
I think this problem is solved by my answer to #1. As for the database, all of your EC2 instances should be pulling from the same RDS location. Again, while you could have MySQL running on each EC2 instance, in the interest of persistence, having a separate database makes more sense.
You, again, have most everything right. Local development should always precede live deployment. Upgrading locally then pushing to the live servers will make sure all of your instances remain identical.
Truth-be-told, I am still in the process of learning all of this too, as I said, I'm not an expert. Hopefully someone else will come along and give a more informed answer. However, the key conceptual hurdle here is the idea of elastic scalability--and the salient point of this idea is the separation of elements between what is elastic/scalable/disposable and what is persistent.
Hopefully that helps.
I have deployed a small Wordpress site on EB, S3 and RDS. S3 holds all static data, such as media uploads. This works through a plugin. RDS holds the database. EB holds the latest deployed application. The application is deployed from my dev environment, with a build script. This way, I just have to press one button and I redeploy.
I wrote an article about it here: http://www.cortexcode.com/wordpress-to-aws-code-example/
While it was at first annoying to work with, the speed of AWS is nice and now it's easier than ever. It used to be that I had to upload a bunch of files over FTP, this is way more efficient. :-)
As an addition to all the great answers already:
1) I can highly recommend EFS but also S3 for media files, so they are served from high availability regions in combination with cloudfront. For Wordpress there is one plugin that really speeds up this ( not affiliated to them, just really like the plugin ). There is also an assets plugin, if you'd like to serve JS, CSS files from S3. For the EFS solution, take a look at the AWSlabs docs on git, and specifically this file on how they mount the uploads file.
In general, EBS is really great for Wordpress, but you'll need to think in a different mindset as compared to other hosting solutions ( shared hosting, managed hosting ).
OK I researched a lot on this particular issue, and this is what I learned--
(1) If a wordpress user uploads some files, then his files will be uploaded only to the virtual machine that is actually serving his request at that time. Eg if currently the wordpress site is cloud-deployed and is using 5 virtual machines, now when user makes request he is directed to one virtual machine-- the one with the lowest load at that point... His uploads are stored only on that server. Current Platform-as-a-service solutions (like Amazon Elastic Beanstalk and App Fog) do not have the ability to propagate the changes to all the running instances. Either that (propagate changes to all servers) or use a common storage by all running instances-- these are the only 2 solutions to this problem. (Eg of common storage would be all 5 running virtual machines using Network-Attached-Storage (NAS)... )
(2) With ref to platforms available currently like Amazon Elastic Beanstalk and App Fog, for example, even if user made changes directly to running app- these platforms rely on the local version of code (which the admin deployed initially to cloud)- and there is no way to update the local version of code (on admin's PC) with the changes made by a user to running app-- hence these changes viz, files are lost-- Similarly, changes in database by user to running app are also lost-- unless the admin is using exactly the same database for his local app (that he deployed to cloud)
(3) Any changes to running apps first have to be made to the local app on admin's PC and then pushed to cloud.
I am working on a Cloud PaaS that addresses all these concerns-- viz updates can be made to running apps, code changes made to running app are also updated in code repository accessible by user...The Proof of concept is ready, hopefully it will be as good as I hope it should be :) -- currently the only thing that is actually there is the website (anyacloudpanel.com) -- design work is going on :)
If there is some rule that I should not mention my website( Anya Cloud Panel) -- then I am sorry -- pls feel free to edit and remove my website URL from my answer :)
Thanks,
Arvind.
Deploying WordPress to AWS Elastic Beanstalk does require some change to the normal WordPress deployment as mentioned here a few times. To answer your questions, here is a great tutorial explaining stateless applications and how to deploy to Elastic Beanstalk:
Deploying WordPress to Amazon Web Services AWS EC2 and RDS via ElasticBeanstalk
Be careful if you use a theme from themeforest for example. Some of them are incompatible with wordpress S3 plugin. Then you're screwed, you can not deploy your wordpress on the cloud.