I tried to deploy my Django project to beanstalk (no docker). It seems to me that I can only choose AWS RDS as my database choice. Could I install PostgreSQL in the same beanstalk instance?
If so, how could I install PostgreSQL myself? Using RDS is just an additional cost for me. So, I am looking for cheap solution. Possibly SQLite3 is a solution? But I hope to use PostgreSQL.
Could I install PostgresSQL in the same beanstalk instance?
Yes, you could. But this will require a bit of "manual" setup and it will be not-scalable nor really fault tolerant. With RDS you pay premium, but you get fully managed, highly scalable and reliable database.
But of course, not all use cases require using RDS. In this case you could install PostgreSQL on your EB instance (I assume single-instance EB environment). For this you would need to setup a number of configuration options in .ebextensions. However, this process is not that easy as you would highly-couple your application deployments with the DB.
As a middle ground, I think it would be better to install PostgreSQL on a separate, dedicated instance. This way your EB instance and the DB are de-coupled, easier to manage, update, backup and scale.
Related
Can some one please advise the steps required for migrating a web application which is currently running on tomcat server at onpremise to AWS ec2 instance. I understand this is not a straight forward and requires some detailed process.
The code is wrriten in Java and database used as oracle.
So it would be helpfull if someone can suggest me any relavent document or any website which gives some demo to refer me and proceed with this scenario.
If it's a personal project then I would recommend Lightsail as the simplest way to deploy existing Java application.
For a database a small instance of MySQL or if relational database is not needed then a document database like DynamoDB. https://aws.amazon.com/products/databases/?nc2=h_m1
There are multiple choices one how to migrate a Java application to AWS.
You could potentially use existing AWS services like:
Lightsail - https://aws.amazon.com/lightsail/
Beanstock - https://aws.amazon.com/elasticbeanstalk/
or
EC2 instance and install Tomcat manually
Use ECS with Docker https://aws.amazon.com/getting-started/tutorials/deploy-docker-containers/?nc2=type_a
As for Database solution Oracle is an option but quite expensive one.
When moving to AWS it's better to use one of the RDS managed databases like MySQL, Postgress or more expensive like Aurora.
In order to propose an architecture some details would be needed on predicted load, the size of the application and volume of data. Is the product regional or global, are there any additional issues that need to be addressed while moving to a cloud (performance, availability etc), how users are authenticated (are any other services needed).
I am trying to automate the creation of the temporary environment, I am struggling with the creation of a related RDS instance inside Elastic Beanstalk.
I would like that when I call eb create envName the environment also spawns an RDS database.
One solution would be to manually do it. Another solution seems to involve '.ebextensions' Using Elastic Beanstalk .ebextensions to specify an RDS database , though .ebextensions are to be run at each deployment, this could fit through a specific hook, but I would like to have my config in .elasticbeanstalk/config.yml because this is the file that seems to have preconfiguration for when I call eb commands.
One of the reason to not but it in .ebextensions is also that in the project configuration, those are not environment specific, they describe the requirements for the app in any environment.
I personally believe it is a bad idea to create the database as part of the environment, as it is a lot harder to deal with stopping and recreating the environment if you need to. I believe it is a lot easier and safer to keep the database running even if you want to completely destroy the environment and start all over. I believe most experience users will recommend you to manage the RDS on your own (outside EB). It is really easy anyway.
That said (and given you didn't ask for that), you can simply create a database as part of the eb create command itself:
eb create -db -db.engine postgres -db.version 9.4 -db.user dbroot -db.i ...
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb3-create.html
I'm new to both Elastic Beanstalk, EC2 and Docker and spent the last couple of weeks researching and playing around with it. I have a few questions that I'm finding difficult to find answers to elsewhere.
One thing I like is that I am able to run eb local run to boot a local environment of what will be running in production. This seems to work well until it comes to databases.
1) As far as I understand Elastic Beanstalk spawns instances running the containers inside, which could result in having multiple databases if Elastic Beanstalk spawns multiple instances? Is this correct?
2) Is it better to use AWS RDS in production and then have an external database container locally?
3) In terms of persisting data, I read that EBS can only mount to one EC2 instance, how do people handle storing user files, or do they have their application push to a service such as S3 directly?
I don't know if this is stated anywhere but I am fairly sure AWS does not intent for you to use EB's multi-container to run databases or anything that should run only once on your system. As their examples show, it is for you to have better control what the front end server will be.
If you want to run databases, or store files, you will either move to AWS ECS where you can better control this, or use multiple EB environment (e.g. create a worker-tier, single instance environment for running the database)
One thing I like is that I am able to run eb local run to boot a local environment of what will be running in production. This seems to work well until it comes to databases.
I have not used eb local run and instead use docker-compose, which allows me to properly run a proper environment locally, including my databases. Yes, you may need to duplicate some information between the docker-compose file, and the Dockerrun file, but once you set it up, you will see how powerful it is. Because you are still sharing the Dockerfiles, you can still assume things will run in a similar enough way once deployed.
1) As far as I understand Elastic Beanstalk spawns instances running the containers inside, which could result in having multiple databases if Elastic Beanstalk spawns multiple instances? Is this correct?
Yes, I think that is correct. EB assumes you will use RDS or dynamodb or something else, already centralized and managed.
2) Is it better to use AWS RDS in production and then have an external database container locally?
Yes, and by the way, rather than having EB manage the creation of the database, I find it a better practice for you to manually instantiate it so that it stays persistent after you kill your EB environments.
3) In terms of persisting data, I read that EBS can only mount to one EC2 instance, how do people handle storing user files, or do they have their application push to a service such as S3 directly?
Yes, using S3 is the way to go for multiple reasons, but mostly because AWS manages and you can scale without you having to worry about it. In fact, you want your client to get or even post the files directly on S3, so your server does not have to do any work (note the server may need to sign the URL but that is about it).
If you really have an issue against S3 (for whatever reason), then you will also (like with the database) create a second, single instance EB environment with EBS to ensure you have a single instance. But compared to the S3 solution it won't scale very far, and will in fact be much more expensive than using S3.
I've been following the official Amazon documentation on deplaying to the Elastic Bean Stalk.
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python.html
and the customization environment
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers.html#customize-containers-format
however, I am stuck. I do not want to use the built in RDS database I want to use mongodb but have my django/python application scale as a RESTful frontend or rather API endpoint for my users.
Currently I am running one EC2 instance to test out my django application.
Some problems that I have with the Elastic Bean:
1. I cannot figure out how to run commands such as
pip install git+https://github.com/django-nonrel/django#nonrel-1.5
Since I cannot install the device mongo driver for use by django I cannot run my mongodb commands.
I was wondering if I am just skipping over some concepts or just not understanding how deploying on the beanstalk works. I can see that beanstalk just launches EC2 instances and possibly need to write custom scripts or something I don't know.
I've searched around but I don't exactly know what to ask in regards to this. Top results of google are always Amazon documents which are less than helpful in customization outside of their RDS environment. I know that Django traditionally uses RDS environments but again I don't want to use those as they are not flexible enough for the web application I am writing.
You can create a customize AMI to your specific needs the steps are outline in the AWS documentation below. Basically you would create a custom AMI with the packages needed to host your application and then update the Beanstalk config to use your customize AMI.
Using Custom AMIs
As a sysadmin, i'm looking for an efficient way or best practices that you do on managing an ec2 instances with autoscaling.
How you manage automate this following scenario: (our environment is running with autoscaling, Elastic Load Balancing and cloudwatch)
patching the latest version of the rpm packages of the server for security reasons? like (yup update/upgrade)
making a configuration change of the Apache server like a change of the httpd.conf and apply it to all instances in the auto-scaling group?
how do you deploy the latest codes to your app to the server with less disruption in production?
how do you use puppet or chef to automate your admin task?
I would really appreciate if you have anything to share on how you automate your administration task with aws
Check out Amazon OpsWorks, the new Chef based DevOps tool for Amazon Web Services.
It gives you the ability to run custom Chef recipes on your instances in the different layers (Load Balancer, App servers, DB...), as well as to manage the deployment of your app from various source repositories (Git, Subversion..).
It supports auto-scaling based on load (like the auto-scaling that you are already using), as well as auto-scaling based on time, which is more complex to achieve with standard EC2 auto-scaling.
This is relatively a young service and not all functionality is available already, but it might be useful for your.
patching the latest version of the rpm packages of the server for
security reasons? like (yup update/upgrade)
You can use puppet or chef to create a cron job that takes care of this for you (the cron would in its most basic form download and or install updates via a bash script). You may want to automatically upgrade, or simply notify an admin via email so you can evaluate before apply updates.
making a configuration change of the Apache server like a change of
the httpd.conf and apply it to all instances in the auto-scaling
group?
I usually handle all of my configuration files through my Puppet manifest. You could setup each EC2 instance to pull updates from a Puppet Server, then you can roll out changes on demand. Part of this process should be updating the AMI stored in your AutoScale group (this is done with the Amazon Command Line tools).
how do you deploy the latest codes to your app to the server with less
disruption in production?
Test it in staging first! Also a neat trick is to versioned deployments, so each time you do a deployment it gets its own folder (/var/www/v1 /var/www/v2 etc) and once you have verified the deployment was successful you simply update a symlink to point to the lastest version (/var/www/current points to /var/www/v2).
OpsWorks handles all this sort of stuff for you so you can look into that if you don't want to do it all yourself.
how do you use puppet or chef to automate your admin task?
You can use Chef or Puppet to do all sorts of things, and anything they can't (or you don't know how to) do can be done via a bash/python script that you invoke from Chef or Puppet.
I normally do things like install packages, build custom packages, set permissions, download things, start services, manage configuration files, setup cron jobs etc
I would really appreciate if you have anything to share on how you automate your administration task with aws
Look into CloudFormation. This can help you setup all your servers and related services (think EC2, LBS, CloudWatch) through configuration files, thus helping you to automate your entire stack (not just the EC2's Operating System).