I have a Django project. I am considering adding Docker to it before deploying to Elastic Beanstalk. I am very new to Django and Docker and want to know what are the benefits of using Docker when deploying a Django app to Elastic Beanstalk. Thanks!
The general benefits of using Docker in EB, as compared to regular Python EB environment portability and reproducibility.
If you bundle your django app as Docker container, you know that you your development environment will be exactly same as your production one. All the dependencies, package versions, tools will be same in the container, regardless if it runs on your local workstation, home laptop or on EB platform.
However, when you use regular Python platform, the portability and reproducibility can be difficult to guarantee. The current Python platform is based on Amazon Linux 2. So the question is, is your development environment at home or work exactly same? Usually this is not the case, which often leads to issues in the vain of "It works on my local ubuntu workstation, but not on EB".
Also, one day you may decide to migrate your app out of EB or even AWS. It will be much easier to do that when using docker. This is because EB is a custom product from AWS, not available in other could providers with its own settings and requirements.
EB supports two types of docker-based environments:
single-docker
multi-docker
Depending on your requirements, you would have to use one of them. Each of them has its own use-cases, which I think are out of the scope to discuss for this question.
Related
I have recently deployed my Django application on Elastic Beanstalk.
I have everything working now, but I'm curious what the best way to develop locally is.
Currently, after I make a change locally, I have to commit the changes via git and then run eb deploy. This process takes 1-3 minutes which is not ideal for making changes.
The Django application will not run on my local machine, as it is configured for EB.
You are right, having to deploy remotely during development isn't best practice.
Have you considered Docker?
To run a typical Django app locally using Docker, you'll need to dockerize:
The Django app
Database eg Postgres
Worker eg Celery
Local mailer eg Mailhog
Not a very long list.
Obviously you'll add or remove from that list depending on how complex or simple your app is.
I recently started using AWS EBS and was confused as to what environment and application is and what is the difference?
Could I use environment to have a development testing and production environment and then have several applications in each environment?
Thanks!
Quoting the docs:
Application: An Elastic Beanstalk application is a logical collection of Elastic Beanstalk components, including environments, versions, and environment configurations. In Elastic Beanstalk an application is conceptually similar to a folder.
Environment: An environment is a collection of AWS resources running an application version. Each environment runs only one application version at a time, however, you can run the same application version or different application versions in many environments simultaneously.
Back to your question:
Could I use environment to have a development testing and production environment and then have several applications in each environment?
No. Its the other way around. One application can have multiple environments.
For example. If you application is called: my-first-web-application, then you could have three independent environments in it:
prod - environment stable serving real life requests, running stable version of my-first-web-application code base.
dev - development environment when you develop and tests new version of your my-first-web-application code base.
staging - final testing of a new version of your app, before it gets put into production environment.
I made a Django project and have successfully deployed it to an Elastic Beanstalk environment, let's say it's called app_name. However, I realized I needed 2 environments: development and production. The purpose of this is so I can try things out in development, and when I know it's fully working, I can deploy it in production for the public to use.
I tried looking around their documentations and found Managing multiple Elastic Beanstalk environments as a group with the EB CLI . It basically says you can create a group of environments in one project with the command:
~/workspace/project-name$ eb create --modules component-a component-b --env-group-suffix group-name
However, I'm not sure what a group means. I mean, I just need a development and production environment.
I'm fairly new at this. How do I create and manage development and production environments for this purpose? I would ever be so grateful if someone were to shed some light to my problem.
Running a group of environments is more for different services doing different things. You would have an environment that handles Service One, and an environment that handles Service Two etc. This isn't really what you want.
You just need an environment in the same application as your production environment. It doesn't have to be in the same application but I like it that way because its useful for deploying an app version to dev, and then deploy the app version to prod once it's tested.
An easy way to do this is run
eb clone app_name (where app_name is the name of your production environment)
This will clone your production environment and prompt you to give it a name, which you might set to app_name_dev. From there you can edit your dev environment to make it more suitable for dev (maybe you'd make the instances smaller, change software variables like MAIL_DRIVER=mailgun to MAIL_DRIVER=mailtrap, connect it to a dev database instead of your prod database, etc)
The downside of doing this would be if your production environment is currently running jobs or doing anything meaningful, you may not want to clone it right away since the new dev environment could start doing these things too, before you manage to update its config to point to a dev DB etc! If this is the case, you would just run eb create my_app_dev and configure it from scratch.
We push a java app on cloud foundry using cf push with below manifest file
applications:
- name: xyz-api
instances: 1
memory: 1G
buildpack: java_buildpack_offline
path: target/xyz-api-0.1-SNAPSHOT.jar
I understand that, PAAS (ex: cloud foundry) is a layer on top of IAAS(ex:vcenter hosting linux and windows VM's).
In manifest file, buildpack just talks about userspace runtime libraries required to run an app.
Coming from non-cloud background, and reading this manifest file, I would like to understand...
1) How to understand the operating system(OS) environment, that an app is running? On which operating system...
2) How app running on bosh instance different from docker container?
1) How to understand the operating system(OS) environment, that an app is running? On which operating system...
The stack determines the operating system on which your app will run. There is a stack attribute in the manifest or you can use cf push -s to indicate the stack.
You can run cf stacks to see all available stacks.
In most environments at the time of writing, you will have cflinuxfs2. This is Ubuntu Trusty 14.04. It will be replaced by cflinuxfs3 which is Ubuntu Bionic 18.04, because Trusty is only supported through April of 2019. You will always have some cflinuxfs* stack though, the number will just vary depending on when you read this.
In some environments you might also have a Windows based stack. The original Windows based stack is windows2012r2. This is quite old as I write this so you probably won't see it any more. What you're likely to see is windows2016 or possibly something even newer depending on when you read this.
If you need more control than that, you can always push a docker container. That would let you pick the full OS image for your app.
2) How app running on bosh instance different from docker container?
Apps running on Cloud Foundry aren't deployed by BOSH directly. The app runs in a container. The container is scheduled and run by Diego. Diego is a BOSH deployed VM. So there's an extra layer in there.
At the core, the difference between running your app on Cloud Foundry and running an app in a docker container is minimal. They both run in a Linux "container" which has limitations put on it by kernel namespaces & cgroups.
The difference comes in a.) how you build the container and b.) how the container is deployed.
With Cloud Foundry, you don't build the container. You provide your app to CF & CF builds the container image based on the selected stack and the additional software added by buildpacks. The output in CF terminology is called a "droplet", but it basically an OCI image (this will be even more so with buildpacks v3). When you need to upgrade or add new code, you just repeat the process and push again. The stack and buildpacks, which are automatically updated by the platform, will in turn provide you with a patched & up-to-date app image.
With Docker, you manually create your image building it up from scratch or from some trusted base image. You add you own runtimes & application code. When you need to upgrade, that's on you to pull in updates from the base image & runtimes or worse to update your from-scratch image.
When it comes to deployment, CF handles this all for you automatically. It can run any number of instance of your app that you'd like & it will automatically place those so that your app is resilient to failures in the infrastructure & in CF.
With Docker, that's on you or increasingly often on some other tool like Kubernetes.
Hope that helps!
I want to setup a unit test environment for my product. I have a web application build on nginx in Lua which use mysql and redis. I think docker will be good for this although i am new to docker. My application runs on centos server (production server).
I am planning to setup different container for mysql,redis and webapp and then write UT application (unit test for Lua using Busted framework) in my mac (My development machine is MAC) or VM to test it. The UT application will talk to docker container nginx and nginx will use container mysql and redis. Is this good ? If yes ,can someone guide me how to do this? maybe some good link? If no , what could be better way. I have already tried using vagrant but that took too much time which shouldn't be in my UT case.
For an example how we setup our project template you may have a look at phundament/app and its testing setup.
We are using a dockerized GitLab installation with a customized runner, which is able to execute docker-compose.
Note! The runner itself is running on a separate Docker host.
We are using docker-compose.yml to define the services in a stack with adjustments for development and testing.
The CI configuration is optimized to handle multiple concurrent tests of isolated stacks, this is just done by specifying a custom COMPOSE_PROJECT_NAME.
Some in-depth documentation about our testing process and useful information about docker-compose and dockerized CI.
#testing README
#testing Docs
CI builds
Extending services and Compose files
Docker-in-Docker for CI?
Finally, Travis CI also supports Docker since a while, but I haven't tested this approach at all.
If you are new to Docker based CI, please look at Drone:
Official page
Github repo
Tutorial
There some are drawbacks to this solution (like size of images), but it will get you off the grounds.