We are migrating some of our J2EE based application from on-prem to the AWS cloud. I am trying to find some good document on what steps to be considered for the App migration. Since we already have an AWS account, and some of the applications have been migrated earlier, I don't have to worry about those aspects.. However I am thinking more towards
- Which App-server to use?
- Do i need to migrate DB as well..or just the App?
- Any licensing requirements for app.. we use mostly Open source.. So that should be fine..
- Operational monitoring after migrating to cloud..
Came across some of these articles.
https://serverguy.com/cloud/aws-migration/
Migration Scenario: Migrating Web Applications to the AWS Cloud : https://d36cz9buwru1tt.cloudfront.net/CloudMigration-scenario-wep-app.pdf
I would like to know If you have worked on this kind of work.. and If you point me to some helpful document/links.. or your pwn experience?
So theres 2 good resources I'd recommend for migration:
AWS Whitepaper for migration
AWS Well-Architected Framework.
The key is planning, but not being afraid to experiment. This is cloud so don't be afraid of setting an instance size in stone, you can easily change it.
Related
I currently have aws dynamodb (port: 8000) and elasticsearch (port: 9200)+kibana(port: 5601) running locally.I pretty much spent almost half a day figuring out how to sync these services together. To provide context, I have a nextJS app running thats integrated with both clients however, when I upsert data into dynamo, Id like to get it synced with elasticsearch right away.
Note, for upper environments I plan on using aws lambdas integrated with dynamodb streams to update indexes in elasticsearch instances (I am not a fan of the aws elasticsearch managed service).
Here is what I have tried thus far for local env syncing:
Logstash w/dynamodb plugin (https://github.com/awslabs/logstash-input-dynamodb) - the repo hasnt been updated in 4 years and issues support newer version of ES. Been a headache all morning trying to get the thing running due to this and other issues --> https://github.com/awslabs/logstash-input-dynamodb/issues/10
result: its a lost cause as far as I know... ive even tried their docker images but not workin..
node-scheduler: oh jeez... given Im using nextJS Id have to create a custom server just to get this extra piece synced... not to mention it removes a lot of the important features of nextJS -> https://nextjs.org/docs/advanced-features/custom-server
host the development dynamodb in aws (not local) + provision ES on ec2 and point local app there. I think this is overkill imo as now pretty much everything is hosted except my app, which could be ok. Let me know your thoughts as I already have dev environment using a separate cognito pool to authenticate.
Just chain ES calls to any updates made on dynamodb. Can someone tell me why we cant do this locally and in general? I get that data can get out of sync but maybe we can do hourly, or even twice a day (offpeak) cron jobs to bulk update the dynamo records to ES.
Would really appreciate your thoughts. Thanks!
I'm developing small single user applications in Django. Currently I do so with Heroku, which works just fine. I would like to deploy the application on Google Cloud run to have in the future a bit more flexibility.
In order to keep the overhead as small as possible I was considering using Sqlite. To keep persistency all I would need is a persistent volume, which could be achived via Google Cloud storage that is mounted into the docker container via gcsfuse. But here is the issue. I can't find a small Image with python and gcsfuse. I'm not a docker pro. Just getting started...
Any help is appreciated.
Google itself provides a HowTo Deploy a (simple) Django app to Google Cloud Run:
https://codelabs.developers.google.com/codelabs/cloud-run-django/
I myself as a Google Cloud rookie followed it step by step and it worked without any problems. By the way, I found that via the episode of the podcast Django Chat "Advanced Deployment" with Katie McLaughlin.
The HowTo use a Cloud SQL instance instead of Sqlite, but this seems to be a good choice:
„Generally speaking SQLite is not a good database choice for
professional websites. So while it is fine to use SQLite locally while
prototyping an idea, it is rare to actually use SQLite as the database
on a production project.“
William S. Vincent. „Django for Professionals, Chapter 2
Letting this apart it shouldn't be so hard to skip the Cloud SQL step and keep SQLite.
I have an application developed in Meteor Framework.
We are planning to move it to AWS withmulti AZ deployment
need Master Slave configuration for the Mongo DB
My question is how to achieve this, i believe mongo db comes bundled in with the Framework itself,
never worked on it so any help will be appriciated.
Thanks
Welcome to Stack Overflow.
Mongo is bundled into the development environment, but not the server.
It is normal to host the database either on a different server of your own, or using a database service (there are many around, such as compose.io, Mongolab etc) So Mongo can be set up for load balancing and scaling independently of the app itself.
Amazon Web Services offer a number of continuous deployment and management tools such as Elastic Beanstalk, OpsWorks, Cloud Formation and Code Deploy depending on your needs. The basic idea being to facilitate code deployment and upgrade with zero downtime. They also help manage best architectural practice using AWS resources.
For simplicity lets assuming a basic architecture where you have a 2 tear structure; a collection of application servers behind a load balancer and then a persistence layer using a multi-zone RDS DB.
The actual code upgrade across a fleet of instances (app servers) is easy to understand. For a very simplistic overview the AWS service upgrades each node in turn handing connections off so the instance in question is not being used.
However, I can't understand how DB upgrades are managed. Assume that we are going from version 1.0.0 to 2.0.0 of an application and that there is a requirement to change the DB structure. Normally you would use a script or a library like Flyway to perform the upgrade. However, if there is a fleet of servers to upgrade there is a point where both 1.0.0 and 2.0.0 applications exist across the fleet each requiring a different DB structure.
I need to understand how this is actually achieved (high level) to know what the best way/time of performing the DB migration is. I guess there are a couple of ways they could be achieving this but I am struggling to see how they can do it and allow both 1.0.0 and 2.0.0 to persist data without loss.
If they migrate the DB structure with the first app node upgrade and at the same time create a cached version of the 1.0.0. Users connected to the 1.0.0 app persist using the cached version of the DB and users connected to the 2.0.0 app persist to the new migrated DB. Once all the app nodes are migrated, the cached data is merged into the DB.
It seems unlikely they can do this as the merge would be pretty complex but I can't see another way. Any pointers/help would be appreciated.
This is a common problem to encounter once your application infrastructure gets into multiple application nodes. In the olden days, you could take your application offline for "maintenance windows" during which you could:
Replace application with a "System Maintenance, back soon" page.
Perform database migrations (schema and/or data)
Deploy new application code
Put application back online
In 2015, and really for several years this approach is not acceptable. Your users expect 24/7 operation, so there must be a better way. Of course there is, the answer is a series of patterns for Database Refactorings.
The basic concept to always keep in mind is to assume you have to maintain two concurrent versions of your application, and there can be no breaking changes between these two versions. This means that you have a current application (v1.0.0) currently in production and (v2.0.0) that is scheduled to be deployed. Both these versions must work on the same schema. Once v2.0.0 is fully deployed across all application servers, you can then develop v3.0.0 that allows you to complete any final database changes.
Finally the time has come and I'm ready to deploy my first django project.
I'm a newbie in web development stuff and now the real fun begins.
This is a low scale site for computer jobs.
I want to start with a free tier and grow from there as need emerges.
I've read some guides regarding django project deployment but could not find all answers.
so hope some guys here could help me out:
I've been thinking on getting Amazon EC2 free tier VPS, is this a good option?
my local development machine runs Ubuntu, I've read that i could install 10GB Ubuntu image, do you recommend such image?
should I go with apache or lighter web server?
My project is hosted on bitbucket, I just need to checkout my project on my VPS right?
What about data backups? I would like to backup my mySQL DB
How do you recommend me serving the static files?
I'm looking for a good tutorial on how to setup AWS with django and mysql
10x guys!
I've been thinking on getting Amazon EC2 free tier VPS, is this a good
option?
If it fufills your technology requirements, ram, cpu, memory; it is a good option.
my local development machine runs Ubuntu, I've read that i could
install 10GB Ubuntu image, do you recommend such image?
Might as well keep your environments the same if you can. If you can match up versions that is another plus
should I go with apache or lighter web server?
Either, Apache would probably be easier to deploy at this point because you don't have to worry about running it as a servicer ( using a program like like supervisor to manage it).
Whichever one you choose, there is an abundance of tutorials online describing how to set up django.
My project is hosted on bitbucket, I just need to checkout my project
on my VPS right?
That is one way. There are lots of ways to deploy. I like syncing the actual files using fabric. That way your production server doesn't need to know about your bitbucket account. Once again, there are so many tutorials online describing deploying django. Fabric is a great place to start.
What about data backups? I would like to backup my mySQL DB
There exists lots of tools for this. Plenty of premade tools and shell scripts. I have used automysqlbackup and it works great http://sourceforge.net/projects/automysqlbackup/
How do you recommend me serving the static files?
Make sure the webserver serves them. If you deploy through apache you can set up an alias to serve static files very easily. You can come up with a collectstatic deployment scheme to put your static on s3, but for a simple site apache would be just fine
I'm looking for a good tutorial on how to setup AWS with django and
mysql
Perhaps you can find a tutorial that covers this, most likely you might just find a tutorial :
how to setut aws with ubuntu
Installing django / mysql on ubuntu