I have Django server deployed on Google App Engine, I am doing simple GET request which is taking around 2 seconds, while the same request takes around 300ms when run locally. Both servers are using the same mysql database on Google Cloud SQL. I am testing this on my home wifi (100mbps), so don't think it's a network issue, anyway the payload is pretty small (2.5kb).
Anyone seen this slowness when deployed to Google App Engine? Is there any configuration change I could make to Google App Engine, that would make it faster?
Any suggestions are welcome.
Thanks!
When comparing Google App Engine’s performance with the local one you should keep in mind that deploying on GAE needs more time in order to import all the necessary libraries and set up the Django framework.
Here , it is stated that Instance Startup Time for Standard Environment is up to seconds and for Flexible up to minutes. Additionally, I found some StackOverflow posts that shed some light on this here and here.
You may profile your application by using Cloud Trace to analyze the requests and isolate what causes the issue so that you may improve it afterwards.
In addition to that there are various ways to optimize your application’s performance, as the following typical ones:
Scaling configuration, by setting up “min_idle_instances” to be kept running and ready to serve traffic.
using Warm Up Requests to reduce request and response latency during the time when your app's code is being loaded to a newly created instance.
Furthermore, here and here you may find the official Running Django on the App Engine environments tutorials so that you can spot any details you may have missed.
Finally, during my investigation, I came across PageSpeed Insights, which analyzes the content of a web page, then generates suggestions to make that page faster and could be handy.
I hope this information is helpful and point you in the right direction.
Related
Apologies in advance for my little knowledge of AWS
I'm trying to draw parallels between my current setup on Heroku to a move to AWS. I've run into some memory issues on Heroku because of some machine learning models I'm running and Heroku seems too expensive for my needs.
I was recommenced to move to aws using fargate which would be a better fit for my app. Below is my whole architecture, I'm hoping for some guidance on my direction of what I have and where I plan to go.
A django application running on heroku.
The base of functionality is the user uploads a video from their mobile device and uploads it to s3. A message from SNS is sent to my Heroku server that the upload is completed. The server kicks off a celery task that downloads the video from s3 and uses a machine learning model to do some natural language processing, then saves the results to my postresql database. Obviously this is very compute intensive, so I've run into some memory issues and can for-see scaling issues to come.
After lots of tweaking and attempts to no avail, I've decided to move over to AWS and leverage some of the cost benefits that I've seen in comparison to heroku of running more memory intensive tasks.
I should also mention there is a web interface involved with this django project and it isn't just a REST Api.
As far as AWS goes, I'm looking for a bit of direction. Possibly just a rough outline of the architecture I should look deeper into.
My first plan is to dockerize my application and go from there...but I'm a bit stuck on how my application fits (website, rest api, worker threads) into the AWS ecosystem.
AWS is a great fit for the application you describe. AWS Fargate / RDS will host your Django application. You have the option of using AWS Batch to handle your processing. One huge advantage is the ability to scale according to the needs of your application.
This image is one possible way to structure your application. It's a lot of work to get to this point, but AWS offers a lot of power and flexibility for reasonable costs IMO.
I am facing slow page load time (list page in Django admin) for a simple Django app, deployed on Google App Engine, with postgres Cloud SQL. There are fewer than 20 records. SQL time is negligible, even with 30 queries.
Majority of the time is shown as domLoading
I assume that domLoading is probably referring to the initial loading of the case/ page, which took 3 seconds.
Most of the solutions online refer to tweaking apache/nginx settings. But since I am using Google App Engine (PaaS), I cannot directly control webserver settings. gcloud app deploy is supposed to handle the load-balancing and code versioning.
How do I improve the basic load time of Django App on GAE? Both GAE and Cloud SQL are hosted in the same region.
PS: I did find some answers like Optimizing my Django app on Google App Engine, but they refer to optimizing SQL queries, which is not the case here.
This could be due to not having any available instances since that's about the time it takes for an instance to be deployed. You could use warmup requests to improve performance in this case. If that doesn't work for you, you can use Cloud Trace to analyze the requests and isolate the issue.
This is my first attempt at looking into cloud hosting and I'm feeling like a complete idiot. I have always had my own dedicated server with which I would would remote in and install/manage everything myself. So this cloud thing is completely new for me. I just can't seem to grasp basic things... like how I would get Tomcat and PostgreSQL installed in a way that they could talk to each other or get my domain and SSL cert on there, etc.
If I could just get a feel for where I should start, then I could probably calculate my costs and jump into the free trial where hopefully things will click for me.
Here are my basic, high-level requirements...
My web app running in Tomcat over HTTPS
Let's say approximately 1,000 page views per day
PostgreSQL supporting my web app.
Let's say approximately 10GB database storage
Throughout the day, a fairly steady stream of inbound SFTP data (~ 100MB per day)
The processing load on the app server side should be fairly light. The heaving lifting will be on the DB side sorting through and processing lots of data.
I'm having trouble figuring out which options I would install and calculating costs. If someone could help me get started by saying something like "You would start with a std-xyz-med server, install ABC located here at http://blahblah, then install XYZ located at http://XYZ.... etc.. etc. You can expect to pay somewhere around $100-$200 per month"....
Thoughts?
I would be eternally grateful. It seems like they should have some free sales support channel to ask someone at Google about this, but I don't see it.
Thank You!
I'll try to give you some tips where to start looking.
I will be referring to some products, here are the links
If you want to stick to your old ways, you can always spin up an instance on Compute Engine and set it up the same way you did before, these are just regular virtual machines. For some use cases this is completely valid.
You can split different components of your stack to different products:
For example, if your app is fine with postgresql, you can spin up a fully managed service in Cloud SQL, which might make it easier to manage backup or have several apps access the same db.
Alternatively, have a look at the different DB offerings to see if any of them matches your needed workload better. Perhaps have a look at BigQuery?
If you want to turn your app into a microservice, which is then easier to autoscale and is more fault tolerant, have a look at App Engine. That way you don't need to manage a virtual machine. The docs here will lead you through some easy to follow examples on how to set up SSL.
For the services to talk to each other, refer to docs of the individual components. It's usually very simple.
With pricing, try https://cloud.google.com/products/calculator/
Things like BigQuery have different pricing models - you don't pay for server uptime, but for amounts of data stored & processed with your queries.
I m running my python base REST services on DJango free hosting site called "pythonanywhere".
So far each web service was ~100ms in response thus my frontend was super fast, but last few days response is drastically changed, now for same REST API I am getting 30 seconds.
With above performance I cannot schedule my demo, I am planning to setup new django/python/mysql base environment for myself.
What are the best ways to host/setup Django based application (with mysql), preferably free hosting but I dont mind spending few bucks for better service.
For production deployment setups the recommended deployment is with wsgi.
StackOverflow is not the right place to solicit blanket recommendations especially since you haven't given any idea of what your expected load/usage is.
If you just need something to run your application "online" quickly; a PaaS provider should provide the quickest ramp up time. I have used heroku before and its very simple to get started.
I have a django web app that uses postgres db.It allows users to login and make some posts which get saved to db and later on the user can list how many posts he made on a particular day etc and list the posts belonging to a particular category etc.While this worked without any delay in my machine,it is taking a lot of time to load each page when hosted on a free host.
How do you find out why this is happening?which part of the app should I look first?Is there any meaning in using a profiler since this app used to run with no delays in my local machine?
I would like to find out how to approach this problem in general.I was able to access other apps hosted on the same free host without much delays ..so this may be a problem specific to my app
I would like some advice regarding this..If anyone can help..
thank you
p.s:(I intentionally left out the host's name because ,since that was a free service ,there was no point in complaining and also other apps on the same host works well)
The here is the free host bit, when on a free host you could be sharing a box with hundreds of other sites (that can equate to a very small amount of ram or CPU). Pay a little money, ($30 dollars / £22 a year) and get your self a better host.
You will find the performance and reliability so much better.
Failing that I would firstly find out what the latency between you and the server is, on a local machine there is no / little network traffic so your pages will appear to load a lot faster.
Next i would look at the actual download speeds you are getting. It could be that your site is limited to 20-30k, which means even a small site will take over a second to load.
Are you hosting many images? If this is the case are you serving them through django or is the webserver doing this. If it is django then make the webserver take this load.
Finally check the processing speed of the pages. Analise the queries which are being run and find out what is taking the time. Make sure that postgres is correctly configured and has enough resources. You can analyses the query speed using the django debug toolbar.