what things are limited when using django on app engine?
is setup the same or you have to change things?
are there many libs/modules that use c?
You cannot use the entire ORM - Use their custom BigTable API.
You cannot use the urllib library - Use their custom urlfetch for fetching anything.
You cannot use the file system - Use their blobstore API.
There is a time limit for the completion of every request, every db query.
You cannot use any modules that use C. Memcached, PIL, many other useful libraries out of question.
You modify your app for their custom authentication, custom caching, custom image-thumbnailing etc.
Essentially, you have to write the whole application to their custom API and infrastructure. All you get from django really is the template engine and forms library and there is no lack of good ones for either of these in Python.
Using django's builtin standards provides you portability between databases, between caching systems, between authentication mechanisms etc. - You typically forgo and get locked into the Appengine's infrastructure when you code for it. There is an effort to make Appengine specifics as one of the backend for each of these, in the project django-nonrel. It also supports ORM queries, but supports specific types of joins only and fails in not-so-easily predictable ways. - The authentication and caching backends seem to be solid.
Django and Appengine is definitely not a match made in heaven. You, the developer should take the heat of their discomfort. If you are looking to host on AppEngine, may I suggest Flask (or bottle) as a development tool and if you are looking for Django as the development tool, may I suggest ep.io (or djangy) as a cloud hosting destination.
Related
I would like to use ArangoDB in Django, but I don't know which of the following options is better: using the ArangoDB Python driver or building a new API with Foxx. I think that the ArangoDB Python driver is not based on Foxx and I don't know the pros and cons of building a new API from scratch, even if it is made easier by Foxx. In addition, I'm afraid that using javascript in the interface between Foxx and the backend could make things slower. Would it be faster if I used Guacamole ODM together with Ruby on Rails?
Better option for your case is to use ArangoDB Python driver.
Here is couple of reasons:
easy-to-start - just install driver and move on with development
some similarity to Django ORM API
have some documentation
all your business logic will be in place and in Python which should be great advantage
And here is why Foxx is not the best option for your case:
you have to build your own API which means:
bunch of code in JavaScript
some documentation to describe API
additional logic level (in Foxx and in Django project) which increase tangling in your project
it probably not increase performance because you still retrieve your data using HTTP
Foxx is good option when you build Single page APP using ArangoDB as data layer. Also probably Foxx will be great for building hight-level API's with Foxx as preprocessed/aggregated data provider.
I made a python ArangoDB driver (https://github.com/saeschdivara/ArangoPy)
and I created on top of that kind of a bridge for Django (https://github.com/saeschdivara/ArangoDjango). So you can use kind of an orm for ArangoDB and still use the Django Restframework to create your API.
In my django application I have essentially joins ("__"). If I were to use Google App Engine, how would my app work in this instance? From what I've read, Bigtable doesn't allow this. Any advice on how to work with this?
Simple answer: it wouldn't.
You would presumably be using django-nonrel in any case. An extension to that, django-dbindexer, provides some support for emulating certain joins. But it's far from comprehensive. You generally need to architect your application in a different way to use the GAE datastore.
I'm writing Django application (social network) and thinking about dividing monolithic project to two projects: UI and API. For example, Django will be used only to render pages, interacting with and taking data from API, written on web.py.
Pros are following:
I can develop and test API independently.
In the future, other UI can appears (mobile, for example), it will require service.
I plan to outsource web UI developing, so, if my application will have two modules, I can provide outside only UI one, not sharing logic of application.
Cons are following:
I'm working alone, and developing two projects are harder, then one.
I will not be able to use cool Django admin panel. I will need to write my own.
web.py is more low-level comparing with Django.
It's like a brain dump, but I will be really appreciated if you share your experience in creating web application with UI module and independent API module.
Update (more specific question, as Mike asked)
What Python framework will you use for creating REST API of social network, which can be used by different client applications? Is using web.py that returns JSON only and rendering it by Django for web is good idea?
Thanks,
Boris.
I've been in a situation similar to yours. I ended up writing both, the UI and the API part in Django. Currently, I am serving them both out of the same process/project. You mentioned you wanted to be able to outsource the UI development, but do hear me out.
In the meantime, I have used django-piston to implement the RESTful front end, but a bit of preparation went into it:
Encapsulate all DB and ORM accesses into a library. You can do that either for your entire project, or on an app by app basis. The library is not just a low-level wrapper around your DB accesses, but also can be for higher-level 'questions', such as "all_comments_posted_by_friends()" or something. This accomplishes two things:
You can call your pre-canned queries from UI views as well as API views without having to re-implement them in multiple places.
You will later be able to replace some - if not all - of the underlying DB logic if you ever feel like going to a NoSQL database, for example, to some other distributed storage model. You can setup your entire app of this ahead of time, without actually having to worry about the complicated details of this right at the start.
The authentication layer for the API was able to accept an HMAC/token based header for programmatic access and normal Django auth. I setup the views in such a way that they would render plain JSON for the programmatic clients (based on content-type), and would render the data structure in HTML (with clickable links and clickable docstrings) if browsed by a human from a browser. This makes it possible that the API is fully explorable and clickable by a human without having to read any docs, while at the same time it can be easily processed by a client just via JSON.
On effect, the database layer I build serves as the internal API. This same database layer can be used from multiple applications, multiple processes, if you wish to do so. The UI views and the REST views were both implemented in Django. They can either be in the same process or in separate processes (as long as they have access to the same database right now).
I am looking for a guide to migrate Django project to Google App Engine and use Google's datastore. The most of the guides I found were linked to Django-Appengine using Django-nonrel (but I want to use GAE's native support).
Going through GAE getting started guide, it says:
Google App Engine supports any framework written in pure Python that speaks CGI (and any WSGI-compliant framework using a CGI adaptor), including Django, CherryPy, Pylons, web.py, and web2py. You can bundle a framework of your choosing with your application code by copying its code into your application directory.
I understand that I won't be able to use some features of Django in that case (majorly the admin feature) and would also need to restructure the models.
From other reading, I also found that latest SDK of GAE now includes Django 1.3 on Python 2.5.
I tried to put all files from my Django application to a GAE project, but couldn't get it all to work together.
Please provide some basic guide using which I may migrate my Django project to Google App Engine's code.
Thanks.
For an existing Django app, using django-nonrel is the simplest approach; it is very popular so you should be able to find help with specific errors you get quickly.
Another approach is written up in this article: http://code.google.com/appengine/articles/pure_django.html -- it goes the other way, taking an App Engine app that uses Django for dispatch, templates, and forms, but not for models, and describes how to make it run in a native Django environment. Maybe you can glean some useful hints for your situation from it.
I've used django-nonrel, which behaves pretty much like django, except that operations with JOINs will return errors. I've basically worked around this by avoiding ManyToMany fields, and essentially building that functionality manually with an intermediate table.
So far I've ran into two problems with Django-nonrel:
1. No access to ancestor queries, which can be run in a transaction. There's a pending pull request for this feature though.
2. You can't specify fields that are not indexed. This could significantly increase your write costs. I have an idea to fix this, but I haven't done so yet.
(Edit: You CAN specify fields that are not indexed, and I've verified this works well).
2 (new). Google is pushing a new database backend called ndb that does automatic caching and batching, which will not be available with django-nonrel.
If you decide not to use django-nonrel, the main differences are that Django models do not run under App Engine. You'll have to rewrite your models to inherit from App Engine's db.Model. Your forms that use Django's ModelForm will need to inherit from google.appengine.ext.db.djangoforms instead. Once you're on App Engine, you'd have to port back Django if you ever take your app somewher else.
If you already have a Django application you might want to check this out. You won't work with App Engine's datastore but Google Cloud SQL might fit your needs.
I am building a application that services users via a web front end, which I have chosen to use Django for. I also have to choose a framework/libraries to provide management of and abstracted access to a bunch of embedded systems that provide information the web user gets to see in one form or another.
I like the idea of sticking with a restful approach to access the backend application which provides the hardware generated resources. Does it make sense to use Django for the front end and CherryPy for the backend? Or should I just use Django for both and ignore the stuff I don't need in django for the backend.
I guess another way to ask, is what do I gain by using CherryPy as the backend that out ways having to know two sets of libraries/frameworks.
I don't see any real benefit, only added complexity in the long run. If you are using Django's ORM, you'll want to build the REST interface around that anyways. For building REST interfaces with Django I like using django-tastypie which makes it easy to build RESTful APIs supporting authentication/authorization, validation, various types of serialization, throttling, caching, etc. I also rather like django-piston, which is also quite popular.