In my django application I have essentially joins ("__"). If I were to use Google App Engine, how would my app work in this instance? From what I've read, Bigtable doesn't allow this. Any advice on how to work with this?
Simple answer: it wouldn't.
You would presumably be using django-nonrel in any case. An extension to that, django-dbindexer, provides some support for emulating certain joins. But it's far from comprehensive. You generally need to architect your application in a different way to use the GAE datastore.
Related
I am trying to make django interact with the cassandra database. I was going through the forums and found this
Can this interact with cassandra in django? and are there any libraries that can interact with cassandra in django?
In order to use cassandra (or any other NoSql db) with django you need to have a backend which can support it. Luckily for you though, you're not the first to come up with the idea/need.
Take a look at these resources:
Django NoSql Support
django_cassandra_backend
django-nonrel
nosql-to-people
making django and nosql play nice
I haven't tried any on my own so can't say if anything is stable enough or gives an answer out of the box, but it should send you in the right direction.
Hope this helps.
The best Python client is https://github.com/pycassa/pycassa. You can easily use this from django; https://github.com/twissandra/twissandra is one example.
In my opinion the best way to use Cassandra with Django is cqlengine. It provides beautiful, django-like ORM and is really easy to use. The project seems active, last commit is from few days ago.
You can use it as is or with simple custom django database wrapper django-cassandra-engine.
I'm developing an app that interfaces with an iPhone app via an API. I am building it on Google App Engine, using django-nonrel. I'm looking for a solution that allows me to build read-write APIs that perform CRUD operations.
I've tried django-tastypie, but it doesn't work with django-nonrel.
The biggest restriction of django-nonrel is no JOINs.
What is the best way to write those APIs? Should I just write the views from scratch, or is there some library that I could use?
Thanks in advance!
I got tastypie working, with two small code changes. It was pretty simple.
See here: http://eatdev.tumblr.com/post/12076483835/tastypie-on-django-nonrel-on-app-engine
what things are limited when using django on app engine?
is setup the same or you have to change things?
are there many libs/modules that use c?
You cannot use the entire ORM - Use their custom BigTable API.
You cannot use the urllib library - Use their custom urlfetch for fetching anything.
You cannot use the file system - Use their blobstore API.
There is a time limit for the completion of every request, every db query.
You cannot use any modules that use C. Memcached, PIL, many other useful libraries out of question.
You modify your app for their custom authentication, custom caching, custom image-thumbnailing etc.
Essentially, you have to write the whole application to their custom API and infrastructure. All you get from django really is the template engine and forms library and there is no lack of good ones for either of these in Python.
Using django's builtin standards provides you portability between databases, between caching systems, between authentication mechanisms etc. - You typically forgo and get locked into the Appengine's infrastructure when you code for it. There is an effort to make Appengine specifics as one of the backend for each of these, in the project django-nonrel. It also supports ORM queries, but supports specific types of joins only and fails in not-so-easily predictable ways. - The authentication and caching backends seem to be solid.
Django and Appengine is definitely not a match made in heaven. You, the developer should take the heat of their discomfort. If you are looking to host on AppEngine, may I suggest Flask (or bottle) as a development tool and if you are looking for Django as the development tool, may I suggest ep.io (or djangy) as a cloud hosting destination.
How much of a pain is it to run a Django App on App Engine? Also, does the Datastore work as-is with Django?
I spent some time trying to answer the same question... it seems to me that the most difficult thing to transfer to GAE are django's models... in the sense that they require various modifications and rethinking, mainly because GAE's backend is not a standard relational DB, but google's BigTable. I found a nice intro to this here:
http://www.youtube.com/watch?v=rRCx9e38yr8
Anyways, it's worth downloading one of those 'patches' and have a go with it!
For me the best solution is the 'app-engine-patch'. I downloaded the sample project and it worked straightaway! (Mind that you need to have GAE's SDK installed separately) A killer-feature for me is the fact that the django-admin and many other classic django functionalities have been ported too!!!
http://code.google.com/p/app-engine-patch/
The documentation is still quite minimal in my opinion, but it's good enough to get you going. It'll help you to skim though the official GAE docs though!
Just Yesterday (depending on your time zone), Google released a new SDK for Python on Appengine that supports django 1.0 out of the box.
You need to use django-nonrel (source).
You will still find loads of issues:
Many2Many relations not supported
Fake joins increase number of queries
App Engine doesn't allow any python lib with socket or C dependencies (sentry, lxml...)
You can try to get early access to CloudSQL.
Otherwise you are not constraint to use App Engine, you can think about using:
Heroku
Gondor
Cheaper and more control with support requirement files like pip.
31.01.2012, Google released App Engine 1.6.2 that supports Django out-of-the box.
App Engine includes version 0.96 of Django out-of-the-box, but it is quite crippled.
App Engine Helper and app-engine-patch supposedly fix this problem to some degree, but I haven't tried either myself.
http://code.google.com/appengine/articles/appengine_helper_for_django.html
http://code.google.com/appengine/articles/app-engine-patch.html
The amount of pain depends on how much existing code you want to reuse. Unfortunately because of the Datastore does not support SQL, you often cannot just take any django-pluggable app and use it on your GAE project.
App-engine-patch http://code.google.com/p/app-engine-patch/ looks to be ahead of the other django helpers in bringing the standard applications (Sites, ContentTypes, Flatpages) over to GAE. I have used app-engine-patch on several gae projects, and once you understand how to port a django-sql model to a django-gae model and converting sql to datastore queries things can be done very quickly - but there is always a learning curve.
appengine-helper tries to bridge the Datastore gap by providing a model so you don't have to change your model superclasses, but I've found that you end up having to change ManyToMany relationships and any sql anyway, so the advantage ends up being minimal. ae-patch has a roadmap to try to provide an ae-datastore backend, but it probably won't happen for a while.
Google has now launched their Cloud SQL storage. That is actually MySQL 5.5 in the cloud. IMO that's a very nice way to migrate your Django app into the cloud. They have a free trial up to June 1, 2013.
If you need some tips how to set up your Django project for Appengine and Cloud SQL I've written a tutorial for that.
Given that gae & django persistence layers are quite similar, I'm wondering whether someone has tried creating a wrapper? Say, could be useful for utilizing django.forms.ModelForm.
Are there any fundamental problems with this?
App-engine-helper provides limited support for this, but you're still dealing with thinly-disguised App Engine models. I'm not hugely familiar with Django's model framework, but the basic issue is that Django's model framework is simply too tied to relational databases to easily move to the App Engine backend.
You can use a variant of ModelForms, built right into the App Engine SDK, though - see here for details.
Django norel is better alternative to app-engine-helper. It is a much better abstraction that supports django queries (well many of them -- there is no JOIN support), foregin keys (no ManyToMany relations, though).
Django norel supports admin app (app-engine-patch doesn't).
What, you mean like the Google App Engine Helper or Google App Engine Patch?
The design of Django models implicitly assumes that you're using a relational database. Unfortunately, Datastore does not support all the features of a relational database. For example, there is no equivalent of a join query in Datastore. General transactions aren't supported either. As such, it would be really hard (if not impossible) to adapt Datastore to work as a drop-in replacement for Django models.
At 4:45 of this YouTube video, Guido Von Rossum briefly describes how you need to rethink the way you design your database to fit Datastore's model. The benefit is that you will be able to scale up very well. At 48:30, someone asks why the Datastore API wasn't designed to be more like Django models. Guido explains that the fundamental differences between Datastore and a relational database make this impractical. 13:20 also discusses this.