I am developing a gis application using Google maps API. Currently I am using Postgis db.
I am considering switching to mongodb and I have following questions about this,
Is mongodb a viable choice for storing GIS data. ( Is any other NoSQL engine viable?)
Does django-nonrel has modified django.contrib.gis module for mongodb support? and how well does it work?
Thanks in advance :)
As stated by Sergio Tulentsev, MongoDB has spatial indexes. But, not all geometry types can be stored. As far as I know, currently only points can be stored. You can, however, query using a polygon.
Since MongoDB is very flexible, you could store geometry as a text, but also as a JSON object. For example, you could store a coordinate like: { lon: 52.1234, lat: 14.1245 }. You can interpret this is on your own application as you like.
The downside to this is that there is no native index. If you never have to query based on location/spatial relation, you are fine. If you do have query based on location/spatial relation, you would have to write your own index. Can be a hard thing do...
There is CouchDB which has a GeoSpatial addition. Don't know how it works/what it supports though.
A hybrid approach is also often taken by some large-data-based companies. For example, use MongoDB to store/query normal data, and PostGIS to store/query geospatial data.
Django MongoDB Engine developer here.
We haven't got any sort of real support for MongoDB's GIS features. It's possible to do geospatial queries by "working around Django" though (see http://django-mongodb.org/topics/lowerlevel.html - the first example is actually about Geo indexes).
It should be possible, however, to implement native Django GIS support into the MongoDB backend. We're really happy to mentor anyone who wants to approach that task!
I've been working with MongoDb storing points of interest in the DB, then doing point in poly queries, along with radius queries and Mongo has been FAR superior to PostGres. I've had much faster response times with hundreds of thousands of items.
I would say however depending on the complexity of your queries (ie if you're doing something beyond point in poly, radius around a point), you might want a more heavy GIS db like postgres.. you're just going to get the extra weight of postgres as well.
Sorry I can't speak to anything on django as well.
Related
So I implemented Haystack with ElasticSearch a week ago within our BETA application. One thing I can notice is that getting some data (large amount) back to our users (for example listing all the users within the application) is much faster by going through Haystack then Django's ORM. Now, I will be releasing a REST service (with TastyPie) to serve the possible tablets within the next weeks, as I want to be able to access the information from iPads, Nexus tablets and so on.
One thing I was wondering, is when should I be querying the ORM vs Haystack/ElasticSearch? For example, if the user on the tablet is requesting a specific set of users, should we let TastyPie query the ORM, or go to ElasticSearch?
If we look at this answer Django: Haystack or ORM, we can all agree that a DB is made to retrieve and write data. However, could we say that retrieving faster can be faster with Haystack/ElasticSearch once the search engine was updated?
I am a bit confused as to when, should we not be querying Haystack if it is much faster?!
To make things clear I guess you're talking about querying Elasticsearch via Haystack without later instantiating any objects for your search results with data from you database.
Some points to consider besides the points mentioned in the other post:
A search engine like Elasticsearch is highly optimized when dealing with full-text searches (When doing something with SQL it highly depends on the database/engine you are using)
Queries that are involving a lot of relations/joins will most like be easier to handle with the ORM, but on the other hand you can eg save data from foreign-key relations in a denormalized fashion when using ES which could give you a performance boost. Of course you can denormalize your database tables as well but this is quite often considered as a bad practice as long as you know what you are doing, eg when solving a performance bottleneck.
ES is somehow quite easy to scale while scaling your SQL DB might be more complicated.
Most likely this is a decision that depends very much on your use case, the amount of data to process and the queries you are intending to run. So the best thing of course is - as always - to do some benchmarking yourself and compare this two solutions. But don't do any premature optimisations as one big advantage of the ORM is to keep things simple - you don't have to care much about the integrity of your data and maintain an additional system.
I'm build a Django app with Neo4j (along with Postgres), I found this Django integration called neo4django, I was wondering if it's possible to use neo4restclient only, like, what would be the disadvantages of not using Neo4django? Does using neo4-rest-client only, give me more flexibility?
When I was creating my models with Neo4Django, it seemed that there is no difference between modeling a graph db and relational db. Am I missing anything?
Thanks!
You can absolutely go ahead with neo4j-rest-client or py2neo, without using neo4django. In the same way, you can use any other database driver you'd like any time using Django, any REST client, etc.
What'll you lose? The model DSL, the built-in querying (eg, Person.objects.filter(name="Mohamed")), the built-in indexing, and the Lucene, Gremlin and Cypher behind that. Some things will be much easier- like setting an arbitrary property on a node- but you'll need to learn more about how Neo4j works.
You'll also lose some of the shortcuts Django provides that work with neo4django, like get_object_or_404() and some of the class-based views that work with querysets.
What'll you gain? Absolute power over the DB, and an easier time tweaking DB performance. Though neo4django isn't nearly as good a lib as some traditional ORMs in the Python sphere, the trade-off of power vs provided ease is similar.
That said, the two can work together- you can drop down from neo4django to the underlying REST client nodes and relationships anytime. Just use model_instance.node to get the underlying neo4j-rest-client node object from a model, and from neo4django.db import connection to get a wrapped neo4j-rest-client GraphDatabase.
On whether you're missing something: neo4django was written to re-use a powerful developer interface- the Django ORM- so it should feel similar to writing models for Postgres. I've written a bit about that odd feeling in the past. I think part of the problem might be that the lib doesn't highlight the graph terminology new graph-interested devs expect- like traversals and pattern matching- and instead dresses those techniques in Django query clothing.
I'd love your thoughts, or to know anything you'd like the library to do that it isn't doing :) Good luck!
I'm currently building my first Django-based app, which is proceeding
fairly well, thus far. One of the goals is to allow users to post
information (text, pics, video) and for the app to be able to
automatically detect the location where they posted this information
(i.e., pulling location info from the browser). That data would then ideally
be able to be filtered later, such as viewing the posts that were made within
a specific radius.
I've been reading a bit about GeoDjango and it sounds intriguing,
if perhaps more sophisticated than the requirements of this project.The querying aspects appear promising.
A colleague, though, suggested everything that can be done using
GeoDjango is equally efficient utilizing the Google Maps API with
JavaScript or JQuery to obtain the proper coordinates.
Essentially, I'm looking to see what benefits GeoDjango would offer
this fairly straightforward project over using just the Google Maps API. If I've already
started a project in basic Django; is incorporating GeoDjango
problematic? I'm still attempting to master the basics of Django and
venturing into GeoDjango may be too much for a novice developer. Or not.
Any insight appreciated.
To accurately find geolocated posts within a given radius of a location, you need to calculate distances between geographic locations. The calculations are not trivial. To quote the Django docs (with a minor grammatical correction):
Distance calculations with spatial data are tricky because,
unfortunately, the Earth is not flat.
Fortunately using GeoDjango hides this complexity. Distance queries are as simple as the following:
qs = SouthTexasCity.objects.filter(point__distance_lte=(pnt, 7000))
Although one could program this logic using JavaScript/JQuery, I don't see a reason to because you are already using Django. Unless if you are:
unable to use a spatial database. GeoDjango distance queries are only available if you use PostGIS, Oracle, or SpatiaLite as your database. (i.e. MySQL does not support distance queries)
unable to install the geospatial libraries that GeoDjango depends on.
Well, here is my problem, I'll make an e-commerce project for my thesis, I'll make it using Django, and I wanna use NoSql solution, since I'll imagine that this e-commerce website will grow (...) the idea of that website is a social-e-commerce without money transactions, I'll use two solutions: MongoDB to store users information (since it's a schema-free) and Neo4j to make relations between users.
Ok, maybe someone will say: why not using Neo4j to handle everything since Neo4j is also a schema-free, but because it's Django, so I said that something that is a C-program will be better and faster for the web application than a 100% Java solution (when dealing with C-Django), I'll use CPython and not Jython, and I've installed JPype, so I imagine that calling Java from Python is something that will take time?
So my question is:
is it better to use that solution: _id123456789012 is a (friend/best client/best seller...) with _id122234567890 as you can see the _id is generated from mongodb (the harder work) but only the relations are made using neo4j, so all the 80% of the work is done from mongoDB, or do i make everything with one of them: neo4j or mongo?
second, if the solution of using both, how about scaling? from the graph that speaks about NoSql it seems that Neo4j is the latest in Scaling in size (while the best in Scaling in complexity) so will there a problem of "synchronizing"?
It looks like the question has been answered in the comments, so I'm providing this answer to allow the question to show up as being answered.
I'm building a real estate site in Django and have a Home model, which stores various information including the address. Database backend is using MySQL.
Want to create a Yelp like search.
A search where users can enter in zip code or city name, then get Home results in that area.
Users can also choose the radius(5 mi, 10 mi...) from the point and get more/less results.
Search results will be on google map and users can zoom in/out to get new search results within the map.
Is Geo Django a good fit here? Guessing not too many people use GeoDjango, because I can't find many docs to solve problems mentioned above.
After looking at its official doc for about couple hours, I can't really find relevant example for my problems and not sure how well it integrates with existing website using MySQL. Maybe too sophisticated for my simple location usage?
I'm really curious if Geo Django is a good fit, if so..then I'll look into it more aggressively and deeply. If not, I'll try to build it on my own.
Any guidance or tips on GeoDjango or on how to build the system would be very helpful and will be appreciated so much.
GeoDjango could handle something like that. Supposing your Home model looks like this:
from django.contrib.gis.db.models import PointField
class Home(Model):
location = PointField()
.. etc
You could fetch all Home models within x miles of a point like this:
from django.contrib.gis.measure import D
from django.contrib.gis.geos import Point
Home.objects.filter(location__distance_lte=(Point([...]), D(mi=x)))
Basically, what you need to give to the filter is a tuple containing a Geometry (Point in your case), and the distance(with the D object. You have a large range of units to represent the distance, in your case mi).
Also, this won't work in MySQL because there is no backend function to handle it. Your best bet is PostgreSQL.
I built a site on Django with similar features (MySQL database full of addresses and lat/long coordinates). I talked with cohorts that are more familiar with Django and GeoDjango and they recommended that it might be a bit overkill for my project.
So if migrating the database is a huge pain and not necessary, it seems you may be well off just sticking to your current database and just using a Haversine formula query to find closest points to a location.
GeoDjango makes a lot of sense if you're already using Django. However, you should also consider using Lucene for your use case. It can handle geographical queries, and probably performs better than mysql queries if you need to do multi-faceted queries and fuzzy search (something I would want to use on a site with a lot of homes).
GeoDjango works but it is very hard to install for novice user. Also GeoDjango has some limitations on MySQL. As I know it shows the best on the postgresql.
What exactly problem do you experience in geodjango installation?
I've wasted about 10-20 hours before I began understand the geodjango installation process :)