I am looking for a simple way to create geographical maps in Django, in which I could then select, highlight and annotate countries or groups thereof.
"Annotate": insert a label displaying textual information about the said country.
Is there anything that comes to mind?
Many thanks
EDIT: I checked GeoDjango already and it looks like much work in order to get where I need to. Don't get me wrong: I'm not trying to minimize my own investment in learning new tools, but for this project, I have a trade-off between time allocated to learning and the relative importance of this geographical feature in my app. It's more of a nice-to-have feature I'd like to add to an already 'complete' app. So I wondered whether there exists a 'simpler' python library for this task.
I think this is more of a question for if there is a front-end library to elegantly handle this. However if you need to generate the maps you could try something like this
https://kartograph.org/
I have personally used this http://jvectormap.com/ and found it to be really good.
In your database you could just have a Countries model with any associated information you might need to display, and create a view to handle that appropriately.
I'm build a Django app with Neo4j (along with Postgres), I found this Django integration called neo4django, I was wondering if it's possible to use neo4restclient only, like, what would be the disadvantages of not using Neo4django? Does using neo4-rest-client only, give me more flexibility?
When I was creating my models with Neo4Django, it seemed that there is no difference between modeling a graph db and relational db. Am I missing anything?
Thanks!
You can absolutely go ahead with neo4j-rest-client or py2neo, without using neo4django. In the same way, you can use any other database driver you'd like any time using Django, any REST client, etc.
What'll you lose? The model DSL, the built-in querying (eg, Person.objects.filter(name="Mohamed")), the built-in indexing, and the Lucene, Gremlin and Cypher behind that. Some things will be much easier- like setting an arbitrary property on a node- but you'll need to learn more about how Neo4j works.
You'll also lose some of the shortcuts Django provides that work with neo4django, like get_object_or_404() and some of the class-based views that work with querysets.
What'll you gain? Absolute power over the DB, and an easier time tweaking DB performance. Though neo4django isn't nearly as good a lib as some traditional ORMs in the Python sphere, the trade-off of power vs provided ease is similar.
That said, the two can work together- you can drop down from neo4django to the underlying REST client nodes and relationships anytime. Just use model_instance.node to get the underlying neo4j-rest-client node object from a model, and from neo4django.db import connection to get a wrapped neo4j-rest-client GraphDatabase.
On whether you're missing something: neo4django was written to re-use a powerful developer interface- the Django ORM- so it should feel similar to writing models for Postgres. I've written a bit about that odd feeling in the past. I think part of the problem might be that the lib doesn't highlight the graph terminology new graph-interested devs expect- like traversals and pattern matching- and instead dresses those techniques in Django query clothing.
I'd love your thoughts, or to know anything you'd like the library to do that it isn't doing :) Good luck!
I am developing a gis application using Google maps API. Currently I am using Postgis db.
I am considering switching to mongodb and I have following questions about this,
Is mongodb a viable choice for storing GIS data. ( Is any other NoSQL engine viable?)
Does django-nonrel has modified django.contrib.gis module for mongodb support? and how well does it work?
Thanks in advance :)
As stated by Sergio Tulentsev, MongoDB has spatial indexes. But, not all geometry types can be stored. As far as I know, currently only points can be stored. You can, however, query using a polygon.
Since MongoDB is very flexible, you could store geometry as a text, but also as a JSON object. For example, you could store a coordinate like: { lon: 52.1234, lat: 14.1245 }. You can interpret this is on your own application as you like.
The downside to this is that there is no native index. If you never have to query based on location/spatial relation, you are fine. If you do have query based on location/spatial relation, you would have to write your own index. Can be a hard thing do...
There is CouchDB which has a GeoSpatial addition. Don't know how it works/what it supports though.
A hybrid approach is also often taken by some large-data-based companies. For example, use MongoDB to store/query normal data, and PostGIS to store/query geospatial data.
Django MongoDB Engine developer here.
We haven't got any sort of real support for MongoDB's GIS features. It's possible to do geospatial queries by "working around Django" though (see http://django-mongodb.org/topics/lowerlevel.html - the first example is actually about Geo indexes).
It should be possible, however, to implement native Django GIS support into the MongoDB backend. We're really happy to mentor anyone who wants to approach that task!
I've been working with MongoDb storing points of interest in the DB, then doing point in poly queries, along with radius queries and Mongo has been FAR superior to PostGres. I've had much faster response times with hundreds of thousands of items.
I would say however depending on the complexity of your queries (ie if you're doing something beyond point in poly, radius around a point), you might want a more heavy GIS db like postgres.. you're just going to get the extra weight of postgres as well.
Sorry I can't speak to anything on django as well.
I'm building a real estate site in Django and have a Home model, which stores various information including the address. Database backend is using MySQL.
Want to create a Yelp like search.
A search where users can enter in zip code or city name, then get Home results in that area.
Users can also choose the radius(5 mi, 10 mi...) from the point and get more/less results.
Search results will be on google map and users can zoom in/out to get new search results within the map.
Is Geo Django a good fit here? Guessing not too many people use GeoDjango, because I can't find many docs to solve problems mentioned above.
After looking at its official doc for about couple hours, I can't really find relevant example for my problems and not sure how well it integrates with existing website using MySQL. Maybe too sophisticated for my simple location usage?
I'm really curious if Geo Django is a good fit, if so..then I'll look into it more aggressively and deeply. If not, I'll try to build it on my own.
Any guidance or tips on GeoDjango or on how to build the system would be very helpful and will be appreciated so much.
GeoDjango could handle something like that. Supposing your Home model looks like this:
from django.contrib.gis.db.models import PointField
class Home(Model):
location = PointField()
.. etc
You could fetch all Home models within x miles of a point like this:
from django.contrib.gis.measure import D
from django.contrib.gis.geos import Point
Home.objects.filter(location__distance_lte=(Point([...]), D(mi=x)))
Basically, what you need to give to the filter is a tuple containing a Geometry (Point in your case), and the distance(with the D object. You have a large range of units to represent the distance, in your case mi).
Also, this won't work in MySQL because there is no backend function to handle it. Your best bet is PostgreSQL.
I built a site on Django with similar features (MySQL database full of addresses and lat/long coordinates). I talked with cohorts that are more familiar with Django and GeoDjango and they recommended that it might be a bit overkill for my project.
So if migrating the database is a huge pain and not necessary, it seems you may be well off just sticking to your current database and just using a Haversine formula query to find closest points to a location.
GeoDjango makes a lot of sense if you're already using Django. However, you should also consider using Lucene for your use case. It can handle geographical queries, and probably performs better than mysql queries if you need to do multi-faceted queries and fuzzy search (something I would want to use on a site with a lot of homes).
GeoDjango works but it is very hard to install for novice user. Also GeoDjango has some limitations on MySQL. As I know it shows the best on the postgresql.
What exactly problem do you experience in geodjango installation?
I've wasted about 10-20 hours before I began understand the geodjango installation process :)
The story so far:
Decided to go with Xapian as search backend because it has all search-engine features I was looking for, knows about Unicode, stemming, has few dependencies and requires no bloated app-server installation on top of it.
Tried Django and Haystack (plus xapian-haystack, the backend glue code to tie Haystack to Xapian) because it was advertised on quite some blogs as "working". Did not work. Neither django-haystack nor the xapian-haystack project provide a version combination that actually works together. MASTER from both projects yields an error from Xapian, so it's not stable at all. Haystack 1.0.1 and xapian-haystack 1.0.x/1.1.0 are not API-compatible. Plus, in a minimally working installation of Haystack 1.0.1 and xapian-haystack MASTER, any complex query yields zero results due to errors in either django-haystack or xapian-haystack (I double-verified this), maybe because the unit-tests actually test very simple cases, and no edge-cases at all.
Tried Djapian. The source-code is riddled with spelling errors (mind you, in variable names, not comments), documentation is also riddled with ambiguities and outdated information that will never lead to a working installation. Not surprisingly, users rarely ask for features but how to get it working in the first place.
Next on the plate: exploring Solr (installing a Java environment plus Tomcat gives me headaches, the machine is RAM- and CPU-constrained), or Lucene (slightly less headaches, but still).
Before I proceed spending more time with a solution that might or might not work as advertised, I'd like to know: Did anyone ever get an actual, real-world search solution working in Django? I'm serious. I find it really frustrating reading about "large problems mostly solved", and then realizing that you will never get a working installation from the source-code because, actually, all bloggers dealing with those "mostly solved problems" never went past basic installation and copy-pasting the official tutorials.
So here are the requirements:
must be able to search for 10-100 terms in one query
must handle + (term must be present) and - (term must not be present), AND/OR
must handle arbitrary grouping (i.e. parentheses around AND/OR)
must allow for Django-ORM filtering before or after fulltext-search (i.e. pre-/post-processing of results with the full set of filters that Django knows about)
alternatively, there must be a facility to bulk-fetch the result set and transform it into a QuerySet
should be light on the machine, so preferably no humongous JVM and Java-based app-server installation
Is there anything out there that does this? I'm not interested in anecdotal evidence, or references to some blog posts that claim it should be working. I'd like to hear from someone who actually has a fully-functional setup working in the real world, under real conditions, with real queries.
EDIT:
Let me repeat again that I'm not so much interested in anecdotal evidence that someone, somewhere has a somewhat running installation working with unspecified properties. I already went there, I read all the blog posts, mailing lists, I contacted the authors, but when it came to actual implementation of real-world scenarios, nothing ever worked as advertised.
Also, and a user below brought that point up as well, considering the TCO of any project, I'm definitely not interested in hearing that someone, somewhere was able to pull it off once a vendor parachuted in an unknown number of specialists to monkey-patch the whole installation with specific domain-knowledge that's documented nowhere.
So, please, if you claim you have a working installation that actually satisfies minimum requirements for a full-fledged search (see requirements above), please provide the following so that we can all benefit from a search solution for Django that actually solves the problem:
exact Linux distribution, release version,
exact release version of Haystack (or equivalent) and release version of search backend,
exact release version of the search engine
publicly (!) available documentation how to set up all components exactly in the way that your installation was set up such that the minimal requirements above are met.
Thank you.
I have developed some Django applications with xapian support too. The biggest of them has a xapian database with an index of 8G storing 2.4M documents (including forum posts, wiki entries, planet entries and blog entries) - still growing.
Overall I am quite happy with xapian. It performs extremely well and is easy to use. The only thing I don't like is that xapian won't work with mod_wsgi (except of the global mode) because of a deadlock. So you are forced to use fastcgi (or connect to xapian-tcpsrv or write your own service).
I recommend you, to use the xapian-bindings directly. Xapian nowadays offers quite a lot of useful helpers (TermGenerator, QueryParser etc), which makes both the indexing and the querying simple. In fact, there is nothing I can imaging which would justify an additional library. In my opinion they are all more complicated and don't allow you to index efficiently.
The only thing you need, is some understanding of the way how xapian is working. (What are terms? What are values? What is stemming and where should I use it? and so on). You can find all those topics on the xapian website, and as soon as you understand those concepts, dealing with xapian will become easy.
Also, the xapian API is extremly stable. I've started using it a long time before the 1.0 release and never had any problems with API changes or version conflicts. The only thing which has changed is that all those helpers (query parser, tokenizer, etc.) I have once written for my Django project are now useless, because similar classes have made their way into the xapian core.
So, to summarize, just give the direct usage of xapian-bindings a try.
I can vouch for Django-Haystack with the Xapian backend (In the interest of full disclosure, I am the author of the xapian-haystack backend) in a real life, production environment. We currently use Haystack/Xapian on several sites, the largest of which has more than 20,000 registered users and a Xapian database with 20,000+ documents containing more than 143,000 unique terms for a total size of ~141mb.
As for not being able to get any combination of Haystack and the Xapian backend running, I'll admit that I was not as diligent as I should have been with my tagging and so there is some confusion with the versions. You should, however, be able to use the current master of both codebases without any issue. If this is not case, I'd be more than happy to assist with problems. You'll need to be a little bit more specific about the issue though. Simply saying "it did not work" is not enough information.
Daniel and I both do our best to respond to any issues opened on Github within a timely manner. Also, we're both usually available on the #haystack IRC channel during the day and the django-haystack Google Group.
Versions used:
Haystack 1.0BETA with Xapian-Haystack 1.1.0BETA
Haystack 1.0.1FINAL with Xapian-Haystack 1.1.3BETA
Most of the sites we've deployed with Haystack have been running Ubuntu 8.04 LTS with Xapian 1.0.5
Short answer: No.
We bailed and went with a Google Custom Search. Although the site has over 10,000 possible page views, we keep the sitemap feed down to the main 4,000 pages or so and it costs $250/year, which is about 2 hours of my time. The customer is happy and he feels comfortable with the results.
I'd love to see someone come up with a good FOSS solution, but in a commercial situation the TCO has got to make economic sense.
The details you requested.
exact Linux distribution, release version - Ubuntu 9.04 & 9.10
exact release version of Haystack (or equivalent) - Haystack 1.0 as well as master
release version of search backend - The Solr & Whoosh backends included with Haystack
exact release version of the search engine - Solr 1.3, Solr 1.4 & Whoosh 0.3.15
publicly (!) available documentation how to set up all components exactly in the way that your installation was set up such that the minimal requirements above are met.
http://docs.haystacksearch.org/dev/installing_search_engines.html#solr (or #whoosh)
Beyond this, it's the standard configuration bits from the tutorial, plus any additional overrides from (which I can't link to, thanks Stack Overflow) as needed.
As the maintainer of Haystack, I'm actively running all of the above previous setups. The smallest Haystack installation (Haystack 1.0 + Whoosh) is ~600 documents. A slightly larger one (Haystack master + Solr 1.4) is ~4000 documents. The largest deployment I'm aware of (Haystack master + Solr 1.4) is ~3 million documents.
I generally try to avoid Stack Overflow, so don't be surprised if you see nothing further from me. The mailing list is the best place for support, but given your responses thus far, I'm sure you'd rather just trash me here.
I (and my colleagues) have successfully used Haystack to achieve a fairly good search functionality.
It is easy to start with haystack and whoosh backend; and change to the Apache-Solr backend when performance of whoosh is not acceptable.
We really got to get around to write a detailed post about it with links to the projects where it works.
For now I can suggest you to have a look at this search: http://www.webdevjobshq.com/search/?q=rails implemented using Haystack with Apache-Solr backend. Or this: http://www.govbuddy.com/search/?q=Roy
Have you considered Sphinx? What are you using as you data store? It has a MySQL engine that works terrific. I think it meet most of your requirements except I'm not exactly certain how nicely it can be tied into Django-ORM.
I'm heavily considering using Sphinx in one of my own Django Apps to improve performance on an auto-suggest field that does a prefix and infix search on a corpus of 3.5 million records. But I haven't got around to implementing it yet, so I can't speak to Django+Sphinx integration. My only Sphinx experience is with the MySQL Engine and directly querying MySQL.
I use Djapian. It was quite simple to install and works great. There is an actual tutorial that covers basic use-cases and shows entire integration process.
Yes, it has some ambiguities but issue tracker is open and authors rapidly fixes bugs and add features.