Django models architecture for full-text search in PostgreSQL - django

I'm building a web-site for a social interaction on a particular topic, based on threads, think of gmail, only public. There will also be some static info in vocabularies, as well as blog, how-tos, knowledge base, etc. It is django+postgres.
One of the most important requirements is a full-text search over all information, regardless of the type of a model. If the exact search phrase appears in the blog, and its twisted sister in messages, than a snippet from the blog entry should appear first in the search results and be followed by a snippet from the message. So, i need a table with all the texts indexed, and the links to _any_other_table_ in the db.
My first idea is to create a separate model with "loose reference", e.g.:
class Content(models.Model):
obj_id= CharField() # An id of the object of a given model.
model= CharField(choices=("Message", "BlogEntry", "HowTo", "EntityProfile",))
content_type= CharField(choices=("subject", "body", "description", "tags",))
body= TextField()
But it feels kind of wrong... This promises an unnecessary hassle around integrity of references when creating and re-linking instances.
So, the question is - is there any elegant solution that django would provide? What might be the most efficient architecture to solve the problem?
I am not asking for a direct answer, but rather a hint.
Thanks in advance!

I've had a lot of success working with this snippet, which uses PostgreSQL's tsearch2. I've tweaked it for various different purposes in various different ways, but basically it works very well, and is very easy to implement.

Thanks a lot for the hints!
while contenttypes is an ideal for this kind of task, it is really tempting to be DB independent
from what i've read so far, Solr is a reliable search engine, but i think, i'll go and try ElasticSearch via Haystack
thanks again!

Related

Django Master/Detail

I am designing a master/detail solution for my app. I have searched for ever in the django docs, also here and elsewhere I could, so I guess the answer is not that obvious, despite being an answer many people look for - not only in django, but in every language, I think.
Generally, in most cases, the master already exists: for example, the Django Docs illustrate the Book example, where we already have an Author and we want to add several Books for that Author.
In my case, the parent is not yet present on the database think of a purchase order, for instance.
I have thought to divide the process in two steps: the user would start to fill in the info for the master model (regular form) and then proceed to another view to add the lines (inline formset). But I don't think this is the best process at all - there are a lot of possible flaws in it.
I also thought about creating a temporary parent object in a different table and only having a definitive master when the children are finally created. But it still doesn't look clean.
Because of that, for my app it would be ideal to create the master object at the same time as the detail objects (lines) - again, like an order.
Is there a way where I can have the same view to manage both master and detail? Like this I would receive both in the same POST request and it would make a lot more sense, not to say it would be much cleaner.
Sorry if it's too long, and thank you in advance!
So I found out that in my case the process could actually be split in two phases.
For this I simply use the traditional model form and inline formset.
But! I also found out that there could be several answers to this:
We could get crazy and build some spaceship in AJAX that would get the job done, simply by sending a JSON object (in which the lines could be an array of objects)
Django also has its ways and it's possible to send multiple forms in the same request! (thank you #mousetail for the tip).
Of course, be there as it may, there are many ways to build a house, these are just the ones I found out.

Query on a Many to Many relationship using through in Django

I'm new in Django and I'm giving myself a big headhache trying to structure this query.
I have a BaseProfile connected with a OneToOne field to User.
I'm specializing the profile in CustomerProfile connected with a OneToOne field to BaseProfile.
A CustomerProfile has a ManyToMany relationship with other CustomerProfile (so itself) through a RelatedCustomer model.
In the RelatedCustomer I specify the from_customer and to_customer Foreign Keys.
Maybe with an image you can understand better.
My problem:
Given a user.id I need to know all the other user.id of the customers that he is connected to (so passing through from_customer and to_customer):
So basically, first I need to dig from User to RelatedCustomer using reverse lookup, take all the set, and then going back to know the user.id of each customer in the set.
EDIT2:
What I've reached so far:
# This gives me back a customer profile given a user.id (2)
cm = CustomerProfile.objects.get(base_profile__user=2)
# M2M lookup. Given one customer returns all the RelatedCustomer relations
# that he has as a part of the 'from' M2M
cm.from_photographer.all()
Chaining the previous two: given a user.id I obtain a queryset of CustomerRelated relations:
rel = CustomerProfile.objects.get(base_profile__user=2).from_photographer.all()
This gives me back something like:
[<CustomerRelated: from TestCustomer4 to TestCustomer2 >,
<CustomerRelated: from TestCustomer4 to TestCustomer3 >]
Where in this case the user having a user.id=2 is the TestCustomer4.
My question:
So far so good, but now having this set how can I get all the user.id of the to_customer?
That is, how do I get the user.id of TestCustomer2 and TestCustomer3?
Firstly, this is not how you query the database in django. Secondly (since you're learning), it would be good to point out that you can run dbshell to try out different things. And lastly, this kind of problem is described in the documentation.
I am telling you this, because as a beginner, I also felt that it was a little difficult to navigate through the whole thing. The best way to find things is just to use google, and add a django at the end.
I know how you feel, the documentation search sucks, right? Heh, I feel you, that is why you always search the way I described it. Once you get a hang of the documentation, you will feel that the documentation title page is a little more intuitive.
Okay, so now to the answer:
To access a ManyToMany, OneToOne or ForeignKey field, you need to use a __ commonly known as dunder.
So, this is how I would go about doing this. Please note that there are other ways, and potentially better ways of doing this:
thing_I_want = RelatedCustomer.objects.get(to_customer__id=2)
Note, however that if you wanted to get a list of customers you would use filter(). Here is an example (which uses number of purchases as an example):
things_I_want = RelatedCustomer.objects.filter(to_customer__no_of_purchases=16)
Also note that the great thing about filter is that you stack one filter on top of another. You can read more about these features in the documentation link I provide below.
That will get you what you want. Now, you might have more queries regarding this, and how it all works together. Not to fear, please click this documentation link to check it out.
EDIT
Seems like what you want to do can be done by django, but if you want to do it using sql, then that is possible too. For example, SomeModel.objects.raw("SQL_HERE"). The name of the tables are usually <app>_<model>.
However, what you are asking can also be done in django, using the ORM. But it will be tricky.
Ok, as usual whenever you get the answer it always look much more easier than what you were expecting.
I guess this worked for me:
User.objects.filter(base_profile__customer_profile__to_customer__in=
User.objects.get(id=2).base_profile.customer_profile.from_customer.all())
Many thanks to #Games Brainiac

Single model for storing django application-wide options

I have a set of top-level configuration data fields that I want to be able to set within django admin for each deployment of my django app. I only want one set of these.
Example fields: site_logo, contact_person, address, facebook_url, twitter_url
The problem is that Django Admin is geared towards tables (lists) of models, so its not a good fit for this type of singular configuration model. I really only want one of these models to exist for the whole site, and to be able to click into it from admin and edit the various fields.
It seems i've come across a 3rd party app in the past to accomplish this but can't find it anywhere. Part of the problem is I'm finding it difficult to find the right words to google. Any ideas?
It looks like django-values will do what you're looking for.
Other possible contenders:
http://github.com/sciyoshi/django-dbsettings (doesn't look maintained)
http://github.com/jqb/django-settings
Have a look at django-livesettings it sounds like it might fit.
Not that i have used it, but i have heard good things about django-constance.
And there are even some more options listed in the Configuration-Grid on Django Packages.

Satchmo: list all active products

I'd like to list all active products (from all or a specific category) in a template. I've looked almost everywhere and I simply cannot find a way to do this.
I want to display them in the footer of the shop (10 products from 1 category). That means show them without selecting product category.
Is this even possible? Products are only listed in the category template...
I'm using Satchmo 0.9.2
EDIT: Somehow I've missed this:
http://www.satchmoproject.com/docs/dev/customization.html
So it's solved...
Thank you!
this is a more general answer since there isnt any answer yet, so dont beat me. You also have to know that I never used satchmo, I never had a look at it.
But despite this, if I had to deal with your situation, I would have a look at the source code.
You might find answers there to develop something custom for your situation. This can be a tricky task but at least its worth a try.
There have to be models which store the data for your product and categories. Have a look at them and the views that retrieve the products from the database to render them. Also a look into the database cant hurt (think of phpmyAdmin to have a nice webbased interface).
It can be helpful to fire up your ./manage.py shell, import your/satchmos product and category models and play around with them.
A possible solution then could be to write a custom context_processor which retrieves the needed products/categories and passes these products from a category to your footer on a more global basis. Have a look here https://docs.djangoproject.com/en/1.3/ref/templates/api/#writing-your-own-context-processors. Maybe a custom middleware could also be a possibility. https://docs.djangoproject.com/en/dev/topics/http/middleware/#writing-your-own-middleware
I hope this helps. At least worth a try :)

Django end-user defined fields, how to? [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Django dynamic model fields
Good Morning guys!
Scenario is the following. For some models on Django, I would like to allow the end user to define his own fields. It would be great if I could keep all Django awesome features like the ORM, so I can still do calls like field__gte to search on the model, still have field validation according to field type, etc. I've thought about two ways of doing this, and I'm more than open for new suggestions. Any feedback would be VERY appreciated.
The first approach, is the Entity-Attribute-Value ( http://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model ), which django already has an app for. See http://code.google.com/p/django-custom-field/
I think this would be an OK solution, but I lose the ability to do "mymodel.objects.filter(custom_field_x=something)". Maybe there's a way to regain the ORM, any ideas? But I've heard so many bad stories about this method that I'm little scared to use it.
The second approach would be to have a database table for each of the users (probably no more than a 1000). I've read django has something in the lines of inspectdb, which actually checks which fields are there and produces the model for you. This could be useful but I think maybe I should store the fields this particular user has created and somehow dinamically tell django, hey, we also have this fields in this model. Is this possible? I know it's generally bad to have different tables for each user, but considering this scenario, how would you guys rate this method, would it be ok to have one table for each user?
The model that requires custom fields is for example Person. They might want a custom field to store address, blood type, or any other thing.
MANY THANKS in advance! Have a nice sunday!
Very similar: How to create user defined fields in Django -- but only talks about the EAV, which I would like to avoid. I'm open for new ideas!
One approach is to use a NoSQL document-based solution such as MongoDB which allows you to store objects that have a fluid structure (no such restrictions as pre-defined columns).
Pros:
No restriction on custom field types, number of types of fields, etc.
Retains ORM functionality (django-mongodb)
Other various benefits of NoSQL - which you can read about online
Avoids EAV
Cons:
Need to setup NoSQL server
Additional knowledge required on NoSQL concepts (documents vs. tables)
You may have to maintain two databases - if you decide not to migrate your entire solution to NoSQL (multi-db)
EDIT:
After reading the comments its worth pointing out that depending on which NoSQL solution you go with, you may not need reversion support. CouchDB, for example has built in support for document versioning.
what about creating another model for storing user_defined_fields?
class UserDefinedField(models.Model):
#..................
user = models.ForeignKey(User)
field_name = models.CharField(max_length=50)
field_value = models.TextField()
Then you can do UserDefinedField.objects.filter(field_name=some_name,field_value=somevalue)