I can't get Django to INNER JOIN properly :( - django

Despite having looked everywhere for similar issues I still cannot make the query working using INNER JOIN with the Django ORM... Sorry if this might sound stupid, but this is my first time with Django on a project and especially the ORM.
I have an Articles table with a Users table (named Fellows in my case), the Articles table has it's foreign key on author and references the user_id in Fellows table.
class Fellow(models.Model):
id = models.AutoField(db_column='ID', primary_key=True) # ID
user_id = models.PositiveBigIntegerField(db_column='User_ID', unique=True) # Global User ID.
nickname = models.CharField(db_column='Name', max_length=64, db_collation='utf8mb4_general_ci') # Display Name
user_password = models.CharField(db_column='User_Password', max_length=256, blank=True, null=True) # Passwd
gold = models.IntegerField(db_column='Gold') # Credits
faction = models.ForeignKey('Faction', models.RESTRICT, db_column='Faction', default=1) # ID Faction
class Meta:
managed = False
db_table = 'Fellows'
def __str__(self):
return self.nickname # Test.
class Article(models.Model):
id = models.AutoField(db_column='ID', primary_key=True) # ID
author = models.ForeignKey('Fellow', models.CASCADE, db_column='ID_User', default=1) # Global User ID
title = models.CharField(db_column='Title', max_length=32) # Title
content = models.TextField(db_column='Content') # Content
posted = models.DateTimeField(db_column='Posted') # Date Posted
source = models.CharField(db_column='Source', max_length=64, blank=True, null=True) # Source picture url of the article.
class Meta:
db_table = 'Articles'
I tried to get the display name of the related author that posted the article without success.
This is my views.py:
from .models import Article
def index(request):
"""
And then Vue.JS will take care of the rest.
"""
# articles = Article.objects.order_by('-posted')[:5] # Returns everything inside Articles table but nothing inside Fellows table.
# articles = Article.objects.select_related() # No Result.
# Still can't get display_name in index.html with this one.
articles = Article.objects.raw('SELECT Fellows.Name AS Display_Name, Articles.ID, Articles.Title, Articles.Content, Articles.Posted, Articles.Source FROM Articles INNER JOIN Fellows ON Fellows.User_ID = Articles.ID_User ORDER BY Articles.ID DESC LIMIT 5;')
data = {
'articles': articles,
}
return render(request, 'home/index.html', data)
The raw request returns everything fine only with sql interpreter, so there is two options:
Django won't perform the INNER JOIN.
I didn't figured out how to read the Display_Name in the template (index.html).
This is how I retrieve the data using VueJS (even with the raw query I can't get the display_name, it's empty).
<script>
const store = new Vuex.Store({
state: {
articles: [
{% for article in articles %}
{
title: '{{ article.title }}',
content: '{{ article.content | linebreaksbr }}',
source: "{% static 'home/img/' %}" + '{{article.source}}',
display_name: '{{article.display_name}}', // Maybe this is not how to retrieve the display_name?
},
{% endfor %}
],
},
});
// Components.
ArticleList = Vue.component('article-list', {
data: function () { return { articles: store.state.articles } },
template: '#article-list-template',
});
ArticleItem = Vue.component('article-item', {
delimiters: ['[[', ']]'],
props: ['id', 'title', 'content', 'source', 'display_name'],
template: '#article-item-template',
});
...
</script>
if someone could help me with this I would appreciate immensely! TT

Problem solved,
I had to change the foreign key constraint Articles.ID_User which now leads to Fellows.ID.
Previously the constraint led to Fellows.User_ID.
I can finally use:
articles = Article.objects.select_related('author').order_by('-posted')[:5]
And indeed finally accessing it in the front by article.author, simple as that.
Yet I still don't really understand why the raw sql query (using the mysql interpreter) with the INNER JOIN worked fine tho when referencing Fellows.User_ID, which was apparently not the case in the ORM.
Although it is working, my sql relational might be wrong or not ideal, therefore I am still open to suggestions!

Related

How to create Article tags from splitting the Title in Django

I want to create articles tags from the title of a RSS feed post. Then save the tags into a DB with a post_id of the title i got the tags from at the same time. Something like this:
Title = "Voyant raises $15M to scale production of its tiny, inexpensive lidar tech"
Tags = ['Voyant', 'Raises', '$15M', 'To', 'Scale', 'Production', 'Of', 'Its', 'Tiny', 'Inexpensive', 'Lidar', 'Tech']
Assuming the post_id is 1, the Tags table should look like:
id | tag | post_id
--------------------------------
1 | Voyant | 1
2 | Raises | 1
I have 3 models in my table(Source, Posts & Tags).
class Source(models.Model):
name = models.CharField(max_length=500, verbose_name='Website Name')
class Posts(models.Model):
post_title = models.CharField(max_length=500, verbose_name='Post Title')
source = models.ForeignKey(Source, on_delete=models.CASCADE, verbose_name='Source')
class Tags(models.Model):
name = models.CharField(max_length=500)
post = models.ForeignKey(Posts, on_delete=models.CASCADE, verbose_name='Posts')
So so far i was able to split the title above.
title = item.title
strip_away = title.replace(",", "").replace(":", "").replace("(", "").replace(")", "").replace("'", "").replace("[", "").replace("]", "").replace("!", "").replace("?", "").replace("-", " ")
capital = strip_away.title()
article_tags = capital.split()
But now my problem comes during the saving part.
def fetch_articles():
feed = feedparser.parse("my_site_of_preference")
source = Source.objects.get(pk=1)
source_id = int(source.pk)
source_name = source
save_new_articles(source_id, source_name, feed)
def save_new_articles(source_id, source_name, feed):
selected_source_id = source_id
for item in feed.entries:
title = item.title
""" The splitting code """
if not Posts.objects.filter(post_title=title).exists():
post = Posts(post_title = title, source_id = selected_source_id)
post.save()
for i in range(len(article_tags)):
tags = Tags.objects.create(name = article_tags[i], post_id = source_name.pk)
tags.save()
I keep getting the error:
django.db.utils.IntegrityError: insert or update on table "Posts_tags" violates foreign key constraint "Posts_tags_post_id_3e6ae939_fk_Posts_posts_id"
DETAIL: Key (post_id)=(1) is not present in table "Posts_posts".
The post hasn't been saved to create a post_id that it can be used as a PK when saving the tags. How can i go about this to save the tags after saving the post title?
When saving tags, you should reference the post with the object, not the pk of it. Django ORM will do that for you. And when using create, you do not need to save it again as create() already saves it. Try the following within your loop:
Tags.objects.create(name=article_tags[i], post=post)

Django Admin - Show form json input value as text insted string

Considering I have a model like:
MyStore = (
id = 1,
name = 'Foobar',
information_as_json = {
'open_at': datetime.now(),
'close_at': datetime.now() + timedelta('+1 day'),
'workers' : {
'Person1' : 'Owner',
'Person2' : 'Boss',
'Person3' : 'Boss',
}
})
Inside Django admin forms, for every field is generated an input, but for the field "information_as_json", I don't want to show it as a string or as JSON. That is because the users who are accessing this store admin page, need to read the field 'information_as_json' easier since no one can edit these values because it is generated in another part of the application.
Is it possible to convert these values to a "div" or a plain text? The contents would be:
This store opens at: {information_as_json.open_at}
This store close at: {information_as_json.close_at}
And for the workers, iterate through keys and values:
for key, value in information_as_json.workers:
Worker {key} has the role: {value}
I'm a beginner at Django, so I'm struggling a little with this part.
Every help would be appreciated :D
I would suggest approaching the model a little differently. Rather than storing the opening and closing hours as JSON they can just be fields directly on the store model. The the workers can be a JSONfield [docs] containing name/role pairs. If you're using PostgreSQL for your database you could even use HStoreField [docs], which might be more appropriate.
Here's how I would write a similar model.
class Store(models.Model):
name = models.CharField(max_length=512, unique=True)
workers = models.JSONField(blank=True, default=dict, editable=False)
closing = models.TimeField(blank=True, null=True, editable=False)
opening = models.TimeField(blank=True, null=True, editable=False)
To display the details in the Django admin we just need to define a property which returns the correct string.
#mark_safe
def details(self):
roles = [
f'{x} has the role: {y}'
for x, y in self.workers.items()
]
return '<br>'.join([
f'This store opens at: {self.opening:%-H:%M}',
f'This store closes at: {self.closing:%-H:%M}',
] + roles)
This method can then be referenced in the ModelAdmin and used like a read-only field.
#admin.register(Store)
class StoreAdmin(admin.ModelAdmin):
list_display = ['name', 'opening', 'closing']
fields = ['name', 'details']
readonly_fields = ['details']

django rest datatables, filter experts by meeting objectives

I need to filter all Experts by past objectives.
I have a minimal runnable example at https://github.com/morenoh149/django-rest-datatables-relations-example (btw there are fixtures you can load with test data).
my models are
class Expert(models.Model):
name = models.CharField(blank=True, max_length=300)
class Meeting(models.Model):
user = models.ForeignKey(User, on_delete=models.SET_NULL, blank=True, null=True)
expert = models.ForeignKey(Expert, on_delete=models.SET_NULL, blank=True, null=True)
objective = models.TextField(null=True, blank=True)
my datatables javascript
$("#table-analyst-search").DataTable({
serverSide: true,
ajax: "/api/experts/?format=datatables",
ordering: false,
pagingType: "full_numbers",
responsive: true,
columns: [
{
data: "objectives",
name: "objectives",
visible: false,
searchable: true,
render: (objectives, type, row, meta) => {
return objectives;
}
},
],
});
My serializer
class ExpertSerializer(serializers.ModelSerializer):
id = serializers.IntegerField(read_only=True)
objectives = serializers.SerializerMethodField()
class Meta:
model = Expert
fields = (
"id",
"objectives",
)
def get_objectives(self, obj):
request = self.context["request"]
request = self.context["request"]
meetings = Meeting.objects.filter(
analyst_id=request.user.id, expert_id=obj.id
).distinct('objective')
if len(meetings) > 0:
objectives = meetings.values_list("objective", flat=True)
objectives = [x for x in objectives if x]
else:
objectives = []
return objectives
When I begin to type in the datatables.js searchbar I get an error like
FieldError at /api/experts/
Cannot resolve keyword 'objectives' into field. Choices are: bio, company, company_id, created_at, description, email, favoriteexpert, first_name, id, is_blocked, last_name, meeting, middle_name, network, network_id, position, updated_at
Request Method: GET
Request URL: http://localhost:8000/api/experts/?format=datatables&draw=3&columns%5B0%5D%5Bdata%5D=tags&columns%5B0%5D%5Bname%5D=favoriteexpert.tags.name&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=false&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D=desc&columns%5B1%5D%5Bname%5D=&columns%5B1%5D%5Bsearchable%5D=false&columns%5B1%5D%5Borderable%5D=false&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%
fwiw, in pure django orm what I want to accomplish would be something like
Expert.objects.filter(
pk__in=Meeting.objects.filter(
objective__icontains='Plastics', user=request.user
).values('expert')
)
How can I filter experts by historical meeting objectives?
The reason for the error is that django-rest-framework-datatables is trying to translate the request into a query which can be run against the Expert table.
In your JS, you're asking for a field called 'objectives' to be returned, but there is no such field on the Expert model.
You could probably achieve what you are trying to do using the django-filter integration. In this case, you could set up a filter on the FK reference to the Meeting table. The example app demonstrates how to do this.
I think the best way to understand what's going on is to get the example application running, and if possible, set breakpoints and step through.
Incidentally, if you want to get the search box to work correctly, then you need to define a global_q() method. This is also covered in the example app.
I ended up authoring a custom django-filter
class AssociatedMeetingCharFilter(filters.CharFilter):
def global_q(self):
"""
Uses the global filter to search a meeting field of meetings owned by the logged in user
"""
if not self._global_search_value:
return Q()
kw = "meeting__{}__{}".format(self.field_name, self.lookup_expr)
return Q(**{
kw: self._global_search_value,
"meeting__user_id": self.parent.request.user.id or -1,
})
class ExpertGlobalFilterSet(DatatablesFilterSet):
name = GlobalCharFilter(lookup_expr='icontains')
objectives = AssociatedMeetingCharFilter(field_name='objective', lookup_expr='icontains')
full example at https://github.com/morenoh149/django-rest-datatables-relations-example

Django DateTimeField User Input

So I'm trying to populate a model in django using a postgres (postgis) database. The problem I'm having is inputting the datetimefield. I have written a population script but every time I run it I get the error django.db.utils.IntegrityError: null value in column "pub_date" violates not-null constraint. The code below shows my model and the part of the population script that applies to the table.
The model:
class Article(models.Model):
authors = models.ManyToManyField(Author)
location = models.ForeignKey(Location)
article_title = models.CharField(max_length=200, unique_for_date="pub_date")
pub_date = models.DateTimeField('date published')
article_keywords = ArrayField(ArrayField(models.CharField(max_length=20, blank=True), size=8), size=8,)
title_id = models.CharField(max_length=200)
section_id = models.CharField(max_length=200)
And the population script:
def populate():
add_article(
id = "1",
article_title = "Obama scrambles to get sceptics in Congress to support Iran nuclear deal",
pub_date = "2015-04-06T20:38:59Z",
article_keywords = "{obama, iran, debate, congress, america, un, republican, democrat, nuclear, isreal}",
title_id = "white-house-scrambles-sceptics-congress-iran-nuclear-deal",
section_id = "us-news",
location_id = "1"
)
def add_article(id, article_title, pub_date, article_keywords, title_id, section_id, location_id):
article = Article.objects.get_or_create(article_title=article_title)[0]
article.id
article.article_title
article.pub_date
article.article_keywords
article.title_id
article.section_id
article.location_id
article.save()
return article
if __name__ == '__main__':
print "Starting Newsmap population script..."
populate()
I've searched around for ages but there seems to be no solution to this specific problem. Any help much appreciated!!
The issue is that you do not pass to Article.objects.get_or_create the data needed to create a new object in case none already exists.
What you need to do is (see the documentation for get_or_create):
article = Article.objects.get_or_create(
article_title=article_title,
pub_date=pub_date,
defaults={
'id': id,
'article_keywords': article_keywords,
# etc...
}
)[0]
The data passed using the defaults argument will only be used to create a new object. The data passed using other keyword arguments will be used to check if an existing object matches in the database.

models.py with ManyToMany and progmatically adding data via a shell script

First post to stackoverflow I did do a search and came up dry. I also own
the django book (Forcier,Bissex,Chun) and they don't explain how to do
this. In short I can't figure out how to progmatically add a data via
a python shell script to the ManyToMay model..
from django.db import models
from django.contrib import admin
class Client(models.Model):
client = models.CharField(max_length=256, primary_key=True)
access = models.DateField()
description = models.TextField()
host = models.CharField(max_length=256)
lineEnd = models.CharField(max_length=256)
options = models.TextField()
owner = models.CharField(max_length=100)
root = models.CharField(max_length=256)
submitOptions = models.CharField(max_length=256)
update = models.DateField()
def __unicode__(self):
return str(self.client)
admin.site.register(Client)
class Change(models.Model):
"""This simply expands out 'p4 describe' """
change = models.IntegerField(primary_key=True)
client = models.ManyToManyField(Client)
desc = models.TextField()
status = models.CharField(max_length=128)
def __unicode__(self):
return str(self.change)
admin.site.register(Change)
Here is what I have which works but I don't know how to add the
ManyToMany. I can't seem to figure out how to progmatically call it.
I know the row in SQL exists.
--- massImport.py ---
# Assume the client "clientspec" exists. I know how to create that if
neeeded.
changes = [ { 'change': 123, 'desc': "foobar", status': "foobar",
client': "clientspec", }]
for item in changes:
entry = Change(
change = item['change'],
desc = item['desc'],
status = item['status'],
# client = Client.objects.filter(client=item['client'])
)
entry.save()
Can anyone show me where the error of my ways is. I would really
appreciate it.
Thanks!!
Turns out Tiago was very close..
# Assume the client "clientspec" exists. I know how to create that if
neeeded.
changes = [ { 'change': 123, 'desc': "foobar", status': "foobar",
client': "clientspec", }]
for item in changes:
entry = Change()
entry.change = item['change']
entry.desc = item['desc']
entry.status = item['status']
entry.time = datetime.datetime.fromtimestamp(float(item['time']))
entry.client.add(Client.objects.get(client=item['client']))
entry.save()
So.. I will give props to Tiago
.filter returns a list, when you need is a single object, so you should use .get(client=item['client'])
I tried the code but i got error
ValueError: "<Change: 123 -- foobar>" needs to have a value for field "change" before this many-to-many relationship can be used
Manytomany(entry.client.add) can be used only after saving the field ie entry.save()
There may be a lot of clients so you can use:
changes = [{'change': 123, 'desc': "foobar", 'status': "foobar",
'client': ("client1","client2"),},{......]
for item in changes:
entry = Change(
change = item['change'],
desc = item['desc'],
status = item['status'],)
entry.save()
for c in item['client']:
entry.client.add(Client.objects.get(client=c))