So I'm trying to populate a model in django using a postgres (postgis) database. The problem I'm having is inputting the datetimefield. I have written a population script but every time I run it I get the error django.db.utils.IntegrityError: null value in column "pub_date" violates not-null constraint. The code below shows my model and the part of the population script that applies to the table.
The model:
class Article(models.Model):
authors = models.ManyToManyField(Author)
location = models.ForeignKey(Location)
article_title = models.CharField(max_length=200, unique_for_date="pub_date")
pub_date = models.DateTimeField('date published')
article_keywords = ArrayField(ArrayField(models.CharField(max_length=20, blank=True), size=8), size=8,)
title_id = models.CharField(max_length=200)
section_id = models.CharField(max_length=200)
And the population script:
def populate():
add_article(
id = "1",
article_title = "Obama scrambles to get sceptics in Congress to support Iran nuclear deal",
pub_date = "2015-04-06T20:38:59Z",
article_keywords = "{obama, iran, debate, congress, america, un, republican, democrat, nuclear, isreal}",
title_id = "white-house-scrambles-sceptics-congress-iran-nuclear-deal",
section_id = "us-news",
location_id = "1"
)
def add_article(id, article_title, pub_date, article_keywords, title_id, section_id, location_id):
article = Article.objects.get_or_create(article_title=article_title)[0]
article.id
article.article_title
article.pub_date
article.article_keywords
article.title_id
article.section_id
article.location_id
article.save()
return article
if __name__ == '__main__':
print "Starting Newsmap population script..."
populate()
I've searched around for ages but there seems to be no solution to this specific problem. Any help much appreciated!!
The issue is that you do not pass to Article.objects.get_or_create the data needed to create a new object in case none already exists.
What you need to do is (see the documentation for get_or_create):
article = Article.objects.get_or_create(
article_title=article_title,
pub_date=pub_date,
defaults={
'id': id,
'article_keywords': article_keywords,
# etc...
}
)[0]
The data passed using the defaults argument will only be used to create a new object. The data passed using other keyword arguments will be used to check if an existing object matches in the database.
Related
I have a History model like below
class History(models.Model):
class Meta:
app_label = 'subscription'
ordering = ['-start_datetime']
subscription = models.ForeignKey(Subscription, related_name='history')
FREE = 'free'
Premium = 'premium'
SUBSCRIPTION_TYPE_CHOICES = ((FREE, 'Free'), (Premium, 'Premium'),)
name = models.CharField(max_length=32, choices=SUBSCRIPTION_TYPE_CHOICES, default=FREE)
start_datetime = models.DateTimeField(db_index=True)
end_datetime = models.DateTimeField(db_index=True, blank=True, null=True)
cancelled_datetime = models.DateTimeField(blank=True, null=True)
Now i have a queryset filtering like below
users = get_user_model().objects.all()
queryset = users.exclude(subscription__history__end_datetime__lt=timezone.now())
The issue is that in the exclude above it is checking end_datetime for all the rows for a particular history object. But i only want to compare it with first row of history object.
Below is how a particular history object looks like. So i want to write a queryset filter which can do datetime comparison on first row only.
You could use a Model Manager method for this. The documentation isn't all that descriptive, but you could do something along the lines of:
class SubscriptionManager(models.Manager):
def my_filter(self):
# You'd want to make this a smaller query most likely
subscriptions = Subscription.objects.all()
results = []
for subscription in subscriptions:
sub_history = subscription.history_set.first()
if sub_history.end_datetime > timezone.now:
results.append(subscription)
return results
class History(models.Model):
subscription = models.ForeignKey(Subscription)
end_datetime = models.DateTimeField(db_index=True, blank=True, null=True)
objects = SubscriptionManager()
Then: queryset = Subscription.objects().my_filter()
Not a copy-pastable answer, but shows the use of Managers. Given the specificity of what you're looking for, I don't think there's a way to get it just via the plain filter() and exclude().
Without knowing what your end goal here is, it's hard to say whether this is feasible, but have you considered adding a property to the subscription model that indicates whatever you're looking for? For example, if you're trying to get everyone who has a subscription that's ending:
class Subscription(models.Model):
#property
def ending(self):
if self.end_datetime > timezone.now:
return True
else:
return False
Then in your code: queryset = users.filter(subscription_ending=True)
I have tried django's all king of expressions(aggregate, query, conditional) but was unable to solve the problem so i went with RawSQL and it solved the problem.
I have used the below SQL to select the first row and then compare the end_datetime
SELECT (end_datetime > %s OR end_datetime IS NULL) AS result
FROM subscription_history
ORDER BY start_datetime DESC
LIMIT 1;
I will select my answer as accepted if not found a solution with queryset filter chaining in next 2 days.
In this model:
class Rank(models.Model):
User = models.ForeignKey(User)
Rank = models.ForeignKey(RankStructure)
date_promoted = models.DateField()
def __str__(self):
return self.Rank.Name.order_by('promotion__date_promoted').latest()
I'm getting the error:
Exception Value:
'str' object has no attribute 'order_by'
I want the latest Rank as default. How do I set this?
Thanks.
Update #1
Added Rank Structure
class RankStructure(models.Model):
RankID = models.CharField(max_length=4)
SName = models.CharField(max_length=5)
Name = models.CharField(max_length=125)
LongName = models.CharField(max_length=512)
GENRE_CHOICES = (
('TOS', 'The Original Series'),
('TMP', 'The Motion Picture'),
('TNG', 'The Next Generation'),
('DS9', 'Deep Space Nine'),
('VOY', 'VOYAGER'),
('FUT', 'FUTURE'),
('KTM', 'KELVIN TIMELINE')
)
Genre = models.CharField(max_length=3, choices=GENRE_CHOICES)
SPECIALTY_OPTIONS = (
('CMD', 'Command'),
('OPS', 'Operations'),
('SCI', 'Science'),
('MED', 'Medical'),
('ENG', 'Engineering'),
('MAR', 'Marine'),
('FLT', 'Flight Officer'),
)
Specialty = models.CharField(max_length=25, choices=SPECIALTY_OPTIONS)
image = models.FileField(upload_to=image_upload_handler, blank=True)
This is the Rank_structure referenced by Rank in Class Rank.
THe User Foreign key goes to the standard User table.
The reason that you’re getting an error is because self.Rank.Name is not a ModelManager on which you can call order_by. You’ll need an objects in there somewhere if you want to call order_by. We can’t help you with the django formatting for the query you want unless you also post the model definitions as requested by several commenters. That said, I suspect that what you want is something like:
def __str__(self):
return self.objects.filter(Rank_id=self.Rank_id).order_by('date_promoted').latest().User.Name
How to get only recently added Order?
class Order(models.Model):
STATUSS = (
(u'E', u'Expected'),
(u'S', u'Sent'),
(u'F', u'Finished'),
)
who = models.ForeignKey(User, related_name='Owner')
products = models.TextField()
date = models.DateTimeField(auto_now_add=True)
send = models.ForeignKey(Send)
status = models.CharField(max_length=2, null=True, choices=STATUSS, default='O')
I prefer auto-increment pk, then
Order.objects.latest('pk')
It's simpler, indexed and is ready as long as the default surrogate primary key is used.
If by recently you mean the most recent order regarding the date it was added then you can use:
Order.objects.order_by('-date')[0]
If your definition of "recently" is "the last added", you can use latest()
Order.objects.latest('date')
or just
Order.objects.latest()
if you have
class Meta:
get_latest_by = 'date'
in your model. This is from the django docs, https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.latest
If you want more than one of the most recent, say everything from the last 5 days:
import datetime
Order.objects.filter(date__gte=datetime.date.today() - datetime.timedelta(days=5))
or if you want the last 10 records regardless of how recent, then:
Order.objects.order_by('-date')[10]
In brief: A model's method performs a query (returning the output of objects.filter()), but when the objects' values are changed in the database, the results of objects.filter() don't update until I bounce the server. How can I force the query to evaluate each time the method is called?
The details:
At the model level, I've defined a method to return all non-expired Announcement objects:
class AnnouncementManager(models.Manager):
# this is the method
def activeAnnouncements(self, expiry_time):
activeAnnouncements = self.filter(expires_at__gt=expiry_time).all()
return activeAnnouncements
class Announcement(models.Model):
...
expires_at = models.DateTimeField("Expires", null=True)
objects = AnnouncementManager()
I call this from a view with:
activeAnnouncements = Announcement.objects.activeAnnouncements()
However, when an Announcement object's data is updated in the database (e.g. expires_at is changed), the query still reflects the old data until the server is bounced. After reading http://docs.djangoproject.com/en/dev/ref/models/querysets/#when-querysets-are-evaluated, I tried to force the query to reevalute by updating the method as follows:
def activeAnnouncements(self, expiry_time):
# use boolean evaluation to force reevaluation of queryset
if self.filter(expires_at__gt=expires):
pass
activeAnnouncements = self.filter(expires_at__gt=expiry_time).all()
return activeAnnouncements
This had no effect.
Thanks for your help!
Update:
Can you please show the full code of where you are calling it?
This is the view which calls it:
#never_cache
def front_page(request):
'''
Displays the current announcements
'''
announcements = ''
activeAnnouncements = Announcement.objects.activeAnnouncements().order_by('-id')
if not request.user.get_profile().admin:
hide_before = request.user.get_profile().suppress_messages_before
if hide_before is not None:
activeAnnouncements = activeAnnouncements.filter(created_at__gt=hide_before)
if activeAnnouncements.count() > 0:
announcements = activeAnnouncements
else:
announcements = ""
return render_to(
request
, "frontpage.html"
, {
'announcements' : announcements
})
And here's the full version of the Announcement and AnnouncementManager models (excerpted above):
class AnnouncementManager(models.Manager):
# Get all active announcements (i.e. ones that have not yet expired)
def activeAnnouncements(self, expires=datetime.datetime.now()):
activeAnnouncements = self.filter(expires_at__gt=expires).all()
return activeAnnouncements
class Announcement(models.Model):
text = models.TextField()
subject = models.CharField(max_length=100)
expires_at = models.DateTimeField("Expires", null=True)
created_at = models.DateTimeField("Creation Time", auto_now_add=True)
created_by = models.ForeignKey(User, related_name="created_announcements")
updated_at = models.DateTimeField("Update Time", auto_now=True)
updated_by = models.ForeignKey(User, related_name="updated_announcements")
objects = AnnouncementManager()
def __unicode__(self):
return self.subject
Aha. The full version of the Manager method has a big difference from the one you originally posted, and it's there that the trouble is.
def activeAnnouncements(self, expires=datetime.datetime.now()):
This is one of the biggest Python gotchas: default function parameters are evaluated when the function is defined, not when it is called. So the default value for expiry will be set to whenever the server process was first started. Read the effbot's explanation of the problem. (Note it's a Python problem, not anything to do with Django querysets.)
Instead, do this:
def activeAnnouncements(self, expires=None):
if expires is None:
expires = datetime.datetime.now()
activeAnnouncements = self.filter(expires_at__gt=expires).all()
return activeAnnouncements
Is this an answer to your question?
First post to stackoverflow I did do a search and came up dry. I also own
the django book (Forcier,Bissex,Chun) and they don't explain how to do
this. In short I can't figure out how to progmatically add a data via
a python shell script to the ManyToMay model..
from django.db import models
from django.contrib import admin
class Client(models.Model):
client = models.CharField(max_length=256, primary_key=True)
access = models.DateField()
description = models.TextField()
host = models.CharField(max_length=256)
lineEnd = models.CharField(max_length=256)
options = models.TextField()
owner = models.CharField(max_length=100)
root = models.CharField(max_length=256)
submitOptions = models.CharField(max_length=256)
update = models.DateField()
def __unicode__(self):
return str(self.client)
admin.site.register(Client)
class Change(models.Model):
"""This simply expands out 'p4 describe' """
change = models.IntegerField(primary_key=True)
client = models.ManyToManyField(Client)
desc = models.TextField()
status = models.CharField(max_length=128)
def __unicode__(self):
return str(self.change)
admin.site.register(Change)
Here is what I have which works but I don't know how to add the
ManyToMany. I can't seem to figure out how to progmatically call it.
I know the row in SQL exists.
--- massImport.py ---
# Assume the client "clientspec" exists. I know how to create that if
neeeded.
changes = [ { 'change': 123, 'desc': "foobar", status': "foobar",
client': "clientspec", }]
for item in changes:
entry = Change(
change = item['change'],
desc = item['desc'],
status = item['status'],
# client = Client.objects.filter(client=item['client'])
)
entry.save()
Can anyone show me where the error of my ways is. I would really
appreciate it.
Thanks!!
Turns out Tiago was very close..
# Assume the client "clientspec" exists. I know how to create that if
neeeded.
changes = [ { 'change': 123, 'desc': "foobar", status': "foobar",
client': "clientspec", }]
for item in changes:
entry = Change()
entry.change = item['change']
entry.desc = item['desc']
entry.status = item['status']
entry.time = datetime.datetime.fromtimestamp(float(item['time']))
entry.client.add(Client.objects.get(client=item['client']))
entry.save()
So.. I will give props to Tiago
.filter returns a list, when you need is a single object, so you should use .get(client=item['client'])
I tried the code but i got error
ValueError: "<Change: 123 -- foobar>" needs to have a value for field "change" before this many-to-many relationship can be used
Manytomany(entry.client.add) can be used only after saving the field ie entry.save()
There may be a lot of clients so you can use:
changes = [{'change': 123, 'desc': "foobar", 'status': "foobar",
'client': ("client1","client2"),},{......]
for item in changes:
entry = Change(
change = item['change'],
desc = item['desc'],
status = item['status'],)
entry.save()
for c in item['client']:
entry.client.add(Client.objects.get(client=c))