I want to show 20 reviews per page on my website.
Unfortunately whole reviews count is about 260,000, It takes 3~4sec to show at page 10,000~
I used to use queryset like this:
reviews = Review.objects.all()[(page_num-1)*20:20*page_num]
.
.
.
obj['reviews'] = reviews
Now I think using iterator(chunk_size=) is better than upper one for efficiency.
But I dont know how to use iterator() properly.
I definitely need your helps
Please let me know...
Django pagination:
https://docs.djangoproject.com/en/3.1/topics/pagination/
p = Paginator(objects, 20)
>>> p.count
20*N
>>> p.num_pages
N
page1 = p.page(1)
page2 = p.page(2)
...
pageN = p.page(n)
Related
I am learning Django and still a beginner. For practising, i am trying to make a demo social media website. In my project, users can create groups, then they can post and comment there. In the home page, i am trying to add a section like 'recent activities' where a user can see recent activities in that website like "John created a group 'Javascript', Tim posted a comment in 'Python', Sarah posted in 'CSS'" Now i have made some queries like:
groups = Group.objects.all().order_by('-created')[0:5]
posts = Post.objects.all().order_by('-created')[0:5]
comments = Comment.objects.all().order_by('-created')[0:5]
I want to mix them all in a single queryset. Then order them all by the time they were created. I know it's a silly question and i have been stuck here since morning. Can you help me and show me the process please?
You can chain these together and order by the created field with:
from operator import attrgetter
groups = Group.objects.order_by('-created')[:5]
posts = Post.objects.order_by('-created')[:5]
comments = Comment.objects.order_by('-created')[:5]
all_items = sorted(
[*groups, *posts, *comments],
key=attrgetter('created'),
reversed=True
)
Now all_items is a hetrogenous list with different types of objects. This will thus make the rendering process a bit more complicated since a comment probably has different fields than a Post for example.
You can also use chain function from itertools module to combine the querysets and then sort them in reverse order using the created field as key.
from itertools import chain
groups = Group.objects.all()[0:5]
posts = Post.objects.all()[0:5]
comments = Comment.objects.all()[0:5]
queryset = sorted(
chain(groups, posts, comments),
key=lambda instance: instance.created,
reverse=True
)
I have two models, ParentPage and ChildPage. I want to find the set of ParentPages where a field is_completed is True on the ChildPage.
Normally in Django, I could do something like
ParentPage.objects.filter(child_page__is_completed=True)
However, I don't think there is a join here for the Wagtail/Treebeard hierarchy.
I also thought you might be able to filter ChildPages by multiple ParentPages, e.g. ChildPage.objects.children_of([parent_ids]), but I can't see a way to do that either.
Is there a simpler way?
The Page table has path and url_path columns. If you find all children and strip the last part of the path or url_path, you can use that result to query the parent pages.
Path:
child_paths = ChildPage.objects.filter(is_completed=True).values_list("path", flat=True)
parent_paths = set([cp[:-4] for cp in child_paths])
pages = Page.objects.filter(path__in=parent_paths)
Url path:
child_urls = ChildPage.objects.filter(is_completed=True).values_list("url_path", flat=True)
parent_urls = set([url.rsplit('/', 1)[0] for url in child_urls])
pages = Page.objects.filter(url_path__in=parent_urls)
Disclaimer: untested code.
What I'm trying to do
I'm trying to enter a list of tags in flask that should become passable as a list but I can't figure out how to do it in flask, nor can I find documentation to add lists (of strings) in flask_wtf. Has anyone have experience with this?
Ideally I would like the tags to be selectively delete-able, after you entered them. So that you could enter.
The problem
Thus far my form is static. You enter stuff, hit submit, it gets processed into a .json. The tags list is the last element I can't figure out. I don't even know if flask can do this.
A little demo of how I envisioned the entry process:
How I envisioned the entry process:
The current tags are displayed and an entry field to add new ones.
[Tag1](x) | [Tag2](x)
Enter new Tag: [______] (add)
Hit (add)
[Tag1](x) | [Tag2](x)
Enter new Tag: [Tag3__] (add)
New Tag is added
[Tag1](x) | [Tag2](x) | [Tag3](x)
Enter new Tag: [______]
How I envisioned the deletion process:
Hitting the (x) on the side of the tag should kill it.
[Tag1](x) | [Tag2](x) | [Tag3](x)
Hit (x) on Tag2. Result:
[Tag1](x) | [Tag3](x)
The deletion is kind of icing on the cake and could probably be done, once I have a list I can edit, but getting there seems quite hard.
I'm at a loss here.
I basically want to know if it's possible to enter lists in general, since there does not seem to be documentation on the topic.
Your description is not really clear (is Tag1 the key in the JSON or is it Tag the key, and 1 the index?)
But I had a similar issue recently, where I wanted to submit a basic list in JSON and let WTForms handle it properly.
For instance, this:
{
"name": "John",
"tags": ["code", "python", "flask", "wtforms"]
}
So, I had to rewrite the way FieldList works because WTForms, for some reason, wants a list as "tags-1=XXX,tags-2=xxx".
from wtforms import FieldList
class JSONFieldList(FieldList):
def process(self, formdata, data=None):
self.entries = []
if data is None or not data:
try:
data = self.default()
except TypeError:
data = self.default
self.object_data = data
if formdata:
for (index, obj_data) in enumerate(formdata.getlist(self.name)):
self._add_entry(formdata, obj_data, index=index)
else:
for obj_data in data:
self._add_entry(formdata, obj_data)
while len(self.entries) < self.min_entries:
self._add_entry(formdata)
def _add_entry(self, formdata=None, data=None, index=None):
assert not self.max_entries or len(self.entries) < self.max_entries, \
'You cannot have more than max_entries entries in this FieldList'
if index is None:
index = self.last_index + 1
self.last_index = index
name = '%s-%d' % (self.short_name, index)
id = '%s-%d' % (self.id, index)
field = self.unbound_field.bind(form=None, name=name, id=id, prefix=self._prefix, _meta=self.meta,
translations=self._translations)
field.process(formdata, data)
self.entries.append(field)
return field
On Flask's end to handle the form:
from flask import request
from werkzeug.datastructures import ImmutableMultiDict
#app.route('/add', methods=['POST'])
def add():
form = MyForm(ImmutableMultiDict(request.get_json())
# process the form, form.tags.data is a list
And the form (notice the use of JSONFieldList):
class MonitorForm(BaseForm):
name = StringField(validators=[validators.DataRequired(), validators.Length(min=3, max=5)], filters=[lambda x: x or None])
tags = JSONFieldList(StringField(validators=[validators.DataRequired(), validators.Length(min=1, max=250)], filters=[lambda x: x or None]), validators=[Optional()])
I found a viable solution in this 2015 book, where a tagging system is being build for flask as part of a blog building exercise.
It's based on Flask_SQLAlchemy.
Entering lists therefore is possible with WTForms / Flask by submitting the items to the database via, e.g. FieldList and in the usecase of a tagging system, reading them from the database back to render them in the UI.
If however you don't want to deal with O'Rielly's paywall (I'm sorry, I can't post copyrighted material here) and all you want is a solution to add tags, check out taggle.js by Sean Coker. It's not flask, but javascript, but it does the job.
Using django 1.7 and python 2.7, in a views I have:
page = 0
sex = [u'\u0632\u0646'] #sex = زن
url = "/result/%s/%d" % (sex, page)
return HttpResponseRedirect(url)
Which needs to return:
/result/زن/0
However the resulting url turns out to be:
/result/[u'\u0632\u0646']/0
Which is not what envisage in the pattern:
`url(r'^result/(?P<sex>\w+)/(?P<page>\d+)','userprofile.views.profile_search_result')`,
I also tried
return HttpResponseRedirect( iri_to_uri(url))
but does not solve the problem.
I got really confused and appreciate your help to fix this.
Since sex is a list, you simply need to use the actual element you want:
url = "/result/%s/%d" % (sex[0], page)
Although note that to construct URLs in Django, you should really use the reverse function:
from django.core.urlresolvers import reverse
...
url = reverse('userprofile.views.profile_search_result', kwargs={'sex': sex[0], 'page': page})
url should also be an unicode string for that to work:
page = 0
sex = u'\u0632\u0646' #sex=زن
url = u"/result/%s/%d" % (sex, page)
return HttpResponseRedirect(url)
I'm trying to figure out how to show something like "Showing 1-10 of 52" using django pagination in my templates.
I accomplished the pagination itself but nothing comes to my mind about this requirement.
Any ideas?
As the documentation shows, Paginator objects have a count attribute which shows the total number of objects across all pages (in the queryset or list they're paginating) and Page objects have start_index() and end_index() methods.
So, assuming you pass the page to the template context:
Showing {{ page.start_index }}-{{ page.end_index }} of {{ page.paginator.count }}
You'll need to do something moderately more complex behind the scenes. And please note that while I am a python dev, i've been using werkzeug and jinja2 for a long time now and so my django syntax is a little rusty. Also this was dry-coded (as in I just typed it here in the browser) and should be tested to make sure that it works as intended.
Generally I'll create a pagination object, and pass it in a query object that isn't filtered by pages, you can also tell it how many per page and what page you're on.
So something vaguely similar to:
Paginator(query, objects_per_page, current_page_number)
And then pass the resulting paginator object into the template.
Inside the paginator's init you'd want to do something similar to:
def __init__(self, query, objects_per_page, current_page_number):
self.total = query.count()
self.per_page = objects_per_page
self.current_page_number = current_page_number
self.lower_limit = objects_per_page * current_page_number
self.upper_limit = objects_per_page * (current_page_number + 1)
if self.upper_limit > self.total:
self.upper_limit = self.total
self.objects = query[self.lower_limit - 1:self.upper_limit - 1]
Then in the template you'd do something like
Showing {{paginator.lower_limit}}-{{paginator.upper_limit}} of {{paginator.total}}
And later when you are ready to iterate over the objects you could just iterate over paginator.objects.
I hope that gives you a general idea of how you can cleanly do this.
The django Paginator object has a num_pages property. In your view, you can just pass the correct data to your template and show it there.
So as an (rough) example:
view
current_page = ## input param
items_per_page = 10
paginator = Paginator(result, items_per_page)
...
return render_to_response('template.html',
{
'num_pages': paginator.num_pages,
'start_count': items_per_page * current_page + 1,
'end_count': (items_per_page * current_page + 1) + len(paginator(current_page).object_list)
'items_per_page': items_per_page
}
template
showing {{start_count} - {{end_count}} of {{num_pages}}
(I wrote this code without the benefit of a compiler, so test it)
paginator = Paginator(query, settings.SEARCH_DATA_PER_PAGE)
For number of Pages :
num_of_pages = paginator.num_pages
For total number of items across all pages :
num_of_objects = paginator.count
You can calculate number of items per page from this data.
totalPages = math.ceil(float(totalRecords)/recordsPerPage)
or
paginatorInstance.num_pages # this is there by default
Wasn't that hard, was it?