A lot of websites will display:
"1.8K pages" instead of "1,830 pages"
or
"43.2M pages" instead of "43,200,123 pages"
Is there a way to do this in Django?
For example, the following code will generate the quantified amount of objects in the queryset (i.e. 3,123):
Books.objects.all().count()
Is there a way to add a custom count filter to return "3.1K pages" instead of "3,123 pages?
Thank you in advance!
First off, I wouldn't do anything that alters the way the ORM portion of Django works. There are two places this could be done, if you are only planning on using it in one place - do it on the frontend. With that said, there are many ways to achieve this result. Just to spout off a few ideas, you could write a property on your model that calls count then converts that to something a little more human readable for the back end. If you want to do it on the frontend you might want to find a JavaScript lib that could do the conversion.
I will edit this later from my computer and add an example of the property.
Edit: To answer your comment, the easier one to implement depends on your skills in python vs in JavaScript. I prefer python so I would probably do it in there somewhere on the model.
Edit2: I have wrote an example to show you how I would do a classmethod on a base model or on the model that you need these numbers on. I found a python package called humanize and I took its function that converts these to readable and modified it a bit to allow for thousands and took out some of the super large number conversion.
def readable_number(value, short=False):
# Modified from the package `humanize` on pypy.
powers = [10 ** x for x in (3, 6, 9, 12, 15, 18)]
human_powers = ('thousand', 'million', 'billion', 'trillion', 'quadrillion')
human_powers_short = ('K', 'M', 'B', 'T', 'QD')
try:
value = int(value)
except (TypeError, ValueError):
return value
if value < powers[0]:
return str(value)
for ordinal, power in enumerate(powers[1:], 1):
if value < power:
chopped = value / float(powers[ordinal - 1])
chopped = format(chopped, '.1f')
if not short:
return '{} {}'.format(chopped, human_powers[ordinal - 1])
return '{}{}'.format(chopped, human_powers_short[ordinal - 1])
class MyModel(models.Model):
#classmethod
def readable_count(cls, short=True):
count = cls.objects.all().count()
return readable_number(count, short=short)
print(readable_number(62220, True)) # Returns '62.2K'
print(readable_number(6555500)) # Returns '6.6 million'
I would stick that readable_number in some sort of utils and just import it in your models file. Once you have that, you can just stick that string wherever you would like on your frontend.
You would use MyModel.readable_count() to get that value. If you want it under MyModel.objects.readable_count() you will need to make a custom object manager for your model, but that is a bit more advanced.
Related
I want to concatenate two queryset obtained from two different models and i can do it using itertools like this:
ci = ContributorImage.objects.all()
pf = Portfolio.objects.all()
cpf = itertools.chain(ci,pf)
But the real fix is paginating results.If i pass a iterator(cpf, or our concatenated queryset) to Paginator function, p = Paginator(cpf, 10), it works as well but fails at retrieving first page page1 = p.page(1) with an error which says:
TypeError: object of type 'itertools.chain' has no len()
What can i do in case like this ?
The itertools.chain() will return a generator. The Paginator class needs an object implementing __len__ (generators, of course do not support it since the size of the collection is not known).
Your problem could be resolved in a number of ways (including using list to evaluate the generator as you mention) however I recommending taking a look at the QuerySetChain mentioned in this answer:
https://stackoverflow.com/a/432666/119071
I think it fits exactly to your problem. Also take a look at the comments of that answer - they are really enlightening :)
I know it's too late, but because I encountered this error, I would answer to this question.
you should return a list of objects:
ci = ContributorImage.objects.all()
pf = Portfolio.objects.all()
cpf = itertools.chain(ci,pf)
cpf_list = list(cpf)
I want a ChoiceField with these choices:
choices = [(1, '1 thing'),
(2, '2 things'),
(3, '3 things'),
...]
and I want to have it translated.
This does not work:
choices = [(i, ungettext_lazy('%s thing', '%s things', i) % i) for i in range(1,4)]
because as soon as the lazy object is interpolated, it becomes a unicode object - since ChoiceField.choices is evaluated at startup, its choices will be in the language active during Django's startup.
I could use ugettext_lazy('%s things' % i), but that would require a translation for each numeral, which is silly. What is the right way to do this?
In the Django documentation, Translation… Working with lazy translation objects, I see a remark which seems to address your concern here.
Using ugettext_lazy() and ungettext_lazy() to mark strings in models and utility functions is a common operation. When you're working with these objects elsewhere in your code, you should ensure that you don't accidentally convert them to strings, because they should be converted as late as possible (so that the correct locale is in effect). This necessitates the use of the helper function described next.
Then they present django.utils.functional.lazy(func, *resultclasses), which is not presently covered by the django.utils.functional module documentation. However, according to the django.utils.functional.py source code, it "Turns any callable into a lazy evaluated callable.… the
function is evaluated on every access."
Modifying their example from Other uses of lazy in delayed translations to incorporate your code, the following code might work for you.
from django.utils import six # Python 3 compatibility
from django.utils.functional import lazy
from django.utils.safestring import mark_safe
choices = [
(i, lazy(
mark_safe(ungettext_lazy('%s thing', '%s things', i) % i),
six.text_type
)# lazy()
for i in range(1,4)
]
Also, the django.utils.functional module documentation does mention a function decorator allow_lazy(func, *resultclasses). This lets you write your own function which takes a lazy string as arguments. "It modifies the function so that if it's called with a lazy translation as the first argument, the function evaluation is delayed until it needs to be converted to a string." lazy(func, *resultclasses) is not a decorator, it modifies a callable.
N.B. I haven't tried this code in Django. I'm just passing along what I found in the documentation. Hopefully it will point you to something you can use.
For those who encounter this question. Unfortunately, #Jim DeLaHunt's answer doesn't completely work - it's almost there, but not exactly what needs to be done.
The important distinctions are:
What you need to warp with lazy is a function that'd return you a text value, not another lazy translation object, or you'll likely see weird <django.utils.functional.__proxy__ at ...> instead of the actual text (IIRC Django won't go deep down the chain of lazy objects). So, use ungettext, not ungettext_lazy.
You want to do string interpolation only when the wrapped function runs. If you write lazy(f("%d" % 42)) the interpolation would happen too early - in this case Python evaluates eagerly. And don't forget about variable scopes - you can't just refer to the iterator from the wrapped function.
Here, I've used a lambda that receives a number argument and does the interpolation. The code inside lambda is only executed when lazy object is evaluated, that is, when the choice is rendered.
So, the working code is:
choices = [
(
(i, lazy(
lambda cnt: ungettext(u"%(count)d thing",
u"%(count)d things", cnt)
% {"count": cnt},
six.text_type
)(i))
)
for i in [1, 2, 3]
]
This will correctly have the same intended effect as
choices = [
(1, _("1 thing")),
(2, _("2 things")),
(3, _("3 things")),
]
But there will be just a single entry for this in translation database, not multiple ones.
This looks like a situation where you could benefit from the trick taught by Ilian Iliev's blog, Django forms ChoiceField with dynamic values….
Iliev shows a very similar initialiser:
my_choice_field = forms.ChoiceField(choices=get_my_choices())
He says, "the trick is that in this case my_choice_field choices are initialized on server (re)start. Or in other words once you run the server the choices are loaded(calculated) and they will not change until next (re)start." Sounds like the same difficulty you are encountering.
His trick is: "fortunately the form`s class has an init method that is called on every form load. Most of the times you skipped it in the form definition but now you will have to use it."
Here is his sample code, blended with your generator expression:
class MyForm(forms.Form):
def __init__(self, *args, **kwargs):
super(MyForm, self).__init__(*args, **kwargs)
self.fields['my_choice_field'] = forms.ChoiceField(
choices=(
(i, ungettext_lazy('%s thing', '%s things', i) % i)
for i in range(1,4)
)# choices=
)# __init__
The generator expression is enclosed in parentheses so that it is treated as a generator object, which is assigned to choices.
N.B. I haven't tried this code in Django. I'm just passing along Iliev's idea.
lcount = Open_Layers.objects.all()
form = SearchForm()
if request.method == 'POST':
form = SearchForm(request.POST)
if form.is_valid():
data = form.cleaned_data
val=form.cleaned_data['LayerName']
a=Open_Layers()
data = []
for e in lcount:
if e.Layer_name == val:
data = val
return render_to_response('searchresult.html', {'data':data})
else:
form = SearchForm()
else:
return render_to_response('mapsearch.html', {'form':form})
This just returns back if a particular "name" matches . How do to change it so that it returns when I give a search for "Park" , it should return Park1 , Park2 , Parking , Parkin i.e all the occurences of the park .
You can improve your searching logic by using a list to accumulate the results and the re module to match a larger set of words.
However, this is still pretty limited, error prone and hard to maintain or even harder to make evolve. Plus you'll never get as nice results as if you were using a search engine.
So instead of trying to manually reinvent the wheel, the car and the highway, you should spend some time setting up haystack. This is now the de facto standard to do search in Django.
Use woosh as a backend at first, it's going to be easier. If your search get slow, replace it with solr.
EDIT:
Simple clean alternative:
Open_Layers.objects.filter(name__icontains=val)
This will perform a SQL LIKE, adding %` for you.
This going to kill your database if used too often, but I guess this is probably not going to be an issue with your current project.
BTW, you probably want to rename Open_Layers to OpenLayers as this is the Python PEP8 naming convention.
Instead of
if e.Layer_name == val:
data = val
use
if val in e.Layer_name:
data.append(e.Layer_name)
(and you don't need the line data = form.cleaned_data)
I realise this is an old post, but anyway:
There's a fuzzy logic string comparison already in the python standard library.
import difflib
Mainly have a look at:
difflib.SequenceMatcher(None, a='string1', b='string2', autojunk=True).ratio()
more info here:
http://docs.python.org/library/difflib.html#sequencematcher-objects
What it does it returns a ratio of how close the two strings are, between zero and 1. So instead of testing if they're equal, you chose your similarity ratio.
Things to watch out for, you may want to convert both strings to lower case.
string1.lower()
Also note you may want to impliment your favourite method of splitting the string i.e. .split() or something using re so that a search for 'David' against 'David Brent' ranks higher.
I'm in a situation where I must output a quite large list of objects by a CharField used to store street addresses.
My problem is, that obviously the data is ordered by ASCII codes since it's a Charfield, with the predictable results .. it sort the numbers like this;
1, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 2, 20, 21....
Now the obvious step would be to change the Charfield the proper field type (IntegerField let's say), however it cannot work since some address might have apartments .. like "128A".
I really don't know how I can order this properly ..
If you're sure there are only integers in the field, you could get the database to cast it as an integer via the extra method, and order by that:
MyModel.objects.extra(
select={'myinteger': 'CAST(mycharfield AS INTEGER)'}
).order_by('myinteger')
Django is trying to deprecate the extra() method, but has introduced Cast() in v1.10. In sqlite (at least), CAST can take a value such as 10a and will cast it to the integer 10, so you can do:
from django.db.models import IntegerField
from django.db.models.functions import Cast
MyModel.objects.annotate(
my_integer_field=Cast('my_char_field', IntegerField())
).order_by('my_integer_field', 'my_char_field')
which will return objects sorted by the street number first numerically, then alphabetically, e.g. ...14, 15a, 15b, 16, 16a, 17...
If you're using PostgreSQL (not sure about MySQL) you can safely use following code on char/text fields and avoid cast errors:
MyModel.objects.extra(
select={'myinteger': "CAST(substring(charfield FROM '^[0-9]+') AS INTEGER)"}
).order_by('myinteger')
Great tip! It works for me! :) That's my code:
revisioned_objects = revisioned_objects.extra(select={'casted_object_id': 'CAST(object_id AS INTEGER)'}).extra(order_by = ['casted_object_id'])
I know that I’m late on this, but since it’s strongly related to the question, and that I had a hard time finding this:
You have to know that you can directly put the Cast in the ordering option of your model.
from django.db import models
from django.db.models.functions import Cast
class Address(models.Model):
street_number = models.CharField()
class Meta:
ordering = [
Cast("street_number", output_field=models.IntegerField()),
]
From the doc about ordering:
You can also use query expressions.
And from the doc about database functions:
Functions are also expressions, so they can be used and combined with other expressions like aggregate functions.
The problem you're up against is quite similar to how filenames get ordered when sorting by filename. There, you want "2 Foo.mp3" to appear before "12 Foo.mp3".
A common approach is to "normalize" numbers to expanding to a fixed number of digits, and then sorting based on the normalized form. That is, for purposes of sorting, "2 Foo.mp3" might expand to "0000000002 Foo.mp3".
Django won't help you here directly. You can either add a field to store the "normalized" address, and have the database order_by that, or you can do a custom sort in your view (or in a helper that your view uses) on address records before handing the list of records to a template.
In my case i have a CharField with a name field, which has mixed (int+string) values, for example. "a1", "f65", "P", "55" e.t.c ..
Solved the issue by using the sql cast (tested with postgres & mysql),
first, I try to sort by the casted integer value, and then by the original value of the name field.
parking_slots = ParkingSlot.objects.all().extra(
select={'num_from_name': 'CAST(name AS INTEGER)'}
).order_by('num_from_name', 'name')
This way, in any case, the correct sorting works for me.
In case you need to sort version numbers consisting of multiple numbers separated by a dot (e.g. 1.9.0, 1.10.0), here is a postgres-only solution:
class VersionRecordManager(models.Manager):
def get_queryset(self):
return super().get_queryset().extra(
select={
'natural_version': "string_to_array(version, '.')::int[]",
},
)
def available_versions(self):
return self.filter(available=True).order_by('-natural_version')
def last_stable(self):
return self.available_versions().filter(stable=True).first()
class VersionRecord(models.Model):
objects = VersionRecordManager()
version = models.CharField(max_length=64, db_index=True)
available = models.BooleanField(default=False, db_index=True)
stable = models.BooleanField(default=False, db_index=True)
In case you want to allow non-numeric characters (e.g. 0.9.0 beta, 2.0.0 stable):
def get_queryset(self):
return super().get_queryset().extra(
select={
'natural_version':
"string_to_array( "
" regexp_replace( " # Remove everything except digits
" version, '[^\d\.]+', '', 'g' " # and dots, then split string into
" ), '.' " # an array of integers.
")::int[] "
}
)
I was looking for a way to sort the numeric chars in a CharField and my search led me here. The name fields in my objects are CC Licenses, e.g., 'CC BY-NC 4.0'.
Since extra() is going to be deprecated, I was able to do it this way:
MyObject.objects.all()
.annotate(sorting_int=Cast(Func(F('name'), Value('\D'), Value(''), Value('g'), function='regexp_replace'), IntegerField()))
.order_by('-sorting_int')
Thus, MyObject with name='CC BY-NC 4.0' now has sorting_int=40.
All the answeres in this thread did not work for me because they are assuming numerical text. I found a solution that will work for a subset of cases. Consider this model
Class Block(models.Model):
title = models.CharField()
say I have fields that sometimes have leading characters and trailing numerical characters If i try and order normally
>>> Block.objects.all().order_by('title')
<QuerySet [<Block: 1>, <Block: 10>, <Block: 15>, <Block: 2>, <Block: N1>, <Block: N12>, <Block: N4>]>
As expected, it's correct alphabetically, but makes no sense for us humans. The trick that I did for this particular use case is to replace any text i find with the number 9999 and then cast the value to an integer and order by it.
for most cases that have leading characters this will get the desired result. see below
from django.db.models.expressions import RawSQL
>>> Block.objects.all()\
.annotate(my_faux_integer=RawSQL("CAST(regexp_replace(title, '[A-Z]+', '9999', 'g') AS INTEGER)", ''))\
.order_by('my_faux_integer', 'title')
<QuerySet [<Block: 1>, <Block: 2>, <Block: 10>, <Block: 15>, <Block: N1>, <Block: N4>, <Block: N12>]>
Nearly every kind of lookup in Django has a case-insensitive version, EXCEPT in, it appears.
This is a problem because sometimes I need to do a lookup where I am certain the case will be incorrect.
Products.objects.filter(code__in=[user_entered_data_as_list])
Is there anything I can do to deal with this? Have people come up with a hack to work around this issue?
I worked around this by making the MySQL database itself case-insensitive. I doubt that the people at Django are interested in adding this as a feature or in providing docs on how to provide your own field lookup (assuming that is even possible without providing code for each db backend)
Here is one way to do it, admittedly it is clunky.
products = Product.objects.filter(**normal_filters_here)
results = Product.objects.none()
for d in user_entered_data_as_list:
results |= products.filter(code__iexact=d)
If your database is MySQL, Django treats IN queries case insensitively. Though I am not sure about others
Edit 1:
model_name.objects.filter(location__city__name__in': ['Tokio','Paris',])
will give following result in which city name is
Tokio or TOKIO or tokio or Paris or PARIS or paris
If it won't create conflicts, a possible workaround may be transforming the strings to upper or lowercase both when the object is saved and in the filter.
Here is a solution that do not require case-prepared DB values.
Also it makes a filtering on DB-engine side, meaning much more performance than iterating over objects.all().
def case_insensitive_in_filter(fieldname, iterable):
"""returns Q(fieldname__in=iterable) but case insensitive"""
q_list = map(lambda n: Q(**{fieldname+'__iexact': n}), iterable)
return reduce(lambda a, b: a | b, q_list)
The other efficient solution is to use extra with quite portable raw-SQL lower() function:
MyModel.objects.extra(
select={'lower_' + fieldname: 'lower(' + fieldname + ')'}
).filter('lover_' + fieldname + '__in'=[x.lower() for x in iterable])
Another solution - albeit crude - is to include the different cases of the original strings in the list argument to the 'in' filter. For example: instead of ['a', 'b', 'c'], use ['a', 'b', 'c', 'A', 'B', 'C'] instead.
Here's a function that builds such a list from a list of strings:
def build_list_for_case_insensitive_query(the_strings):
results = list()
for the_string in the_strings:
results.append(the_string)
if the_string.upper() not in results:
results.append(the_string.upper())
if the_string.lower() not in results:
results.append(the_string.lower())
return results
A lookup using Q object can be built to hit the database only once:
from django.db.models import Q
user_inputed_codes = ['eN', 'De', 'FR']
lookup = Q()
for code in user_inputed_codes:
lookup |= Q(code__iexact=code)
filtered_products = Products.objects.filter(lookup)
A litle more elegant way would be this:
[x for x in Products.objects.all() if x.code.upper() in [y.upper() for y in user_entered_data_as_list]]
You can do it annotating the lowered code and also lowering the entered data
from django.db.models.functions import Lower
Products.objects.annotate(lower_code=Lower('code')).filter(lower_code__in=[user_entered_data_as_list_lowered])