I want to concatenate two queryset obtained from two different models and i can do it using itertools like this:
ci = ContributorImage.objects.all()
pf = Portfolio.objects.all()
cpf = itertools.chain(ci,pf)
But the real fix is paginating results.If i pass a iterator(cpf, or our concatenated queryset) to Paginator function, p = Paginator(cpf, 10), it works as well but fails at retrieving first page page1 = p.page(1) with an error which says:
TypeError: object of type 'itertools.chain' has no len()
What can i do in case like this ?
The itertools.chain() will return a generator. The Paginator class needs an object implementing __len__ (generators, of course do not support it since the size of the collection is not known).
Your problem could be resolved in a number of ways (including using list to evaluate the generator as you mention) however I recommending taking a look at the QuerySetChain mentioned in this answer:
https://stackoverflow.com/a/432666/119071
I think it fits exactly to your problem. Also take a look at the comments of that answer - they are really enlightening :)
I know it's too late, but because I encountered this error, I would answer to this question.
you should return a list of objects:
ci = ContributorImage.objects.all()
pf = Portfolio.objects.all()
cpf = itertools.chain(ci,pf)
cpf_list = list(cpf)
Related
Is there a way in Django to achieve the following in one DB hit (Debug Toolbar shows 2 queries)?
q = SomeModel.objects.filter(name=name).order_by(some_field)
if q.count() == 0:
q = SomeModel.objects.all().order_by(some_field)
I want to check if there are objects with a given name. If yes, then return them. If not, return all objects. All done in one query.
I've checked Subquery, Q, conditional expressions but still don't see how to fit it into one query.
Ok, much as I resisted (I still think it's premature optimization), curiosity got the better of me. This is not pretty but does the trick:
from django.db.models import Q, Exists
name_qset = SomeObject.objects.filter(name=name)
q_func = Q(name_exists=True, name=name) | Q(name_exists=False)
q = SomeModel.objects.annotate(
name_exists=Exists(name_qset)
).filter(q_func).order_by(some_field)
Tried it out and definitely only one query. Interesting to see if it is actually appreciably faster for large datasets...
You best bet is to use .exists(), otherwise your code is fine
q = SomeModel.objects.filter(name=name).order_by(some_field)
if not q.exists():
q = SomeModel.objects.all().order_by(some_field)
I'm new to coding and have searched as best I can to find out how to solve this before asking.
I'm trying to pull information from poloniex.com REST api, which is in JSon format I believe. I can import the data, and work with it a little bit, but when I try to call and use the elements in the contained dictionaries, I get "'unicode' object not callable". How can I use this information? The end goal with this data is to pull the "BTC: "(volume)" for each coin pair and test if it is <100, and if not, append it to a new list.
The data is presented like this or you can see yourself at https://poloniex.com/public?command=return24hVolume:
{"BTC_LTC":{"BTC":"2.23248854","LTC":"87.10381314"},"BTC_NXT":{"BTC":"0.981616","NXT":"14145"}, ... "totalBTC":"81.89657704","totalLTC":"78.52083806"}
And my code I've been trying to get to work with currently looks like this(I've tried to iterate the information I want a million different ways, so I dunno what example to give for that part, but this is how I am importing the data):
returnvolume = urllib2.urlopen(urllib2.Request('https://poloniex.com/public?command=return24hVolume'))
coinvolume = json.loads(returnvolume.read())
coinvolume = dict(coinvolume)
No matter how I try to use the data I've pulled, I get an error stating:
"unicode' object not callable."
I'd really appreciate a little help, I'm concerned I may be approaching this the wrong way as I haven't been able to get anything to work, or maybe I'm just missing something rudimentary, I'm not sure.
Thank you very much for your time!
Thanks to another user, downshift, I have discovered the answer!
d = {}
for k, v in coinvolume.items():
try:
if float(v['BTC']) > 100:
d[k] = v
except KeyError:
d[k] = v
except TypeError:
if v > 100:
d[k] = k
This creates a new list, d, and adds every coin with a 'BTC' volume > 100 to this new list.
Thanks again downshift, and I hope this helps others as well!
A lot of websites will display:
"1.8K pages" instead of "1,830 pages"
or
"43.2M pages" instead of "43,200,123 pages"
Is there a way to do this in Django?
For example, the following code will generate the quantified amount of objects in the queryset (i.e. 3,123):
Books.objects.all().count()
Is there a way to add a custom count filter to return "3.1K pages" instead of "3,123 pages?
Thank you in advance!
First off, I wouldn't do anything that alters the way the ORM portion of Django works. There are two places this could be done, if you are only planning on using it in one place - do it on the frontend. With that said, there are many ways to achieve this result. Just to spout off a few ideas, you could write a property on your model that calls count then converts that to something a little more human readable for the back end. If you want to do it on the frontend you might want to find a JavaScript lib that could do the conversion.
I will edit this later from my computer and add an example of the property.
Edit: To answer your comment, the easier one to implement depends on your skills in python vs in JavaScript. I prefer python so I would probably do it in there somewhere on the model.
Edit2: I have wrote an example to show you how I would do a classmethod on a base model or on the model that you need these numbers on. I found a python package called humanize and I took its function that converts these to readable and modified it a bit to allow for thousands and took out some of the super large number conversion.
def readable_number(value, short=False):
# Modified from the package `humanize` on pypy.
powers = [10 ** x for x in (3, 6, 9, 12, 15, 18)]
human_powers = ('thousand', 'million', 'billion', 'trillion', 'quadrillion')
human_powers_short = ('K', 'M', 'B', 'T', 'QD')
try:
value = int(value)
except (TypeError, ValueError):
return value
if value < powers[0]:
return str(value)
for ordinal, power in enumerate(powers[1:], 1):
if value < power:
chopped = value / float(powers[ordinal - 1])
chopped = format(chopped, '.1f')
if not short:
return '{} {}'.format(chopped, human_powers[ordinal - 1])
return '{}{}'.format(chopped, human_powers_short[ordinal - 1])
class MyModel(models.Model):
#classmethod
def readable_count(cls, short=True):
count = cls.objects.all().count()
return readable_number(count, short=short)
print(readable_number(62220, True)) # Returns '62.2K'
print(readable_number(6555500)) # Returns '6.6 million'
I would stick that readable_number in some sort of utils and just import it in your models file. Once you have that, you can just stick that string wherever you would like on your frontend.
You would use MyModel.readable_count() to get that value. If you want it under MyModel.objects.readable_count() you will need to make a custom object manager for your model, but that is a bit more advanced.
I've seen answers to both halves of my question, but I can't work out how to marry the two.
I have a book model, and a translatedBook model.
The translatedBook has a langage set up as model choices in the usual way:
LANGUAGES = (
(u'it', u'Italian'),
(u'ja', u'Japanese'),
(u'es', u'Spanish'),
(u'zh-cn', u'Simplified Chinese'),
(u'zh-tw', u'Traditional Chinese'),
(u'fr', u'French'),
(u'el', u'Greek'),
(u'ar', u'Arabic'),
(u'bg', u'Bulgarian'),
(u'bn', u'Bengali'),
etc
I know that to get "Italian" I have to do translatedBook.get_language_display on a Book object.
But how do I get a list of distinct languages in their long format?
I've tried:
lang_avail = TargetText.objects.values('language').distinct().order_by('language')
lang_avail = TargetText.objects.distinct().order_by('language').values('language').
lang_avail = TargetText.objects.all().distinct('language').order_by('language')
but I can't seem to get what I want - which is a list like:
"English, Italian, Simplified Chinese, Spanish"
The final lang_avail listed above didn't return the list of 5, it returned the list of 355 (ie, # of books) with multiple repeats....
-- EDIT --
Daniel's answer almost got me there - as it turns out, that throws a "dicts are unhashable" error. Thanks to Axiak on the django irc, we use Daniel's solution with this line instead:
langs = TargetText.objects.values_list('language', flat=True).distinct().order_by('language')
and it works.
There isn't a built-in way. You could do something like this:
lang_dict = dict(LANGUAGES)
langs = TargetText.objects.values('language').distinct().order_by('language')
long_langs = [lang_dict[lang] for lang in langs]
which simply makes a dictionary out of the LANGUAGE choices and then looks up each language ID.
Nearly every kind of lookup in Django has a case-insensitive version, EXCEPT in, it appears.
This is a problem because sometimes I need to do a lookup where I am certain the case will be incorrect.
Products.objects.filter(code__in=[user_entered_data_as_list])
Is there anything I can do to deal with this? Have people come up with a hack to work around this issue?
I worked around this by making the MySQL database itself case-insensitive. I doubt that the people at Django are interested in adding this as a feature or in providing docs on how to provide your own field lookup (assuming that is even possible without providing code for each db backend)
Here is one way to do it, admittedly it is clunky.
products = Product.objects.filter(**normal_filters_here)
results = Product.objects.none()
for d in user_entered_data_as_list:
results |= products.filter(code__iexact=d)
If your database is MySQL, Django treats IN queries case insensitively. Though I am not sure about others
Edit 1:
model_name.objects.filter(location__city__name__in': ['Tokio','Paris',])
will give following result in which city name is
Tokio or TOKIO or tokio or Paris or PARIS or paris
If it won't create conflicts, a possible workaround may be transforming the strings to upper or lowercase both when the object is saved and in the filter.
Here is a solution that do not require case-prepared DB values.
Also it makes a filtering on DB-engine side, meaning much more performance than iterating over objects.all().
def case_insensitive_in_filter(fieldname, iterable):
"""returns Q(fieldname__in=iterable) but case insensitive"""
q_list = map(lambda n: Q(**{fieldname+'__iexact': n}), iterable)
return reduce(lambda a, b: a | b, q_list)
The other efficient solution is to use extra with quite portable raw-SQL lower() function:
MyModel.objects.extra(
select={'lower_' + fieldname: 'lower(' + fieldname + ')'}
).filter('lover_' + fieldname + '__in'=[x.lower() for x in iterable])
Another solution - albeit crude - is to include the different cases of the original strings in the list argument to the 'in' filter. For example: instead of ['a', 'b', 'c'], use ['a', 'b', 'c', 'A', 'B', 'C'] instead.
Here's a function that builds such a list from a list of strings:
def build_list_for_case_insensitive_query(the_strings):
results = list()
for the_string in the_strings:
results.append(the_string)
if the_string.upper() not in results:
results.append(the_string.upper())
if the_string.lower() not in results:
results.append(the_string.lower())
return results
A lookup using Q object can be built to hit the database only once:
from django.db.models import Q
user_inputed_codes = ['eN', 'De', 'FR']
lookup = Q()
for code in user_inputed_codes:
lookup |= Q(code__iexact=code)
filtered_products = Products.objects.filter(lookup)
A litle more elegant way would be this:
[x for x in Products.objects.all() if x.code.upper() in [y.upper() for y in user_entered_data_as_list]]
You can do it annotating the lowered code and also lowering the entered data
from django.db.models.functions import Lower
Products.objects.annotate(lower_code=Lower('code')).filter(lower_code__in=[user_entered_data_as_list_lowered])