i'm running into issues with the following, and I'm wondering if it is even possible.
I have a flask-admin adminview setup, with an extra form field which shows a dropdown based on a specific column (category) in the sql model. See code for clarification:
model:
class Item(db.Model):
id = db.Column(db.Integer, primary_key = True)
name = db.Column(db.String(128), index = True)
category = db.Column(db.String(16))
I have the extra form field in Flask-Admin as follows:
form_extra_fields = {
'category': QuerySelectField(
label='Categories',
query_factory = lambda: db.session.query(Item),
get_label = 'category',
)
}
This all works fine except if there are duplicates in the category column, then the dropdown is populated with these duplicate values. Is it possible to remove those duplicates from dropdown, or at least only show the unique values?
Basically I solved this by overriding a class method in QuerySelectField class as follows, by appending unique labels to a list and check if every next label is in that list. I'm still thinking there should be a better way though...
def iter_choices(self):
labels = []
if self.allow_blank:
yield ('__None', self.blank_text, self.data is None)
for pk, obj in self._get_object_list():
if self.get_label(obj) not in labels:
labels.append(self.get_label(obj))
yield (pk, self.get_label(obj), obj == self.data)
Related
I have a programme where users should be able to filter different types of technologies by their attributes. My question is, how would I filter the technologies when there's potential conflicts and empty values in the parameters I use to filter?
Forms.py:
class FilterDataForm(forms.ModelForm):
ASSESSMENT = (('', ''),('Yes', 'Yes'),('No', 'No'),)
q01_suitability_for_task_x = forms.ChoiceField(label='Is the technology suitable for x?',
choices=ASSESSMENT, help_text='Please select yes or no', required=False,)
q02_suitability_for_environment_y = forms.ChoiceField(label='Is the technology suitable for environment Y?',
choices=ASSESSMENT, help_text='Please select yes or no', required=False)
There are many fields in my model like the ones above.
views.py
class TechListView(ListView):
model = MiningTech
def get_queryset(self):
q1 = self.request.GET.get('q01_suitability_for_task_x', '')
q2 = self.request.GET.get('q02_suitability_for_environment_y', '')
object_list = MiningTech.objects.filter(q01_suitability_for_task_x=q1).filter(
q02_suitability_for_environment_y=q2)
return object_list
The difficulty is that not all technology db entries will have data. So in my current setup there's times where I will filter out objects that have one attribute but not another.
For instance if my db has:
pk1: q01_suitability_for_task_x=Yes; q02_suitability_for_environment_y=Yes;
pk2: q01_suitability_for_task_x=Yes; q02_suitability_for_environment_y='';
In the form, if I don't select any value for q01_suitability_for_task_x, and select Yes for q02_suitability_for_environment_y, I get nothing back in the queryset because there are no q01_suitability_for_task_x empty fields.
Any help would be appreciated. I'm also ok with restructuring everything if need be.
The problem is that your self.request.GET.get(...) code defaults to an empty string if there is no value found, so your model .filter() is looking for matches where the string is ''.
I would restructure the first part of get_queryset() to build a dictionary that can be unpacked into your filter. If the value doesn't exist then it doesn't get added to the filter dictionary:
filters = {}
q1 = self.request.GET.get('q01_suitability_for_task_x', None)
q2 = self.request.GET.get('q02_suitability_for_environment_y', None)
if q1 is not None:
filters['q01_suitability_for_task_x'] = q1
... etc ...
object_list = MiningTech.objects.filter(**filters)
If you have a lot of q1, q2, etc. items then consider putting them in a list, looping through and inserting into the dictionary if .get(...) returns anything.
Edit: Because there are indeed a lot possible filters, the final solution looks as follows:
def get_queryset(self):
filters = {}
for key, value in self.request.GET.items():
if value != '':
filters[key] = value
object_list = Tech.objects.filter(**filters)
I am trying to create a filter search bar that I can customize. For example, if I type a value into a search bar, then it will query a model and retrieve a list of instances that match the value. For example, here is a view:
class StudentListView(FilterView):
template_name = "leads/student_list.html"
context_object_name = "leads"
filterset_class = StudentFilter
def get_queryset(self):
return Lead.objects.all()
and here is my filters.py:
class
StudentFilter(django_filters.FilterSet):
class Meta:
model = Lead
fields = {
'first_name': ['icontains'],
'email': ['exact'],
}
Until now, I can only create a filter search bar that can provide a list of instances that match first_name or email(which are fields in the Lead model). However, this does now allow me to do more complicated tasks. Lets say I added time to the filter fields, and I would like to not only filter the Lead model with the time value I submitted, but also other Lead instances that have a time value that is near the one I submitted. Basically, I want something like the def form_valid() used in the views where I can query, calculate, and even alter the values submitted.
Moreover, if possible, I would like to create a filter field that is not necessarily an actual field in a model. Then, I would like to use the submitted value to do some calculations as I filter for the list of instances. If you have any questions, please ask me in the comments. Thank you.
You can do just about anything by defining a method on the filterset to map the user's input onto a queryset. Here's one I did earlier. Code much cut down ...
The filter coat_info_contains is defined as a CharFilter, but it is further parsed by the method which splits it into a set of substrings separated by commas. These substrings are then used to generate Q elements (OR logic) to match a model if the substring is contained in any of three model fields coating_1, coating_2 and coating_3
This filter is not implicitly connected to any particular model field. The connection is through the method= specification of the filter to the filterset's method, which can return absolutely any queryset on the model that can be programmed.
Hope I haven't cut out anything vital.
import django_filters as FD
class MemFilter( FD.FilterSet):
class Meta:
model = MyModel
# fields = [fieldname, ... ] # default filters created for these. Not required if all declarative.
# fields = { fieldname: [lookup_expr_1, ...], ...} # for specifying possibly multiple lookup expressions
fields = {
'ft':['gte','lte','exact'], 'mt':['gte','lte','exact'],
...
}
# declarative filters. Lots and lots of
...
coat_info_contains = FD.CharFilter( field_name='coating_1',
label='Coatings contain',
method='filter_coatings_contains'
)
...
def filter_coatings_contains( self, qs, name, value):
values = value.split(',')
qlist = []
for v in values:
qlist.append(
Q(coating_1__icontains = v) |
Q(coating_2__icontains = v) |
Q(coating_3__icontains = v) )
return qs.filter( *qlist )
Goal was to implement a simple View for the Users to Select columns dynamically with some added calculated info (annotations on some specific columns) and also let them filter on fields.
Thankful for any comments, since this took me quite a few hours to get it working properly I thought I would provide a short writeup for anyone looking at a similar problem :)
Used Modules/Libraries etc:
Django-Filter
Django_Tables2
Bootstrap-Select to properly display Multiple Choice Fields
Example Model which we would like to use:
class Summary(models.Model):
billing_date = models.DateField(verbose_name='Billing Date')
period = models.CharField(max_length=10, verbose_name='Period')
operator = models.CharField(max_length=50, verbose_name='Operator')
product = models.CharField(max_length=30, verbose_name='Product')
...
The filters are really straightforward, the only special case here is that the we want an empty queryset initially and some fields should be required.
"info" will hold the select columns of our "Summary" Model, "product" and "operator" are just fields in Summary.
class AdHocReportFilter(django_filters.FilterSet):
info = django_filters.MultipleChoiceFilter(choices=report_field_choices, label='Available Fields', required=True)
product = django_filters.ModelChoiceFilter(queryset=Product.objects.all(), label='Products', required=True)
operator = django_filters.CharFilter(field_name="operator", lookup_expr='contains', label='Operator')
....
def __init__(self, *args, **kwargs):
super(AdHocReportFilter, self).__init__(*args, **kwargs)
if self.data == {}:
self.queryset = self.queryset.none()
Template:
Nothing interesting to show here, you can use Bootstrap-Select to tidy up your Multi Select Fields (there are quite a few nice writeups about that available).
Make sure to put your Table into an "if" as the object may or may not exist depending on your view (if someone wants an example of the template let me know)
View:
Extract your GET requests accordingly (either as list or simple value depending on your available filters).
The actual filter itself depends on what you want the user to be able to filter, make sure to either make all necessary fields required or replace them with some standard value as the filter will not accept None Types.
"field__contains" is your friend here since it will also show values on not selected fields!
Special Case, if there can actually be "Null" Values in the DB for specific fields, move them to another filter likethe below example of the Q - query!
Fortunately "values" accepts "*list" which is a simple list of all our available columns.
The annotations are just dependant on what you want to achieve.
Call the Table Object with the added argument "user_columns" which holds our list so we can build the required Table.
#login_required
def ad_hoc_report(request):
template_name = 'non_voice/ad_hoc_report.html'
filter = AdHocReportFilter(request.GET, queryset=Summary.objects.all())
info = request.GET.getlist('info', None)
product = request.GET.get('product', '')
operator = request.GET.get('operator', '')
start_date = request.GET.get('start_date', None)
end_date = request.GET.get('end_date', None)
if operator is None:
operator = ''
result_object = Summary.objects.filter(product__contains=product, ).filter((Q(operator__contains=operator)|Q(operator__isnull=True)).values(*info).annotate(
Amount=Sum("amount"), Count=Sum("count"))
table = AdHocReportTable(data=result_object, user_columns=info)
return render(request, template_name, {'filter': filter, 'table': table})
Table:
This was the difficult part and only possible with lots and lots of reading various stack overflow comments :)
First of all define your calculated annotation columns and set the required Meta info, the '...' is a built in placeholder without knowing the column name in advance (which helps us to move our calculated columns to the end of the Table)
In the init we first check if our "self.base_columns" are consistent with what we provided and remove columns which were deselected by our user, otherwise it would still show them empty even after filtering. (Maybe there is a nicer way to do this, haven't found it yet)
In the next step add the columns selected by our user dynamically from the mentioned above "user_columns" which we passed in the views.py
class AdHocReportTable(tables.Table):
Amount = tables.Column(verbose_name='Amount')
Count = tables.Column(verbose_name='Count')
class Meta:
# '...' is a built in placeholder!
sequence = ('...', 'Amount', 'Count')
template_name = "django_tables2/bootstrap4.html"
attrs = {'class': 'table table-hover', }
# This makes it possible to pass a dynamic list of columns to the Table Object
def __init__(self, data, user_columns, *args, **kwargs):
if user_columns:
calulated_columns = ['Amount', 'Count']
# Removes deselected columns from the table (otherwise they are shown empty)
for key, val in self.base_columns.items():
if key not in user_columns and key not in calulated_columns:
del self.base_columns[key]
# Add the Selected Columns dynamically to the Table
for col in user_columns:
self.base_columns[col] = tables.Column(verbose_name=col)
super(AdHocReportTable, self).__init__(data, user_columns, *args, **kwargs)
I'm trying to populate a django modelform with initial data provided from an external source. To achieve that I start by pull all the needed data from the external source:
url =('http://myapi.example.com')
data = urllib2.urlopen(url)
result = json.load(data)
api_data_name = result['properties']['name']
api_data_type = result['properties']['type']
Followed by populating a dict which will serve as initial data to my form:
data = {}
for field in my_model._meta.fields:
if field.name == 'name':
data[field.name] = api_data_name
form = MyEditForm(initial=data)
Then I'm passing the form to the template and the initial data is populating my text fields as expected, but now I need to be able to set a value of a select field based on a string I receive from my external source and Im not getting how can I achieve that, since doing something like:
if field.name == 'type':
data[field.name] = api_data_type
Wont do the job cause the select element has "0", "1", "2", etc as options value and not the long description i get from api_data_type variable.
How can I get the long_description from all the options <option value="1">long_description</option> of my select field in my view so i can compare each one with api_data_type?
Heres a sample of my models.py and forms.py:
#models.py
TYPE = (
('0',_(u'Type1')),
('1',_(u'Type2')),
('2',_(u'Type3')),
)
class MyModel(models.Model):
...
type=models.CharField(max_length=30,choices=TYPE,blank=True)
...
#forms.py
class MyEditForm(forms.ModelForm):
class Meta:
model = MyModel
widgets = {
...
'type': Select(attrs={'class':'select-small span2'}),
...
}
Found out how to accomplish what I asked.
# For select fields
if field.name == 'classification':
for choice in field.choices:
if choice[1].lower() == api_poi_classification.lower():
data[field.name] = choice[0]
And for any of ya trying to populate many-to-many fields (as checkboxes in my case)
# Many to many fields populate
for field in hotel_poi._meta.many_to_many:
if field.name == 'views':
if u'Vista' in api_poi_review_fields:
api_vistas = api_poi_review[u'Vista']
# The api_vistas string comes from api in the format (v1; v2; v3; v4)
views = api_vistas.split(';')
choices = field.get_choices()
temp = []
for view in views:
for choice in choices:
if view.lower().strip() == choice[1].lower().strip():
temp.append(choice[0])
data[field.name]=temp
All of this could be avoided if I had direct database access... In that case i would just need to set an object instance like m = MyModel.objects.filter(id=1) and call form = MyEditForm(instance=m)
But that was not the case and that's what makes this question a bit particular.
EDIT:
It turns out the real question is - how do I get select_related to follow the m2m relationships I have defined? Those are the ones that are taxing my system. Any ideas?
I have two classes for my django app. The first (Item class) describes an item along with some functions that return information about the item. The second class (Itemlist class) takes a list of these items and then does some processing on them to return different values. The problem I'm having is that returning a list of items from Itemlist is taking a ton of queries, and I'm not sure where they're coming from.
class Item(models.Model):
# for archiving purposes
archive_id = models.IntegerField()
users = models.ManyToManyField(User, through='User_item_rel',
related_name='users_set')
# for many to one relationship (tags)
tag = models.ForeignKey(Tag)
sub_tag = models.CharField(default='',max_length=40)
name = models.CharField(max_length=40)
purch_date = models.DateField(default=datetime.datetime.now())
date_edited = models.DateTimeField(auto_now_add=True)
price = models.DecimalField(max_digits=6, decimal_places=2)
buyer = models.ManyToManyField(User, through='Buyer_item_rel',
related_name='buyers_set')
comments = models.CharField(default='',max_length=400)
house_id = models.IntegerField()
class Meta:
ordering = ['-purch_date']
def shortDisplayBuyers(self):
if len(self.buyer_item_rel_set.all()) != 1:
return "multiple buyers"
else:
return self.buyer_item_rel_set.all()[0].buyer.name
def listBuyers(self):
return self.buyer_item_rel_set.all()
def listUsers(self):
return self.user_item_rel_set.all()
def tag_name(self):
return self.tag
def sub_tag_name(self):
return self.sub_tag
def __unicode__(self):
return self.name
and the second class:
class Item_list:
def __init__(self, list = None, house_id = None, user_id = None,
archive_id = None, houseMode = 0):
self.list = list
self.house_id = house_id
self.uid = int(user_id)
self.archive_id = archive_id
self.gen_balancing_transactions()
self.houseMode = houseMode
def ret_list(self):
return self.list
So after I construct Itemlist with a large list of items, Itemlist.ret_list() takes up to 800 queries for 25 items. What can I do to fix this?
Try using select_related
As per a question I asked here
Dan is right in telling you to use select_related.
select_related can be read about here.
What it does is return in the same query data for the main object in your queryset and the model or fields specified in the select_related clause.
So, instead of a query like:
select * from item
followed by several queries like this every time you access one of the item_list objects:
select * from item_list where item_id = <one of the items for the query above>
the ORM will generate a query like:
select item.*, item_list.*
from item a join item_list b
where item a.id = b.item_id
In other words: it will hit the database once for all the data.
You probably want to use prefetch_related
Works similarly to select_related, but can deal with relations selected_related cannot. The join happens in python, but I've found it to be more efficient for this kind of work than the large # of queries.
Related reading on the subject