Importing many to many relationship (Django) - django

I want to import data to many to many fields.
My code below imports the data, but only applies one relationship to the model (not two, as I've asked it to do below by importing it twice- where I wrote "cat = ...")
In my example below, I want it to import cat (category) from columns 4 & 11. My code below only applies one category to the model (not both).
How can I get it to apply both fields to the model? Im using python 2.7
import csv
l = list(csv.reader(open('test_data.csv', 'rb')))
Gender_CHOICES = {
'Male': 1,
'Female': 2,
'Unisex': 3,
}
Stock_CHOICES = {
'in stock': 1,
'low stock': 2,
'out of stock': 3,
'discountinued': 4
}
for i in l[1:]:
cat = m.Category.objects.get_or_create(category_name = i[4],[11])[0]
prod = m.Product(
name = i[0],
link = i[1],
description = i[6],
brand = i[7],
colour = i[10],
gender = Gender_CHOICES[i[8]] if i[8] in Gender_CHOICES else 3,
store = m.Store.objects.get_or_create(store_name = i[2])[0]
)
prod.save()
var = m.Variation(
product = prod,
variation = "default"
)
var.save()
img = m.Image(
variation = var,
image = i[5]
)
img.save()
size = m.Size(
variation = var
)
size.save()
price = m.Price(
variation = var,
price = float(i[3])
)
price.save()
stock = m.Stock(
size = size,
stock = Stock_CHOICES[i[9]] if i[9] in Stock_CHOICES else 4
)
stock.save()
prod.category.add(cat)
Sample of CSV:
prod_name,prod_link,store_name,prod_price,category,image_default,prod_description,prod_brand,gender,prod_stock,category1
Bliss Firm Baby Firm Lifting & Volumising Serum 30ml - Serum,http://click.linksynergy.com/link?id=dnw*50nuNL8&offerid=287549.2554637&type=15&murl=http%3A%2F%2Fwww.asos.com%2Fau%2FBliss%2FBliss-Firm-Baby-Firm-Lifting-Volumising-Serum-30ml%2FProd%2Fpgeproduct.aspx%3Fiid%3D3936070%26istCompanyId%3Df448b47d-6b90-4b9b-a52d-eb6058c99b1c%26istItemId%3Dwxqqpxxmi%26istBid%3Dt,Asos,117,Skin Care Body Creams & Moisturisers,http://images.asos-media.com/inv/media/0/7/0/6/3936070/serum/image1xxl.jpg,Firm Baby Firm Lifting & Volumising Serum by Bliss Designed to boost collagen and elasticity Concentrated formula with a water-free aloe base Aims to plump skin from the inside out,Bliss,Female,
Yes To Carrots Day Cream 50ml - Carrots,http://click.linksynergy.com/link?id=dnw*50nuNL8&offerid=287549.2825448&type=15&murl=http%3A%2F%2Fwww.asos.com%2Fau%2FYES-TO%2FYes-To-Carrots-Day-Cream-50ml%2FProd%2Fpgeproduct.aspx%3Fiid%3D4254119%26istCompanyId%3Df448b47d-6b90-4b9b-a52d-eb6058c99b1c%26istItemId%3Dwiqwwawpm%26istBid%3Dt,Asos,21,Skin Care Body Creams & Moisturisers,http://images.asos-media.com/inv/media/9/1/1/4/4254119/carrots/image1xxl.jpg,Day cream by Yes To Carrots 95% natural ingredients Including carrots and sweet almond oil Rich moisturising formula Naturally nourishes to promote softer skin Suitable for normal to dry skin types Product size: 50ml,YES TO,Female,Belts

I believe, the problem is with Object creation. I think the following is what you are looking for :
....
....
for i in l[1:]:
cat_1 = m.Category.objects.get_or_create(category_name = i[4])
cat_2 = m.Category.objects.get_or_create(category_name = i[11])
....
....
prod.category.add(cat_1)
prod.category.add(cat_2)

Related

Joining of curve fitting models

I have this 7 quasi-lorentzian curves which are fitted to my data.
and I would like to join them, to make one connected curved line. Do You have any ideas how to do this? I've read about ComposingModel at lmfit documentation, but it's not clear how to do this.
Here is a sample of my code of two fitted curves.
for dataset in [Bxfft]:
dataset = np.asarray(dataset)
freqs, psd = signal.welch(dataset, fs=266336/300, window='hamming', nperseg=16192, scaling='spectrum')
plt.semilogy(freqs[0:-7000], psd[0:-7000]/dataset.size**0, color='r', label='Bx')
x = freqs[100:-7900]
y = psd[100:-7900]
# 8 Hz
model = Model(lorentzian)
params = model.make_params(amp=6, cen=5, sig=1, e=0)
result = model.fit(y, params, x=x)
final_fit = result.best_fit
print "8 Hz mode"
print(result.fit_report(min_correl=0.25))
plt.plot(x, final_fit, 'k-', linewidth=2)
# 14 Hz
x2 = freqs[220:-7780]
y2 = psd[220:-7780]
model2 = Model(lorentzian)
pars2 = model2.make_params(amp=6, cen=10, sig=3, e=0)
pars2['amp'].value = 6
result2 = model2.fit(y2, pars2, x=x2)
final_fit2 = result2.best_fit
print "14 Hz mode"
print(result2.fit_report(min_correl=0.25))
plt.plot(x2, final_fit2, 'k-', linewidth=2)
UPDATE!!!
I've used some hints from user #MNewville, who posted an answer and using his code I got this:
So my code is similar to his, but extended with each peak. What I'm struggling now is replacing ready LorentzModel with my own.
The problem is when I do this, the code gives me an error like this.
C:\Python27\lib\site-packages\lmfit\printfuncs.py:153: RuntimeWarning:
invalid value encountered in double_scalars [[Model]] spercent =
'({0:.2%})'.format(abs(par.stderr/par.value))
About my own model:
def lorentzian(x, amp, cen, sig, e):
return (amp*(1-e)) / ((pow((1.0 * x - cen), 2)) + (pow(sig, 2)))
peak1 = Model(lorentzian, prefix='p1_')
peak2 = Model(lorentzian, prefix='p2_')
peak3 = Model(lorentzian, prefix='p3_')
# make composite by adding (or multiplying, etc) components
model = peak1 + peak2 + peak3
# make parameters for the full model, setting initial values
# using the prefixes
params = model.make_params(p1_amp=6, p1_cen=8, p1_sig=1, p1_e=0,
p2_ampe=16, p2_cen=14, p2_sig=3, p2_e=0,
p3_amp=16, p3_cen=21, p3_sig=3, p3_e=0,)
rest of the code is similar like at #MNewville
[![enter image description here][3]][3]
A composite model for 3 Lorentzians would look like this:
from lmfit import Model, LorentzianModel
peak1 = LorentzianModel(prefix='p1_')
peak2 = LorentzianModel(prefix='p2_')
peak3 = LorentzianModel(prefix='p3_')
# make composite by adding (or multiplying, etc) components
model = peak1 + peaks2 + peak3
# make parameters for the full model, setting initial values
# using the prefixes
params = model.make_params(p1_amplitude=10, p1_center=8, p1_sigma=3,
p2_amplitude=10, p2_center=15, p2_sigma=3,
p3_amplitude=10, p3_center=20, p3_sigma=3)
# perhaps set bounds to prevent peaks from swapping or crazy values
params['p1_amplitude'].min = 0
params['p2_amplitude'].min = 0
params['p3_amplitude'].min = 0
params['p1_sigma'].min = 0
params['p2_sigma'].min = 0
params['p3_sigma'].min = 0
params['p1_center'].min = 2
params['p1_center'].max = 11
params['p2_center'].min = 10
params['p2_center'].max = 18
params['p3_center'].min = 17
params['p3_center'].max = 25
# then do a fit over the full data range
result = model.fit(y, params, x=x)
I think the key parts you were missing were: a) just add models together, and b) use prefix to avoid name collisions of parameters.
I hope that is enough to get you started...

How to make sorting in DB by the number of items in the ListField?

I have the next model:
class Departments(Document):
_id = fields.ObjectIdField()
name = fields.StringField(blank=True, null=True)
department_id = fields.StringField(blank=True, null=True) # Added
list_of_users = fields.ListField(blank=True, null=True)
list_of_workstations = fields.ListField(blank=True, null=True)
As you can see list_of_users and list_of_workstations are lists of items.
I wrote a code in Python, which takes all data from DB, put it into dict and then sorts as I need, but it works too slow.
How can I sort Departments right in the DB by the length of list_of_users or list_of_workstations or by ratio of list_of_users/list_of_workstations, something like:
departments = DepartmentStats.objects.order_by(len(list_of_users)).dsc
or
departments = DepartmentStats.objects.order_by(len(list_of_users)/len(list_of_workstations)).dsc
?
For your first request, use annotation like Umut Gunebakan told you in his comment. But I'm know sure about Count() on ListField
departments = DepartmentStats.objects.all().annotate(num_list_users=Count('list_of_users')).order_by('-num_list_users')
For a desc order by, you just need to add the sign '-' (minus).
https://docs.djangoproject.com/en/1.10/ref/models/querysets/#order-by
The second request will be :
departments = DepartmentStats.objects.all().annotate(user_per_workstation=(Count('list_of_users')/Count('list_of_workstations')).order_by('-user_per_workstation')
UPDATE: (Mongoengine used)
With mongoengine you need to get item frequencies and sorted the result :
Check this part of documentation - futher aggregation
list_user_freqs = DepartmentStats.objects.item_frequencies('list_of_users', normalize=True)
from operator import itemgetter
list_user_freqs_sroted = sorted(list_user_freqs.items(), key=itemgetter(1), reverse=True)
If someone needs raw query:
departments = DepartmentStats._get_collection().aggregate([
{"$project": {
"department_id": 1,
"name": 1,
"list_of_users": 1,
}},
{"$sort": {"list_of_users": -1}},
])
and the case, when the result must be sorted by the ratio list_of_users/list_of_workstations
departments = DepartmentStats._get_collection().aggregate([
{"$project": {
"department_id": 1,
"name": 1,
"list_of_users": 1,
"len_list_of_items": {"$divide": [{"$size": "$list_of_users"},
{"$size": "$list_of_workstations"}]}
}},
{"$sort": {"len_list_of_items": -1}},
])

Filtering over a list of date ranges in one ORM call

I am having the following structure:
class Player(models.Model):
name = models.CharField(max_length=50)
dob = models.DateField(max_length=80)
class Game(models.Model):
name = models.CharField(max_length=50)
class Attempts(models.Model):
player = models.ForeignKey("Player")
game = models.ForeignKey("Game")
won = models.BooleanField()
The class Attempts logs the attempts of each player and whether he has won or lost a game.Now I want to get number of people attempting a particular game in agr groups.For example people aged 0-18 have attempted a game 5 times and so on.I have consulted the following this post in django users and made the following code:
ranges = [(0,18),(18,25),(25,50)]
now = datetime.now()
groups = {}
for (a,b) in ranges:
# print str(a) + " " +str(b)
newer_date = (now - relativedelta(years=a)).date()
older_date = (now - relativedelta(years=b)).date()
print str(newer_date)
print str(older_date)
groups[str(a)+","+str(b)] = Attempts.objects.filter(game__pk = "101",player__dob__range=(older_date,newer_date)).count()
This also gives result correctly as:
{"0,18": 2, "25,50": 1, "18,25": 1}
How ever the queries made by this code are equal to number of categories in the ranges.I want to some how get this data in one single ORM query.Is it possible to do so ?

Select values from two tables using Django

I have two tables as follows,
cabinet
cabinet_id cabinet_name
1 Test1
2 Test2
3 Test3
cabinet_maping
id product_id cabinet_id size
1 34 1 20
2 34 3 25
How can I select all cabinet_name from cabinet table where cabinet_maping.product_id = 34 in Django
Expected Output as follows,
cabinet_id cabinet_name size
1 Test1 20
2 Test2 0
3 Test3 25
I think that yours models could look like
class Cabinet(models.Model):
name = models.CharField(max_length = 30)
class cabinet_maping(models.Model):
product = models.ForeignKey(Product)
cabinet = models.ForeignKey(Cabinet)
size = .....
You should do sth like this:
cabinet_name = cabinet_maping.objects.filter(product__id = 34)
for item in cabinet_name:
print item.cabinet.id
print item.cabinet.name
print item.size
I didn't check it but i think that should work (that works :))
I agree with Silwestpl that your models are wrong as using Foreign Key would make this query a lot easier. But just an answer to your question would be. I am assuming that you dont have any relationships between the two tables as you haven't mentioned any.
x = Cabinet_mapping.objects.filter(product_id = somevalue).distinct('cabinet_id')
response_list = []
for y in x:
response_dict = {}
response_dict['cabinet_id'] = y.cabinet_id
response_dict['cabinet_name'] = cabinet.objects.get(id = y.cabinet_id).cabinet_name
response_dict['size'] = y.size
response_list.append(response_dict)
return response_list
This would be the answer according to the details you have provided.
But Ideally I would do something like this
class Product(models.Model):
# id is generated automatically.
## any other fields that you want for product.
class Cabinet(models.Model):
# again id is auto-genearated.
cabinet_name = models.CharField(max_length=100)
cabinet_size = models.IntegerField()
products = models.ManytoManyField(Product)#This automatically maps product_id n cabinet_id.
And your query would be.
x = Cabinet.objects.filter(product.id=some_value)
print (x.id , x.cabinet_name, x.cabinet_size) # I used print, but you can use it anyway you want.
This would be what you require. Please use the second solution if you are looking into something serious.

Django charts - date&time axis

I have one model which looks like this:
class Measurement(models.Model):
date = models.DateField('date')
time = models.TimeField('time')
Q = models.DecimalField(max_digits=10, decimal_places=6)
P = models.DecimalField(max_digits=10, decimal_places=6)
f = models.DecimalField(max_digits=10, decimal_places=6)
In my views, I would like to represent it. So I made this function:
def plotMeas(request):
# Count the events
c = Measurement.objects.all()
c = c.count()
# Variables
i = 0
a = [0]
P = a*c
Q = a*c
t = a*c
# Save dP_L1 & dQ_L1 in lists
for i in range(c):
meas = Measurement.objects.get(pk = i+1)
P [i] = meas.P
Q [i] = meas.Q
t [c-1-i] = i*10
if c > 100:
P = P[-100:]
Q = Q[-100:]
t [i] = t[-100:]
# Construct the graph
fig = Figure()
q = fig.add_subplot(211)
q.set_xlabel("time (minutes ago)")
q.set_ylabel("Q (VAR)")
p = fig.add_subplot(212)
p.set_xlabel("time (minutes ago)")
p.set_ylabel("P (W)")
p.plot(t,P, 'go-')
q.plot(t,Q, 'o-')
canvas = FigureCanvas(fig)
response = HttpResponse(content_type='image/png')
canvas.print_png(response)
return response
However, I would like that the horizontal axis would show the date and the time (saved in the model). Does anyone know how to do it?
Have a look at the documentation for plot_date. Conveniently plot_date takes similar arguments to plot. A call might look like:
p.plot_date(sequence_of_datetime_objects, y_axis_values, 'go-')
Using matplotlib.dates you can then customize the format of your x-axis labels.
A simple example:
The following will specify that the x-axis displays only every third month in the format Jan '09 (assuming English-speaking locale).
p.xaxis.set_major_locator(mdates.MonthLocator(interval=3))
p.xaxis.set_major_formatter(mdates.DateFormatter("%b '%y"))
Since you have dates and times stored separately you may either want to
change your model to use a DateTimeField, or
use Python to combine them.
For example:
import datetime as dt
t1 = dt.time(21,0,1,2) # 21:00:01.2
d1 = dt.date.today()
dt1 = dt.datetime.combine(d1,t1)
# result: datetime.datetime(2011, 4, 15, 21, 0, 1, 2)
To iterate over two sequences and combine them you might use zip (code for illustrative purposes only, not necessarily optimized):
sequence_of_datetime_objects = []
for a_date, a_time in zip(sequence_of_date_objects, sequence_of_time_objects):
sequence_of_datetime_objects.append(dt.datetime.combine(a_date, a_time))
Feel free to open another question if you get stuck implementing the specifics.