Related
Please need help with the following
I am trying to update database in comparison to previous uploaded csv file. I need to update all fields except the vin if it changes (vin is the unique value), delete the item if it is no longer in the csv file and create one if one is new
vin. stock_no make model trim miles
12345789098765432 4535 honda civic lx 89000
j4j4jj49098765432 3453 toyota corolla DX 54555
12345345438765432 6254 ford mustang es 101299
When I change any value and the csv is uploaded it makes a duplicate:
def upload_file__view(request):
form = form(request.POST or None, request.FILES or
None)
company = Comp_info.objects.last()
if form.is_valid():
form.save()
obj = c.objects.get(activated=False)
with open(obj.file_name.path, 'r+') as f:
reader = c.reader(f)
for i, row in enumerate(reader):
if i==0:
pass
else:
# row = "".join(row)
# row = row.replace(",", " ")
# row = row.split()
print(row)
print(type(row))
vin = row[0].upper()
condition = row[1].replace("U", "Used").replace("N", "New")
stock_no = row[2]
year = int(row[5])
make = row[3]
model = row[4]
trim = row[6]
mileage = row[8]
mpg_city = row[18]
mpg_hwy = row[19]
engine = row[9]
transmission = row[12]
fuel_type = row[11]
vehicle_type = row[7]
drive_type = row[20].replace("4X2", "2WD").replace("4X4", "4WD")
exterior_color = row[15]
interior_color = row[16]
price = row[13].replace("0", "")
description = row[22]
features_2 = row[21]
images_data = row[23]
raw_images_list = images_data.split(',')
images_list = [""] * 25
for x in range(image_count):
if x == 25:
break
images_list[x] = raw_images_list[x]
for x in images_list:
print(x)
if images_list[0] == "":
images_list[0] = "https://www.beverlyhillscarclub.com/template/images/ina_f.jpg"
car_photo = images_list[0]
car_photo_1 = images_list[1]
car_photo_2 = images_list[2]
car_photo_3 = images_list[3]
car_photo_4 = images_list[4]
car_photo_5 = images_list[5]
car_photo_6 = images_list[6]
car_photo_7 = images_list[7]
car_photo_8 = images_list[8]
car_photo_9 = images_list[9]
car_photo_10 = images_list[10]
car_photo_11 = images_list[11]
car_photo_12 = images_list[12]
car_photo_13 = images_list[13]
car_photo_14 = images_list[14]
car_photo_15 = images_list[15]
car_photo_16 = images_list[16]
car_photo_17 = images_list[17]
car_photo_18 = images_list[18]
car_photo_19 = images_list[19]
car_photo_20 = images_list[20]
car_photo_21 = images_list[21]
car_photo_22 = images_list[22]
car_photo_23 = images_list[23]
car_photo_24 = images_list[24]
# notes = pip._vendor.requests(images_list[0], stream=True)
#car_photo = row[23]
# user = User.objects.get(username=row[3])
Cars.objects.update_or_create(
vin = vin,
condition = condition,
stock_no = stock_no,
year = year,
make = make,
model = model,
trim = trim,
mileage = mileage,
mpg_city = mpg_city,
engine = engine,
transmission = transmission,
fuel_type = fuel_type,
vehicle_type = vehicle_type,
drive_type = drive_type,
exterior_color = exterior_color,
interior_color = interior_color,
price = price,
description = description,
company_name = company.company_name,
address = company.company_address,
city = company.city,
state = company.state,
zip = company.zip_code,
phone_number = company.phone_number,
email = company.fax_number,
features_2 = features_2,
car_photo = downloadFile(car_photo),
car_photo_1 = downloadFile(car_photo_1),
car_photo_2 = downloadFile(car_photo_2),
car_photo_3 = downloadFile(car_photo_3),
car_photo_4 = downloadFile(car_photo_4),
car_photo_5 = downloadFile(car_photo_5),
car_photo_6 = downloadFile(car_photo_6),
car_photo_7 = downloadFile(car_photo_7),
car_photo_8 = downloadFile(car_photo_8),
car_photo_9 = downloadFile(car_photo_9),
car_photo_10 = downloadFile(car_photo_10),
car_photo_11 = downloadFile(car_photo_11),
car_photo_12 = downloadFile(car_photo_12),
car_photo_13 = downloadFile(car_photo_13),
car_photo_14 = downloadFile(car_photo_14),
car_photo_15 = downloadFile(car_photo_15),
car_photo_16 = downloadFile(car_photo_16),
car_photo_17 = downloadFile(car_photo_17),
car_photo_18 = downloadFile(car_photo_18),
car_photo_19 = downloadFile(car_photo_19),
car_photo_20 = downloadFile(car_photo_20),
car_photo_21 = downloadFile(car_photo_21),
car_photo_22 = downloadFile(car_photo_22),
car_photo_23 = downloadFile(car_photo_23),
car_photo_24 = downloadFile(car_photo_24)
#car_photo = car_photo,
# quantity = int(row[2]),
# salesman = user
)
obj.activated = True
obj.save()
data = {
'form' : form,
'now' : now,
}
return render(request, 'uploads.html', data)
Thanks in advance for any help!
Thank you
Step 1
An empty list was created to compare with uploaded data:
imported_cars = []
Step 2
Created a filter of unique value (primary Key) and checked if it existed and used the method get to update items. Created car (item variable) to update or create ubject.
if Cars.objects.filter(vin=vin).exists():
car = Cars.objects.get(vin=vin)
Step 3
Used else statement to create item if it did not exist.
else:
car = Cars.objects.create(vin=vin, condition=condition...)
Last, out of the loop populated empty list with updated and created cars and deleted items that were in the database but not in the csv file.
imported_cars_vin_numbers = [car.vin for car in imported_cars]
for car in Cars.objects.all():
if car.vin not in imported_cars_vin_numbers:
car.delete()
Special thanks and credit to Zack Plauché who was extremely helpful and professional in helping me and teaching me how to solve this issue.
Your issue is in the model.py
you should write the Cars object with the following.
vin = models.CharField(primary_key=True, editable=False)
Confirm this works, since I am suggesting solution without actually seeing the model.py
This should handle the update aspect of your logic. The part where you delete a vin if its not in the CSV will have to be done with new process I don't see written here.But a suggestion would be to clear the DB and repopulate, or create function that compares DB with CSV and delete object if not in CSV.
I want to prefill a form with values taken in a table.
First I pass the PK relative to the line where I wan't to get values and build the kwargs list:
views.py
def NavetteToFicheCreateView(request, pk):
navette = Navette.objects.get(id=pk)
ref = navette.id
attribute_set = navette.famille.pk
cost = navette.cost
qty = navette.qty
etat = navette.etat
etat_ebay = navette.etat.etat_ebay
ean = get_last_ean()
form = NavetteToFicheForm(
request.POST,
ref=ref,
attribute_set=attribute_set,
cost=cost,
qty=qty,
etat=etat,
etat_ebay=etat_ebay,
ean=ean,
)
[...]
then I retrieve the kwargs in the form.py and setup my initial values
class NavetteToFicheForm(forms.ModelForm):
def __init__(self, *args, **kwargs):
self.ref = kwargs.pop('ref', 'noref')
self.attribute_set = kwargs.pop('attribute_set', 9999)
self.cost = kwargs.pop('cost', 0)
self.qty = kwargs.pop('qty', 0)
self.etat = kwargs.pop('etat', 0)
self.etat_ebay = kwargs.pop('etat_ebay', 9999)
self.ean = kwargs.pop('ean', 9999)
super(NavetteToFicheForm, self).__init__(*args, **kwargs)
self.fields['ref'].initial = self.ref
self.fields['attribute_set'].initial = self.attribute_set
self.fields['cost'].initial = self.cost
self.fields['qty'].initial = self.qty
self.fields['etat'].initial = self.etat
self.fields['etat_ebay'].initial = self.etat_ebay
self.fields['ean'].initial = self.ean
[...]
My problem : some fields like "ref" or "attribute_set" are foreignKeys and are not transmitted when i display the form.
For checking my values :
print(self.ref)
print(self.attribute_set)
output
34
2
noref
9999
questions :
Why does the "print" displays 2 couples of values ? This looks like as if the "noref" and "999" are taken in account.
Why if i set manually 34 and 2 values, it works ?
self.fields['ref'].initial = 34
self.fields['attribute_set'].initial = 2
There's maybe a better way of doing this but I don't know it yet .
I have two question. First question: Python script that shows me the precipitation for a certain period.For example, I'm getting an initial year-month and a final year-month.
Initial:
year:2000
month:3
Final
year1:2005
month:4
Now, instead of seeing:
2000/3,2000/4,2000/5,2000/6..........2005/1,2005/2,2005/3,2005/4
she works like this(look in the hooked picture):
2000/3, 2000/4, 2001/3, 2001/4........2005/3,2005/4.
I want to work for me like the first case.
def period_month_prec(year,month,year1,month1):
for i in range (year,year1+1,1):
for j in range(month,month1+1,1):
......................
Second question: How to write the output(picture) from the script in csv.fileenter image description here
This is what my views.py script looks like , which saves me only the first result:
def monthly_period(request):
if request.method == "POST" :
form = PeriodMonthlyForm(request.POST)
if form.is_valid():
data = form.cleaned_data
year = data.get('year')
month = data.get('month')
year1 = data.get('year1')
month1 = data.get('month1')
lon = data.get('lon')
lat = data.get ('lat')
inter = data.get('inter')
point = period_month_prec(year,month,year1,month1,lon,lat)
args = {'point':point}
response = HttpResponse(content_type='text/txt')
response['Content-Disposition'] = 'attachment; filename="precipitation.txt"'
writer = csv.writer(response)
writer.writerow([point])
return response
else:
form = PeriodMonthlyForm()
active_period_monthly = True
return render (request, 'carpatclimapp/home.html',{'form':form, 'active_period_monthly': active_period_monthly})
Ok, i have forms like this:
Forms
You set initial values(red color) and end interval(blue color). For this given interval, the lon and lat are defined for the point in which we want to perform interpolation. When you press the submit button, it starts with interpolation for a defined period. The loop problem is because it works only for the defined months (we see from the 2nd picture that it only works in the interval 1-6) but not for 7,8,9,10,11,12 months between these years.
Initial: year:2000, month:3
Final: year1:2001, month:4
for this she's doing it like this: 2000/3,2000/4,2001/3,2001/4
I do not want that, I want this: 2000/3,2000/4,2000/5,2000/6,2000/7.....2000/12,2001/1,2001/2,2001/3,2001/4.
this is me code :
def period_month_prec(year,month,year1,month1,lon,lat):
cnx = sqlite3.connect(DB1)
cursor = cnx.cursor()
table = 'monthly'
year = int(year)
year1 = int(year1)
month = int(month)
month1 = int(month1)
for i in range (year,year1+1,1):
for j in range(month,month1+1,1):
query = '''
SELECT dates, cell, prec FROM %s WHERE dates = "%s-%s" ;
''' % (table,i,j)
df = pd.read_sql_query(query, cnx)
tacka = '''SELECT id, lon, lat,country,altitude FROM %s;''' % 'grid1'
grid1 = pd.read_sql_query(tacka, cnx)
podaci = pd.merge(df,grid1,left_on='cell',right_on='id')
podaci_a = podaci.drop(['cell','id','country','altitude'],axis=1)
lon_n = podaci_a['lon'].values
lat_n = podaci_a['lat'].values
prec =podaci_a['prec'].values
x_masked, y_masked, prec_p = remove_nan_observations(lon_n, lat_n, prec)
xy = np.vstack([x_masked,y_masked]).T
xi = ([lon,lat])
inter_point = interpolate_to_points(xy,prec_p,xi, interp_type='linear'
return (i,j,lon,lat,inter_point)
The results that come out look like this:
loop with calculations
The second question was how to save these results(2nd picture) in the csv file, how to write correctly views.py. Currently she looks like this :
def monthly_period(request):
if request.method == "POST" :
form = PeriodMonthlyForm(request.POST)
if form.is_valid():
data = form.cleaned_data
year = data.get('year')
month = data.get('month')
year1 = data.get('year1')
month1 = data.get('month1')
lon = data.get('lon')
lat = data.get ('lat')
inter = data.get('inter')
point = period_month_prec(year,month,year1,month1,lon,lat)
args = {'point':point}
response = HttpResponse(content_type='text/txt')
response['Content-Disposition'] = 'attachment; filename="precipitation.txt"'
writer = csv.writer(response)
writer.writerow([point])
return response
else:
form = PeriodMonthlyForm()
active_period_monthly = True
return render (request, 'carpatclimapp/home.html',{'form':form, 'active_period_monthly': active_period_monthly})
I hope I'm a little clearer now
Hei all,
I am trying to set up an abstract model for a very simple QP of the form
min (x-x0)^2
s.t.
A x = b
C x <= d
I would like to use an abstract model, as I need to resolve with changing parameters (mainly x0, but potentially also A, b, C, d). I am right now struggeling with simply setting the parameters in the model instance. I do not want to use an external data file, but rather internal python variables. All examples I find online use AMPL formatted data files.
This is the code I have right now
import pyomo.environ as pe
model = pe.AbstractModel()
# the sets
model.n = pe.Param(within=pe.NonNegativeIntegers)
model.m = pe.Param(initialize = 1)
model.ss = pe.RangeSet(1, model.n)
model.os = pe.RangeSet(1, model.m)
# the starting point and the constraint parameters
model.x_hat = pe.Param(model.ss)
model.A = pe.Param(model.os, model.ss)
model.b = pe.Param(model.os)
model.C = pe.Param(model.os, model.os)
model.d = pe.Param(model.ss, model.os)
# the decision variables
model.x_projected = pe.Var(model.ss)
# the cosntraints
# A x = b
def sum_of_elements_rule(model):
value = model.A * model.x_projected
return value == model.d
model.sumelem = pe.Constraint(model.os, rule=sum_of_elements_rule)
# C x <= d
def positivity_constraint(model):
return model.C*model.x_projected <= model.d
model.bounds = pe.Constraint(model.ss, rule=positivity_constraint)
# the cost
def cost_rule(model):
return sum((model.x_projected[i] - model.x[i])**2 for i in model.ss)
model.cost = pe.Objective(rule=cost_rule)
instance = model.create_instance()
And somehow here I am stuck. How do I set the parameters now?
Thanks and best, Theo
I know this is an old post but a solution to this could have helped me so here is the solution to this problem:
## TEST
data_init= {None: dict(
n = {None : 3},
d = {0:0, 1:1, 2:2},
x_hat = {0:10, 1:-1, 2:-100},
b = {None: 10}
)}
# create instance
instance = model.create_instance(data_init)
This creates the instance in an equivalent way than what you did but in a more formal way.
Ok, I seemed to have figured out what the problem is. If I want to set a parameter after I create an instance, I need the
mutable=True
flag. Then, I can set the parameter with something like
for i in range(model_dimension):
getattr(instance, 'd')[i] = i
The model dimension I need to choose before i create an instance (which is ok for my case). The instance can be reused with different parameters for the constraints.
The code below should work for the problem
min (x-x_hat)' * (x-x_hat)
s.t.
sum(x) = b
x[i] >= d[i]
with x_hat, b, d as parameters.
import pyomo.environ as pe
model = pe.AbstractModel()
# model dimension
model.n = pe.Param(default=2)
# state space set
model.ss = pe.RangeSet(0, model.n-1)
# equality
model.b = pe.Param(default=5, mutable=True)
# inequality
model.d = pe.Param(model.ss, default=0.0, mutable=True)
# decision var
model.x = pe.Var(model.ss)
model.x_hat = pe.Param(model.ss, default=0.0, mutable=True)
# the cost
def cost_rule(model):
return sum((model.x[i] - model.x_hat[i])**2 for i in model.ss)
model.cost = pe.Objective(rule=cost_rule)
# CONSTRAINTS
# each x_i bigger than d_i
def lb_rule(model, i):
return (model.x[i] >= model.d[i])
model.state_bound = pe.Constraint(model.ss, rule=lb_rule)
# sum of x == P_tot
def sum_rule(model):
return (sum(model.x[i] for i in model.ss) == model.b)
model.state_sum = pe.Constraint(rule=sum_rule)
## TEST
# define model dimension
model_dimension = 3
model.n = model_dimension
# create instance
instance = model.create_instance()
# set d
for i in range(model_dimension):
getattr(instance, 'd')[i] = i
# set x_hat
xh = (10,1,-100)
for i in range(model_dimension):
getattr(instance, 'x_hat')[i] = xh[i]
# set b
instance.b = 10
# solve
solver = pe.SolverFactory('ipopt')
result = solver.solve(instance)
instance.display()
I'm generating a form based on meta data in the following manner
class MeasureForm(forms.Form):
def __init__(self,measure_id,*args,**kwrds):
super(MeasureForm,self).__init__(*args,**kwrds)
m = Measure.objects.get(pk=measure_id);
if (m):
self.fields["measure_id"] = forms.IntegerField(initial = m.id , widget=forms.HiddenInput())
for mp in MeasureParameters.objects.filter(measure = m):
# get the NVL'ed copy of the parameter
p = mp.get_parameter_for_measure()
if not p.is_modifiable:
# the file has a constant value
if (p.values and p.default): # contant must have both values and default index
value_ = p.values[p.values.keys()[p.default-1]];
self.fields[p.name] = forms.IntegerField(label = p.description , initial = value_);
self.fields[p.name].widget.attrs['readonly'] = True;
else:
raise Exception("Parameter set as unmodifiable but has no value. [measure: %s, parameter: %s, measureparameter %s]"
% (measure_id , p.id , mp.__unicode__()))
elif (p.values):
# convert hstore dict to list of tuples for the choices to read
values_ = [(v, k) for k, v in p.values.iteritems()];
# set default if exists , else take the first item
default_ = values_[p.default-1][0] if p.default else values_[0][0]
self.fields[p.name] = forms.ChoiceField(label = p.description , choices = values_ , initial = default_)
else:
self.fields[p.name] = forms.IntegerField(label = p.description)
if (not p.is_visible):
self.fields[p.name].widget = forms.HiddenInput()
else:
raise Exception ("Could not find measure. [measure %s]" % (measure_id));
This is my view
def index(request,measure_id = None):
owners = Owner.objects.all()
form = None
result = None
title = None
# handle the form
if request.method == 'POST': # the form has been submitted
form = MeasureForm(request.POST)
if form.is_valid(): # All validation rules pass
result = 100;
else:
if (measure_id):
title = Measure.objects.get(pk=measure_id).name;
# make an unbound form
form = MeasureForm(measure_id)
return render(request, 'calc/index.html' ,
{'owners' : owners,
'form' : form ,
'title' : title ,
'result' : result})
I followed this tutorial.
I'm running into a problem when i issue submit back to the index view. i get the following error
int() argument must be a string or a number, not 'QueryDict'
I can see in the dump that it's fail in this line
m = Measure.objects.get(pk=measure_id);
So I looked around and found this post . I tried to change the call to the form like this
form = MeasureForm(request.POST, measure_id = request.POST.get('measure_id'))
and i got the following error
__init__() got multiple values for keyword argument 'measure_id'
My first question is - Why is django executing the init again. a form object was already created - the fields just need to get bound form the post data.
Second , how can I overcome this ?
**django newbie. Thanks.
Remove measure_id from init method and instead use kwrds['measure_id'] to retrieve it inside, then use them appropriately for keyword argument:
MeasureForm(request.POST, measure_id=measure_id)
MeasureForm(measure_id=measure_id)