I have a number stored in mongo as 15000.245263 with 6 numbers after decimal point but when I use pymongo to get this number I got 15000.24. Is the pymongo reduced the precision of float?
I can't reproduce this. In Python 2.7.13 on my Mac:
>>> from pymongo import MongoClient
>>> c = MongoClient().my_db.my_collection
>>> c.delete_many({}) # Delete all documents
>>> c.insert_one({'x': 15000.245263})
>>> c.find_one()
{u'x': 15000.245263, u'_id': ObjectId('59525d32a08bff0800cc72bd')}
The retrieved value of "x" is printed the same as it was when I entered it.
This could happen if you trying to print out a long float value, and i think it is not related to mongodb.
>>> print 1111.1111
1111.1111
>>> print 1111111111.111
1111111111.11
>>> print 1111111.11111111111
1111111.11111
# for a timestamp
>>> import time
>>> now = time.time()
>>> print now
1527160240.06
For python2.7.10 it will just display 13 character(for my machine), if you want to display the whole value, use a format instead, like this:
>>> print '%.6f' % 111111111.111111
111111111.111111
And this is just a display problem, the value of the variable will not be affected.
>>> test = 111111111.111111 * 2
>>> test
222222222.222222
>>> print test
222222222.222
Related
I have a pyomo ConcreteModel() which I solve repeatedly within another stochastic optimization process whereas one or more parameters are changed on the model.
The basic process can be described as follows:
# model is created as a pyomo.ConcreteModel()
for i in range(0, 10):
# change some parameter on the model
opt = SolverFactory('gurobi', solver_io='lp')
# how can I check here if the changed model/lp-file is valid?
results = opt.solve(model)
Now I get an error for some cases where the model and LP file (see gist) seems to contain NaN values:
ERROR: Solver (gurobi) returned non-zero return code (1)
ERROR: Solver log: Academic license - for non-commercial use only Error
reading LP format file /tmp/tmp8agg07az.pyomo.lp at line 1453 Unrecognized
constraint RHS or sense Neighboring tokens: " <= nan c_u_x1371_: +1 x434
<= nan "
Unable to read file Traceback (most recent call last):
File "<stdin>", line 5, in <module> File
"/home/cord/.anaconda3/lib/python3.6/site-
packages/pyomo/solvers/plugins/solvers/GUROBI_RUN.py", line 61, in
gurobi_run
model = read(model_file)
File "gurobi.pxi", line 2652, in gurobipy.read
(../../src/python/gurobipy.c:127968) File "gurobi.pxi", line 72, in
gurobipy.gurobi.read (../../src/python/gurobipy.c:125753)
gurobipy.GurobiError: Unable to read model Freed default Gurobi
environment
Of course, the first idea would be to prevent setting these NaN-values. But I don't know why they occur anyhow and want to figure out when the model breaks due to a wrong structure caused by NaNs.
I know that I can catch the solver status and termination criterion from the SolverFactory() object. But the error obviously occurs somewhere before the solving process due to the invalid changed values.
How can I can catch these kinds of errors for different solvers before solving i. e. check if the model/lp-file is valid before applying a solver? Is there some method e.g. check_model() which delivers True or False if the model is (not) valid or something similar?
Thanks in advance!
If you know that the error is taking place when the parameter values are being changed, then you could test to see whether the sum of all relevant parameter values is a valid number. After all, NaN + 3 = NaN.
Since you are getting NaN, I am going to guess that you are importing parameter values using Pandas from an Excel spreadsheet? There is a way to convert all the NaNs to a default number.
Code example for parameter check:
>>> from pyomo.environ import *
>>> m = ConcreteModel()
>>> m.p1 = Param(initialize=1)
>>> m.p2 = Param(initialize=2)
>>> for p in m.component_data_objects(ctype=Param):
... print(p.name)
...
p1
p2
>>> import numpy
>>> m.p3 = Param(initialize=numpy.nan)
>>> import math
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
True
Indexed, Mutable Parameters:
>>> from pyomo.environ import *
>>> m = ConcreteModel()
>>> m.i = RangeSet(2)
>>> m.p = Param(m.i, initialize={1: 1, 2:2}, mutable=True)
>>> import math
>>> import numpy
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
False
>>> m.p[1] = numpy.nan
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
True
I have an app that want to serialize only classes/functions which are:
no Python primitive data type
no Numpy data type
no pandas data type.
So, is it possible to filter object to be saved in dill ?
(filter by loop on the type)
Thanks,
While this is not a complete solution (i.e. you'd probably want to include more of the modules with pandas data types, numpy data types… and also you might want to be more selective for the built-in types by filtering by type instead of module)… I think it sort of gets you what you want.
>>> import dill
>>> import numpy
>>> import pandas
>>>
>>> target = numpy.array([1,2,3])
>>> dill.dumps(target) if not dill.source.getmodule(type(target)) in [numpy, pandas.core.series, dill.source.getmodule(int)] else None
>>>
>>> target = [1,2,3]
>>> dill.dumps(target) if not dill.source.getmodule(type(target)) in [numpy, pandas.core.series, dill.source.getmodule(int)] else None
>>>
>>> target = lambda x:x
>>> dill.dumps(target) if not dill.source.getmodule(type(target)) in [numpy, pandas.core.series, dill.source.getmodule(int)] else None
>>>
>>> class Foo(object):
... pass
...
>>> target = Foo()
>>> dill.dumps(target) if not dill.source.getmodule(type(target)) in [numpy, pandas.core.series, dill.source.getmodule(int)] else None
'\x80\x02cdill.dill\n_create_type\nq\x00(cdill.dill\n_load_type\nq\x01U\x08TypeTypeq\x02\x85q\x03Rq\x04U\x03Fooq\x05h\x01U\nObjectTypeq\x06\x85q\x07Rq\x08\x85q\t}q\n(U\r__slotnames__q\x0b]q\x0cU\n__module__q\rU\x08__main__q\x0eU\x07__doc__q\x0fNutq\x10Rq\x11)\x81q\x12}q\x13b.'
>>>
However, if you are asking if dill has such a filtering method, then the answer is no.
I am currently using Python 2.7.5 on Aix 5.1 with cx_Oracle version 5.2 to connect to Oracle 12c
I am trying to execute a SQL query, and put its output in a csv file using the csv module. The query I am running is:
Select 1.563/100, 0.38/100 from dual; - - simplified query
However the output in file is:
0.015629999999999998,0.0038
When I expect it to be
0.01563,0.0038
After doing some research, I believe this is because floating point numbers are represented in Binary Base 2.
But I don't know how to resolve this?
I also tried
from __future__ import division
But it did not help.
function ROUND is your friend
SELECT ROUND(3.1415926,4),ROUND(3.1415926,5) FROM DUAL;
ROUND(3.1415926,4) ROUND(3.1415926,5)
------------------ ------------------
3.1416 3.14159
or, in Python:
print round (3.1415926, 4)
print round (3.1415926, 5)
3.1416
3.14159
Thank You Zsigmond Lőrinczy.
It worked, by using a to_char(round())
>>> import cx_Oracle
>>> con = cx_Oracle.connect(xxx/xxx#xxx)
>>> cur = con.cursor()
>>> cur.execute("select 1.563/100, round(1.563/100,5), to_char(round(1.563/100,5)) from dual")
>>> l_result = cur.fetchall()
>>> l_result
[(0.015629999999999998, 0.015629999999999998, '0.01563')]
Suppose model is a defaultdict, and num is a set
>>> model
>>> defaultdict(<function <lambda> at 0x11076f758>, {1: 3, 2: 2, 4: 1})
>>> num
>>> {1, 2, 3, 4, 5, 6}
I want to get the item from num that has maximum value in model, and the following code works fine in Python2
>>> # python 2.7.6
>>> max(num, key=model.get)
>>> 1
But it doesn't work in Python3,
>>> # python 3.3.3
>>> max(num, key=model.get)
>>> TypeError: unorderable types: NoneType() > int()
I can use max(num, key=lambda k:model[k]) to get it work in Python3, but if the item in num is not in the model, it will be added. This will modify model.
I am wondering why model.get doesn't work in Python3, and how can I do it without modifying model.
Use key=lambda x: model.get(x, 0).
defaultdict.get by default returns None if the item is not found. Python 2 allows ordered comparisons (like less-than and greater-than) on different types, but Python 3 doesn't. When Python 3 tries to find the max, it tries to see if the value for one key is greater than another. If one of the values is None, it fails with the error you saw. The solution is to make your key function return zero instead of None for missing values.
could u please help me to Find correlation for these two lists importing stats-model in python.
a=[1.0,2.0,3.0,2.0]
b=[789.0,786.0,788.0,785.0]
using some built-in functions
>>> import numpy as np
>>> a = np.array([1.0,2.0,3.0,2.0])
>>> b = np.array([789.0,786.0,788.0,785.0])
>>> np.corrcoef(a,b)
array([[ 1. , -0.2236068],
[-0.2236068, 1. ]])
Just use indexing to extract the right one:
np.corrcoef(a,b)[0,1]