I am currently using Python 2.7.5 on Aix 5.1 with cx_Oracle version 5.2 to connect to Oracle 12c
I am trying to execute a SQL query, and put its output in a csv file using the csv module. The query I am running is:
Select 1.563/100, 0.38/100 from dual; - - simplified query
However the output in file is:
0.015629999999999998,0.0038
When I expect it to be
0.01563,0.0038
After doing some research, I believe this is because floating point numbers are represented in Binary Base 2.
But I don't know how to resolve this?
I also tried
from __future__ import division
But it did not help.
function ROUND is your friend
SELECT ROUND(3.1415926,4),ROUND(3.1415926,5) FROM DUAL;
ROUND(3.1415926,4) ROUND(3.1415926,5)
------------------ ------------------
3.1416 3.14159
or, in Python:
print round (3.1415926, 4)
print round (3.1415926, 5)
3.1416
3.14159
Thank You Zsigmond Lőrinczy.
It worked, by using a to_char(round())
>>> import cx_Oracle
>>> con = cx_Oracle.connect(xxx/xxx#xxx)
>>> cur = con.cursor()
>>> cur.execute("select 1.563/100, round(1.563/100,5), to_char(round(1.563/100,5)) from dual")
>>> l_result = cur.fetchall()
>>> l_result
[(0.015629999999999998, 0.015629999999999998, '0.01563')]
Related
I have a pyomo ConcreteModel() which I solve repeatedly within another stochastic optimization process whereas one or more parameters are changed on the model.
The basic process can be described as follows:
# model is created as a pyomo.ConcreteModel()
for i in range(0, 10):
# change some parameter on the model
opt = SolverFactory('gurobi', solver_io='lp')
# how can I check here if the changed model/lp-file is valid?
results = opt.solve(model)
Now I get an error for some cases where the model and LP file (see gist) seems to contain NaN values:
ERROR: Solver (gurobi) returned non-zero return code (1)
ERROR: Solver log: Academic license - for non-commercial use only Error
reading LP format file /tmp/tmp8agg07az.pyomo.lp at line 1453 Unrecognized
constraint RHS or sense Neighboring tokens: " <= nan c_u_x1371_: +1 x434
<= nan "
Unable to read file Traceback (most recent call last):
File "<stdin>", line 5, in <module> File
"/home/cord/.anaconda3/lib/python3.6/site-
packages/pyomo/solvers/plugins/solvers/GUROBI_RUN.py", line 61, in
gurobi_run
model = read(model_file)
File "gurobi.pxi", line 2652, in gurobipy.read
(../../src/python/gurobipy.c:127968) File "gurobi.pxi", line 72, in
gurobipy.gurobi.read (../../src/python/gurobipy.c:125753)
gurobipy.GurobiError: Unable to read model Freed default Gurobi
environment
Of course, the first idea would be to prevent setting these NaN-values. But I don't know why they occur anyhow and want to figure out when the model breaks due to a wrong structure caused by NaNs.
I know that I can catch the solver status and termination criterion from the SolverFactory() object. But the error obviously occurs somewhere before the solving process due to the invalid changed values.
How can I can catch these kinds of errors for different solvers before solving i. e. check if the model/lp-file is valid before applying a solver? Is there some method e.g. check_model() which delivers True or False if the model is (not) valid or something similar?
Thanks in advance!
If you know that the error is taking place when the parameter values are being changed, then you could test to see whether the sum of all relevant parameter values is a valid number. After all, NaN + 3 = NaN.
Since you are getting NaN, I am going to guess that you are importing parameter values using Pandas from an Excel spreadsheet? There is a way to convert all the NaNs to a default number.
Code example for parameter check:
>>> from pyomo.environ import *
>>> m = ConcreteModel()
>>> m.p1 = Param(initialize=1)
>>> m.p2 = Param(initialize=2)
>>> for p in m.component_data_objects(ctype=Param):
... print(p.name)
...
p1
p2
>>> import numpy
>>> m.p3 = Param(initialize=numpy.nan)
>>> import math
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
True
Indexed, Mutable Parameters:
>>> from pyomo.environ import *
>>> m = ConcreteModel()
>>> m.i = RangeSet(2)
>>> m.p = Param(m.i, initialize={1: 1, 2:2}, mutable=True)
>>> import math
>>> import numpy
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
False
>>> m.p[1] = numpy.nan
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
True
I have a number stored in mongo as 15000.245263 with 6 numbers after decimal point but when I use pymongo to get this number I got 15000.24. Is the pymongo reduced the precision of float?
I can't reproduce this. In Python 2.7.13 on my Mac:
>>> from pymongo import MongoClient
>>> c = MongoClient().my_db.my_collection
>>> c.delete_many({}) # Delete all documents
>>> c.insert_one({'x': 15000.245263})
>>> c.find_one()
{u'x': 15000.245263, u'_id': ObjectId('59525d32a08bff0800cc72bd')}
The retrieved value of "x" is printed the same as it was when I entered it.
This could happen if you trying to print out a long float value, and i think it is not related to mongodb.
>>> print 1111.1111
1111.1111
>>> print 1111111111.111
1111111111.11
>>> print 1111111.11111111111
1111111.11111
# for a timestamp
>>> import time
>>> now = time.time()
>>> print now
1527160240.06
For python2.7.10 it will just display 13 character(for my machine), if you want to display the whole value, use a format instead, like this:
>>> print '%.6f' % 111111111.111111
111111111.111111
And this is just a display problem, the value of the variable will not be affected.
>>> test = 111111111.111111 * 2
>>> test
222222222.222222
>>> print test
222222222.222
Surely I am missing something obvious - but I am baffled by this result:
Environment:
Ubuntu 16.04.1 LTS
Python 2.7.12
pandas 0.18.1
CSV File:
Date,Open,High,Low,Close,Volume
12-Aug-16,107.78,108.44,107.78,108.18,18660434
11-Aug-16,108.52,108.93,107.85,107.93,27484506
10-Aug-16,108.71,108.90,107.76,108.00,24008505
Code:
import pandas as pd
aapl = pd.DataFrame.from_csv('aapl.csv',index_col=None)
print aapl.columns
print aapl.Low.dtype
print aapl['Low'].dtype
# Fails - KeyError
print aapl['Date'].dtype
Output:
Index([u'Date', u'Open', u'High', u'Low', u'Close', u'Volume'], dtype='object')
float64
float64
KeyError: 'Date'
The mystery to me is that 'Date' appears in the columns list, but I cannot address the column. What am I missing?
To close this out, #Boud answered the question. Printing the columns with repr(), i.e. print(repr(aapl.columns[0]) showed encoded characters in the key string which prevented it from being found.
I'm attempting to output the result into a pandas data frame. When I print the data frame, the object values appear correct, but when I use the to_csv function on the data frame, my csv output has only the first character for every string/object value.
df = pandas.DataFrame({'a':[u'u\x00s\x00']})
df.to_csv('test.csv')
I've also tried the following addition to the to_csv function:
df.to_csv('test_encoded.csv', encoding= 'utf-8')
But am getting the same results:
>>> print df
a
0 us
(output in csv file)
u
For reference, I'm connecting to a Vertica database and using the following setup:
OS: Mac OS X Yosemite (10.10.5)
Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, Sep 15 2015,
14:29:08)
pyodbc 3.0.10
pandas 0.16.2
ODBC: Vertica ODBC 6.1.3
Any help figuring out how to pass the entire object string using the to_csv function in pandas would be greatly appreciated.
I was facing the same problem and found this post UTF-32 in Python
To fix your problem, I believe that you need to replace all '\x00' by empty. I managed to write the correct CSV with the code below
fixer = dict.fromkeys([0x00], u'')
df['a'] = df['a'].map(lambda x: x.translate(fixer))
df.to_csv('test.csv')
To solve my problem with Vertica I had to change the encoding to UTF-16 in the file /Library/Vertica/ODBC/lib/vertica.ini with the configuration below
[Driver]
ErrorMessagesPath=/Library/Vertica/ODBC/messages/
ODBCInstLib=/usr/lib/libiodbcinst.dylib
DriverManagerEncoding=UTF-16
Best regards,
Anderson Neves
could u please help me to Find correlation for these two lists importing stats-model in python.
a=[1.0,2.0,3.0,2.0]
b=[789.0,786.0,788.0,785.0]
using some built-in functions
>>> import numpy as np
>>> a = np.array([1.0,2.0,3.0,2.0])
>>> b = np.array([789.0,786.0,788.0,785.0])
>>> np.corrcoef(a,b)
array([[ 1. , -0.2236068],
[-0.2236068, 1. ]])
Just use indexing to extract the right one:
np.corrcoef(a,b)[0,1]