I have three variables stored as number, string and string, as shown below.
load_id = 100
t_date = '2014-06-18'
p_date = '19-JUN-14 10.51.45.378196'
I would like to insert them into a SQL Server table using Python 2.7. The SQL Server table structure is as follows
load_id = float
t_date = date
p_date = timestamp
In Oracle, we tend to use TO_DATE or TO_TIMESTAMP to convert the string to DATE or TIMESTAMP field.
I would like to know how I can do similar conversion while inserting into an SQL Server table.
Thanks in advance.
convert with :
import datetime
import calendar
thedate=datetime.datetime.strptime(p_date,'%d-%b-%y %H.%M.%S.%f')
thetimestamp=calendar.timegm(thedate.utctimetuple())
https://community.toadworld.com/platforms/sql-server/b/weblog/archive/2012/04/18/convert-datetime-to-timestamp
DECLARE #DateTimeVariable DATETIME
SELECT #DateTimeVariable = GETDATE()
SELECT #DateTimeVariable AS DateTimeValue,
CAST(#DateTimeVariable AS TIMESTAMP) AS DateTimeConvertedToTimestampCAST
SELECT CAST(CAST(#DateTimeVariable AS TIMESTAMP) AS DATETIME) AS
TimestampToDatetime
Do the conversion with SQL instead of trying to get Python to match the SQL format.
Neither format matches yours, however the DATETIME type should be adequate.
Related
In Power BI I am using a native query to return a table with a datetime field I would like that field to become a date field without using Power Query because I am using a native query that does not support query folding, how can I change the datetime field into a date field.
points of consideration: using a oracle database, have already tried using trunc which still returns a datetime field but with the time part trunced to 00:00:00
p.s could create it in my datasource but for the purpose of the report may be a bit excessive
I wrote a query for one of my Big Query table called historical and I would like to copy the result of this query into a new Big Query table called historical_recent. I have difficulties to figure out how to do this operation with Python. Right now, I am able to execute my query and get the expected result:
SELECT * FROM gcp-sandbox.dailydev.historical WHERE (date BETWEEN '2015-11-05 00:00:00' AND '
2015-11-07 23:00:00')
I am also able to copy a my Big Query table without making any changes with this script:
from google.cloud import bigquery
client = bigquery.Client()
job = client.copy_table(
'gcp-sandbox.dailydev.historical',
'gcp-sandbox.dailydev.historical_copy')
How can I combine both using Python?
You can use INSERT statement as in below example
INSERT `gcp-sandbox.dailydev.historical_recent`
SELECT *
FROM `gcp-sandbox.dailydev.historical`
WHERE date BETWEEN '2015-11-05 00:00:00' AND '2015-11-07 23:00:00'
Using Python to save your query result.
from google.cloud import bigquery
client = bigquery.Client()
# Target table to save results
table_id = "gcp-sandbox.dailydev.historical_recent"
job_config = bigquery.QueryJobConfig(
allow_large_results=True,
destination=table_id,
use_legacy_sql=True
)
sql = """
SELECT * FROM gcp-sandbox.dailydev.historical
WHERE (date BETWEEN '2015-11-05 00:00:00' AND '2015-11-07 23:00:00')
"""
query = client.query(sql, job_config=job_config)
query.result()
print("Query results loaded to the table {}".format(table_id))
This example is based on the Google documentation.
I'm having a problem converting this varchar into an AWS Athena datetime
"2012-06-10T11:33:25.202615+00:00"
I've tried some like date_parse(pickup, %Y-%m-%dT%T)
I want to make a view like this using the timestamp already converted
CREATE OR REPLACE VIEW vw_ton AS
(
SELECT
id,
date_parse(pickup, timestamp) as pickup,
date_parse(dropoff, timestamp) as dropoff,
FROM "table"."ton"
)
You can use parse_datetime() function:
presto> SELECT parse_datetime('2012-06-10T11:33:25.202615+00:00', 'YYYY-mm-dd''T''HH:mm:ss.SSSSSSZ');
_col0
-----------------------------
2012-01-10 11:33:25.202 UTC
(1 row)
(Verified on Presto 339)
I am trying to use Athena to query some data I have stored in an s3 bucket in parquet format. I have field called datetime which is defined as a date data type in my AWS Glue Data Catalog.
When I try running the following query in Athena, I get the error below:
SELECT DISTINCT datetime
FROM "craigslist"."pq_craigslist_rental_data_parquet"
WHERE datetime > '2018-09-14'
ORDER BY datetime DESC;
And the error:
Your query has the following error(s):
SYNTAX_ERROR: line 3:16: '>' cannot be applied to date, varchar(10)
What am I doing wrong here? How can I properly filter this data by date?
the string literal that you provide has to be casted to a date, in order to compare to a date.
where datetime = date('2019-11-27')
its having issue with the string literal used for date filter. Use WHERE datetime > date '2018-09-14'
from_iso8601_date or date should work.
SELECT DISTINCT datetime
FROM "craigslist"."pq_craigslist_rental_data_parquet"
WHERE datetime > from_iso8601_date('2018-09-14')
ORDER BY datetime DESC;
both return a proper date object.
SELECT typeof(from_iso8601_date('2018-09-14'))
Bit late here, but I had the same issue and the only workaround I have found is:
WHERE datetime > (select date '2018-09-14')
Is it possible to retrieve a List<Date> instead of a List<SomeBean> using Ebean ?
For example, I have this model:
Days(id, name, day);
I'd like to do something like:
List<Date> dates = Ebean.createQuery(Date.class, "SELECT day FROM days").findList();
Of course, this doesn't work and returns this:
PersistenceException: java.util.Date is NOT an Entity Bean registered with this server?
How can I do that?
you can use sqlQuery see SqlQuery in Ebean :
The database retrieves time and date data with a type LocalDateTime
String sql = "select day from days";
List<SqlRow> row = DB.sqlQuery(sql).findList();
List<LocalDateTime> days = row.stream().map(r->(LocalDateTime) r.get("day")).collect(Collectors.toList());