I am trying to populate TIMESTAMP_TZ column in Snowflake using the Informatica IICS from the Taskflow. In the taskflow I am using function fn:current-dateTime() which I am trying then in the mapping to convert to TIMESTAMP_TZ using the to_date() function but I am getting error
MAPPING> TE_7002 [2023-01-17 11:07:50.987] Transformation stopped due to a fatal error in the mapping. The expression [To_Date($$inp_load_dttm)] contains the following errors [<<PM Parse Error>> [$$inp_load_dttm)]: Invalid date value string: >>>2023-01-17T16:06:58.019Z<<<.
... To_Date(>>>>$$inp_load_dttm<<<<)].
MANAGER> PETL_24005 [2023-01-17 11:07:50.988] Starting post-session tasks. : (Tue Jan 17 11:07:50 2023)
fn:current-dateTime() returns date (ref 1). to_date expects a string (ref 2). Here in this case you should not try to conver a date to date - it should simply work. If what you need is just the date, without the time, try using fn:current-date().
Related
I'm developing a data fusion pipeline. It contains a wrangler node where I'm trying to create a new field that will contain the system date in timestamp format (yyyy-MM-dd'T'HH-mm-ss).
I've tried using the sentence:
set-column :sysdate (${logicalStartTime(yyyy-MM-dd'T'HH-mm-ss)})
But I receive the error:
Caused by: io.cdap.wrangler.api.DirectiveParseException: Error encountered while parsing 'set-column' : Error encountered while compiling '( 2022 -12-01T16-29-32 ) ' at line '1' and column '14'. Make sure a valid jexl transformation is provided.
Which would be the correct sentence?
I've tried:
set-column :sysdate (${logicalStartTime(yyyy-MM-ddHH-mm-ss)})
Which will result in something like "1877", as it substracts the numbers, and also tried:
set-column :sysdate (${logicalStartTime(yyyyMMddHHmmss)})
but the format isn't correct and can only be written if the field is a String.
You have the correct method, just incorrect syntax. The syntax you are looking for is set-column :sysdate ${logicalStartTime(yyyy-MM-dd'T'HH-mm-ss)}, you have to remove (). Then you can convert the string in datetime pattern in this format parse-as-datetime :sysdate "yyyy-MM-dd'T'HH-mm-ss".
I seem to keep getting 'conversion' errors when calling and passing values into a stored procedure....
Im trying to write a dead simple procedure to insert data collected from a form here:
void orderform::on_btnSave_clicked()
{
dbhandler().TestEntry(ui->edtOrderDate->date().toString());
};
this is the code in the dbhandler.cpp:
void dbhandler::TestEntry(QString PltfrmOrderDate)
{
QSqlQuery query;
query.prepare("{CALL [DSDB].[dbo].[Test](PltfrmOrderDate)}");
query.bindValue("#PltfrmOrderDate", PltfrmOrderDate);
if(!query.exec())
qDebug() << query.lastError();
}
** Just a note, I know this is a date and that i can use QDate but that also returns a conversion error - Please see below **
The stored procedure:
USE [DSDB]
GO
/****** Object: StoredProcedure [dbo].[Test] Script Date: 1/20/2022 7:37:33 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: <Author,,Name>
-- Create date: <Create Date,,>
-- Description: <Description,,>
-- =============================================
ALTER PROCEDURE [dbo].[Test]
-- Add the parameters for the stored procedure here
#PltfrmOrderDate nvarchar
AS
BEGIN
INSERT INTO [dbo].[OrdersTable]
(
PltfrmOrderDate
)
VALUES
(
#PltfrmOrderDate
)
END
So, when i run my code like this I get the following error:
QODBCResult::exec: Unable to execute statement: "[Microsoft][ODBC SQL Server Driver][SQL Server]Conversion failed when converting date and/or time from character string."
QSqlError("241", "QODBC3: Unable to execute statement", "[Microsoft][ODBC SQL Server Driver][SQL Server]Conversion failed when converting date and/or time from character string.")
now to address the before sated QDate issue, That was how i had it innitially with the code as follows:
void orderform::on_btnSave_clicked()
{
dbhandler().TestEntry(ui->edtOrderDate->date());
};
dbhandler.cpp:
void dbhandler::TestEntry(QDate PltfrmOrderDate)
{
QSqlQuery query;
query.prepare("{CALL [DSDB].[dbo].[Test](PltfrmOrderDate)}");
query.bindValue("#PltfrmOrderDate", PltfrmOrderDate);
if(!query.exec())
qDebug() << query.lastError();
}
and finally the Stored Procedure:
USE [DSDB]
GO
/****** Object: StoredProcedure [dbo].[Test] Script Date: 1/20/2022 7:37:33 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: <Author,,Name>
-- Create date: <Create Date,,>
-- Description: <Description,,>
-- =============================================
ALTER PROCEDURE [dbo].[Test]
-- Add the parameters for the stored procedure here
#PltfrmOrderDate date
AS
BEGIN
INSERT INTO [dbo].[OrdersTable]
(
PltfrmOrderDate
)
VALUES
(
#PltfrmOrderDate
)
END
and this resulted in the following error:
QODBCResult::exec: Unable to execute statement: "[Microsoft][ODBC SQL Server Driver][SQL Server]Error converting data type nvarchar to date."
QSqlError("8114", "QODBC3: Unable to execute statement", "[Microsoft][ODBC SQL Server Driver][SQL Server]Error converting data type nvarchar to date.")
What i dont understand is why is there conversions happening.... I'm passing a YYYY-MM-DD date into a YYYY-MM-DD date format, and QT documents seems to suggest that the format/data type is supported.
I ran into the same issue using Double form QT into Money in SQL, Int from QT into Int in SQL and even Bool in QT into Bit in SQL.
I'm sure its something small that I'm missing but its the first time I'm trying to use this and cant seem to find the issue!
For the first example, never declare nvarchar without length. As a stored procedure parameter, that becomes nvarchar(1), which means any string passed in gets truncated silently to a single character. An easy way to debug this yourself would have been to call the procedure manually with the same arguments and simply PRINT #PltfrmOrderDate; instead of INSERT. This will have printed 2 for any date this century.
Always use YYYYMMDD since YYYY-MM-DD is not a safe format in SQL Server - it can be interpreted as YYYY-DD-MM depending on your language settings and any data type it hits via implicit or explicit conversion. It may be worth showing us exactly how C++ is presenting PltfrmOrderDate to SQL Server.
Lots more background on using date and time data in SQL Server here: Dating Responsibly
So I have managed to resolve this by using QDateTime instead of QDate.
Here the input has changed from date to datetime
void orderform::on_btnSave_clicked()
{
dbhandler().TestEntry(ui->edtOrderDate->dateTime());
};
Function parameter changed from QDate to QDateTime, as well as the ":" for the parameter in the query.prepare statement and the query.bindValue statement was missing
void dbhandler::TestEntry(QDateTime PltfrmOrderDate)
{
QSqlQuery query;
query.prepare("{CALL [DSDB].[dbo].[Test](:PltfrmOrderDate)}");
query.bindValue(":PltfrmOrderDate", PltfrmOrderDate);
if(!query.exec())
qDebug() << query.lastError();
}
In the SP all that had to change was date to datetime
USE [DSDB]
GO
/****** Object: StoredProcedure [dbo].[Test] Script Date: 1/20/2022 7:37:33 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: <Author,,Name>
-- Create date: <Create Date,,>
-- Description: <Description,,>
-- =============================================
ALTER PROCEDURE [dbo].[Test]
-- Add the parameters for the stored procedure here
#PltfrmOrderDate datetime
AS
BEGIN
INSERT INTO [dbo].[OrdersTable]
(
PltfrmOrderDate
)
VALUES
(
#PltfrmOrderDate
)
END
I'm trying to execute a stored procedure in Power BI, where a user can input a data value in the parameter.
let
Source = Sql.Database("nj1tstsql12", "GI_MASTER", [Query="exec
dal_ANALYTICS_TIME_SERIES_SUMMARY_BY_SECTOR " &AsOfDate])
in
Source
I keep on receiving this error:
Expression.Error: We cannot apply operator & to types Text and DateTime.
Details:
Operator=&
Left=exec dal_ANALYTICS_TIME_SERIES_SUMMARY_BY_SECTOR
Right=12/10/2018 12:00:00 AM
Thanks!
As the others said on comments, you need to change the data into Text format. You made a syntax error when you did, that's why you got another error. Please try below:
let
Source = Sql.Database("nj1tstsql12",
"GI_MASTER",
[Query="exec dal_ANALYTICS_TIME_SERIES_SUMMARY_BY_SECTOR '" & Date.ToText(AsOfDate) & "'"])
in
Source
I want to query a HDF5-file. I do
df.to_hdf(pfad,'df', format='table')
to write the dataframe on disc.
To read I use
hdf = pandas.HDFStore(pfad)
I have a list that contains numpy.datetime64 values called expirations and try to read the portion of the hd5 table into a dataframe, that has values between expirations[1] and expirations[0] in column "expiration". Column expiration entries have the format Timestamp('2002-05-18 00:00:00').
I use the following command:
df = hdf.select('df',
where=['expiration<expiration[1]','expiration>=expirations[0]'])
However, this fails and produces a value error:
ValueError: The passed where expression: [expiration=expirations[0]]
contains an invalid variable reference
all of the variable refrences must be a reference to
an axis (e.g. 'index' or 'columns'), or a data_column
The currently defined references are: index,columns
Can you try this code:
df = hdf.select('df', where='expiration < expirations[1] and expiration >= expirations[0]')
or, as a query:
df = hdf.query('expiration < #expirations[1] and expiration >= #expirations[0]')
Not sure which one fits best your case, I noticed you are trying to use 'where' to filter rows, without a string or a list, does it make sense ?
I have some code that reads a datetime from a sqlite database, the datetime is returned as a string. when I try to convert it to a date using QDateTime::FromString it returns an invalid date. Below is the time as returned from the database and conversion.
Why is this failing to parse?
// -this is the value returned from the DB currentServerTime=2012-01-17 19:20:27.0
QString format("yyyy/MM/dd hh:mm:ss");
QString qCurrentServerTime(currentServerTime);
now = QDateTime::fromString(qCurrentServerTime, format);
No expert in QT, but if QDateTime::fromString() works as one would (reasonably) expect and according to this, you're not using the correct pattern.
You indicate the string read from the sqllite database is like "2012-01-17 19:20:27.0", then your format should be like yyyy-MM-dd HH:mm:ss.z.
In detail:
Your separator should by '-' not '/' (as you show in the example)
The time seems to be in 24 hours format (19 -> 7 p.m.) (so use HH instead of hh)
You have one digit for milliseconds, so add .z.