How do you use Informatica to load data into a target table whose name is a SQL reserved keyword?
I have a situation where I am trying to use Informatica to populate a table called Union which is failing with the following error:
SQL Server Message: Incorrect syntax near the keywork 'Union'
Database driver error...
Function Name : Execute Multiple
SQL Stmt : INSERT INTO UNION (UnionCode, UnionName, etc )
I have been told that changing the database properties to use quoted identifier would solve this problem; however, I have tried that and it only appears to work for sources, not targets.
And before anyone states the obvious - I cannot change the name of the target table.
Can you please try overriding the table name in session properties as "Union" with the quotes.
Load your data in a table with valid name ,having same structure as union .
And in Post Sql of that target, you can rename the table with whatever name required .
Ex .
Click on the target (XUnion) ,
go to Post Sql and put statement below --
RENAME XUnion to 'UNION' ;
If any table name or column name contains a database reserved word, such as MONTH or YEAR, the session fails with database errors when the Integration Service executes SQL against the database. You can create and maintain a reserved words file, reswords.txt, in the server/bin directory. When the Integration Service initializes a session, it searches for reswords.txt. If the file exists, the Integration Service places quotes around matching reserved words when it executes SQL against the database.
Use the following rules and guidelines when working with reserved words.
The Integration Service searches the reserved words file when it generates SQL to connect to source, target, and lookup databases.
If you override the SQL for a source, target, or lookup, you must enclose any reserved word in quotes.
You may need to enable some databases, such as Microsoft SQL Server and Sybase, to use SQL-92 standards regarding quoted identifiers. Use connection environment SQL to issue the command. For example, use the following command with Microsoft SQL Server:
SET QUOTED_IDENTIFIER ON
Sample reswords.txt File
To use a reserved words file, create a file named reswords.txt and place it in the server/bin directory. Create a section for each database that you need to store reserved words for. Add reserved words used in any table or column name. You do not need to store all reserved words for a database in this file. Database names and reserved words in reswords.txt are not case sensitive.
Following is a sample reswords.txt file:
[Teradata] MONTH DATE INTERVAL [Oracle] OPTION START [DB2] [SQL Server] CURRENT [Informix] [ODBC] MONTH [Sybase]
Related
I am running this workflow which is supposed to load a file, but i am having the following error messages.
Internal error. Failed to initialize transformation [LKP_FILE_LOAD_ID{{BLD}}]. Contact Informatica Global Customer Support". There is a field 'TOT_CLM' which is a string that i've used LTRIM/RTRIM, but I used 'TO_INTEGER(v_TOT_CLM)' because the field is defined as integer in the target table.
The second error message reads ‘can’t create file in the directory /utils/infa_cache/USS/vnd_HPS/.
The last error message is that ‘check file system permission and free space for possible failure’.
Answers -
When you define a DB lookup in informatica and use SQL override, please use alias to columns. Alias name should be same as lookup port name.
So, if a lookup port is 'TOT_CLM', then select clause should be -
SELECT LTRIM(RTRIM((TOT_CLM)) as TOT_CLM,... FROM claim_table ORDER BY KEY --
If you dont put alias, you get this kind of error. If you still have error pls let me know.
It looks obvious you haven't set cache directory. You can define it in session properties as well as globally. Pls refer to below pic, you can set it for your session.
This seems related to number 2 and should go away if you set correct directory. Please note if your table size is huge, you may run out of spaces. Please connect to informatica admin for this.
HTH
I'm working on a project with C++Builder. I need to create a program that allows the user to check if the structure of a database (Firebird or SQL Server) is good or bad.
I have many XML files for Firebird, and I have many XML files for SQL Server. In those files there are many SQL queries defined.
The idea that I want to implement is to execute every query with ExecSQL(). If there are any problems in the structure of a table, ExecSQL() would throw an exception with all the details of the error, and I just have to take all the error messages and generate a report PDF.
The types of queries that I have in the XML files are ALTER, INSERT, and CREATE.
When executing a CREATE, if the table does not exist I can execute the query, but if the table does exist and there is a column that does not exist, the error that I get from ExecSQL() only tells me the table exists but does not tell me the column is missing.
For exmple, if I have this SQL code:
CREATE TABLE CUSTOMERS(ID INT NOT NULL, NAME VARCHAR (20) NOT NULL);
After connecting to the database and executing the query, there are 3 cases:
if the table doesn't exist, ExecSQL() runs without problem.
if the table exists and all the columns that I need to create already exist, the exception that I get is "TABLE ALREADY EXIST".
if the table exists and one or more columns do not exist, the exception that I get is "TABLE ALREADY EXIST". It's not telling me about the column(s).
I have the following concern regarding SAS/ACCESS facility.
Let's imagine that we have an external DB (i.e. Oracle), which we have assigned to a certain libname.
Next, we do a simple operation on one of the tables within this DB, i.e.
data db.table_new;
set db.table_old(keep=var1 var2 var3);
if var1>0 then new_var1=5;
run;
My question is the following:
Will the whole table table_old be pulled from external DB to SAS Server in order to process the data?
Will SAS/ACCESS transform the data step into DBMS operation or SQL so the whole processing will be performed outside SAS?
The documentation is unclear about it . See page 62.
Usually the rule of thumb is: if the SAS functions that are used in DATA step can be converted to native db sql functions, then SAS will let the DB server do the data processing. In your case, this seems to be the situation.
You can answer this question on any piece of code through a set of non-syntax-highlighted options that need to be simplified:
options sastrace=',,,d' sastraceloc=saslog nostsuffix;
When you run the data step, check the log. You will see information about whether SAS is able to successfully translate the code or not. If it was unsuccessful, you will see:
ACCESS ENGINE: SQL statement was not passed to the DBMS, SAS will do the processing.
If this occurs, SAS will usually send out a select * to the server and pull everything before filtering. When you see that error, try doing explicit passthrough, or redesign your query so that it can do everything on the server. It is possible to bring down the SAS server, or severely degreade performance on the Oracle server, if the table is large enough.
Some common functions you'll want to avoid using directly in the query, especially with Oracle:
datepart()
intnx()
intck()
today()
put()
input()
If I have to use any of those functions, I usually play it safe and create a macro variable of static ones beforehand (e.g. today()), filter the raw data at the lowest level first to get it into the SAS server, or use explicit SQL passthrough.
In summary, I would say it depends on your method. On the second page of Chapter 1 of the SAS/Access 9.2 document in your above link, there are two methods (among the older DBLOAD procedure) of the SAS/ACCESS facility:
LIBNAME reference - assign SAS librefs to DBMS objects such
as schemas and databases; you can then work with the table or view as you would with a SAS data set...You can use such SAS procedures as PROC SQL or DATA step programming on any libref that references DBMS data.
SQL Pass-through facility - to interact with a data source using its
native SQL syntax without leaving your SAS session. SQL statements are passed directly to the data source for processing...The DBMS optimizer can take advantage of indexes on DBMS columns to process a query more quickly and
efficiently
Hence, for the first method SAS handles processing and second method DBMS handles processing. Like most clients (Java, C#, Python script or PHP webpage) that connect to external RDMS sources, unless a direct ODBC/OLEDB or other API connection is explicitly employed and request sent, processing is handled in the frontend (i.e., calculating parameters) and the end result is updated to the backend via transactions. All SAS's libraries would live in memory (or temporary hard disk) during the appointed session and depending on the code handles data itself and passes results to external source or passes data handling entirely to another source.
Comparative Example: Microsoft Access
One good comparative example would be Microsoft Access which like SAS too provides a linked table connection and pass-through query for any ODBC-compliant RDMS including SQL Server, Oracle, MySQL, etc. It is often a misnomer to tag Access as a database when actually it is a GUI program and collection of objects, one of which is the default Windows JET/ACE engine (a .dll file) not at all restricted to Access but available to all Office programs. Notice the world default as this can be switched out to any ODBC database source.
Linked tables are essentially Access GUI objects (specifically special tabledefs) not unlike SAS's libname refs that are loaded into a JET/ACE table container with data pointing externally. One can then use a linked table like any other Access local table and use anything of the ACE SQL dialect. This special linked table (much like SAS's libname refs are established by ODBC or other connection type) points to the external source and the driver translates query command for the migration action. Therefore, an exact same Access linked table query may perform differently than same RDMS query.
Analogy
I imagine SAS behaves the same way and exists as a front-end with libname ref as local objects with pointers to the backend. All data step handling is processed locally and simply the resultset are imported or extracted by the engine. To use an analogy. A database would be the home and SAS is the garbage man, home decorator, or move-in helper. SAS (like Java's JDBC, PHP's PDO, Python's cursors, R's libraries) knocks on the door which the database answers (annoyed by so many requests). "Hey buddy, we need to take out the garbage and here are the exact items...or we need to remodel the basement and here are the exact specs...or we have new furniture to add in the truck ready for drop off...with credentials signed please carry out immediately." And like in both, pass-through methods are requests carried out on the backend engine. So SAS leaves instructions, maybe a note on the door (without exactness) for homeowner to carry out.
Background:
We have an application that uses the ODBC API to interact with Access and SQL Server (dynamically, depending on user's configuration).
I have discovered a bug which might be in the ODBC SQL driver, or may be a misconfiguration issue with the ODBC DSN we create, or may be a bug somehow in our code.
When a document is edited and saved, we query the database to see if this file has a corresponding record in the database - if so, we update the record with the updated data from the document; if not, we do an insert to create the necessary record for it.
We use the filename as the unique primary key on our table, and this works fine normally.
The bug is that if the filename contains characters outside of the current ANSI code page, then the select indicates no matches:
SQL: SELECT * FROM "My Designs" WHERE "PATHNAME" = '\\FILE-SERVER\Home Folders\User Files\狭すぎて丸め処理が出来ません!!.foo' [# matches = 0]
However, when the insert is attempted, we get a unique key violation (of course) - since there already is a record with that filename.
Database error: Violation of PRIMARY KEY constraint 'PK__My Desig__1B3D5B4BF643706B'. Cannot insert duplicate key in object 'dbo.My Designs'. The duplicate key value is (\\FILE-SERVER\Home Folders\User Files\狭すぎて丸め処理が出来ません!!.foo).
The statement has been terminated.
I've been over the code with a fine-tooth comb, and I can see nothing wrong. :(
The SQL statement that is being generated produces the correct Unicode output of the filename. Our application is compiled for Unicode. The column is SQL_WVARCHAR in ODBC speak.
I've tried adding AutoTranslate=no to the DSN configuration string, but that appears to have no effect.
I've tried logging the database connection from ODBC control panel. Sadly, that interface produces an ANSI log file - so I cannot verify UNICODE / ANSI issues using that tool.
Questions:
Is there a tool I can use to verify that these statements are being
created / issued correctly by the ODBC driver to the SQL Server
database?
Is there a better way to use ODBC so that the driver doesn't get canoodled by a simple UNICODE string in a SELECT query vs. an INSERT request?
Any other ideas for how to approach this problem (short of replacing our technology)
In the select statement, make sure you enclose the where clause string with a N to tell SQL it's unicode:
..."PATHNAME" = N'\\FILE-SERVER\Home Folders\User Files\狭すぎて丸め処理が出来ません!!.foo'
Also, MFC converts the data to MCBS or UNICODE depending on your configuration. Make sure you use CStringT in recordset.
This may seem a odd question as I have a solution, I just dont understand why and that limits me.
I am copying data from various sources into SQL and am using a ADO connection in C++ Builder XE2.
When the data is from MSAccess or MSExcel the code is similar to the following:
//SetupADO..
ADOConn->ConnectionString="Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:/temp/testdb.mdb";
//Then open it..
ADOConn->Connected = true;
//Build SQL
UnicodeString sSQL = "SELECT * INTO [ODBC;DSN=PostgreSQL30;DATABASE=admin_db;SERVER=192.168.1.10;PORT=5432;UID=user1;PWD=pass1;SSLmode=disable;ReadOnly=0;Protocol=7.4;].[table1] FROM [accesstb]";
//And finally I use the EXCEUTE() function of the ADO Connection
ADOConn->Execute(sSQL, iRA, TExecuteOptions() << TExecuteOption::eoExecuteNoRecords);
This works fine for Excel too but not for CSV files. I'm using the same driver must can only get it working by changing the syntax around.
//SetupADO..
ADOConn->ConnectionString="Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\temp;Extended Properties=\"Text;HDR=Yes;\";Persist Security Info=False";
//Then open it..
ADOConn->Connected = true;
//Build SQL with the IN keyword and start internal ODBC connection with 2 single quotes
UnicodeString sSQL = "SELECT * INTO [table1] IN '' [ODBC;DSN=PostgreSQL30;DATABASE=admin_db;SERVER=192.168.1.10;PORT=5432;UID=user1;PWD=pass1;SSLmode=disable;ReadOnly=0;Protocol=7.4;] FROM [test.csv]";
//And finally EXCEUTE() again
ADOConn->Execute(sSQL, iRA, TExecuteOptions() << TExecuteOption::eoExecuteNoRecords);
When using the same SQL as the Access query the error "Query input must contain at least one table or query" would be returned.
Intrestingly, one escaped quote, i.e. \' fails when used in place of the 2 single ones. I have also tried writing to another Access database in case the problem was with PG but I had the same results.
Can someone tell me why the IN keywork is required and what the single quotes do?
Extended Properties=\"Text;HDR=Yes;\" specifies text as the datasource, so the connection string is different. IN '' tells the database to map table1 to the first column of the CSV file, since there is no relational model in CSV.
References
Importing CSV Data and saving it in database - CodeProject