I'm trying to use a Lookup transformation to extract ACCT_ID from ACCT table based on the port CUST_DDA which is an output port from an expression.
I'm using an sqloverride as below. The initial lookup condition :
SUBSTR_ACCT_ID = IN_CUST_DDA
Override:
SELECT
ACCT.ACCT_ID as ACCT_ID,
ACCT.ALT_ACCT_ID as ALT_ACCT_ID,
substr(acct.acct_id,-1*(length(IN_CUST_DDA))) as SUBSTR_ACCT_ID
FROM ACCT
WHERE ACCT.ALT_ACCT_ID LIKE '%'||TO_CHAR(IN_CUST_DDA)
AND ACCT.ACCT_ID LIKE '%'||TO_CHAR(IN_CUST_DDA)
The above sql override is failing due to the error : ORA-00904: "IN_CUST_DDA": invalid identifier
Is there a way to use the value from CUST_DDA port as an input port for the lookup. CUST_DDA is not a field that belongs to the ACCT table. Is there a way to do this.
Thanks.
From the override I can see that you are trying to convert IN_CUST_DDA into CHAR, also at the same time your using IN_CUST_DDA in the length.
Might be the length function causing the issue, because length function can be used along with a string.
In order to use CUST_DDA(from source) in your lookup override. You need to join the lookup table with source with a common field in the override.
You cant use the port in the way you mentioned. When you run the workflow informatica integration service will run the lookup override query in the database and get the data into the cache file (that is the reason you are receiving the error "IN_CUST_DDA": invalid identifier.) . Once the cache file is ready it will apply the conditions and then get the output for you.
Let me know if you are not clear on this
Regards
Raj
To achieve this you need to configure your lookup as non-cached, so the query will be executed for each input row. Note, that this degrades performance a lot.
Next, you need to use a bit different syntax, enclosing the input port in question marks. Here's an example. In your case it should be something like (this might need a little adjustment):
SELECT
ACCT.ACCT_ID as ACCT_ID,
ACCT.ALT_ACCT_ID as ALT_ACCT_ID,
substr(acct.acct_id,-1*(length(?IN_CUST_DDA?))) as SUBSTR_ACCT_ID
FROM ACCT
WHERE ACCT.ALT_ACCT_ID LIKE '%'||TO_CHAR(?IN_CUST_DDA?)
AND ACCT.ACCT_ID LIKE '%'||TO_CHAR(?IN_CUST_DDA?)
Related
I am running this workflow which is supposed to load a file, but i am having the following error messages.
Internal error. Failed to initialize transformation [LKP_FILE_LOAD_ID{{BLD}}]. Contact Informatica Global Customer Support". There is a field 'TOT_CLM' which is a string that i've used LTRIM/RTRIM, but I used 'TO_INTEGER(v_TOT_CLM)' because the field is defined as integer in the target table.
The second error message reads ‘can’t create file in the directory /utils/infa_cache/USS/vnd_HPS/.
The last error message is that ‘check file system permission and free space for possible failure’.
Answers -
When you define a DB lookup in informatica and use SQL override, please use alias to columns. Alias name should be same as lookup port name.
So, if a lookup port is 'TOT_CLM', then select clause should be -
SELECT LTRIM(RTRIM((TOT_CLM)) as TOT_CLM,... FROM claim_table ORDER BY KEY --
If you dont put alias, you get this kind of error. If you still have error pls let me know.
It looks obvious you haven't set cache directory. You can define it in session properties as well as globally. Pls refer to below pic, you can set it for your session.
This seems related to number 2 and should go away if you set correct directory. Please note if your table size is huge, you may run out of spaces. Please connect to informatica admin for this.
HTH
I am writing simple SELECT queries which involve parsing out date from a string.
The dates are typed in by users manually in a web application and are recorded as string in database.
I am having CASE statement to handle various date formats and use correct format specifier accordingly in TO_DATE function.
However, sometimes, users enter something that's not a valid date(e.g. 13-31-2013) by mistake and then the entire query fails. Is there any way to handle such rougue records and replace them with some default date in query so that the entire query does not fail due to single invalid date record?
I have already tried regular expressions but they are not quite reliable when it comes to handling leap years and 30/31 days in months AFAIK.
I don't have privileges to store procedures or anything like that. Its just plain simple SELECT query executed from my application.
This is a client task..
The DB will give you an error for an invalid date (the DB does not have a "TO_DATE_AND_FIX_IF_NOT_CORRECT" function).
If you've got this error- it means you already tried to cast something to an invalid date.
I recommend doing the migration to date on your application server, and in the case of exception from your code - send a default date to the DB.
Also, that way you send to the DB an object of type DbDate and not a string.
That way you achieve two goals:
1. The dates will always be what you want them to be (from the client).
2. You close the door for SQL Injection attacks.
It sounds like in your case you should write the function I mentioned...
it should look something like that:
Create or replace function TO_DATE_SPECIAL(in_date in varchar2) return DATE is
ret_val date;
begin
ret_val := to_date(in_date,'MM-DD-YYYY');
return ret_val;
exception
when others then
return to_date('01-01-2000','MM-DD-YYYY');
end;
within the query - instead of using "to_date" use the new function.
that way instead of failing - it will give you back a default date.
-> There is not IsDate function .. so you'll have to create an object for it...
I hope you've got the idea and how to use it, if not - let me know.
I ended up using crazy regex that checks leap years, 30/31 days as well.
Here it is:
((^(0?[13578]|1[02])[\/.-]?(0?[1-9]|[12][0-9]|3[01])[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^(0?[469]|11)[\/.-]?(0?[1-9]|[12][0-9]|30)[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^([0]?2)[\/.-]?(0?[1-9]|1[0-9]|2[0-8])[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^([0]?2)[\/.-]?29[\/.-]?(((18|19|20){0,1}(04|08|[2468][048]|[13579][26]))|2000|00)$))
It is modified version of the answer by McKay here.
Not the most efficient but it works. I'll wait to see if I get a better alternative.
Requirement: I am suppose to use an existing Integration Object for my requirement. As this IO consists of ICs that I do not need in my requirement, I would like to avoid them in my IO query output.
I observe that passing Id = '' returns no result in Siebel 8.0. Can I use it as a feature and pass SearchSpec => [Integration Component.Id]='' to EAI Siebel Adapter query to suppress ICs that I don't want in output?
How good is this query Id=''? Will Siebel ignore this query completely? or will it attempt and return no output?
As per my understanding Siebel ignores the query where row_id is passed as ''
(Not true for siebel 6.0) Please share your opinion.
Not sure of using Id = '', when you club the condition with other conditions, Siebel might try to find actual matching records. Also , not sure if future upgrading will keep the same system.
If yours is the only code using the IO, you could straightaway inactivate the ICs you dont want.
If you are unsure of IC inactivation, best way should be to a DatMapper. Set up an EAI Datamapper, source and target IOs of same name. In this datamapper, map only the ICs you need. After querying from EAI Siebel Adapter, send your output to this DataMapper.
Siebel will keep only the ICs mapped and remove all the rest.
Since this is a non-repository change, you can modify the DataMapper in future too.
Hope this helps !
Answering it myself with my opinion..
As per my understanding, querying with Id='' still queries the database for row_id = ''. Including this in IO query reduces the query scope to the parent's context..
Though this won't improve any performance, IO query output looks cleaner.
Update: I'm using a Indexed column based field Id (ROW_ID) with search spec as "[Id] IS NULL". It's a next to impossible case in database having ROW_ID = NULL, unless it's intentionally and manually updated. Again no one would do it unless really wants to messup that data .. because without ROW_ID record is literally invalid..
Adding a null query to the IC will inherently result in an empty property set for the IC in question. But if you don't need the IC, and the ICs in the IO are not hierarchically connected (no hierarchy key )(eg- independant BCs with same base table in the same BO), you just have to remove the IC mapping in the datamap editor and the IC wont show in the IO propset
How I can encode/escape a varchar to be more secure without using cfqueryparam? I want to implement the same behaviour without using <cfqueryparam> to get around "Too many parameters were provided in this RPC request. The maximum is 2100" problem. See: http://www.bennadel.com/blog/1112-Incoming-Tabular-Data-Stream-Remote-Procedure-Call-Is-Incorrect.htm
Update:
I want the validation / security part, without generating a prepared-statement.
What's the strongest encode/escape I can do to a varchar inside <cfquery>?
Something similar to mysql_real_escape_string() maybe?
As others have said, that length-related error originates at a deeper level, not within the queryparam tag. And it offers some valuable protection and therefore exists for a reason.
You could always either insert those values into a temporary table and join against that one or use the list functions to split that huge list into several smaller lists which are then used separately.
SELECT name ,
..... ,
createDate
FROM somewhere
WHERE (someColumn IN (a,b,c,d,e)
OR someColumn IN (f,g,h,i,j)
OR someColumn IN (.........));
cfqueryparam performs multiple functions.
It verifies the datatype. If you say integer, it makes sure there is an integrer, and if not, it does nto allow it to pass
It separates the data of a SQL script from the executable code (this is where you get protection from SQL injection). Anything passed as a param cannot be executed.
It creates bind variables at the DB engine level to help improve performance.
That is how I understand cfqueryparam to work. Did you look into the option of making several small calls vs one large one?
It is a security issue. Stops SQL injections
Adobe recommends that you use the cfqueryparam tag within every cfquery tag, to help secure your databases from unauthorized users. For more information, see Security Bulletin ASB99-04, "Multiple SQL Statements in Dynamic Queries," at www.adobe.com/devnet/security/security_zone/asb99-04.html, and "Accessing and Retrieving Data" in the ColdFusion Developer's Guide.
The first thing I'd be asking myself is "how the heck did I end up with more than 2100 params in a single query?". Because that in itself should be a very very big red flag to you.
However if you're stuck with that (either due to it being outwith your control, or outwith your motivation levels to address ;-), then I'd consider:
the temporary table idea mentioned earlier
for values over a certain length just chop 'em in half and join 'em back together with a string concatenator, eg:
*
SELECT *
FROM tbl
WHERE col IN ('a', ';DROP DATABAS'+'E all_my_data', 'good', 'etc' [...])
That's a bit grim, but then again your entire query sounds grim, so that might not be such a concern.
param values that are over a certain length or have stop words in them or something. This is also quite a grim suggestion.
SERIOUSLY go back over your requirement and see if there's a way to not need 2100+ params. What is it you're actually needing to do that requires all this???
The problem does not reside with cfqueryparam, but with MsSQL itself :
Every SQL batch has to fit in the Batch Size Limit: 65,536 * Network Packet Size.
Maximum size for a SQL Server Query? IN clause? Is there a Better Approach
And
http://msdn.microsoft.com/en-us/library/ms143432.aspx
The few times that I have come across this problem I have been able to rewrite the query using subselects and/or table joins. I suggest trying to rewrite the query like this in order to avoid the parameter max.
If it is impossible to rewrite (e.g. all of the multiple parameters are coming from an external source) you will need to validate the data yourself. I have used the following regex in order to perform a safe validation:
<cfif ReFindNoCase("[^a-z0-9_\ \,\.]",arguments.InputText) IS NOT 0>
<cfthrow type="Application" message="Invalid characters detected">
</cfif>
The code will force an error if any special character other than a comma, underscore, or period is found in a text string. (You may want to handle the situation cleaner than just throwing an error.) I suggest you modify this as necessary based on the expected or allowed values in the fields you are validating. If you are validating a string of comma separated integers you may switch to use a more limiting regex like "[^0-9\ \,]" which will only allow numbers, commas, and spaces.
This answer will not escape the characters, it will not allow them in the first place. It should be used on any data that you will not use with <cfqueryparam>. Personally, I have only found a need for this when I use a dynamic sort field; not all databases will allow you to use bind variables with the ORDER BY clause.
I need to make a query like this:
SELECT PNPDeviceID FROM Win32_NetworkAdapter WHERE AdapterTypeId = 0
Trouble is, the AdapterTypeId column isn't always present. In this case, I just want everything, like so:
SELECT PNPDeviceID FROM Win32_NetworkAdapter
My WQL/SQL knowledge is extremely limited. Can anybody tell me how to do this in a single query?
EDIT:
A bit more background seems to be required: I am querying Windows for device information using WMI, which uses an SQL-like syntax. So, in my example, I am querying for network adapters that have an AdapterTypeId of 0.
That column is not always present however, meaning that if I enumerate through the returned values then "AdapterTypeId" is not listed.
EDIT 2:
Changed SQL to WQL; apparantly this is more correct.
I am assuming you mean the underlying schema is unreliable.
This is a highly unconventional situation. I suggest that you resolve the issue that is causing the column to not always be present, because to have the schema changing dynamically underneath your application is potentially (almost certainly) disastrous.
Update:
OK, so WQL lets you query objects with a SQL-like syntax but, unlike SQL, the schema can change underneath your feet. This is a classic example of a leaky abstraction, and I now hate WQL without ever having used it :).
Since the available properties are in flux, I am guessing that WQL provides a way to enumerate the properties for a given adapter. Do this, and choose which query to run depending upon the results.
After some Googling, there is an example here, which shows how to enumerate through the available properties. You can use this to determine if AdapterTypeId exists or not.
SELECT PNPDeviceID FROM Win32_NetworkAdapter WHERE AdapterTypeId = {yourDesire} OR AdapterTypeId IS NULL
I assume that you mean that this field is missing from the table.
Do you know before submitting the query if this field exists?
If yes then just create SQL dynamically, otherwise It think you will get syntax error in case of missing field
This is not an SQL question. SQL does not contemplate records with varying schemas in a single table source. Instead (as you mention) this is a different system using an "SQL-like" syntax. You'll have better luck if you recast the question using the actual product that you're trying to query, and information how that product deals with variable record structures is probably discussed in the documentation.