Our C++ program is using Oracle and OCI to do its database work. Occasionally, the user will trigger a constraint violation, which we detect and then show an error message from OCIErrorGet. OCIErrorGet returns strings like this:
ORA-02292: integrity constraint (MYSCHEMA.CC_MYCONSTRAINT) violated - child record found
ORA-06512: at line 5
I am looking for the cleanest way to extract "MYSCHEMA.CC_MYCONSTRAINT" from the Oracle error. Knowing the name of the constraint, I could show a better error message (our code could look up a very meaningful error message if it had access to the constraint name).
I could use a regex or something and assume that the Oracle message will never change, but this seems a little fragile to me. Or I could look for specific ORA codes and then grab whatever text falls between the parentheses. But I was hoping OCI had a cleaner/more robust way, if a constraint fails, to figure out the actual name of the failed constraint without resorting to hardcoded string manipulation.
Any ideas?
According to the Oracle Docs, a string search is exactly what you need to do:
Recognizing Variable Text in Messages
To help you find and fix errors, Oracle embeds object names, numbers,
and character strings in some messages. These embedded variables are
represented by string, number, or character, as appropriate. For
example:
ORA-00020: maximum number of processes (number) exceeded
The preceding message might actually appear as follows:
ORA-00020: maximum number of processes (50) exceeded
Oracle makes a big point in their docs of saying the strings will be kept up to date in their section on "Message Accuracy." It's a pretty strong suggestion that they intend you to do a string search.
Also, according to this website, the Oracle Error structure also pretty strongly implies that they intend you to do a string search, because the data structure lacks anything else for you to get:
array(4) {
["code"]=>int(942)
["message"]=>string(40) "ORA-00942: table or view does not exist"
["offset"]=>int(14)
["sqltext"]=>string(32) "select * from non_existing_table"
}
This output reveals the following information:
The variable $erris an array with four elements.
The first element is accessible by the key ‘code’ and its value is number 942.
The second value is accessible by the key ‘message’ and the value is string “ORA-00942: table or view does not exist”.
The third value is accessible by the key ‘offset’, and its value is the number 14. This is the character before the name of the
non-existing table.
The fourth member is the problematic SQL message causing the error in the first place.
I agree with you; it would be great if there were a better way to get the constraint name you're violating, but string-matching seems to be the intended way.
Related
I am writing a large MERGE statement in BigQuery.
When I attempt to run this query the validator gives me an error involving a lot of ...'s that hides the useful information as shown below:
Value has type ARRAY<STRUCT<eventName STRING, eventUUID STRING, eventDate DATE, ...>> which cannot be inserted into column Events, which has type ARRAY<STRUCT<eventName STRING, eventUUID STRING, eventDate DATE, ...>> at [535:1]
I am extremely confident these two array objects match exactly, however since I am struggling to get around this I would love to see the full error message.
Is there any way to see the full error?
I have looked into the Google Logging tool and cannot see any additional information.
I have also tried the following Cloud Shell command:
bq --format=prettyjson show -j [Job Id Goes Here]
Again, this seems to provide no additional information.
This approach feels pretty silly but it could be the last resort for really long nest type.
Use INFORMATION_SCHEMA.COLUMNS to get a full string of the target type, in your case, type of column Events.
Use CREATE TABLE <yourDataset>.<yourTempTable> AS SELECT ... to dump one row of the Value into a table. Use 1) again to see its full type string.
In the documentation, https://docs.spring.io/spring-data/neo4j/docs/current/reference/html/
it uses {0} to reference the parameter 'movieTitle'.
#Query("MATCH (movie:Movie {title={0}}) RETURN movie")
Movie getMovieFromTitle(String movieTitle);
However, in my own code, if I use "{title={0}", my IntelliJ always reports a syntax error. I can resolve the issue by changing it to
{title:{movieTitle}
Here I have to use the actual argument name and the colon plus {}.
Is there any trick for this? I don't think the documentation is wrong.
Question 2:
If I want the node label "Movie" to be a parameter, it also shows an error message:
#Query("MATCH (movie:{label} {title={0}}) RETURN movie")
Movie getMovieFromTitle(String movieTitle, String label);
I do not know what version of IntelliJ you are using but the first query is right. There is also a test case for this in the spring-data-neo4j project.
It is not possible to use the second query syntax because there is no support for this on the database level where the query gets executed. If it would be supported in SDN before making the call to the DB the query has to be parsed (and the pattern replaced) every time when the query get executed and SDN will loose the possibility to parse the query once and then just add the parameter values in subsequent calls. This will lower the performance of executing annotated query functions.
I am writing simple SELECT queries which involve parsing out date from a string.
The dates are typed in by users manually in a web application and are recorded as string in database.
I am having CASE statement to handle various date formats and use correct format specifier accordingly in TO_DATE function.
However, sometimes, users enter something that's not a valid date(e.g. 13-31-2013) by mistake and then the entire query fails. Is there any way to handle such rougue records and replace them with some default date in query so that the entire query does not fail due to single invalid date record?
I have already tried regular expressions but they are not quite reliable when it comes to handling leap years and 30/31 days in months AFAIK.
I don't have privileges to store procedures or anything like that. Its just plain simple SELECT query executed from my application.
This is a client task..
The DB will give you an error for an invalid date (the DB does not have a "TO_DATE_AND_FIX_IF_NOT_CORRECT" function).
If you've got this error- it means you already tried to cast something to an invalid date.
I recommend doing the migration to date on your application server, and in the case of exception from your code - send a default date to the DB.
Also, that way you send to the DB an object of type DbDate and not a string.
That way you achieve two goals:
1. The dates will always be what you want them to be (from the client).
2. You close the door for SQL Injection attacks.
It sounds like in your case you should write the function I mentioned...
it should look something like that:
Create or replace function TO_DATE_SPECIAL(in_date in varchar2) return DATE is
ret_val date;
begin
ret_val := to_date(in_date,'MM-DD-YYYY');
return ret_val;
exception
when others then
return to_date('01-01-2000','MM-DD-YYYY');
end;
within the query - instead of using "to_date" use the new function.
that way instead of failing - it will give you back a default date.
-> There is not IsDate function .. so you'll have to create an object for it...
I hope you've got the idea and how to use it, if not - let me know.
I ended up using crazy regex that checks leap years, 30/31 days as well.
Here it is:
((^(0?[13578]|1[02])[\/.-]?(0?[1-9]|[12][0-9]|3[01])[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^(0?[469]|11)[\/.-]?(0?[1-9]|[12][0-9]|30)[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^([0]?2)[\/.-]?(0?[1-9]|1[0-9]|2[0-8])[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^([0]?2)[\/.-]?29[\/.-]?(((18|19|20){0,1}(04|08|[2468][048]|[13579][26]))|2000|00)$))
It is modified version of the answer by McKay here.
Not the most efficient but it works. I'll wait to see if I get a better alternative.
I'm seeing very bizarre behavior with the REGEXP_MATCH function in google big query. The function appears to work perfectly fine for public data but is not working on my dataset. I have a dataset imported from the csv with the first two lines (first is header row which all becomes the schema where everything is a string), there's a lot more but the following is the only relevant data for this case.
"id","common_name","botanical_name","low_hardiness_zone","high_hardiness_zone","type","exposure_min","exposure_max","moisture_min","moisture_max"
"plant1","Abelia","Abelia zanderi 'Conti (Confetti)'","5b","9a","Shrub","Partial Sun","Full Sun","Dry","Dry"
When I run the query:
SELECT * FROM [PlantLink_Plant_Types.plant_data_set]
WHERE REGEXP_MATCH('common_name',r'.*')
I get every result.
However, when I run the query:
SELECT * FROM [PlantLink_Plant_Types.plant_data_set]
WHERE REGEXP_MATCH('common_name',r'A.*')
I get no results, which is really weird because the plant common name Abelia starts with an A.
Now my regex magic is not that strong, but I am pretty sure the pattern is not at fault. Additionally I've run the public dataset test queries with REGEXP_MATCH and they run correctly. Does anyone have any clue why REGEXP_MATCH would not always function as advertised?
Note:
REGEXP_MATCH('common_name',r'.*') matches the string 'common_name'
while
REGEXP_MATCH(common_name,r'.*') matches a field in your table that is called common_name
the 1st one is always true and therefore you get all results.
I guess you wanted to refer the content of the field, so you need to use the second one.
REGEXP_MATCH(common_name,r'A.*') should return all records that field common_name contains "A".
hope this helps.
Issue is the string 'common_name' does not start with 'A'.
Check this:
REGEXP_MATCH('common_name',r'.*'): All results.
REGEXP_MATCH('common_name',r'A.*'): No results.
REGEXP_MATCH('common_name',r'c.*'): All results.
REGEXP_MATCH(common_name,r'A.*'): All results that somewhere have an 'A'.
:)
How I can encode/escape a varchar to be more secure without using cfqueryparam? I want to implement the same behaviour without using <cfqueryparam> to get around "Too many parameters were provided in this RPC request. The maximum is 2100" problem. See: http://www.bennadel.com/blog/1112-Incoming-Tabular-Data-Stream-Remote-Procedure-Call-Is-Incorrect.htm
Update:
I want the validation / security part, without generating a prepared-statement.
What's the strongest encode/escape I can do to a varchar inside <cfquery>?
Something similar to mysql_real_escape_string() maybe?
As others have said, that length-related error originates at a deeper level, not within the queryparam tag. And it offers some valuable protection and therefore exists for a reason.
You could always either insert those values into a temporary table and join against that one or use the list functions to split that huge list into several smaller lists which are then used separately.
SELECT name ,
..... ,
createDate
FROM somewhere
WHERE (someColumn IN (a,b,c,d,e)
OR someColumn IN (f,g,h,i,j)
OR someColumn IN (.........));
cfqueryparam performs multiple functions.
It verifies the datatype. If you say integer, it makes sure there is an integrer, and if not, it does nto allow it to pass
It separates the data of a SQL script from the executable code (this is where you get protection from SQL injection). Anything passed as a param cannot be executed.
It creates bind variables at the DB engine level to help improve performance.
That is how I understand cfqueryparam to work. Did you look into the option of making several small calls vs one large one?
It is a security issue. Stops SQL injections
Adobe recommends that you use the cfqueryparam tag within every cfquery tag, to help secure your databases from unauthorized users. For more information, see Security Bulletin ASB99-04, "Multiple SQL Statements in Dynamic Queries," at www.adobe.com/devnet/security/security_zone/asb99-04.html, and "Accessing and Retrieving Data" in the ColdFusion Developer's Guide.
The first thing I'd be asking myself is "how the heck did I end up with more than 2100 params in a single query?". Because that in itself should be a very very big red flag to you.
However if you're stuck with that (either due to it being outwith your control, or outwith your motivation levels to address ;-), then I'd consider:
the temporary table idea mentioned earlier
for values over a certain length just chop 'em in half and join 'em back together with a string concatenator, eg:
*
SELECT *
FROM tbl
WHERE col IN ('a', ';DROP DATABAS'+'E all_my_data', 'good', 'etc' [...])
That's a bit grim, but then again your entire query sounds grim, so that might not be such a concern.
param values that are over a certain length or have stop words in them or something. This is also quite a grim suggestion.
SERIOUSLY go back over your requirement and see if there's a way to not need 2100+ params. What is it you're actually needing to do that requires all this???
The problem does not reside with cfqueryparam, but with MsSQL itself :
Every SQL batch has to fit in the Batch Size Limit: 65,536 * Network Packet Size.
Maximum size for a SQL Server Query? IN clause? Is there a Better Approach
And
http://msdn.microsoft.com/en-us/library/ms143432.aspx
The few times that I have come across this problem I have been able to rewrite the query using subselects and/or table joins. I suggest trying to rewrite the query like this in order to avoid the parameter max.
If it is impossible to rewrite (e.g. all of the multiple parameters are coming from an external source) you will need to validate the data yourself. I have used the following regex in order to perform a safe validation:
<cfif ReFindNoCase("[^a-z0-9_\ \,\.]",arguments.InputText) IS NOT 0>
<cfthrow type="Application" message="Invalid characters detected">
</cfif>
The code will force an error if any special character other than a comma, underscore, or period is found in a text string. (You may want to handle the situation cleaner than just throwing an error.) I suggest you modify this as necessary based on the expected or allowed values in the fields you are validating. If you are validating a string of comma separated integers you may switch to use a more limiting regex like "[^0-9\ \,]" which will only allow numbers, commas, and spaces.
This answer will not escape the characters, it will not allow them in the first place. It should be used on any data that you will not use with <cfqueryparam>. Personally, I have only found a need for this when I use a dynamic sort field; not all databases will allow you to use bind variables with the ORDER BY clause.