How to load custom sql functions with django - django

I am trying to use Django's initial SQL data functionality to create an SQL function. The docs state I can do this:
https://docs.djangoproject.com/en/1.6/howto/initial-data/#providing-initial-sql-data
Django provides a hook for passing the database arbitrary SQL that’s executed just after the CREATE TABLE statements when you run migrate. You can use this hook to populate default records, or you could also create SQL functions, views, triggers, etc.
After some googling I found that django's customsql code splits any sql files and runs them line by line, creating this error,
Failed to install custom SQL for myapp.somemodel model: unterminated dollar-quoted string at or near "$$ BEGIN;"
Is there an accepted work around for this? Or a better way to load custom sql functions?

Yeah, I've seen this problem before. If you stick a multi-line function in your app's sql/<modelname>.sql like so:
CREATE OR REPLACE FUNCTION increment(i integer) RETURNS integer AS $$
BEGIN
RETURN i + 1;
END;
$$ LANGUAGE plpgsql;
you'll get the error you saw, namely something like:
Failed to install custom SQL for mysite.Poll model: unterminated dollar-quoted string at or near "$$ BEGIN RETURN i + 1;"
LINE 1: ... FUNCTION increment(i integer) RETURNS integer AS $$ BEGIN R...
I think you should be able to work around the problem by squeezing the function definition all onto one line, e.g.
CREATE OR REPLACE FUNCTION increment(i integer) RETURNS integer AS $$BEGIN RETURN i + 1; END; $$ LANGUAGE plpgsql;
Looks like this bug affects any multi-line functions (both dollar-quoted and single-quoted). I tested on Django 1.6, no idea if it's been fixed already.

Related

How to do a dynamic binding to a PL/pgSQL function using SOCI?

I have this PostgreSQL PL/pgSQL function:
CREATE OR REPLACE FUNCTION get_people()
RETURNS SETOF people AS $$
BEGIN
RETURN QUERY SELECT * FROM people;
END;
$$ LANGUAGE plpgsql;
Then I try to read the data in an application using SOCI, with this code:
session sql {"postgresql://dbname=postgres"};
row person {};
procedure proc = (sql.prepare << "get_people()", into(person));
proc.execute(true);
I would expect that person have the data of the first person, but it contains only one column with the name of the stored procedure (i.e., "get_people").
So I don't know what I am doing wrong here, or not doing. Is it the PL/pgSQL code or the SOCI code? Maybe SOCI does not support dynamic binding for stored procedures. Also, this method would allow me to read the first row only, but what about the rest of rows? I know SOCI comes with the rowset class for reading result sets, but the documentation says it only works with queries. Please help.
SELECT get_people() will return a single column, of type people, named after the procedure.
SELECT * FROM get_people() will give you the expected behaviour, decomposing the people records into their constituent fields.
Judging from the source, it looks like the SOCI procedure class (or at least, its Postgres implementation) is hard-wired to run procedures as SELECT ... rather than SELECT * FROM ....
I guess this means you'll need to write your own query, i.e.:
statement stmt = (sql.prepare << "SELECT * FROM get_people()", into(person));

Sqlite 'Unrecognized token: ":" C++

I'm not sure what to do with this as I can't remove the colon from my SQL string.
Basically I am trying to execute an SQL string in Sqlite using the below code.
string database_name = "C:/Programs_C++/Project/Databases/dbase.db";
string exec_string = "SELECT * FROM " + database_name + " WHERE type='table'";
dbase_return=sqlite3_open_v2(database_name.c_str(),&db_handle,SQLITE_OPEN_READWRITE,NULL);
dbase_return_tbl=sqlite3_get_table(db_handle,exec_string.c_str(),&result,&row,&column,&error_msg);
//But I get the error: unrecognized token: ":" ?
How do I get around this? Thanks
You can SELECT from a table, not from a database.
First open the database (using the filename), then execute a valid SQL statement like
SELECT * FROM myTable;
SELECT * FROM C:/Programs_C++/Project/Databases/dbase.db WHERE type = 'table' is not valid SQL. If you are trying to get a list of all tables, you cannot do it that way.
It looks like you have URI filenames switched on - this can be done at compile time or runtime (probably compile time for you if you didn't know about it).
If URI filenames are switched on, you need to change your filename to something like:
file:///C:/Programs_C++/Project/Databases/dbase.db
Edit: If you want to switch it off, I don't think you can do it for this one call (as the call takes a flag as part of the parameters which can only switch it on). Instead you can disable it globally by calling
sqlite3_config(SQLITE_CONFIG_URI, 0)
which tells sqlite to disable the URI filename convention globally. Note: you only need to call this once and it is not thread safe, so probably just put this at the start of your program.
However, it might be worth investigating if URI filenames are useful to you before switching them off entirely.

Handling invalid dates in Oracle

I am writing simple SELECT queries which involve parsing out date from a string.
The dates are typed in by users manually in a web application and are recorded as string in database.
I am having CASE statement to handle various date formats and use correct format specifier accordingly in TO_DATE function.
However, sometimes, users enter something that's not a valid date(e.g. 13-31-2013) by mistake and then the entire query fails. Is there any way to handle such rougue records and replace them with some default date in query so that the entire query does not fail due to single invalid date record?
I have already tried regular expressions but they are not quite reliable when it comes to handling leap years and 30/31 days in months AFAIK.
I don't have privileges to store procedures or anything like that. Its just plain simple SELECT query executed from my application.
This is a client task..
The DB will give you an error for an invalid date (the DB does not have a "TO_DATE_AND_FIX_IF_NOT_CORRECT" function).
If you've got this error- it means you already tried to cast something to an invalid date.
I recommend doing the migration to date on your application server, and in the case of exception from your code - send a default date to the DB.
Also, that way you send to the DB an object of type DbDate and not a string.
That way you achieve two goals:
1. The dates will always be what you want them to be (from the client).
2. You close the door for SQL Injection attacks.
It sounds like in your case you should write the function I mentioned...
it should look something like that:
Create or replace function TO_DATE_SPECIAL(in_date in varchar2) return DATE is
ret_val date;
begin
ret_val := to_date(in_date,'MM-DD-YYYY');
return ret_val;
exception
when others then
return to_date('01-01-2000','MM-DD-YYYY');
end;
within the query - instead of using "to_date" use the new function.
that way instead of failing - it will give you back a default date.
-> There is not IsDate function .. so you'll have to create an object for it...
I hope you've got the idea and how to use it, if not - let me know.
I ended up using crazy regex that checks leap years, 30/31 days as well.
Here it is:
((^(0?[13578]|1[02])[\/.-]?(0?[1-9]|[12][0-9]|3[01])[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^(0?[469]|11)[\/.-]?(0?[1-9]|[12][0-9]|30)[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^([0]?2)[\/.-]?(0?[1-9]|1[0-9]|2[0-8])[\/.-]?(18|19|20){0,1}[0-9]{2}$)|(^([0]?2)[\/.-]?29[\/.-]?(((18|19|20){0,1}(04|08|[2468][048]|[13579][26]))|2000|00)$))
It is modified version of the answer by McKay here.
Not the most efficient but it works. I'll wait to see if I get a better alternative.

Concat strings in SQL Server and Oracle with the same unmodified query

I have a program that must support both Oracle and SQL Server for it's database.
At some point I must execute a query where I want to concatenate 2 columns in the select statement.
In SQL Server, this is done with the + operator
select column1 + ' - ' + column2 from mytable
And oracle this is done with concat
select concat(concat(column1, ' - '), column2) from mytable
I'm looking for a way to leverage them both, so my code has a single SQL query literal string for both databases and I can avoid ugly constructs where I need to check which DBMS I'm connected to.
My first instinct was to encapsulate the different queries into a stored procedure, so each DBMS can have their own implementation of the query, but I was unable to create the procedure in Oracle that returns the record set in the same way as SQL Server does.
Update: Creating a concat function in SQL Server doesn't make the query compatible with Oracle because SQL Server requires the owner to be specified when calling the function as:
select dbo.concat(dbo.concat(column1), ' - '), column2) from mytable
It took me a while to figure it out after creating my own concat function in SQL Server.
On the other hand, looks like a function in Oracle with SYS_REFCURSOR can't be called with a simple
exec myfunction
And return the table as in SQL Server.
In the end, the solution was to create a view with the same name on both RDBMs but with different implementations, then I can do a simple select on the view.
If you want to go down the path of creating a stored procedure, whatever framework you're using should be able to more or less transparently handle an Oracle stored procedure with an OUT parameter that is a SYS_REFCURSOR and call that as you would a SQL Server stored procedure that just does a SELECT statement.
CREATE OR REPLACE PROCEDURE some_procedure( p_rc OUT sys_refcursor )
AS
BEGIN
-- You could use the CONCAT function rather than Oracle's string concatenation
-- operator || but I would prefer the double pipes.
OPEN p_rc
FOR SELECT column1 || ' - ' || column2
FROM myTable;
END;
Alternatively, you could define your own CONCAT function in SQL Server.
Nope, sorry.
As you've noted string concatentaion is implemented in SQL-Server with + and Oracle with concat or ||.
I would avoid doing some nasty string manipulation in stored procedures and simply create your own concatenation function in one instance or the other that uses the same syntax. Probably SQL-Server so you can use concat.
The alternative is to pass + or || depending on what RDBMS you're connected to.
Apparently in SQL Server 2012 they have included a CONCAT() function:
http://msdn.microsoft.com/en-us/library/hh231515.aspx
If you are trying to create a database agnostic application, you should stick to either
Stick to very basic SQL and do anything like this in your application.
Create different abstractions for different databases. If you hope to get any kind of scale out of your application, this is the path you'll likely need to take.
I wouldn't go down the stored procedure path, you can probably get it to work, but but week you'll find out you need to support "database X", then you'll need to rewrite your stored proc in that database as well. Its a recipe for pain.

Syntax error in SQLite query

I am trying to insert a large number of records into a SQLite database. I get the above error if I try to use the sqlite3_exec C-API.
The code looks like this:
ret = sqlite_exec(db_p,".import file.txt table", NULL, NULL, NULL);
I know that the .import is command line, but can there be any way that you can do a extremely large insert of records that takes minimal time. I have read through previous bulk insert code and attempted to make changes but these are not providing the desired results.
Is there not a way to directly insert the string into the tables without having intermediate API's being called?
.import is most probably not available via the API. However there's one crucial thing to speed up inserts: wrap them in a transaction.
BEGIN;
lots of insert statements here;
COMMIT;
Without this, sqlite will need to write to the file after each insert to keep the ACID principle. The transaction let's it write to file later in bulk.
The answer to the syntax error could well be, that your strings are not enclosed in quotes in your SQL statement.