How RedShift Sessions are handled from a Server Connection for TEMP tables - coldfusion

I'm using ColdFusion to connect to a RedShift database and I'm trying to understand how to test/assume myself of how the connections work in relation to TEMP tables in RedShift.
In my CFADMIN for the datasource I have unchecked Maintain connections across client requests. I would assume then each user who is using my website would have their own "Connection" to the DB? Is that correct?
Per the RedShift docs about temp tables:
TEMP: Keyword that creates a temporary table that is visible only within the current session. The table is automatically dropped at the end of the session in which it is created. The temporary table can have the same name as a permanent table. The temporary table is created in a separate, session-specific schema. (You cannot specify a name for this schema.) This temporary schema becomes the first schema in the search path, so the temporary table will take precedence over the permanent table unless you qualify the table name with the schema name to access the permanent table.
Am I to understand that if #1 is true and each user has their own connection to the database and thereby their own session then per #2 any tables that are created will be only in that session even though the "user" is the same as it's a connection from my server that is using the same credentials.
3.If my assumptions in #1 and #2 are correct then if I have ColdFusion code that runs a query like so:
drop if exists tablea
create temp table tablea
insert into tablea
select * from realtable inner join
drop tablea
And multiple users are using that same function that does this. They should never run into any conflicts where one table gets dropped as another request is trying to use it correct?
How do I test that this is the case? Besides throwing it into production and waiting for an error how can I know. I tried running a few windows side by side in different browsers and stuff and didn't notice an issue, but I don't know how to know if the temp tables truly are different between clients. (as they should be.) I imagine I could query some meta data but what meta data about the table would tell me that?

I have a similar situation, but with redbrick database software. I handle it by creating unique table names. The general idea is:
Create a table name something like this:
<cfset tablename = TableText & randrange(1, 100000)>
Try to create a table with that name. If you fail try again with a different name.
If you fail 3 times stop trying and mail the cfcatch information to someone.
I have all this code in a custom tag.
Edit starts here
Based on the comments, here is some more information about my situation. In CFAdmin, for the datasource being discussed, the Maintain Connections box is checked.
I put this code on a ColdFusion page:
<cfquery datasource="dw">
create temporary table dan (f1 int)
</cfquery>
I ran the page and then refreshed it. The page executed successfully the first time. When refreshed, I got this error.
Error Executing Database Query.
** ERROR ** (7501) Name defined by CREATE TEMPORARY TABLE already exists.
That's why I use unique tablenames. I don't cache the queries though. Ironically, my most frequent motivation for using temporary tables is because there are situations where they make things run faster than using the permanent tables.

Related

Redshift Table Ownership And Drop Query

Users & Scope -
write_user - All Access to all Tables
read_user - Read access to all Tables
backup_pruner - All GRANTS to all tables in schema backup.
My Problem Statement -
I have written an automation that has to drop tables of a schema called backup where tables are created by write_user.
Now, for dropping tables I have to user the backup_pruner user and here is the problem.
Since write_user creates table here, it is the owner of all tables in backup & Only Owners/Super-Users can drop tables.
How to Proceed from here ?
To answer the question WHY to use a separate user to DROP Tables -
To tighten the accessibility of tables as DROP if not used properly/any corner case, then can be disastrous for other tables too.
Consider using a Stored Procedure created with the SECURITY DEFINER to drop the tables. An SP created this way runs with the permissions of the creator.
You can define a list of table names allowed to be dropped that the SP checks before taking action.
I created an example of this approach on GitHub: sp_controlled_access

How can I create a model with ActiveRecord capabilities but without an actual table behind?

I think this is a recurrent question in the Internet, but unfortunately I'm still unable to find a successful answer.
I'm using Ruby on Rails 4 and I would like to create a model that interfaces with a SQL query, not with an actual table in the database. For example, let's suppose I have two tables in my database: Questions and Answers. I want to make a report that contains statistics of both tables. For such purpose, I have a complex SQL statement that takes data from these tables to build up the statistics. However the SELECT used in the SQL statement does not directly take values from neither Answers nor Questions tables, but from nested SELECTs.
So far I've been able to create the StatItem model, without any migration, but when I try StatItem.find_by_sql("...nested selects...") the system complains about unexisting table stat_items in the database.
How can I create a model whose instance's data is retrieved from a complex query and not from a table? If it's not possible, I could create a temporary table to store the data in there. In such case, how can I tell the migration file to not create such table (it would be created by the query)?
How about creating a materialized view from your complex query and following this tutorial:
ActiveRecord + PostgreSQL Materialized Views
Michael Kohl and his proposal of materialized views has given me an idea, which I initially discarded because I wrongly thought that a single database connection could be shared by two processes, but after reading about how Rails processes requests, I think my solution is fine.
STEP 1 - Create the model without migration
rails g model StatItem --migration=false
STEP 2 - Create a temporary table called stat_items
#First, drop any existing table created by older requests (database connections are kept open by the server process(es).
ActiveRecord::Base.connection.execute('DROP TABLE IF EXISTS stat_items')
#Second, create the temporary table with the desired columns (notice: a dummy column called 'id:integer' should exist in the table)
ActiveRecord::Base.connection.execute('CREATE TEMP TABLE stat_items (id integer, ...)')
STEP 3 - Execute an SQL statement that inserts rows in stat_items
STEP 4 - Access the table using the model, as usual
For example:
StatItem.find_by_...
Any comments/improvements are highly appreciated.

Conditionally drop temporary table in Redshift

We are using http://aws.amazon.com/redshift/ and I am creating/dropping temporary tables in reports. Occasionally we encounter cases where someone has created a temporary table and failed to drop it.
In other databases, for instance PostgreSQL which Redshift is based on, I could simply:
DROP TEMP TABLE IF EXISTS tblfoo;
But that is a syntax error in Redshift. I can check for the existence of temporary tables myself using http://docs.aws.amazon.com/redshift/latest/dg/r_STV_TBL_PERM.html but that only works if I am a superuser and I am not running as a superuser. I could also go and swallow exceptions, but with my reporting framework I'd prefer not to go there.
So how can I, as a regular user and without generating database errors, conditionally drop a temporary table if it exists?
The test I ran showed that I could see other users' temp tables in stv_tbl_perm using a non-super user id. The cluster version I tested in is 1.0.797. Note that no users can see other users' temp tables in pg_class.

How to check whether sqlite database is attached or not?

I am using sqlite to store my data. I have two databases. In my application, each time a new request comes, I am attaching first db to second db. The problem is, if two request come it is showing the db already in use (it is trying to attach twice with same alias name 'db'). I want to know if there is any way to check whether a database is attached or not?
PRAGMA database_list;
outputs a resultset with full list of available databases. The first column is database name, the second is database file (empty if it is not associated with file). The primary database is always named main, temporary db is always temp.
sqlite> attach "foo.db" as foo;
sqlite> pragma database_list;
0|main|
2|foo|/Users/me/tmp/foo.db
I assume you are reusing the same connection to the database for multiple requests. Because databases are attached to the connection object, attaching fails for the second or further requests with the same connection. The solution I think is thus to attach the database immediately after a new connection is made, and not each time a request is received.

How to Execute 2 or more insert statements using CFQuery in coldfusion?

Is it possible to Execute 2 insert or Update Statements using cfquery?
If yes how?
if no, what is the best way to execute multiple queries in Coldfusion, by opening only one Connection to DB.
I think every time we call cfquery we are opening new connection DB
Is it possible to Execute 2 insert or
Update Statements using cfquery?
Most likely yes. But whether you can run multiple statements is determined by your database type and driver/connection settings. For example, when you create an MS SQL datasource, IIRC multiple statements are allowed by default. Whereas MySQL drivers often disable multiple statements by default. That is to help avoid sql injection. So in that case you must to enable multiple statements explicitly in your connection settings. Otherwise, you cannot use multiple statements. There are also some databases (usually desktop ones like MS Access) that do not support multiple statements at all. So I do not think there is a blanket answer to this question.
If the two insert/update statements are related, you should definitely use a cftransaction as Sam suggested. That ensures the statements are treated as a single unit: ie Either they all succeed or they all fail. So you are not left with partial or inconsistent data. In order to accomplish that, a single connection will be used for both queries in the transaction.
I think every time we call cfquery we
are opening new connection DB
As Sam mentioned, that depends on your settings and whether you are using cftransaction. If you enable Maintain Connections (under Datasource settings in the CF Administrator) CF will maintain a pool of open connections. So when you run a query, CF just grabs an open connection from the pool, rather than opening a new one each time. When using cftransaction, the same connection should be used for all queries. Regardless of whether Maintain Connections is enabled or not.
Within the data source settings you can tell it whether to keep connections open or not with the Maintain Connections setting.
Starting with, I believe, ColdFusion 8 datasources are set up to run only one query at a time due to concerns with SQL injection. To change this you would need to modify with the connection string.
Your best bet is to turn on Maintain Connections and if needed use cftransaction:
<cftransaction>
<cfquery name="ins" datasource="dsn">
insert into table1 values(<cfqueryparam value="#url.x#">)
</cfquery>
<cfquery name="ins" datasource="dsn">
insert into table2 values(<cfqueryparam value="#url.x#">)
</cfquery>
</cftransaction>
And always, always use cfqueryparam for values submitted by users.
the mySQL driver in CF8 does now allow multiple statements.
as Sam says, you can use to group many statements together
or in the coldfusion administrator | Data & Services | Data sources,
add
allowMultiQueries=true
to the Connection String field
I don't have CF server to try, but it should work fine IIRC.
something like:
<cfquery name="doubleInsert" datasource="dsn">
insert into table1 values(x,y,z)
insert into table1 values(a,b,c)
</cfquery>
if you want a more specific example you will have to give more specific information.
Edit: Thanks to #SamFarmer : Newer versions of CF than I have used may prevent this
Sorry for the Necro (I'm new to the site).
You didn't mention what DB you're using. If you happen to use mySQL you can add as many records as the max heap size will allow.
I regularly insert up to ~4500 records at a time with the default heap size (but that'll depend on the amount of data you have).
INSERT INTO yourTable (x,y,z) VALUES ('a','b','c'),('d','e','f'),('g','h','i')
All DBs should do this IMO.
HTH
Use CFTRANSACTION to group multiple queries into a single unit.
Any queries executed with CFQUERY and placed between and tags are treated as a single transaction. Changes to data requested by these queries are not committed to the database until all actions within the transaction block have executed successfully. If an error occurs in a query, all changes made by previous queries within the transaction block are rolled back.
Use the ISOLATION attribute for additional control over how the database engine performs locking during the transaction.
For more information visit http://www.adobe.com/livedocs/coldfusion/5.0/CFML_Reference/Tags103.htm