I have inherited a steep Teradata SQL query which runs on 3 Teradata databases.
Preferring not to get bogged down in the functional aspects of the query (with various windowing statements), I would like to pass the query explicitly through to Teradata (same server).
The construct that I am familiar with connects to only one database, e.g.:
proc sql;
connect to teradata (user="userid" password="password1" mode=teradata
database=DB1 tdpid="MyServer");
create table TD_Results as
select * from connection to TERADATA
(
... TD SQL CODE
... TD SQL CODE
);
quit;
Does anyone have an idea as to how the original TD SQL query referencing 3 databases could be used via passthrough?
Thanks.
Q.
What Teradata calls a DATABASE is what ORACLE calls a SCHEMA. You just use a two level name to reference the tables.
select a.x,b.y,c.z
from db1.table1 a
, db2.table2 b
, db3.table3 c
If you mean that you need to select from multiple servers then I think you need to look into using QueryGrid syntax. In that syntax you can add the server name with a trailing # on the table reference.
select a.x,b.y,c.z
from db1.table1 a
, db2.table2#server2 b
, db3.table3 c
Related
I have just started studying SAS and a little bit confused. This link here show a Query to the DATA SET. I thought that it would be like connecting to a external DATABASE and perform a request query to a DATABASE.
So does the DATA SET is the database and the SQL syntax is just another way of processing data in the DATA SET?
Also can you recommend a better tutorial. A free open source tutorial/book/sources will be much better.
Well I'm still learning and I will appreciate any opinion/answer/recommendation.
I use SAS University Edition in virtually in my computer.
So does the DATA SET is the database and the SQL syntax is just another way of processing data in the DATA SET?
DATA SET is the table (not the database), and yes SQL is another way.
You can think of the native SAS library engine, V9 as the data base. For example:
libname mydata 'c:\projectx\sasdata'; is the same as
libname mydata V9 'c:\projectx\sasdata';
libname mydata <engine> 'c:\projectx\sasdata';
libname mydata <engine> <options for connection parameters>;
V9 is the default engine used when the libname statement does specify one. There are different engines for connecting to almost any remote (non-SAS) data bases, data files or data providers that let a SAS coder code in SAS and not have to learn the language or dialect of the remote environment.
A rough mapping of SAS structure concepts to data base concepts:
V9 engine ~ "data base"
local folder ~ schema, instance, or catalog
data set ~ table
variable ~ column
observation ~ row
You can learn more about engines by searching the help system for "SAS Engines" and "How Engines Work with SAS Files"
Proc SQL lets you code using SQL. A coder can choose the best language for themselves and for the problem at hand; be it SQL, DATA steps and PROC steps.
Do not confuse SQL (query language) with mySQL, postgresql, sqlite or any other database technology.
proc sql is an alternative to the data step.
Mostly you can do the same with both, but one might be able to perform better in certain situation or allow for easier/shorter syntax than the other.
The dataset you use has nothing to do with the language you use to "query" it.
Look into LIBNAME statement to connect to external databases.
As someone said before, do not confuse between SQL (the query language) and DataSet (its the name of the tables in SAS).
Here is an example of the same result using DATA SET syntax and PROC SQL syntax:
With DATA SET:
DATA myNewTable;
SET myTable;
WHERE id = 123;
RUN;
With PROC SQL syntax:
PROC SQL;
CREATE TABLE myNewTable AS
SELECT * FROM myTable
WHERE id = 123;
QUIT;
Hope it makes sense.
I work with SAS on a relationnal database that I can access with a libname odbc statement as below :
libname myDBMS odbc datasrc="myDBMS";
Say the database contains a table named 'myTable' with a numeric variable 'var_ex' which values can be 0,1 or . (missing). Now say I want to exclude all rows for which var_ex=1.
If I use the following :
DATA test1;
SET myDBMS.myTable; /* I call directly the table from the DBMS */
where var_ex NE 1;
run;
I don't get rows for which 'var_ex' is missing. Here is a screenshot of the log, with my actual data :
Whereas if I do the exact same thing after importing the table in the Work :
DATA myTable; /* I put myTable in the Work library */
SET myDBMS.myTable;
run;
DATA test2;
SET myTable; /* I call the table from the work */
where var_ex NE 1;
run;
I select rows for which 'var_ex' is 0 or missing, as intended. Here is a screenshot of the log, with my actual data :
The same happens if I use PROC SQL instead of a DATA step, or another NE-like.
I did some research and more or less understood here that unintended stuff like that can happen if you work directly on a DBMS table.
Does that mean is it simply not recommended to work with a DBMS table, and one has to import table locally as below before doing anything ?
DATA myTable; /* I put myTable in the Work library */
SET myDBMS.myTable;
run;
Or is there a proper way to manipulate such tables ?
The best way to test how SAS is translating the data step code into database code is through the sastrace system option. Before running code, try this:
options sastrace=',,,db' sastraceloc=saslog;
Then run your code tests. When you check the log, you will see precisely how SAS is translating the code (if it can at all). If it can't, you'll see,
ACCESS ENGINE: SQL statement was not passed to the DBMS, SAS will do the processing.
followed by a select * from table.
In general, if SAS cannot translate data step code into dbms-specific code, it will pull everything to locally manipulate the data. By viewing this output, you can determine precisely how to get the data step to translate into what you need.
If all else fails, you can use explicit SQL pass-through. The code in parentheses operates the same way as if you're running SQL directly from some other client.
proc sql;
connect to odbc(datasrc='source' user='username' pass='password');
create table want as
select * from connection to odbc
(<code specific to your dbms language>);
disconnect from odbc;
quit;
I have 2 SQL Join transforms (one using only SELECT, the other using SELECT and WHERE) and I have syntax error near OVER when submitting the following code:
select row_number () over
(
partition by t0.A order by t0.B DESC
) nub,
t0.C,
t0.D
from t0
I am new to SAS DI Studio, but this code works in Teradata, where am I going wrong?
Firstly, I can tell why this error is occurring, but it may fall to someone who has access to DI Studio to answer it fully for you. I've done my best to explain below. Feel free to comment.
The ROW_NUMBER() and PARTITION BY construct in Teradata SQL are not supported in SAS SQL. In order to accommodate this sort of syntax in DI Studio, you'll need to utilise 'pass-through' SQL. The syntax in Base SAS would be as follows:
PROC SQL;
CONNECT TO TERADATA (/* Insert connection details */);
CREATE TABLE sas_table AS
SELECT * FROM CONNECTION TO TERADATA (
/* Insert your Teradata SQL here */
);
DISCONNECT FROM TERADATA;
QUIT;
The syntax above will connect to Teradata and then create a SAS table called sas_table based on the results of your Teradata SQL query.
Dependent upon your version of DI Studio, there may be a specific transform available for Pass Through SQL.
I'm using right now , and its working, a proc sql to connect to my DB(access .accdb). Until now I was only using it to do SELECT query. Here is a example that work and that I use to do so.
proc sql;
/* create an ODBC pass-through connection using the Microsoft Office Access 2007 .accdb driver */
connect to ODBC as savesdb
(required="driver=Microsoft Access Driver (*.mdb, *.accdb); dbq=&dir_BD.;");
create table MYTABLE as select * from connection to savesdb(select MYID from ACCESSTABLE);
/* close the pass-through connection */
disconnect from savesdb;
quit;
Now I want to execute a INSERT INTO query. I know that the next code is working
execute( INSERT INTO ACCESSTABLE ( MYID ) VALUES ( 1 )) by savesdb;
The thing is in the INSERT INTO I want to specify to insert the values that are in a dataset. In other word I have a dataset with 4 records so I want to call my insert into 4 times with the values in dataset.
Is there a way to do so?
All you need to do is loop through the DataSet in a data step and read the columns to a macro variable.. one var for each column.
then call the proc sql code with these macro variables as input to the macro definition which will do the insert into DB using full pass through.
however as pointed out libref is the easier method.
Thanks,
Manish
I am trying to use PROC SQL to query a DB2 table with hundreds of millions of records. During the development stage, I want to run my query on an arbitrarily small subset of those records (say, 1000). I've tried using INOBS to limit the observations, but I believe that this parameter is simply limiting the number of records which SAS is processing. I want SAS to only fetch an arbitrary number of records from the database (and then process all of them).
If I were writing a SQL query myself, I would simply use SELECT * FROM x FETCH FIRST 1000 ROWS ONLY ... (the equivalent of SELECT TOP 1000 * FROM x in SQL Server). But PROC SQL doesn't seem to have any option like this. It's taking an extremely long time to fetch the records.
The question: How can I instruct SAS to arbitrarily limit the number of records to return from the database.
I've read that PROC SQL uses ANSI SQL, which doesn't have any specification for a row limiting keyword. Perhaps SAS didn't feel like making the effort to translate its SQL syntax to vendor-specific keywords? Is there no work around?
Have you tried using the outobs option in your proc sql?
For example,
proc sql outobs=10; create table test
as
select * from schema.HUGE_TABLE
order by n;
quit;
Alternatively, you can use SQL passthrough to write a query using DB2 syntax (FETCH FIRST 10 ROWS ONLY), although this requires you to store all your data in the database, at least temporarily.
Passthrough looks something like this:
proc sql;
connect to db2 (user=&userid. password=&userpw. database=MY_DB);
create table test as
select * from connection to db2 (
select * from schema.HUGE_TABLE
order by n
FETCH FIRST 10 ROWS ONLY
);
quit;
It requires more syntax and can't access your sas datasets, so if outobs works for you, I would recommend that.
When SAS is talking to a database via SAS syntax, part of the query can be translated to DBMS language equivalent - this is called implicit pass through. The rest of the query is "post-processed" by SAS to produce final result.
Depending on SAS version, DBMS vendor and DBMS version, and in some cases even some connection/libname options, different parts of SAS syntax are translatable/considered compatible between SAS and DBMS and thus sent to be performed by DBMS instead of SAS.
With SAS SQL options - INOBS and OUTOBS - I've worked a lot with MS SQL and Oracle via different versions of SAS, but I haven't seen those ever translated to TOP xxx type of queries, so this is probably not supported yet, although when query touches just DMBS data (no joins to SAS data etc), should be quite doable.
So I think you're left with the so called explicit pass-through - specific SAS SQL syntax to connect to database. This type of queries look like this:
proc sql;
connect to oracle as db1 (user=user1 pw=pasw1 path=DB1);
create table test_table as
select *
from connection to db1
( /* here we're in oracle */
select * from test.table1 where rownum <20
)
;
disconnect from db1;
quit;
In SAS 9.3 the syntax can be simplified - if there's already a LIBNAME connection, you can reuse it for explicit pass-through:
LIBNAME ORALIB ORACLE user=...;
PROC SQL;
connect to oracle using ORALIB;
create table work.test_table as
select *
from connection to ORALIB (
....
When connecting using libname be sure to use READBUFF (I usually set some 5000 or so) or INSERTBUFF options (1000 or more) when loading database.
To see if implicit pass-through takes place, set sastrace option:
option sastrace=',,,ds' sastraceloc=saslog nostsuffix;