How can I check if the values are obtained by this condition?
data ad01(keep=str);
length str $1024;
set Address(where=(Type_="1"));
///if resultat not null do something
run;
As #Quentin suggests this becomes a macro question. one way to do this is capturing count into macro variable.
/* an example where count is zero*/
proc sql noprint;
select count(*) into :cnt from sashelp.class
where name = 'unknown';
%put &cnt;
/* Here datastep gets aborted if count = 0 */
data class;
if &cnt = 0 then abort;
set sashelp.class;
run;
/* above query works when there is count gt 0 see the example below*/
proc sql noprint;
select count(*) into :cnt from sashelp.class
where trim(name) = 'Alfred';
%put &cnt;
data class;
if &cnt = 0 then abort;
set sashelp.class (where =(trim(name) = 'Alfred'));
run;
I’m assuming you mean if there are any records that meet your WHERE condition you want to process them somehow.
You can do that with just:
data ad01(keep=str);
length str $1024;
set Address(where=(Type_="1"));
*do something;
run;
If the dataset has no records with TYPE_=“1” the step will end when the SET statement executes and hits the (logical) end of file mark. If there are records that meet the condition, then any statements after the SET statement will be executed.
If by “do something” you mean execute additional steps (not just statements) this could become a macro language question. Where you would want a macro with an %IF statement, something like:
%if %anyobs(Address(where=(Type_=“1”))) %then %do;
*data steps and PROC steps and whatever here;
%end;
For examples of function-style %anyobs macros see e.g. http://www2.sas.com/proceedings/sugi26/p095-26.pdf or https://www.devenezia.com/downloads/sas/macros/index.php?m=anyobs or http://www.datasavantconsulting.com/roland/Spectre/utilmacros/nlobs.sas
If you want to do something when the SET statement return 0 observations you have to do it BEFORE the SET. With 0 obs the SET returns and data step ends. Consider this code where an observation is output when the SET returns 0 obs.
data class;
if 0 then set sashelp.class;
if _n_ eq 1 and eof then do;
name = 'NODATA';
output;
end;
set sashelp.class(where=(sex='X')) end=eof;
run;
proc print; run;
Related
I want to sweep each table in a libname and calculate a hash over each row.
For that purpose, i have already a table with libname, memname, concatenated columns with ',' and number of observations
libname
memname
columns
num_obs
lib_A
table_A
col1a,col2a...colna
1
lib_A
table_B
col1b,col2b...colnb
2
lib_B
table_C
col1c,col2c...colnc
1
I first get all data into ranged macro variables (i think its easier to work, but could be wrong, ofc)
proc sql;
select libname, memname, columns, num_obs
into :lib1-, :tab1-, :column1-, :sqlobs1-
from have
where libname="&sel_livraria"; /*macro var = prompt from user*/
quit;
Just for developing guideline i made the code just to check one specific table without getting the row number of it since with a simple counter doesn't work (i get the order of the rows mess up each time i run) and it works for that purpose
%let lib=lib_A;
%let tab=table_B;
%let columns=col1b,col2b,colnb;
data want;
length check $32.;
format check $hex32.;
set &lib..&tab;
libname="&lib";
memname="&tab";
check = md5(cats(&columns));
hash = put(check,$hex32.);
keep libname memname hash;
put hash;
put _all_;
run;
So, what’s the best approach for getting a MD5 from each row (same order as tables) of all tables in a libname? I saw problems i couldn’t overcame using data steps, procs or macros.
The result i wanted if lib_A was selected in prompt were something like:
libname
memname
obs_row
hash
lib_A
table_A
1
64A29CCA15F53C83A9583841294A26AA
lib_A
table_B
1
80DAC7B9854CF71A67F9C00A7EC4D9EF
lib_A
table_B
2
0AC44CD79DAB2E33C93BB2312D3A9A40
Need some help.
Tks in advance.
You're pretty close. This is how I would approach it. We'll create a macro with three parameters: data, lib, and out. data is the dataset you have with the column information. lib is the library you want to pull from your dataset, and out is the output dataset that you want to have.
We'll read each column into an individual macro variable:
memname1
memname2
memname3
libname1
libname2
libname3
etc.
From here, we simply need to loop over all of the macro variables and apply them where appropriate. We can easily count how many there are in a data step. All we need to do is add double-ampersands to resolve them correctly. For more information on why this is, check out this MWSUG paper.
%macro get_md5(data=, lib=, out=);
/* Save all variables into macro variables:
memname1 memname2 ...
columns1 columns2 ...
*/
data _null_;
set &data.;
where upcase(libname)=upcase("&lib.");
call symputx(cats('memname', _N_), memname);
call symputx(cats('columns', _N_), columns);
call symputx(cats('obs', _N_), obs);
call symputx('n_datasets', _N_);
run;
/* Loop through all the datasets and access each macro variable */
%do i = 1 %to &n_datasets.;
/* Double ampersand needed:
First, resolve &i. to get &memname1
Then resolve &mename1 to get the value stored in the macro variable memname1
*/
%let memname = &&memname&i.;
%let columns = &&columns&i.;
%let obs = &&obs&i.;
/* Calculate md5 in a temporary dataset */
data _tmp_;
length lib $8.
memname $32.
obs_row 8.
hash $32.
;
set &lib..&memname.(obs=&obs.);
lib = "&lib.";
memname = "&memname.";
obs_row = _N_;
hash = put(md5(cats(&columns.)), $hex32.);
keep libname memname obs_row hash;
run;
/* Overwrite the dataset so we don't keep appending */
%if(&i. = 1) %then %do;
data &out.;
set _tmp_;
run;
%end;
%else %do;
proc append base=&out. data=_tmp_;
run;
%end;
%end;
/* Remove temporary data */
proc datasets lib=work nolist;
delete _tmp_;
quit;
%mend;
Example:
data have;
length libname memname columns $15.;
input libname$ memname$ columns$ obs;
datalines;
sashelp cars make,model,msrp 1
sashelp class age,height,name 2
sashelp comet dose,length,sample 1
;
run;
%get_md5(data=have, lib=sashelp, out=want);
Output:
libname memname obs_row hash
sashelp cars 1 258DADA4843E7068ABAF95667E881B7F
sashelp class 1 29E8F4F03AD2275C0F191FE3DAA03778
sashelp class 2 DB664382B88BE7E445418B1A1C8CE13B
sashelp comet 1 210394B77E7696506FDEFD78890A8AB9
I would make a macro that takes as input the four values in your metadata dataset. Note that commas are anathema to SAS programs, especially macro code, so make the macro so it can accept space delimited variable lists (like normal SAS program statements do).
To reduce the risk of name conflict I will name the variable using triple underscores and then rename them back to human friendly names when the dataset is written.
%macro next_ds(libname,memname,num_obs,varlist);
data next_ds;
length ___1 $8 ___2 $32 ___3 8 ___4 $32 ;
___1 = "&libname";
___2 = "&memname";
___3 + 1;
set &libname..&memname(obs=&num_obs keep=&varlist);
___4 = put(md5(cats(of &varlist)),$hex32.);
keep ___1-___4 ;
rename ___1=libname ___2=memname ___3=obs_row ___4=hash;
run;
%mend next_ds;
Let's make some test metadata that reference datasets everyone should have.
data have;
infile cards truncover ;
input libname :$8. memname :$32. num_obs columns $200.;
cards;
sashelp class 3 name,sex,age
sashelp cars 2 make,model
;
And make sure the target dataset does not already exists.
%if %sysfunc(exist(want)) %then %do;
proc delete data=want; run;
%end;
Now you can call that macro once for each observation in your source metadata dataset. There is no need to generated oodles of macro variables. Instead you can use CALL EXECUTE() to generate the macro calls directly from the dataset.
We can replace the commas in the column lists when making the macro call. You can add in a PROC APPEND step after each macro call to aggregate the results into a single dataset.
data _null_;
set have;
call execute(cats(
'%nrstr(%next_ds)(',libname,',',memname,',',num_obs
,',',translate(columns,' ',','),')'
));
call execute('proc append data=next_ds base=want force; run;');
run;
Notice that wrapping the macro call in %NRSTR() makes the SAS log easier to read.
1 + %next_ds(sashelp,class,3,name sex age)
2 + proc append data=next_ds base=want force; run;
3 + %next_ds(sashelp,cars,2,make model)
4 + proc append data=next_ds base=want force; run;
Results:
Obs libname memname obs_row hash
1 sashelp class 1 5425E9CEDA1DDEB71B2692A3C7050A8A
2 sashelp class 2 C532D227D358A3764C2D225DC8C02D18
3 sashelp class 3 13AD5F1517E0C4494780773B6DC15211
4 sashelp cars 1 777C60693BF5E16F38706C89301CD0A8
5 sashelp cars 2 07080C9321145395D1A2BCC10FBE6B83
Note that CATS() might not be the best method for generating the string to pass to the MD5() function. That can generate the same string for different combinations of the source variables. For example 'AB' || 'CD' is the same as 'A' || 'BCD'. Perhaps just use CAT() instead.
Stu's approach is nice, and will work most of the time but will fall over when you have wiiiide variables, a large number of variables, variables with large precision, and other edge cases.
So for the actual hashing part, you might consider this macro, which is extensively tested within Data Controller for SAS:
https://core.sasjs.io/mp__md5_8sas.html
Usage:
data _null_;
set sashelp.class;
hashvar=%mp_md5(cvars=name sex, nvars=age height weight);
put hashvar=;
run;
I have a very large number of datasets that are not consistently formatted - I am trying to read them into SAS and normalize them.
The basic need here is to locate a 'key column' that contains a certain string - from there I know what to do with all the variables to the left and right of that column.
The 'GREP' macro from the sas website (http://support.sas.com/kb/33/078.html) seems like it can handle this, but I need help adapting the code in the following ways:
1 - I only need to search one dataset at a time, already in the 'work' library.
2 - I need to capture the name of the variable (and the position number of it) that prints to the log at the end of this macro. This seems like it would be easy but it just returns the last column in the dataset instead of the (correct) column that prints to the log at the end.
Current code below:
%macro grep(librf,string); /* parameters are unquoted, libref name, search string */
%let librf = %upcase(&librf);
proc sql noprint;
select left(put(count(*),8.)) into :numds
from dictionary.tables
where libname="&librf";
select memname into :ds1 - :ds&numds
from dictionary.tables
where libname="&librf";
%do i=1 %to &numds;
proc sql noprint;
select left(put(count(*),8.)) into :numvars
from dictionary.columns
where libname="&librf" and memname="&&ds&i" and type='char';
/* create list of variable names and store in a macro variable */
%if &numvars > 0 %then %do;
select name into :var1 - :var&numvars
from dictionary.columns
where libname="&librf" and memname="&&ds&i" and type='char';
quit;
data _null_;
set &&ds&i;
%do j=1 %to &numvars;
if &&var&j = "&string" then
put "String &string found in dataset &librf..&&ds&i for variable &&var&j";
%end;
run;
%end;
%end;
%mend;
%grep(work,Source Location);
The log returns: "String Source Location found in dataset WORK.RAW_IMPORT for variable C" (the third), which is correct.
I just need usable macro variables equal to "C" and "3" at the end. This macro will be part of a larger macro (or a prelude to it) so the two macro variables need to reset with each dataset I run through it. Thanks for any help offered.
Please find the modification below, basically what I have done was to create global macro variables for dataset name and variable name which will feed as input to get the variable position using VARNUM function as below, ( change identified by **** )
%macro grep(librf,string);
%let librf = %upcase(&librf);
proc sql noprint;
select left(put(count(*),8.)) into :numds
from dictionary.tables
where libname="&librf";
select memname into :ds1 - :ds&numds
from dictionary.tables
where libname="&librf";
%do i=1 %to &numds;
proc sql noprint;
select left(put(count(*),8.)) into :numvars
from dictionary.columns
where libname="&librf" and memname="&&ds&i" and type='char';
/* create list of variable names and store in a macro variable */
%if &numvars > 0 %then %do;
select name into :var1 - :var&numvars
from dictionary.columns
where libname="&librf" and memname="&&ds&i" and type='char';
quit;
%global var_pos var_nm var_ds;
data _null_;
set &&ds&i;
%do j=1 %to &numvars;
**** ADDED NEW CODE HERE ****;
if &&var&j = "&string" then do; /* IF-DO nesting */;
call symputx("var_nm","&&var&j"); /*Global Macro variable for Variable Name */
call symputx("var_ds","&&ds&i"); /*Global Macro variable for Dataset Name */
put "String &string found in dataset &librf..&&ds&i for variable &&var&j";
%end;
run;
**** ADDED NEW CODE HERE ****;
%let dsid=%sysfunc(open(&var_ds,i)); /* Open Data set */
%let var_pos=%sysfunc(varnum(&dsid,&var_nm)); /* Variable Position */
%let rc=%sysfunc(close(&dsid)); /* Close Data set */;
%end;
%end;
%mend;
%grep(work,Source Location);
%put &=var_nm &=var_ds &=var_pos;
I have multiple tables in a library call snap1:
cust1, cust2, cust3, etc
I want to generate a loop that gets the records' count of the same column in each of these tables and then insert the results into a different table.
My desired output is:
Table Count
cust1 5,000
cust2 5,555
cust3 6,000
I'm trying this but its not working:
%macro sqlloop(data, byvar);
proc sql noprint;
select &byvar.into:_values SEPARATED by '_'
from %data.;
quit;
data_&values.;
set &data;
select (%byvar);
%do i=1 %to %sysfunc(count(_&_values.,_));
%let var = %sysfunc(scan(_&_values.,&i.));
output &var.;
%end;
end;
run;
%mend;
%sqlloop(data=libsnap, byvar=membername);
First off, if you just want the number of observations, you can get that trivially from dictionary.tables or sashelp.vtable without any loops.
proc sql;
select memname, nlobs
from dictionary.tables
where libname='SNAP1';
quit;
This is fine to retrieve number of rows if you haven't done anything that would cause the number of logical observations to differ - usually a delete in proc sql.
Second, if you're interested in the number of valid responses, there are easier non-loopy ways too.
For example, given whatever query that you can write determining your table names, we can just put them all in a set statement and count in a simple data step.
%let varname=mycol; *the column you are counting;
%let libname=snap1;
proc sql;
select cats("&libname..",memname)
into :tables separated by ' '
from dictionary.tables
where libname=upcase("&libname.");
quit;
data counts;
set &tables. indsname=ds_name end=eof; *9.3 or later;
retain count dataset_name;
if _n_=1 then count=0;
if ds_name ne lag(ds_name) and _n_ ne 1 then do;
output;
count=0;
end;
dataset_name=ds_name;
count = count + ifn(&varname.,1,1,0); *true, false, missing; *false is 0 only;
if eof then output;
keep count dataset_name;
run;
Macros are rarely needed for this sort of thing, and macro loops like you're writing even less so.
If you did want to write a macro, the easier way to do it is:
Write code to do it once, for one dataset
Wrap that in a macro that takes a parameter (dataset name)
Create macro calls for that macro as needed
That way you don't have to deal with %scan and troubleshooting macro code that's hard to debug. You write something that works once, then just call it several times.
proc sql;
select cats('%mymacro(name=',"&libname..",memname,')')
into :macrocalls separated by ' '
from dictionary.tables
where libname=upcase("&libname.");
quit;
¯ocalls.;
Assuming you have a macro, %mymacro, which does whatever counting you want for one dataset.
* Updated *
In the future, please post the log so we can see what is specifically not working. I can see some issues in your code, particularly where your macro variables are being declared, and a select statement that is not doing anything. Here is an alternative process to achieve your goal:
Step 1: Read all of the customer datasets in the snap1 library into a macro variable:
proc sql noprint;
select memname
into :total_cust separated by ' '
from sashelp.vmember
where upcase(memname) LIKE 'CUST%'
AND upcase(libname) = 'SNAP1';
quit;
Step 2: Count the total number of obs in each data set, output to permanent table:
%macro count_obs;
%do i = 1 %to %sysfunc(countw(&total_cust) );
%let dsname = %scan(&total_cust, &i);
%let dsid=%sysfunc(open(&dsname) );
%let nobs=%sysfunc(attrn(&dsid,nobs) );
%let rc=%sysfunc(close(&dsid) );
data _total_obs;
length Member_Name $15.;
Member_Name = "&dsname";
Total_Obs = &nobs;
format Total_Obs comma8.;
run;
proc append base=Total_Obs
data=_total_obs;
run;
%end;
proc datasets lib=work nolist;
delete _total_obs;
quit;
%mend;
%count_obs;
You will need to delete the permanent table Total_Obs if it already exists, but you can add code to handle that if you wish.
If you want to get the total number of non-missing observations for a particular column, do the same code as above, but delete the 3 %let statements below %let dsname = and replace the data step with:
data _total_obs;
length Member_Name $7.;
set snap1.&dsname end=eof;
retain Member_Name "&dsname";
if(NOT missing(var) ) then Total_Obs+1;
if(eof);
format Total_Obs comma8.;
run;
(Update: Fixed %do loop in step 2)
I have several databases, one per geographical variables, that I want to append in the end. I am doing some data steps on them. As I have large databases, I select only the variables I need when I first call each table. But on tables in which one variable always equals 0, the variable is not in the table.
So when I select my (keep=var) in a for loop, it works fine if the variable exists, but it produces an error in the other case, so that these tables are ignored.
%do i=1 to 10 ;
data temp;
set area_i(keep= var1 var2);
run;
proc append base=want data=temp force;
run;
%end;
Is there a simple way to tackle that ?
In fact I have found a solution : the DKRICOND (or DKROCOND) options specify the level of error detection to report when a variable is missing from respectively an input (or output) data set during the processing of a DROP=, KEEP=, or RENAME= data set option.
The options are DKRICOND=ERROR | WARN | WARNING | NOWARN | NOWARNING, so you just wave to set
dkricond=warn
/*your program, in my case :*/
%do i=1 to 10 ;
data temp;
set area_i(keep= var1 var2);
run;
proc append base=want data=temp force;
run;
%end;
dkricond=error /* the standard value, probably better to set it back after/ */
How about just adding it to the table if it doesn't already exist?
/*look at dictionary.columns to see if the column already exists*/
proc sql;
select name into :flag separated by ' ' from dictionary.columns where libname = 'WORK' and memname = 'AREA_I' and name = 'VAR1';
run;
/*if it doesn't, then created it as empty*/
%if &flag. ne VAR1 %then %do;
data area_i;
set area_i;
call missing(var1);
run;
%end;
Is there a way to check how many observations are in a SAS data set at runtime OR to detect when you've reached the last observation in a DATA step?
I can't seem to find anything on the web for this seemingly simple problem. Thanks!
The nobs= option to a set statement can give you the number of observations. When the data step is compiled, the header portion of the input datasets are scanned, so you don't even have to execute the set statement in order to get the number of observations. For instance, the following reports 2 as expected:
/* a test data set with two observations and no vars */
data two;
output;
output;
run;
data _null_;
if 0 then set two nobs=nobs;
put nobs=;
run;
/* on log
nobs=2
*/
The end= option sets a flag when the last observation (for the set statement) is read in.
A SAS data set, however, can be a SAS data file or a SAS view. In the case of the latter, the number of observations may not be known either at compile time or at execution time.
data subclass/view=subclass;
set sashelp.class;
where sex = symget("sex");
run;
%let sex=F;
data girls;
set subclass end=end nobs=nobs;
put name= nobs= end=;
run;
/* on log
Name=Alice nobs=9.0071993E15 end=0
Name=Barbara nobs=9.0071993E15 end=0
Name=Carol nobs=9.0071993E15 end=0
Name=Jane nobs=9.0071993E15 end=0
Name=Janet nobs=9.0071993E15 end=0
Name=Joyce nobs=9.0071993E15 end=0
Name=Judy nobs=9.0071993E15 end=0
Name=Louise nobs=9.0071993E15 end=0
Name=Mary nobs=9.0071993E15 end=1
*/
You can also use %sysfunc(attrn( dataset, nlobs)) though it is limited to SAS data sets (i.e. not data views). Credit for the macro to this SUGI paper, which also give great information regarding good macro design.
You can get all sorts of other character and numeric information on a SAS data set.
See the documentation on attrn and attrc.
%macro numobs (data=&syslast ) ;
/* --------------------------------------------
Return number of obs as a function
--------------------------------------------
*/
%local dsid nobs rc;
%let data = &data ; /* force evaluation of &SYSLAST */
%let dsid=%sysfunc(open(&data));
%if &dsid > 0 %then
%do ;
%let nobs=%sysfunc(attrn(&dsid,nlobs));
%let rc=%sysfunc(close(&dsid));
%end ;
%else
%let nobs = -1 ;
&nobs
%mend numobs;
Find the number of observations in a SAS data set:
proc sql noprint;
select count(*) into: nobs
from sashelp.class
;
quit;
data _null_;
put "&nobs";
run;
The SQL portion counts the number of observaions, and stores the number in a macro variable called "nobs".
The data step puts the number for display, but you can use the macro variable like any other.
Performing a certain action when the last observation is processed:
data _null_;
set sashelp.class end=eof;
if eof then do;
put name= _n_=;
end;
run;
The "end" option to the "set" statement defines a variable (here "eof" for end-of-file) that is set to 1 when the last observation is processed. You can then test the value of the variable, and perform actions when its value is 1. For more info, see the documentation for the "set" statement.
data hold;
set input_data end=last;
.
.
.
if last then do;
.
.
.
end;
run;