I don't know if it is the right place to post this, but it seemed to me I would be more likely to get an answer there.
Currrently working on SAS for an internship, I am trying to write a macro in order to automate the process of finding a fitting ARIMA model for my data sets. I am very new to this software, and quite not a specialist in the field of statistics.
However, while I seemingly understood how to import my files and launch the proc arima, I am stuck on a little problem. A part of my code, which is working fine if I write it outside of the macro (I guess it's called open code ?) like this :
data _null_;
set Lib.out; /* Lib.out contains the data of the OUTSTAT statement of the PROC ARIMA */
x = 1000000;
put _STAT_; /* Prints correctly the names of the different lines in the log */
if _STAT_='AIC' then do; /* _STAT_ is a column and AIC the name of a line AFAIK */
if _VALUE_ < x then
x = _VALUE_;
put x;
put _STAT_; /* Here only prints AIC, which I guess is correct inside of the IF loop */
end;
run;
But when running it inside a macro such as :
%macro recherche(poste=, mto=);
--- code ---
data _null_;
set Lib.out; /* Lib.out contains the data of the OUTSTAT statement of the PROC ARIMA */
%let aic0 = 1000000;
%put _STAT_; /* Doesn't recognize the _STAT_ statement and stops */
%if _STAT_='AIC' %then %do;
%if _VALUE_ < &aic0 %then %do;
&aic0 = _VALUE_;
data Lib.chosen;
set Lib.model; /* Contains the OUTMODEL statement of PROC ARIMA */
run;
%end;
end;
run;
--- code ---
I tried to search for similar cases on the internet but couldn't find an explanation for what I am looking for. Plus, being new to SAS, the official documentation is still hard to understand. Thanks in advance.
Most of what you're doing there doesn't need the %. You just need that if it's a statement controlling which lines of code are even sent to the compiler.
%macro recherche(mto=);
data _null_;
set Lib.out; /* Lib.out contains the data of the OUTSTAT statement of the PROC ARIMA */
x = &mto.;
put _STAT_; /* Prints correctly the names of the different lines in the log */
if _STAT_='AIC' then do; /* _STAT_ is a column and AIC the name of a line AFAIK */
if _VALUE_ < x then
x = _VALUE_;
put x;
put _STAT_; /* Here only prints AIC, which I guess is correct inside of the IF loop */
end;
run;
%mend recherche;
That's assuming the MTO parameter is intended to hold the value assigned to x. The only time you would use %IF is if you did something like
%macro recherche(mto=,stat=);
data _null_;
set Lib.out; /* Lib.out contains the data of the OUTSTAT statement of the PROC ARIMA */
x = &mto.;
put _STAT_; /* Prints correctly the names of the different lines in the log */
%if &stat=AIC %then %do;
if _STAT_='AIC' then do; /* _STAT_ is a column and AIC the name of a line AFAIK */
if _VALUE_ < x then
x = _VALUE_;
put x;
put _STAT_; /* Here only prints AIC, which I guess is correct inside of the IF loop */
end;
%end;
%else %if &stat=XYZ %then %do;
*more code ...;
%end;
run;
%mend recherche;
That would only do one or the other of those sections of code. Macro statements don't have access to the data in the proc or data step, and they don't use quotes (unless the quote is an actual important part of the code).
I ran into the same issue when combining a pipe statement do while loop within a macro. I would get the unclosed do loop error, yet the statements would work independently. I had tried every iteration to resolve, but to no avail. Finally, the solution for me was to put the pipe infile statement into its own program and then bring it into the macro using a %include. This worked perfectly! The issue is definitely a bug within SAS.
Related
I want to build a documentation pdf (or html) for a collection of SAS macros. Is there a canonic (or recommended) workflow that I can follow ?
I'm thinking of exporting all my macros and extract title, description, examples, variable descriptions and code using R and Regex then use markdown to build a nicely laid out pdf that I can update in a few steps whenever I add a macro or change descriptions or examples (I want to avoid copy/paste at all cost).
It'd be quite tedious and unflexible though and I might be reinventing the wheel.
My macros all look like this:
*--------------------------------------------------------;
* ASSERT_EXIST ;
* Fails explicitely when a table doesn't exist ;
* Accepts a list of tables as input ;
*--------------------------------------------------------;
/* EXAMPLES
%assert_exist(not_a_table); * prints explicit error and aborts;
%assert_exist(sashelp.class); * does nothing;
%assert_exist(sashelp.cars not_a_table sashelp.class); * prints explicit error and aborts;
*/
%macro assert_exist
(data /* table or list of tables */
);
%local i table;
%do i=1 %to %sysfunc(countw(&data,%str( )));
%let table = %scan(&data,&i,%str( ));
%if not %sysfunc(exist(&table)) %then %do;
%put ERROR: Table &table doesnt exist!;
%abort;
%end;
%end;
%mend;
*----------------------------------------------;
* DROP ;
* Delete table or list of tables ;
* Default deletes all tables starting with _ ;
*----------------------------------------------;
/* EXAMPLES
data _x;
input char1 $ num1;
datalines;
a 1
b 2
;
%put %sysfunc(exist(_x)); * 1;
%drop(_x);
%put %sysfunc(exist(_x)); * 0;
*/
%macro drop
(data /* Name of table to drop, end name of table with ':' to delete all tables with this prefix */
);
%if &data= %then %let data = _:;
proc datasets nolist;
delete &data;
run;
%mend;
*--------------------------------------------------------;
* HEAD ;
* select top rows ;
*--------------------------------------------------------;
/* EXAMPLES
%head(sashelp.class,2) * keep only 2 first rows;
* %drop(_TEMP_); * clean up;
*/
%macro head
(data /* source table */
,n /* number of rows to keep */
,out /* output table */
);
/* default values, checks, initialisations */
%if &data= %then %let data = _TEMP_;
%if &out= %then %let out = _TEMP_;
%if &out=. %then %let out = &data;
%assert_exist(&data)
proc sql inobs=&n;
CREATE TABLE &out AS
SELECT *
FROM &data;
quit;
%mend;
My collection of macros is growing I'd like to respect good practice as much as possible but I've not been able to find much information relative to good documentation in SAS.
#Allan-Bowe gave a great answer which is probably the best practice, but unfortunately I have no way to install doxygen from my work computer, so I'm looking for other solutions not requiring external software.
No need to re-invent the wheel - a great approach for documentation is doxygen.
We use it for the open source SASjs Macro Core library (which also lists a lot of good practices for SAS Macro development).
Simply define your attributes in the header (markdown is accepted), eg:
/**
#file
#brief Logs a key value pair a control dataset
#details If the dataset does not exist, it is created. Usage:
%mp_setkeyvalue(someindex,22,type=N)
%mp_setkeyvalue(somenewindex,somevalue)
#param key Provide a key on which to perform the lookup
#param value Provide a value
#param type= either C or N will populate valc and valn respectively. C is
default.
#param libds= define the target table to hold the parameters
#version 9.2
#author Allan Bowe
#source https://github.com/sasjs/core
**/
Then simply point doxygen at your source folder, tell it which config file to use (a good one for SAS is here) and then choose an output directory for your documentation.
It'll look like this.
There's no pdf option, but it can create files in DOCBOOK format that can be used to generate a pdf: http://www.doxygen.nl/manual/config.html#config_docbook
UPDATE - we recently added doxygen support to SASjs - with a single command (sasjs doc) you can document all your jobs, and even generate a graphviz data lineage diagram, integrated into the output.
Overview: https://www.youtube.com/watch?v=ESNdCtXKRrw
You already seem to be placing the parameter definitions on separate lines. That should help with parsing the source files. Also add the macro name to the %MEND statement so that your parsing code can have a double check that it didn't find the wrong one.
I would also recommend moving your comment blocks into the macro.
%macro assert_exist
(data /* table or list of tables */
);
/*--------------------------------------------------------;
* ASSERT_EXIST ;
* Fails explicitely when a table doesn't exist ;
* Accepts a list of tables as input ;
*--------------------------------------------------------;
EXAMPLES
%assert_exist(not_a_table); * prints explicit error and aborts;
%assert_exist(sashelp.class); * does nothing;
%assert_exist(sashelp.cars not_a_table sashelp.class); * prints explicit error and aborts;
*/
%local i table;
%do i=1 %to %sysfunc(countw(&data,%str( )));
%let table = %scan(&data,&i,%str( ));
%if not %sysfunc(exist(&table)) %then %do;
%put ERROR: Table &table doesnt exist!;
%abort;
%end;
%end;
%mend assert_exist;
Could someone please explain why it is happening?
I'm trying to find objects dependencies tree in a DB.
Let's say view5 is a view sits on top view4 which sits on top view1.
Also,
view3 sits on top view2 sits on top view1.
So,
the when I query the macro for view1, I should get back view4, view5, view2 and view3.
This is the macro:
%macro dependencies(obj=);
%let dependent_objectname =;
proc sql noprint;
select "'"||trim(dependent_objectname)||"'"
into :dependent_objectname separated by ", "
from &_input.
where src_objectname in (&obj.);
quit;
%put &dependent_objectname.;
%let dependent_objectname = (&dependent_objectname.);
%put &dependent_objectname.;
%if %length("&dependent_objectname")>0 %then
%dependencies(obj = &dependent_objectname.);
%mend dependencies;
%let source = 'ditemp.depend_test1';
%put &source.;
%dependencies(obj = &source.);
First iteration works well,
I get the objects sit on top depend_test1
in a form of "('ditemp.depend_test2','ditemp.depend_test3')"
then I'm checking for the length of variable dependent_objectname (greater than zero)
and calling the macro again,
only it never stops...
I see a couple problems.
The statement:
%if %length("&dependent_objectname")>0 %then %do;
will always return true, even if the value of &dependent_objectname is null. Because the quotes are part of the value in the macro language. You probably want:
%if %length(&dependent_objectname)>0 %then %do;
That test for nullness usually works. Or see this paper for better methods. http://support.sas.com/resources/papers/proceedings09/022-2009.pdf
Before that, the statement:
%let dependent_objectname = (&dependent_objectname.);
is adding parentheses to your value. So again, even if &dependent_objectname were null, it would be () after this. It looks like you don't need these parentheses, so I would skip this statement.
I would also add:
%local dependent_objectname ;
to the top of the macro. That way each invocation of the macro will have its own local macro variable, rather than having them all use the macro variable created in the first iteration (or worse yet, all use a global macro variable).
You have sensibly added %PUT statements to help with debugging. I would expect they would show that the value of &dependent_objectname is always non-null as currently written. You could also add:
%put The length is: %length(&dependent_objectname.) ;
Since you are using an SQL query to generate the dependent list you can use the automatic variable SQLOBS in your test to break the recursion.
%if &sqlobs %then %do;
%dependencies(obj = &dependent_objectname.);
%end;
Also do NOT use a comma as the delimiter between the items listed in the OBJ parameter. The IN operator in SAS doesn't need them and they will cause trouble in the macro call.
select * from sashelp.class where name in ('Alfred' 'Alice') ;
So your macro could look like this:
%macro dependencies(object_list);
%local dependent_list ;
proc sql noprint;
select catq('1as',dependent_objectname)
into :dependent_list separated by ' '
from &_input.
where src_objectname in (&object_list)
and dependent_objectname is not null
;
quit;
%put Dependent Objects of (&object_list) = (&dependent_list);
%if &sqlobs %then %dependencies(&dependent_list);
%mend dependencies;
And here is a test case.
%let _input=sample;
data sample;
length src_objectname dependent_objectname $41 ;
input (_all_) (:) ;
cards;
object1 object2
object2 object3
object2 object4
;;;;
%dependencies('object1');
I am an intermediate user of SAS, but I have limited knowledge of arrays and macros. I have a set of code that prompts the user to enter a date range. For example, the user might enter December 1, 2015-December 5,2015. For simplicity, imagine the code looks like:
data new; set old;
if x1='December 1, 2015'd then y="TRUE";
run;
I need to run this same code for every day in the date prompt range, so for the 1st, 2nd, 3rd, 4th, and 5th. My thought was to create an array that contains the dates, but I am not sure how I would do that. My second thought was to create a macro, but I can't figure out out to feed a list through a macro.
Also, just FYI, the code is a lot longer and more complicated than just a data step.
The following macro can be used as a framework for your code:
%MACRO test(startDate, endDAte);
%DO i=&startDate %to &endate;
/* data steps go here */
/* example */
DATA test;
SET table;
IF x1 = &i THEN y = "true";
RUN;
%END;
%MEND;
Look into call execute to call your macro and a data null step using a do loop to loop through the days. Getting the string correct for the call execute can sometimes be tricky, but worth the effort overall.
data sample;
do date='01Jan2014'd to '31Jan2014'd;
output;
end;
run;
%macro print_date(date);
proc print data=sample;
where date="&date"d;
format date date9.;
run;
%mend;
%let date_start=05Jan2014;
%let date_end=11Jan2014;
data _null_;
do date1="&date_start"d to "&date_end"d by 1;
str='%print_date('||put(date1, date9.)||');';
call execute(str);
end;
run;
I'm searching for a while an equivalent of the for in loop (like in Python or in R) in SAS 9.3 macro language. The DO loop seem's to be the solution but did't work exactly as I want.
I founded a way to do it in a data step with a DO loop but it don't work with the macro language.
For example, in a data step, this code is working :
DATA _NULL_;
DO i = 1,3,5,9;
PUT i;
END;
RUN;
And then the log prompt as expected :
1
3
5
9
When I try to do the same with an %DO loop in a Macro, I have an error.
%MACRO test();
%DO i = 1,2,4,9 ;
%PUT i = &i;
%END;
%MEND;
%test();
The log promp these messages :
ERROR: Expected %TO not found in %DO statement.
ERROR: A dummy macro will be compiled
I'm quite new in SAS and stackoverflow so I hope my question is no too stupid. It's so simple to do this in Python and R then it must have a simple way to do it in SAS.
Thank's for help - J. Muller
The closest I've ever come across to this pattern in SAS macro language is this:
%MACRO test();
%let j=1;
%let vals=1 2 4 9;
%do %while(%scan(&vals,&j) ne );
%let i=%scan(&vals, &j);
%put &i;
%let j=%eval(&j+1);
%end;
%MEND;
%test();
(Warning: untested, as I no longer have a SAS installation I can test this out on.)
You can certainly get around it this way:
options mindelimiter=,;
options minoperator;
%MACRO test();
%DO i = 1 %to 9 ;
%if &i in (1,2,4,9) %then %do;
%PUT i = &i;
%END;
%end;
%MEND;
%test();
However, I think you can usually avoid this sort of call by executing your macro multiple times rather than attempting to control the loop inside the macro. For example, imagine a dataset and a macro:
data have;
input x;
datalines;
1
2
4
9
;;;;
run;
%macro test(x);
%put &x;
%mend test;
Now you want to call %test() once for each value in that list. Okay, easy to do.
proc sql;
select cats('%test(',x,')') into :testcall separated by ' ' from have;
quit;
&testcall;
That works just as well as your %do in loop, except it's data driven, meaning if you want to change the calls you just change the dataset (or if your data changes, the call automatically changes!). In general, SAS is more effective when designed as data driven programming rather than as entirely written code.
I wonder if there is a way of detecting whether a data set is empty, i.e. it has no observations.
Or in another saying, how to get the number of observations in a specific data set.
So that I can write an If statement to set some conditions.
Thanks.
It's easy with PROC SQL. Do a count and put the results in a macro variable.
proc sql noprint;
select count(*) into :observations from library.dataset;
quit;
There are lots of different ways, I tend to use a macro function with open() and attrn(). Below is a simple example that works great most of the time. If you are going to be dealing with data views or more complex situations like having a data set with records marked for deletion or active where clauses, then you might need more robust logic.
%macro nobs(ds);
%let DSID=%sysfunc(OPEN(&ds.,IN));
%let NOBS=%sysfunc(ATTRN(&DSID,NOBS));
%let RC=%sysfunc(CLOSE(&DSID));
&NOBS
%mend;
/* Here is an example */
%put %nobs(sashelp.class);
Here's the more complete example that #cmjohns was talking about. It will return 0 if it is empty, -1 if it is missing, and has options to handle deleted observations and where clauses (note that using a where clause can make the macro take a long time on very large datasets).
Usage Notes:
This macro will return the number of observations in a dataset. If the dataset does not exist then -1 will be returned. I would not recommend this for use with ODBC libnames, use it only against SAS tables.
Parameters:
iDs - The libname.dataset that you want to check.
iWhereClause (Optional) - A where clause to apply
iNobsType (Optional) - Either NOBS OR NLOBSF. See SASV9 documentation for descriptions.
Macro definition:
%macro nobs(iDs=, iWhereClause=1, iNobsType=nlobsf, iVerbose=1);
%local dsid nObs rc;
%if "&iWhereClause" eq "1" %then %do;
%let dsID = %sysfunc(open(&iDs));
%end;
%else %do;
%let dsID = %sysfunc(open(&iDs(where=(&iWhereClause))));
%end;
%if &dsID %then %do;
%let nObs = %sysfunc(attrn(&dsID,nlobsf));
%let rc = %sysfunc(close(&dsID));
%end;
%else %do;
%if &iVerbose %then %do;
%put WARNING: MACRO.NOBS.SAS: %sysfunc(sysmsg());
%end;
%let nObs = -1;
%end;
&nObs
%mend;
Example Usage:
%put %nobs(iDs=sashelp.class);
%put %nobs(iDs=sashelp.class, iWhereClause=height gt 60);
%put %nobs(iDs=this_dataset_doesnt_exist);
Results
19
12
-1
Installation
I recommend setting up a SAS autocall library and placing this macro in your autocall location.
Proc sql is not efficient when we have large dataset. Though using ATTRN is good method but this can accomplish within base sas, here is the efficient solution that can give number of obs of even billions of rows just by reading one row:
data DS1;
set DS nobs=i;
if _N_ =2 then stop;
No_of_obs=i;
run;
The trick is producing an output even when the dataset is empty.
data CountObs;
i=1;
set Dataset_to_Evaluate point=i nobs=j; * 'point' avoids review of full dataset*;
No_of_obs=j;
output; * Produces a value before "stop" interrupts processing *;
stop; * Needed whenever 'point' is used *;
keep No_of_obs;
run;
proc print data=CountObs;
run;
The above code is the simplest way I've found to produce the number of observations even when the dataset is empty. I've heard NOBS can be tricky, but the above can work for simple applications.
A slightly different approach:
proc contents data=library.dataset out=nobs;
run;
proc summary data=nobs nway;
class nobs;
var delobs;
output out=nobs_summ sum=;
run;
This will give you a dataset with one observation; the variable nobs has the value of number of observations in the dataset, even if it is 0.
I guess I am trying to reinvent the wheel here with so many answers already. But I do see some other methods trying to count from the actual dataset - this might take a long time for huge datasets. Here is a more efficient method:
proc sql;
select nlobs from sashelp.vtable where libname = "library" and memname="dataset";
quit;