I want to build a documentation pdf (or html) for a collection of SAS macros. Is there a canonic (or recommended) workflow that I can follow ?
I'm thinking of exporting all my macros and extract title, description, examples, variable descriptions and code using R and Regex then use markdown to build a nicely laid out pdf that I can update in a few steps whenever I add a macro or change descriptions or examples (I want to avoid copy/paste at all cost).
It'd be quite tedious and unflexible though and I might be reinventing the wheel.
My macros all look like this:
*--------------------------------------------------------;
* ASSERT_EXIST ;
* Fails explicitely when a table doesn't exist ;
* Accepts a list of tables as input ;
*--------------------------------------------------------;
/* EXAMPLES
%assert_exist(not_a_table); * prints explicit error and aborts;
%assert_exist(sashelp.class); * does nothing;
%assert_exist(sashelp.cars not_a_table sashelp.class); * prints explicit error and aborts;
*/
%macro assert_exist
(data /* table or list of tables */
);
%local i table;
%do i=1 %to %sysfunc(countw(&data,%str( )));
%let table = %scan(&data,&i,%str( ));
%if not %sysfunc(exist(&table)) %then %do;
%put ERROR: Table &table doesnt exist!;
%abort;
%end;
%end;
%mend;
*----------------------------------------------;
* DROP ;
* Delete table or list of tables ;
* Default deletes all tables starting with _ ;
*----------------------------------------------;
/* EXAMPLES
data _x;
input char1 $ num1;
datalines;
a 1
b 2
;
%put %sysfunc(exist(_x)); * 1;
%drop(_x);
%put %sysfunc(exist(_x)); * 0;
*/
%macro drop
(data /* Name of table to drop, end name of table with ':' to delete all tables with this prefix */
);
%if &data= %then %let data = _:;
proc datasets nolist;
delete &data;
run;
%mend;
*--------------------------------------------------------;
* HEAD ;
* select top rows ;
*--------------------------------------------------------;
/* EXAMPLES
%head(sashelp.class,2) * keep only 2 first rows;
* %drop(_TEMP_); * clean up;
*/
%macro head
(data /* source table */
,n /* number of rows to keep */
,out /* output table */
);
/* default values, checks, initialisations */
%if &data= %then %let data = _TEMP_;
%if &out= %then %let out = _TEMP_;
%if &out=. %then %let out = &data;
%assert_exist(&data)
proc sql inobs=&n;
CREATE TABLE &out AS
SELECT *
FROM &data;
quit;
%mend;
My collection of macros is growing I'd like to respect good practice as much as possible but I've not been able to find much information relative to good documentation in SAS.
#Allan-Bowe gave a great answer which is probably the best practice, but unfortunately I have no way to install doxygen from my work computer, so I'm looking for other solutions not requiring external software.
No need to re-invent the wheel - a great approach for documentation is doxygen.
We use it for the open source SASjs Macro Core library (which also lists a lot of good practices for SAS Macro development).
Simply define your attributes in the header (markdown is accepted), eg:
/**
#file
#brief Logs a key value pair a control dataset
#details If the dataset does not exist, it is created. Usage:
%mp_setkeyvalue(someindex,22,type=N)
%mp_setkeyvalue(somenewindex,somevalue)
#param key Provide a key on which to perform the lookup
#param value Provide a value
#param type= either C or N will populate valc and valn respectively. C is
default.
#param libds= define the target table to hold the parameters
#version 9.2
#author Allan Bowe
#source https://github.com/sasjs/core
**/
Then simply point doxygen at your source folder, tell it which config file to use (a good one for SAS is here) and then choose an output directory for your documentation.
It'll look like this.
There's no pdf option, but it can create files in DOCBOOK format that can be used to generate a pdf: http://www.doxygen.nl/manual/config.html#config_docbook
UPDATE - we recently added doxygen support to SASjs - with a single command (sasjs doc) you can document all your jobs, and even generate a graphviz data lineage diagram, integrated into the output.
Overview: https://www.youtube.com/watch?v=ESNdCtXKRrw
You already seem to be placing the parameter definitions on separate lines. That should help with parsing the source files. Also add the macro name to the %MEND statement so that your parsing code can have a double check that it didn't find the wrong one.
I would also recommend moving your comment blocks into the macro.
%macro assert_exist
(data /* table or list of tables */
);
/*--------------------------------------------------------;
* ASSERT_EXIST ;
* Fails explicitely when a table doesn't exist ;
* Accepts a list of tables as input ;
*--------------------------------------------------------;
EXAMPLES
%assert_exist(not_a_table); * prints explicit error and aborts;
%assert_exist(sashelp.class); * does nothing;
%assert_exist(sashelp.cars not_a_table sashelp.class); * prints explicit error and aborts;
*/
%local i table;
%do i=1 %to %sysfunc(countw(&data,%str( )));
%let table = %scan(&data,&i,%str( ));
%if not %sysfunc(exist(&table)) %then %do;
%put ERROR: Table &table doesnt exist!;
%abort;
%end;
%end;
%mend assert_exist;
Related
I'm looking to create a loop such that I run two macros for each dataset
%Let Classification = Data1 data2 data3 data4;
%let index = 1;
%do %until (%Scan(&Classification,&index," ")=);
%Macro1;
%Macro2;
%end;
%let index = %eval(&Index + 1);
The problem is my macros are not pre-loaded and are stored in a macro library, is it possible to do this if I run the above as a macro?
Any advice is appreciate in making this loop of macros work
EDIT:
In my ideal situation the loop would run like a macro
%Macro;
where inside it would look like
%Macro Macro;
%let index = 1;
%do %until (%scan(&classification,&index," ")=);
<Lines of Code>
%end;
%let index = %eval(&Index + 1);
%end;
%mend;
Another problem is my macros enclosed in the loop use the &classification to differentiate between data1, data2, data3, data4 as we process through the different lines of code.
It is probably easier to just iterate over the index. Use the countw() function to find how many iterations to do.
%macro loop(list);
%local index next ;
%do index=1 %to %sysfunc(countw(&list,%str( )));
%let next=%scan(&list,&index,%str( ));
... code to process &NEXT ...
%end;
%mend ;
Then pass in the list to the macro as the parameter value.
%Let Classification = Data1 data2 data3 data4;
%loop(&classification);
SAS does not allow the %DO statement in open-code. When you submit an open code loop you will get log messages
ERROR: The %DO statement is not valid in open code.
...
ERROR: The %END statement is not valid in open code.
as #Tom mentioned the macro %SCAN test should check for null string. Another common and more robust way is to check before token extraction. %do %until will iterate poorly when the classification passed is empty. A %do %while tests the classification scan prior to interior macro invocations. Another common test for null macro value is checking for 0 length and leveraging 0=false ^0=true automatic evaluation.
When the loop is to call other macros with the token value the best practice is to pass the token value instead of having the called macro presume the token symbol (aka macro variable) already exists (in a containing scope) prior the iterated macros invocation.
Example
%macro mydispatch (classification=);
%local index token;
%let index = 1;
%do %while ( %length (%scan (&classification, &index)));
%let token = %scan(&classification,&index));
%* emit code specifically for token;
* this is for &token;
%* iterated invocations, perform two analysis for each data set listed in classification;
%* second analysis is passed another argument specifying the data set that should be used to store output;
%analysis_1 (data=&token)
%analysis_2 (data=&token, out=WORK.results_&token.)
%let index = %eval(&index+1);
%end;
%mend mydispatch;
%mydispatch (classification=data1 data2 data3 data4)
The macro being in an autocall library (which is what I assume you refer to?) does not have any impact on how the above would work. If it's not in an autocall library you'll have to hook up the catalog up to the autocall library first.
In re: your edits; yes, you will need this to be in a macro (I assumed it was a subset of a macro initially). %do is not currently allowed in open code (this may change, but not today).
Note you have several significant issues in your code:
the incrementor is not in the loop
the scan function is wrong; macro language does not use quotations, so
%do %until (%Scan(&Classification,&index," ")=);
needs to be
%do %until (%Scan(&Classification,&index)=);
(space is the default separator), and if you really needed to clarify space:
%do %until (%Scan(&Classification,&index,%str( ))=);
Your macros do not utilize parameters; they should. %macro1; apparently uses &classification and &index; instead you should pass it the thing you want (the "word" from &classification) as a parameter.
I doing like this
libname DZY 'Path';
Proc sql;
select * from DZ.some_table;
run;
Here I have to add an error handling like if something goes wrong in select statement or within the block I have to write error message to an separate text file in the folder.
This is what I tried
%macro sortclass;
Proc sql;
select * from DZ.some_table;
run;
%if &SQLRC gt 0 %then %goto error;
%error:
proc export data=""
run;
%exit:
%mend;
%sortclass;
I am trying to do an try catch like error handling.., How can do this in effective way. Thanks In advance
Base SAS does not have try/catch mechanisms, so an error due to macro generated code can cause problems with attempts to externally log messages.
filename logfile 'mysolution.log' mod;
%macro mylogger(message);
* may fail due to apriori errors;
data _null_;
file log;
put "%superq(message)";
run;
* a more advanced version would instead use macro `%sysfunc'd` invocations of
* `FOPEN FPUT FWRITE FWRITE` to append to a custom external log file;
%mend;
%macro mymacro(...);
... some SQL statement(s) ...
%if &SYSRC > 0 %then %do;
%mylogger(Failer at step 1);
%return;
%end;
... some more SQL statement(s) ...
%if &SYSRC > 0 %then %do;
%mylogger(Failer at step 2);
%return;
%end;
%mend;
Note: You can also submit SAS code from java or .net code that uses a SAS workspace session -- search for SAS integrated object model (iom) for more info. Such a solution would have a rich environment for try/catch/throw and advanced logging models.
I am writing a macro that at some point calls some proc SQL code. I want the user to be able to specify arbitrary proc sql options (e.g. inobs=100 could be one of the input arguments to my macro).
I am having a very hard time quoting an argument that has an equality '=' character.
One of the issues is that I should also check if the macro argument is empty or not, and if it is not empty, only then add the specified options to the sql statement.
Below is an example non-working test that does not work and throws the
ERROR: The keyword parameter INOBS was not defined with the macro.
I have read this (http://www2.sas.com/proceedings/sugi28/011-28.pdf) and other SUGI's and tried many possible ways to quote and call the macro.
If somebody could provide a working example of the below function it would be greatly appreciated.
options mprint mlogic;
data have;
length x $8;
input x;
datalines;
one
two
three
;
proc sql inobs=2;
create table sql_output as
select *
from have;
quit;
%macro pass_parameter_with_equal_sign(table=, sqlOptions=);
proc sql
%if "%left(%trim(&sqlOptions.))" ne "" %then %do;
&sqlOptions.
%end;
/* the semicolon to end the proc sql statement */
;
create table macro_output as
select *
from have;
quit;
%mend;
%pass_parameter_with_equal_sign(table=have, sqlOptions=%str(inobs=2))
title "SQL output:";
proc print data=sql_output; run;
title "Macro output:";
proc print data=macro_output; run;
If you remove the %if condition as follows it should work:
%macro pass_parameter_with_equal_sign(table=, sqlOptions=);
proc sql
&sqlOptions.
/* the semicolon to end the proc sql statement */
;
create table macro_output as
select *
from have;
quit;
%mend;
The %if you have used is to check if &sqlOptions is not blank, this shouldn't matter if you use it as it is because its unconditional usage will give either:
proc sql inobs=2; /* in the case of &sqlOptions=inobs=2 */
or if there is no value supplied for &sqlOptions then you should see:
proc sql; /* i.e. no options specified */
So it should work with or without an argument.
Amir's solution is probably correct for your particular use case. But to answer the more general question, we need to look to the seminal paper on macro parameter testing, Chang Chung's Is This Macro Parameter Blank?.
His example C8 is the right one for you here, though some of the others will also work.
%if %sysevalf(%superq(param)=,boolean) %then ... /* C8 */
For example:
%macro test_me(param=);
%if %sysevalf(%superq(param)=,boolean) %then %put Empty;
%else %put Not Empty;;
%mend test_me;
%test_me(param=);
%test_me(param=MyParam);
%test_me(param=param=5);
%SUPERQ is most useful here because it avoids resolving the macro parameter. Instead, it keeps it as a macro parameter value - fully unresolved - and allows you to work with it in that fashion; so you have no risk of that pesky equal sign bothering you.
His C4 (just using SUPERQ without SYSEVALF) also works in this case, although he explains a few situations where it may have difficulty.
Ahh this was actually a tricky little problem you ran into. The issue was actually being caused by the calls to %trim() and %left().
Removing these results in code that works as intended (note I also removed the macro quoting around the parameter):
%macro pass_parameter_with_equal_sign(table=, sqlOptions=);
proc sql
%if "&sqlOptions" ne "" %then %do;
&sqlOptions
%end;
/* the semicolon to end the proc sql statement */
;
create table macro_output as
select *
from &table;
quit;
%mend;
%pass_parameter_with_equal_sign(table=sashelp.class, sqlOptions= inobs=2);
We can re-create the issue you were experiencing like so:
%put %trim(inobs=1);
Because the parameter was resolving to inobs=1, and %trim() doesn't have any named parameters, it was throwing a hissy fit. To correctly pass in a string that contains "inobs=1" we can do so like this:
%let param = inobs=1;
%put %trim(%str(¶m));
Note: Amir's solution of removing the %if statement altogether is also the best way to design code like this. I'm just providing more details as to why you were having this issue.
Additional Explanation 1 - Why %left() and %trim are not needed
The top code snippet provides the same intended functionality as your original code that had the "%left(%trim(&sqlOptions.))". This is because beginning and ending whitespace is dropped from macro variables (including macro parameters) unless it is explicitly retained by using macro quoting. A simple example to show this is:
%let param = lots of spaces ;
%put ***¶m***;
Gives:
***lots of spaces***
You can see that the internal whitespace is kept, but the left and right padding are gone. To keep whitespace, we can simply use the %str() function.
%let param = %str( lots of spaces );
%put ***¶m***;
Gives:
*** lots of spaces ***
Additional Explanation 2 - Working with macros containing whitespace
If you actually did have whitespace on a macro variable that you needed to remove because it was quoted, and you wanted to use %left() and %trim() to do so, then things get a little wacky. Our variable can be created like so:
%let param = %str( inobs = 2 );
You can see we already have quoted the value with %str() in order to create it. This means we can now call one of the functions without having to quote it again:
%put %trim(¶m); * ALREADY QUOTED AT CREATION SO THIS WORKS FINE;
However, if we then try and feed the result into the %left() function we're back to the original issue:
%put %left(%trim(¶m)); * OOPS. DOESNT WORK;
Now I'm guessing here but I believe this is most likely because the %trim() function removes any macro quoting prior to returning a result. Kind of like this:
%put %unquote(%trim(¶m));
This can be circumvented by re-quoting the returned result using %str() again:
%put %left(%str(%trim(¶m)));
... or wrapping the original parameter with a %nrstr():
%let param = %str( inobs = 2 );
%put %left(%trim(%nrstr(¶m)));
... or using %sysfunc() to call a datastep function:
%put %sysfunc(compress(¶m));
I don't know if it is the right place to post this, but it seemed to me I would be more likely to get an answer there.
Currrently working on SAS for an internship, I am trying to write a macro in order to automate the process of finding a fitting ARIMA model for my data sets. I am very new to this software, and quite not a specialist in the field of statistics.
However, while I seemingly understood how to import my files and launch the proc arima, I am stuck on a little problem. A part of my code, which is working fine if I write it outside of the macro (I guess it's called open code ?) like this :
data _null_;
set Lib.out; /* Lib.out contains the data of the OUTSTAT statement of the PROC ARIMA */
x = 1000000;
put _STAT_; /* Prints correctly the names of the different lines in the log */
if _STAT_='AIC' then do; /* _STAT_ is a column and AIC the name of a line AFAIK */
if _VALUE_ < x then
x = _VALUE_;
put x;
put _STAT_; /* Here only prints AIC, which I guess is correct inside of the IF loop */
end;
run;
But when running it inside a macro such as :
%macro recherche(poste=, mto=);
--- code ---
data _null_;
set Lib.out; /* Lib.out contains the data of the OUTSTAT statement of the PROC ARIMA */
%let aic0 = 1000000;
%put _STAT_; /* Doesn't recognize the _STAT_ statement and stops */
%if _STAT_='AIC' %then %do;
%if _VALUE_ < &aic0 %then %do;
&aic0 = _VALUE_;
data Lib.chosen;
set Lib.model; /* Contains the OUTMODEL statement of PROC ARIMA */
run;
%end;
end;
run;
--- code ---
I tried to search for similar cases on the internet but couldn't find an explanation for what I am looking for. Plus, being new to SAS, the official documentation is still hard to understand. Thanks in advance.
Most of what you're doing there doesn't need the %. You just need that if it's a statement controlling which lines of code are even sent to the compiler.
%macro recherche(mto=);
data _null_;
set Lib.out; /* Lib.out contains the data of the OUTSTAT statement of the PROC ARIMA */
x = &mto.;
put _STAT_; /* Prints correctly the names of the different lines in the log */
if _STAT_='AIC' then do; /* _STAT_ is a column and AIC the name of a line AFAIK */
if _VALUE_ < x then
x = _VALUE_;
put x;
put _STAT_; /* Here only prints AIC, which I guess is correct inside of the IF loop */
end;
run;
%mend recherche;
That's assuming the MTO parameter is intended to hold the value assigned to x. The only time you would use %IF is if you did something like
%macro recherche(mto=,stat=);
data _null_;
set Lib.out; /* Lib.out contains the data of the OUTSTAT statement of the PROC ARIMA */
x = &mto.;
put _STAT_; /* Prints correctly the names of the different lines in the log */
%if &stat=AIC %then %do;
if _STAT_='AIC' then do; /* _STAT_ is a column and AIC the name of a line AFAIK */
if _VALUE_ < x then
x = _VALUE_;
put x;
put _STAT_; /* Here only prints AIC, which I guess is correct inside of the IF loop */
end;
%end;
%else %if &stat=XYZ %then %do;
*more code ...;
%end;
run;
%mend recherche;
That would only do one or the other of those sections of code. Macro statements don't have access to the data in the proc or data step, and they don't use quotes (unless the quote is an actual important part of the code).
I ran into the same issue when combining a pipe statement do while loop within a macro. I would get the unclosed do loop error, yet the statements would work independently. I had tried every iteration to resolve, but to no avail. Finally, the solution for me was to put the pipe infile statement into its own program and then bring it into the macro using a %include. This worked perfectly! The issue is definitely a bug within SAS.
I wonder if there is a way of detecting whether a data set is empty, i.e. it has no observations.
Or in another saying, how to get the number of observations in a specific data set.
So that I can write an If statement to set some conditions.
Thanks.
It's easy with PROC SQL. Do a count and put the results in a macro variable.
proc sql noprint;
select count(*) into :observations from library.dataset;
quit;
There are lots of different ways, I tend to use a macro function with open() and attrn(). Below is a simple example that works great most of the time. If you are going to be dealing with data views or more complex situations like having a data set with records marked for deletion or active where clauses, then you might need more robust logic.
%macro nobs(ds);
%let DSID=%sysfunc(OPEN(&ds.,IN));
%let NOBS=%sysfunc(ATTRN(&DSID,NOBS));
%let RC=%sysfunc(CLOSE(&DSID));
&NOBS
%mend;
/* Here is an example */
%put %nobs(sashelp.class);
Here's the more complete example that #cmjohns was talking about. It will return 0 if it is empty, -1 if it is missing, and has options to handle deleted observations and where clauses (note that using a where clause can make the macro take a long time on very large datasets).
Usage Notes:
This macro will return the number of observations in a dataset. If the dataset does not exist then -1 will be returned. I would not recommend this for use with ODBC libnames, use it only against SAS tables.
Parameters:
iDs - The libname.dataset that you want to check.
iWhereClause (Optional) - A where clause to apply
iNobsType (Optional) - Either NOBS OR NLOBSF. See SASV9 documentation for descriptions.
Macro definition:
%macro nobs(iDs=, iWhereClause=1, iNobsType=nlobsf, iVerbose=1);
%local dsid nObs rc;
%if "&iWhereClause" eq "1" %then %do;
%let dsID = %sysfunc(open(&iDs));
%end;
%else %do;
%let dsID = %sysfunc(open(&iDs(where=(&iWhereClause))));
%end;
%if &dsID %then %do;
%let nObs = %sysfunc(attrn(&dsID,nlobsf));
%let rc = %sysfunc(close(&dsID));
%end;
%else %do;
%if &iVerbose %then %do;
%put WARNING: MACRO.NOBS.SAS: %sysfunc(sysmsg());
%end;
%let nObs = -1;
%end;
&nObs
%mend;
Example Usage:
%put %nobs(iDs=sashelp.class);
%put %nobs(iDs=sashelp.class, iWhereClause=height gt 60);
%put %nobs(iDs=this_dataset_doesnt_exist);
Results
19
12
-1
Installation
I recommend setting up a SAS autocall library and placing this macro in your autocall location.
Proc sql is not efficient when we have large dataset. Though using ATTRN is good method but this can accomplish within base sas, here is the efficient solution that can give number of obs of even billions of rows just by reading one row:
data DS1;
set DS nobs=i;
if _N_ =2 then stop;
No_of_obs=i;
run;
The trick is producing an output even when the dataset is empty.
data CountObs;
i=1;
set Dataset_to_Evaluate point=i nobs=j; * 'point' avoids review of full dataset*;
No_of_obs=j;
output; * Produces a value before "stop" interrupts processing *;
stop; * Needed whenever 'point' is used *;
keep No_of_obs;
run;
proc print data=CountObs;
run;
The above code is the simplest way I've found to produce the number of observations even when the dataset is empty. I've heard NOBS can be tricky, but the above can work for simple applications.
A slightly different approach:
proc contents data=library.dataset out=nobs;
run;
proc summary data=nobs nway;
class nobs;
var delobs;
output out=nobs_summ sum=;
run;
This will give you a dataset with one observation; the variable nobs has the value of number of observations in the dataset, even if it is 0.
I guess I am trying to reinvent the wheel here with so many answers already. But I do see some other methods trying to count from the actual dataset - this might take a long time for huge datasets. Here is a more efficient method:
proc sql;
select nlobs from sashelp.vtable where libname = "library" and memname="dataset";
quit;