So I have created a macro, which works perfectly fine. Within the macro, I set where the observation will begin reading, and then how many observations it will read.
But, in my proc print call, I am not able to simply do:
(firstobs=&start obs=&obs)
Because although firstobs will correctly start where I want it, obs does not cooperate, as it must be a higher number than firstobs. For example,
%testmacro(start=5, obs=3)
Does not work, because it is reading in the first 3 observations, but trying to start at observation 5. What I want the macro to do is, start at observation 5, and then read the next 3. So what I did is this:
(firstobs=&start obs=%eval((&obs-1)+&start))
This works perfectly fine when I use it. But I am just wondering if there is a simpler way to do this, rather than having to use the whole %eval... call. Is there one simple call, something like numberofobservations=...?
I don't think there is. You can only simplify your macro a little, within the %eval(). .
%let start=5;
%let obs=3;
data want;
set sashelp.class (firstobs=&start obs=%eval(&obs-1+&start));
run;
Data set options are listed here:
http://support.sas.com/documentation/cdl/en/ledsoptsref/68025/HTML/default/viewer.htm#p0h5nwbig8mobbn1u0dwtdo0c0a0.htm
You could count the obs inside the data step using a counter and only outputting the records desired, but that won't work on something like proc print and isn't efficient for larger data steps.
You could try the point= option, but I'm not familiar with that method, and again I don't think it will work with proc print.
As #Reeza said - there is not a dataset option that will do what you are looking for. You need to calculate the ending observation unfortunately, and %eval() is about as good a way to do it as any.
On a side-note, I would recommend making your macro parameter more flexible. Rather than this:
%testmacro(start=5, obs=3)
Change it to take a single parameter which will be the list of data-set options to apply:
%macro testmacro(iDsOptions);
data want;
set sashelp.class (&iDsOptions);
run;
%mend;
%testmacro(firstobs=3 obs=7);
This provides more flexibility if you need to add in additional options later, which means fewer future code changes, and it's simpler to call the macro. You also defer figuring out the observation counts in this case to the calling program which is a good thing.
Related
I'm trying to convert a SAS dataset column to a list of macro variables but am unsure of how indexing works in this language.
DATA _Null_;
do I = 1 to &num_or;
set CondensedOverrides4 nobs = num_or;
call symputx("Item" !! left(put(I,8.))
,"Rule", "G");
end;
run;
Right now this code creates a list of macro variables Item1,Item2,..ItemN etc. and assigns the entire column called "Rule" to each new variable. My goal is to put the first observation of "Rule" in Item1, the second observation in that column in Item2, etc.
I'm pretty new to SAS and understand you can't brute force logic in the same way as other languages but if there's a way to do this I would appreciate the guidance.
Much easier to create a series of macro variables using PROC SQL's INTO clause. You can save the number of items into a macro variable.
proc sql noprint;
select rule into :Item1-
from CondensedOverrides4
;
%let num_or=&sqlobs;
quit;
If you want to use a data step there is no need for a DO loop. The data step iterates over the inputs automatically. Put the code to save the number of observations into a macro variable BEFORE the set statement in case the input dataset is empty.
data _null_;
if eof then call symputx('num_or',_n_-1);
set CondensedOverrides4 end=eof ;
call symputx(cats('Item',_n_),rule,'g');
run;
SAS does not need loops to access each row, it does it automatically. So your code is really close. Instead of I, use the automatic variable _n_ which can function as a row counter though it's actually a step counter.
DATA _Null_;
set CondensedOverrides4;
call symputx("Item" || put(_n_,8. -l) , Rule, "G");
run;
To be honest though, if you're new to SAS using macro variables to start isn't recommended, there are usually multiple ways to avoid it anyways and I only use it if there's no other choice. It's incredibly powerful, but easy to get wrong and harder to debug.
EDIT: I modified the code to remove the LEFT() function since you can use the -l option on the PUT statement to left align the results directly.
EDIT2: Removing the quotes around RULE since I suspect it's a variable you want to store the value of, not the text string 'RULE'. If you want the macro variables to resolve to a string you would add back the quotes but that seems incorrect based on your question.
I was using the following code to analyze data:
set taq.cq_&yyyymmdd:;
by symbol date time NOTSORTED ex;
There are are thousands of datasets I am running the code on in the unit of days. When &yyyymmdd only specifies one dataset (for one day. for example, 20130102), it works. However, when I try to run it for multiple datasets (for example, 201301:), SAS returns the following errors:
BY NOTSORTED/NOBYSORTED cannot be used with SET statement when
more than one data set is specified.
If I cannot use NOTSORTED here, what is an equivalent statement that I could use?
My understanding of the keyword NOTSORTED is that you use it when the data is not sorted yet. Therefore, do I need to sort it first? How to do it?
I am also confused by the number of variables that NOTSORTED is referencing. Does it only have an effect on "time", or it has effect on "symbol, data, time"?
Many thanks!
UPDATE#2:
The rest of the process immediately following the set statement is: (pseudo code as i don't have the permission to post the original code)
Data _quotes;
SET STATEMENT HERE
Change the name of a variable in the dataset (Variable name is EXN).
last.EXN in a if statement. If the condition is satisfied, label EXN.
Drop some variables.
Run;
DATA NEWDATASET (sortedby= SYMBOL DATE TIME index=(SYMBOL)
label="WRDS-TAQ NBBO Data");
SET _quotes;
by symbol date time;
....
Run;
NOTSORTED means that SAS can assume the sort order in the data is correct, so it may not have explicitly gone through a PROC SORT but it is in logical order as listed in the BY statement.
All variables in the BY statement are included in the NOTSORTED option. Given that I suspect you fully don't understand BY group processing.
It's usually a bit dangerous to use, especially if you don't understand BY group processing. If your data is in the same group but not adjacent it won't work properly and will not produce an error. The correct workaround depends on your processes to be honest.
I would suggest reviewing the documentation regarding BY group processing. It's quite in depth and has lots of samples to illustrate the different type of calculations.
http://support.sas.com/documentation/cdl/en/lrcon/69852/HTML/default/viewer.htm#n138da4gme3zb7n1nifpfhqv7clq.htm
NOTSORTED is often used in example posts to either avoid a sort or when using a custom sort that's difficult to implement in other ways. Explicitly sorting will remove this issue but you may also be misunderstanding how SAS processes data when you have a SET statement with a BY statement. I believe this is called interleaving.
http://support.sas.com/documentation/cdl/en/lrcon/69852/HTML/default/viewer.htm#n1tgk0uanvisvon1r26lc036k0w7.htm
I suspect that the NOTSORTED keyword is being using to find groups for observations with the same value for the EX variable within the same symbol,date,time. If you only need to find the FIRST then you can use the LAG() function to calculate the FIRST.EX flag.
data want;
set taq.cq_&yyyymmdd:;
by symbol date time;
first_ex = first.time or ex ne lag(ex);
Otherwise then perhaps you want to convert the process to data step views and then set the views together.
data work.view_cq_20130102 / view=work.view_cq_20130102;
set taq.cq_20130102;
by symbol date time ex NOTSORTED;
...
run;
...
data want ;
set work.view_cq_201301: ;
by symbol date time;
...
This code executes fine when Run as a SAS program:
%MyMacro(foo_val, bar_val, bat_val);
I have created a table using:
DATA analyses;
input title : $32. weight : $32. response : $32.;
datalines;
foo1 bar1 bat1
foo2 bar2 bat2
;
I want to execute MyMacro once for each row of the analyses table.
The following code appears to only pass the string values title, weight and response (rather than the data values foo1 etc.) to my macro (tested with calls to the %put command) :
DATA _NULL_ ;
set analyses;
%MyMacro(title, weight, response);
RUN;
How can I invoke the macro once per record of the analyses table whilst passing data values as arguments to the macro? The intention is to actually run this for a very large number of analyses so the solution must scale appropriately to many more records in the analyses table.
This in part depends on what your macro is doing. If we assume that your macro is doing something that is intended to be run outside of a data step (ie, it's not just assigning a data step variable), then you have several options.
CALL EXECUTE
PROC SQL: SELECT INTO macro variable
Write macro calls into an %INCLUDE file
DOSUBL
CALL EXECUTE has already been explained, and is a good option for some cases. It has some downsides, however, particularly with macro timing, that requires some extra care to protect in some cases - particularly when you are creating macro variables inside your macro. Quentin in his comments shows a way to get around this (adding %NRSTR to the call), but I find that I prefer to only use CALL EXECUTE when there's an advantage to doing so over the other methods - particularly, if I want to use SAS data step techniques (such as FIRST or LAST, for example, or some form of looping) in creating my macro calls, or when I have to do things in a data step anyway and can avoid the overhead of reading the file another time. If I'm just writing a data step like yours above - data something, set something, call execute, run - I wouldn't use it.
PROC SQL SELECT INTO is typically what I use for list processing (which is largely what this is). I like SQL's simplicity a bit better when doing things that aren't too complicated; for example, you can get just one version of each macro call easily with DISTINCT without having to explicitly write a proc sort nodupkey or use first/last processing. It also has the advantage for debugging that you can write all of your macro calls to your results window (if you don't add noprint), which is a bit easier to read than the log for me if I'm trying to see why my calls didn't get generated properly (and doesn't take any extra PUT statements).
proc sql;
select catx(',','%macro(',arg1,arg2,arg3)||')'
into :mvarlist separated by ' '
from dataset;
quit;
&mvarlist.
That runs them quite simply, and has no timing issues (As you're just writing a bunch of macro calls out).
The main downside to this method is that you have a maximum of 64k characters in a macro variable, so if you're writing a huge number of these you'll run into that. In that case use CALL EXECUTE or %INCLUDE files.
%INCLUDE files are largely useful either as replacement for SELECT INTO when the call is over the character limit, or if you find it useful to have a text file to look at with your calls (if you're running this in batch mode for example, this could be easier to get to and/or parse than log or listing output). You just write your calls out to a file, and then %INCLUDE that file.
filename myfile temp; *or a real file if you want to look at it.;
data _null_;
set dataset;
file myfile;
length str $200;
str=catx(',','%macro(',arg1,arg2,arg3)||')';
put str;
run;
%include myfile;
I don't really use this much anymore, but it's a common technique used particularly by older SAS programmers so good to know.
DOSUBL is a relatively new method, and to some extent can be used to replace CALL EXECUTE as its default behavior is typically closer to what you expect intuitively than CALL EXECUTE's. The doc page has really the best example for how this works differently; basically, it fixes the timing issue by letting each separate call look import and export the macro variables from/to the calling environment, meaning that each iteration of DOSUBL is run at a distinct time versus CALL EXECUTE where everything is run in one bunch and the macro environment is 'fixed' (ie, any reference to a macro variable is fixed at run time, unless you escape it messily with %NRSTR).
One more thing worth mentioning is RUN_MACRO, a part of the FCMP language. That allows you to completely run a macro and import its contents back to the data step, which is an interesting option in some cases (for example, you could wrap a call around a PROC SQL that selected a count of something, and then import that to the dataset as a variable, all in one datastep). It's applicable if you're doing this for the purpose of calling a macro to assign a data step variable, not to run a process that does things that don't need to be imported into the data step, but it's something worth considering if you do want that data back all in the dataset that called the process.
You could use CALL EXECUTE:
data _null_;
set analyses;
call execute('%nrstr(%MyMacro('||title||','||weight||','||response||'))');
run;
You can put the variables values into macrovariables and then call your %MyMacro many times (the number of obs in your dataset) with the macrovariables as argument:
Data :
DATA analyses;
input title : $32. weight : $32. response : $32.;
datalines;
foo1 bar1 bat1
foo2 bar2 bat2
;
run;
Code to run macro :
data _NULL_;
set analyses end=fine;
call symput("ARGUMENT"||compress(_N_),catx(",",title,weight,response));
if fine then call symput("NLOOPS",compress(_N_));
run;
%*PUT &ARGUMENT1;
%*PUT &ARGUMENT2;
%MACRO MAIN;
%DO L=1 %TO &NLOOPS;
%MyMacro(&&ARGUMENT&L);
%END;
%MEND;
%MAIN;
This code executes fine when Run as a SAS program:
%MyMacro(foo_val, bar_val, bat_val);
I have created a table using:
DATA analyses;
input title : $32. weight : $32. response : $32.;
datalines;
foo1 bar1 bat1
foo2 bar2 bat2
;
I want to execute MyMacro once for each row of the analyses table.
The following code appears to only pass the string values title, weight and response (rather than the data values foo1 etc.) to my macro (tested with calls to the %put command) :
DATA _NULL_ ;
set analyses;
%MyMacro(title, weight, response);
RUN;
How can I invoke the macro once per record of the analyses table whilst passing data values as arguments to the macro? The intention is to actually run this for a very large number of analyses so the solution must scale appropriately to many more records in the analyses table.
This in part depends on what your macro is doing. If we assume that your macro is doing something that is intended to be run outside of a data step (ie, it's not just assigning a data step variable), then you have several options.
CALL EXECUTE
PROC SQL: SELECT INTO macro variable
Write macro calls into an %INCLUDE file
DOSUBL
CALL EXECUTE has already been explained, and is a good option for some cases. It has some downsides, however, particularly with macro timing, that requires some extra care to protect in some cases - particularly when you are creating macro variables inside your macro. Quentin in his comments shows a way to get around this (adding %NRSTR to the call), but I find that I prefer to only use CALL EXECUTE when there's an advantage to doing so over the other methods - particularly, if I want to use SAS data step techniques (such as FIRST or LAST, for example, or some form of looping) in creating my macro calls, or when I have to do things in a data step anyway and can avoid the overhead of reading the file another time. If I'm just writing a data step like yours above - data something, set something, call execute, run - I wouldn't use it.
PROC SQL SELECT INTO is typically what I use for list processing (which is largely what this is). I like SQL's simplicity a bit better when doing things that aren't too complicated; for example, you can get just one version of each macro call easily with DISTINCT without having to explicitly write a proc sort nodupkey or use first/last processing. It also has the advantage for debugging that you can write all of your macro calls to your results window (if you don't add noprint), which is a bit easier to read than the log for me if I'm trying to see why my calls didn't get generated properly (and doesn't take any extra PUT statements).
proc sql;
select catx(',','%macro(',arg1,arg2,arg3)||')'
into :mvarlist separated by ' '
from dataset;
quit;
&mvarlist.
That runs them quite simply, and has no timing issues (As you're just writing a bunch of macro calls out).
The main downside to this method is that you have a maximum of 64k characters in a macro variable, so if you're writing a huge number of these you'll run into that. In that case use CALL EXECUTE or %INCLUDE files.
%INCLUDE files are largely useful either as replacement for SELECT INTO when the call is over the character limit, or if you find it useful to have a text file to look at with your calls (if you're running this in batch mode for example, this could be easier to get to and/or parse than log or listing output). You just write your calls out to a file, and then %INCLUDE that file.
filename myfile temp; *or a real file if you want to look at it.;
data _null_;
set dataset;
file myfile;
length str $200;
str=catx(',','%macro(',arg1,arg2,arg3)||')';
put str;
run;
%include myfile;
I don't really use this much anymore, but it's a common technique used particularly by older SAS programmers so good to know.
DOSUBL is a relatively new method, and to some extent can be used to replace CALL EXECUTE as its default behavior is typically closer to what you expect intuitively than CALL EXECUTE's. The doc page has really the best example for how this works differently; basically, it fixes the timing issue by letting each separate call look import and export the macro variables from/to the calling environment, meaning that each iteration of DOSUBL is run at a distinct time versus CALL EXECUTE where everything is run in one bunch and the macro environment is 'fixed' (ie, any reference to a macro variable is fixed at run time, unless you escape it messily with %NRSTR).
One more thing worth mentioning is RUN_MACRO, a part of the FCMP language. That allows you to completely run a macro and import its contents back to the data step, which is an interesting option in some cases (for example, you could wrap a call around a PROC SQL that selected a count of something, and then import that to the dataset as a variable, all in one datastep). It's applicable if you're doing this for the purpose of calling a macro to assign a data step variable, not to run a process that does things that don't need to be imported into the data step, but it's something worth considering if you do want that data back all in the dataset that called the process.
You could use CALL EXECUTE:
data _null_;
set analyses;
call execute('%nrstr(%MyMacro('||title||','||weight||','||response||'))');
run;
You can put the variables values into macrovariables and then call your %MyMacro many times (the number of obs in your dataset) with the macrovariables as argument:
Data :
DATA analyses;
input title : $32. weight : $32. response : $32.;
datalines;
foo1 bar1 bat1
foo2 bar2 bat2
;
run;
Code to run macro :
data _NULL_;
set analyses end=fine;
call symput("ARGUMENT"||compress(_N_),catx(",",title,weight,response));
if fine then call symput("NLOOPS",compress(_N_));
run;
%*PUT &ARGUMENT1;
%*PUT &ARGUMENT2;
%MACRO MAIN;
%DO L=1 %TO &NLOOPS;
%MyMacro(&&ARGUMENT&L);
%END;
%MEND;
%MAIN;
I started to learn SAS here fairly recently and am getting the basics down pretty well, but have a question regarding something that is a little outside of my current realm of knowledge. Does anyone happen to know of a way to cycle through all variables in a SAS dataset? I know how to run a do loop/array on variables in a range (x1-x99), but ideally would like to look at every variable without having to rename any variables. Basically, I'm looking to run through a dataset and change variable values when the current value = 'True'/'False'. My guess is that I'll need to use proc contents in someway here, but not really sure how to go about using it correctly. Any tips/insight would be greatly appreciated. Thanks!
You can create an array of non-similarly-named variables. You're on the right track with PROC CONTENTS, although you also can use dictionary.columns or sashelp.vcolumn, which contain basically the same information.
proc sql;
select name into :collist separated by ' '
from dictionary.columns
where memname='DATASETNAME' and libname='LIBNAME' and <other criteria>;
quit;
The variables have to be all of the same type (char/numeric) so you may want to include a criterion of variable type in your query, plus any other limiting factor you may need.
That will create a list, &collist., in a macro variable you can use in your array
array vars &collist.;
and now you can loop over the array.
You may also be able to cheat things, if all of your variables are the same type, and you know the order is fixed . The double dash list (x1--x99) is 'in variable order, all variables from x1 to x99' and doesn't require numeric suffixes or anything like that.
Finally, you also might be able to write a format in PROC FORMAT to accomplish what you need, depending on what you are intending to do (mapping TRUE to 1 and FALSE to 0 or something like that).
Adding to Joe's answer: you can overcome the requirement that all variables should be of the same type. For that you can use macro loop instead of array. Firstly you need to define the macro:
%macro loop;
%do i=1 %to %sysfunc(countw(&collist));
....
<here goes your code for changing values, where instead of a variable name
you use macro function %scan(&collist,&i)>
....
%end;
%mend loop;
and now you can paste %loop into the DATA step where you're going to process all variables.