I need to make my transpose conditional. A flow I'm creating in EG allows you to turn some sections of the flow off. It does this by using a macro variable (e.g. &myvariable). When &myvariable is set to 0, that section of the flow will be filtered out, so effectively, no rows of data pass through that section.
This works great, but stacks/ transposes will not work when there is no data.
I need it so that when there is no data to go into the transpose, it instead just makes a padded column to replicate what would have been the output of that program.
It needs to be in base SAS, I'm using Enterprise Guide. I've already tried using the conditional logic functionality in EG, but it's not appropriate because I need an ordered list.
''' some conditional logic?
if &myvariable = 0 then do;
format padded_col1 $10.;
else do;
'''transpose
proc transpose data= some_dataset;
by id;
id year;
var income;
run;
'''
You need to use macro logic, not data step here, if you have SAS 9.4+ you can use %IF/%THEN in open code. If you don't you need to wrap it in a macro.
%if &myVariable = 0 %then %do;
*****sas code*******
%end;
%else %do;
****conditional proc transpose*****
%end;
Related
Suppose there are ten datasets with same structure: date and price, particularly they have same time period but different price
date price
20140604 5
20140605 7
20140607 9
I want to combine them and create a panel dataset. Since there is no name in each datasets, I attempt to add a new variable name into each data and then combine them.
The following codes are used to add name variable into each dataset
%macro name(sourcelib=,from=,going=);
proc sql noprint; /*read datasets in a library*/
create table mytables as
select *
from dictionary.tables
where libname = &sourcelib
order by memname ;
select count(memname)
into:obs
from mytables;
%let obs=&obs.;
select memname
into : memname1-:memname&obs.
from mytables;
quit;
%do i=1 %to &obs.;
data
&going.&&memname&i;
set
&from.&&memname&i;
name=&&memname&i;
run;
%end;
%mend;
So, is this strategy correct? Whether are there a different way to creating a panel data?
There are really two ways to setup repeated measures data. You can use the TALL method that your code will create. That is generally the most flexible. The other would be a wide format with each PRICE being stored in a different variable. That is usually less flexible, but can be easier for some analyses.
You probably do not need to use macro code or even code generation to combine 10 datasets. You might find that it is easier to just type the 10 dataset names than to write complex code to pull the names from metadata. So a data step like this will let you list any number of datasets in the SET statement and use the membername as the value for the new PANEL variable that distinguishes the source dataset.
data want ;
length dsn $41 panel $32 ;
set in1.panel1 in1.panela in1.panelb indsname=dsn ;
panel = scan(dsn,-1,'.') ;
run;
And if your dataset names follow a pattern that can be used as a member list in the SET statement then the code is even easier to write. So you could have a list of names that have a numeric suffix.
set in1.panel1-in1.panel10 indsname=dsn ;
or perhaps names that all start with a particular prefix.
set in1.panel: indsname=dsn ;
If the different panels are for the same dates then perhaps the wide format is easier? You could then merge the datasets by DATE and rename the individual PRICE variables. That is generate a data step that looks like this:
data want ;
merge in1.panel1 (rename=(price=price1))
in1.panel2 (rename=(price=price2))
...
;
by date;
run;
Or perhaps it would be easier to add a BY statement to the data set that makes the TALL dataset and then transpose it into the WIDE format.
data tall;
length dsn $41 panel $32 ;
set in1.panel1 in1.panela in1.panelb indsname=dsn ;
by date ;
panel = scan(dsn,-1,'.') ;
run;
proc transpose data=tall out=want ;
by date;
id panel;
var price ;
run;
I can't comment on the SQL code but the strategy is correct. Add a name to each data set and then panel on the name with the PANELBY statement.
That is a valid way to achieve what you are looking for.
You are going to need 2 . in between the macros for library.data syntax. The first . is used to concatenate. The second shows up as a ..
I assume you will want to append all of these data sets together. You can add
data &going..want;
set
%do i=1 %to &obs;
&from..&&memname&i
%end;
;
run;
You can combine your loop that adds the names and that data step like this:
data &going..want;
set
%do i=1 %to &obs;
&from..&&memname&i (in=d&i)
%end;
;
%do i=1 %to &obs;
if d&i then
name = &&memname&i;
%end;
run;
I have a macro program with a loop (for i in 1 to n). With each i i have a table with many columns - variables. In these columns, we have one named var (who has 3 possible values: a b and c).
So for each table i, I want to check his column var if it exists the value "c". If yes, I want to export this table into a sheet of excel. Otherwise, I will concatenate this table with others.
Can you please tell me how can I do it?
Ok, in your macro at step i you have to do something like this
proc sql;
select min(sum(case when var = 'c' then 1 else 0 end),1) into :trigger from table_i;
quit;
then, you will get macro variable trigger equal 1 if you have to do export, and 0 if you have to do concatenetion. Next, you have to code something like this
%if &trigger = 1 %then %do;
proc export data = table_i blah-blah-blah;
run;
%end;
%else %do;
data concate_data;
set concate_data table_i;
run;
%end;
Without knowing the whole nine yard of your problem, I am at risk to say that you may not need Macro at all, if you don't mind exporting to .CSV instead of native xls or xlsx. IMHO, if you do 'Proc Export', meaning you can't embed fancy formats anyway, you 'd better off just use .CSV in most of the settings. If you need to include column headings, you need to tap into metadata (dictionary tables) and add a few lines.
filename outcsv '/share/test/'; /*define the destination for CSV, change it to fit your real settings*/
/*This is to Cat all of the tables first, use VIEW to save space if you must*/
data want1;
set table: indsname=_dsn;
dsn=_dsn;
run;
/*Belowing is a classic 2XDOW implementation*/
data want;
file outcsv(test.csv) dsd; /*This is your output CSV file, comma delimed with quotes*/
do until (last.dsn);
set want1;
by dsn notsorted; /*Use this as long as your group is clustered*/
if var='c' then _flag=1; /*_flag value will be carried on to the next DOW, only reset when back to top*/
end;
do until (last.dsn);
set want1;
by dsn notsorted;
if _flag=1 then put (_all_) (~); /*if condition meets, output to CSV file*/
else output; /*Otherwise remaining in the Cat*/
end;
drop dsn _flag;
run;
My goal is to create a SAS stored process is to return data for a single dataset and to filter the columns in that dataset based on a multi-value input parameter passed into the stored process.
Is there a simple way to do this?
Is there a way to do this at all?
Here's what I have so far. I'm using a macro to dynamically generate the KEEP statement to define which columns to return. I've defined macro variables at the top to mimic what gets passed into the stored process when called through SAS BI Web Services, so unfortunately those have to remain as they are. That's why I've tried to use the VVALUEX method to turn the column name strings into variable names.
Note - I'm new to SAS
libname fetchlib meta library="lib01" metaserver="123.12.123.123"
password="password" port=1234
repname="myRepo" user="myUserName";
/* This data represents input parameters to stored process and
* is removed in the actual stored process*/
%let inccol0=3;
%let inccol='STREET';
%let inccol1='STREET';
%let inccol2='ADDRESS';
%let inccol3='POSTAL';
%let inccol_count=3;
%macro keepInputColumns;
%if &INCCOL_COUNT = 1 %then
&inccol;
%else
%do k=1 %to (&INCCOL_COUNT);
var&k = VVALUEX(&&inccol&k);
%end;
KEEP
%do k=1 %to (&INCCOL_COUNT);
var&k
%end;
;
%mend;
data test1;
SET fetchlib.Table1;
%keepInputColumns;
run;
/*I switch this output to _WEBOUT in the actual stored process*/
proc json out='C:\Logs\Log1.txt';
options firstobs=1 obs=10;
export test1 /nosastags;
run;
There are some problems with this. The ouput uses var1, var2 and var3 as the column names and not the actual column names. It also doesn't filter by any columns when I change the output to _webout and run it using BI Web Services.
OK, I think I have some understanding of what you're doing here.
You can use KEEP and RENAME in conjunction to get your variable names back.
KEEP
%do k=1 %to (&INCCOL_COUNT);
var&k
%end;
;
This has an equivalent
RENAME
%do k=1 %to (&INCCOL_COUNT);
var&k = &&inccol&k.
%end;
;
and now, as long as the user doesn't separately keep the original variables, you're okay. (If they do, then you will get a conflict and an error).
If this way doesn't work for your needs, and I don't have a solution for the _webout as I don't have a server to play with, you might consider trying this in a slightly different way.
proc format;
value agef
11-13 = '11-13'
14-16 = '14-16';
quit;
ods output report=mydata(drop=_BREAK_);
proc report data=sashelp.class nowd;
format age agef.;
columns name age;
run;
ods output close;
The first part is just a proc format to show that this grabs the formatted value not the underlying value. (I assume that's desired, as if it's not this is a LOT easier.)
Now you have the data in a dataset a bit more conveniently, I think, and can put it out to JSON however you want. In your example you'd do something like
ods output report=work.mydata(drop=_BREAK_);
proc report data=fetchlib.Table1 nowd;
columns
%do k=1 %to (&INCCOL_COUNT);
&&inccol&k.;
%end;
;
run;
ods output close;
And then you can send that dataset to JSON or whatever. It's actually possible that you might be able to go more directly than that even, but I don't know almost anything about PROC JSON.
Reading more about JSON, you may actually have an easier way to do this.
On the export line, you have the various format options. So, assuming we have a dataset that is just a subset of the original:
proc json out='C:\Logs\Log1.txt';
options firstobs=1 obs=10;
export fetchlib.Table1
(
%do k=1 %to (&INCCOL_COUNT);
&&inccol&k.;
%end;
)
/ nosastags FMTCHARACTER FMTDATETIME FMTNUMERIC ;
run;
This method doesn't allow for the variable order to be changed; if you need that, you can use an intermediate dataset:
data intermediate/view=intermediate;
set fetchlib.Table1;
retain
%do k=1 %to (&INCCOL_COUNT);
&&inccol&k.;
%end;
;
keep
%do k=1 %to (&INCCOL_COUNT);
&&inccol&k.;
%end;
;
run;
and then write that out. I'm just guessing that you can use a view in this context.
It turns out that the simplest way to implement this was to change the way that the columns (aka SAS variables) were passed into the stored process. Although Joe's answer was helpful, I ended up solving the problem by passing in the columns to the keep statement as a space-separated column list, which greatly simplified the SAS code because I didn't have to deal with a dynamic list of columns.
libname fetchlib meta library="lib01" metaserver="123.12.123.123"
password="password" port=1234
repname="myRepo" user="myUserName";"&repository" user="&user";
proc json out=_webout;
export fetchlib.&tablename(keep=&columns) /nosastags;
run;
Where &columns gets set to something like this:
Column1 Column2 Column3
I'm trying to efficiently implement a block bootstrap technique to get the distribution of regression coefficients from PROC MIXED. The main outline is as follows:
I have a panel data set, say firm and year are the indices. For each iteration of the bootstrap, I wish to sample with replacement n subjects. From this sample, I need to construct a new data set that is a "stack" (concatenated row on top of row) of all the observations for each sampled subject. With this new data set, I can run the regression and pull out the coefficients of interest. Repeat for a bunch of iterations, say 2000.
Each firm can potentially be selected multiple times, so I need to include its data multiple times in each iteration's data set.
Using a loop and subset approach, seems computationally burdensome.
My real data set quite large (a 2Gb .sas7bdat file).
Example pseudo/explanatory code (please pardon all noob errors!):
DATA subjectlist;
SET mydata;
BY firm;
IF first.firm;
RUN;
%macro blockboot(input=, subjects=, iterations=);
%let numberfirms = LENGTH(&subjects);
%do i = 1 %to &iterations ;
DATA mytempdat;
DO i=1 TO &numberfirms;
rec = ceil(&numberfirms * ranuni(0));
*** This is where I want to include all observations for the randomly selected subjects;
*** However, this code doesn't include the same subject multiple times, which...;
*** ...is what I want;
SET &INPUT subjects IN &subjects;
OUTPUT;
END;
STOP;
PROC MIXED DATA=mytempdat;
CLASS firm year;
MODEL yval= cov1 cov2;
RANDOM intercept /sub=subject type=un;
OUTPUT out=outx cov1=cov1 ***want to output the coefficient estimate on cov1 here;
RUN;
%IF &i = 1 %THEN %DO;
DATA outall;
SET outx;
%END;
%ELSE %DO;
PROC APPEND base=outall data=outx;
%END;
%END; /* i=1 to &REPS loop */
PROC UNIVARIATE data=outall;
VAR cov1;
OUTPUT out=final pctlpts=2.5, 97.5 pctlpre=ci;
%mend;
%blockboot(input=mydata,subjects=subjectlist, reps=2000)
This question is identical to a question I asked previously, found here:
block bootstrap from subject list
Any help is appreciated!
See the following paper for details on the best way to do this in SAS:
http://www2.sas.com/proceedings/forum2007/183-2007.pdf
The general summary is to use PROC SURVEYSELECT with a method that allows sampling with replacement to create your bootstrap sample, then use BY processing with PROC MIXED to run the PROC only once rather than running it 2000 times.
I wonder if there is a way of detecting whether a data set is empty, i.e. it has no observations.
Or in another saying, how to get the number of observations in a specific data set.
So that I can write an If statement to set some conditions.
Thanks.
It's easy with PROC SQL. Do a count and put the results in a macro variable.
proc sql noprint;
select count(*) into :observations from library.dataset;
quit;
There are lots of different ways, I tend to use a macro function with open() and attrn(). Below is a simple example that works great most of the time. If you are going to be dealing with data views or more complex situations like having a data set with records marked for deletion or active where clauses, then you might need more robust logic.
%macro nobs(ds);
%let DSID=%sysfunc(OPEN(&ds.,IN));
%let NOBS=%sysfunc(ATTRN(&DSID,NOBS));
%let RC=%sysfunc(CLOSE(&DSID));
&NOBS
%mend;
/* Here is an example */
%put %nobs(sashelp.class);
Here's the more complete example that #cmjohns was talking about. It will return 0 if it is empty, -1 if it is missing, and has options to handle deleted observations and where clauses (note that using a where clause can make the macro take a long time on very large datasets).
Usage Notes:
This macro will return the number of observations in a dataset. If the dataset does not exist then -1 will be returned. I would not recommend this for use with ODBC libnames, use it only against SAS tables.
Parameters:
iDs - The libname.dataset that you want to check.
iWhereClause (Optional) - A where clause to apply
iNobsType (Optional) - Either NOBS OR NLOBSF. See SASV9 documentation for descriptions.
Macro definition:
%macro nobs(iDs=, iWhereClause=1, iNobsType=nlobsf, iVerbose=1);
%local dsid nObs rc;
%if "&iWhereClause" eq "1" %then %do;
%let dsID = %sysfunc(open(&iDs));
%end;
%else %do;
%let dsID = %sysfunc(open(&iDs(where=(&iWhereClause))));
%end;
%if &dsID %then %do;
%let nObs = %sysfunc(attrn(&dsID,nlobsf));
%let rc = %sysfunc(close(&dsID));
%end;
%else %do;
%if &iVerbose %then %do;
%put WARNING: MACRO.NOBS.SAS: %sysfunc(sysmsg());
%end;
%let nObs = -1;
%end;
&nObs
%mend;
Example Usage:
%put %nobs(iDs=sashelp.class);
%put %nobs(iDs=sashelp.class, iWhereClause=height gt 60);
%put %nobs(iDs=this_dataset_doesnt_exist);
Results
19
12
-1
Installation
I recommend setting up a SAS autocall library and placing this macro in your autocall location.
Proc sql is not efficient when we have large dataset. Though using ATTRN is good method but this can accomplish within base sas, here is the efficient solution that can give number of obs of even billions of rows just by reading one row:
data DS1;
set DS nobs=i;
if _N_ =2 then stop;
No_of_obs=i;
run;
The trick is producing an output even when the dataset is empty.
data CountObs;
i=1;
set Dataset_to_Evaluate point=i nobs=j; * 'point' avoids review of full dataset*;
No_of_obs=j;
output; * Produces a value before "stop" interrupts processing *;
stop; * Needed whenever 'point' is used *;
keep No_of_obs;
run;
proc print data=CountObs;
run;
The above code is the simplest way I've found to produce the number of observations even when the dataset is empty. I've heard NOBS can be tricky, but the above can work for simple applications.
A slightly different approach:
proc contents data=library.dataset out=nobs;
run;
proc summary data=nobs nway;
class nobs;
var delobs;
output out=nobs_summ sum=;
run;
This will give you a dataset with one observation; the variable nobs has the value of number of observations in the dataset, even if it is 0.
I guess I am trying to reinvent the wheel here with so many answers already. But I do see some other methods trying to count from the actual dataset - this might take a long time for huge datasets. Here is a more efficient method:
proc sql;
select nlobs from sashelp.vtable where libname = "library" and memname="dataset";
quit;