I've run into an odd SAS quirk that I can't figure out - hopefully you can help.
I have a simple macro loop that imports CSV files and for some reason if I use a libref statement in the "out=" part of the import procedure, SAS doesn't recognize the libref as a valid name. But if I use the same libref in a data step, it works just fine.
The specific error it gives is: "ERROR: "TESTDB." is not a valid name."
I'd like to figure this out because I work with pretty big files and want to avoid reading through them more times than is necessary.
Here's the code that works, with some comments in it. I got around the issue by reading in the files, then writing them to permanent SAS datasets in a second step, but ideally I'd like to import the files directly into the "TESTDB" library. Any idea how to get SAS to recognize a libref in the "out=" statement of the import procedure?
libname testdb "C:\SAS test";
%let filepath = C:\SAS test\;
%macro loop(values);
%let count=%sysfunc(countw(&values));
%do i = 1 %to &count;
%let value = %qscan(&values,&i,%str(,));
proc import datafile = "&filepath.&value..csv"
out = &value dbms=csv replace; getnames=yes;
/*"out=testdb.&value" in the line above does not work*/
run;
data testdb.&value; set &value; run;
/*here the libref testdb works fine*/
%end;
%mend;
%loop(%str(test_a,test_b,test_c));
Thanks in advance for your help!
john
Perhaps try:
out=testdb.%unquote(&value)
Sometimes the macro language does not unquote values automatically. With result that the extra quoting characters introduced by a quoting function (%qscan %str %bquote %superq etc) cause problems.
Strange error. I am not able to pin it. My guess is that it has something to do with how the value macro variables are being created. When I moved the value variable creation to a data step and used Call Symputx, it works.
%macro loop(files);
/* Create macro variables for files.*/
data _null_;
count = countw("&files.",",");
call symputx("count",count,"L");
do i = 1 to count;
call symputx(cats("file",i),scan("&files.",i,","),"L");
end;
run;
/* Read and save each CSV as a sas table. */
%do i=1 %to &count.;
proc import datafile = "&filepath.&&file&i...csv"
out = testdb.&&file&i. dbms=csv replace; getnames=yes;
run;
%end;
%mend;
%loop(%str(test_a,test_b));
Related
I am very new to SAS and have received my first work assignment. Basically, I need to pull all of the patient ID's (patid) and procedure code (proc_cd) from multiple SAS files and put it into an excel file.
From my research, I believe I need a MACRO with a do loop that will run this search for all of the files
Below is the code I've put together. Again I'm very new to SAS so any help will be appreciated!
libname sas 'P:\H3.2018.DH_StressQuery\dat';
libname optum 'C:\OPTUM Data\Zip5';
data libname.filename;
set libname.filename;
%MACRO LOOP * I don't know what to put here.
%DO i = 1 %TO
("zip5_r2018q1.sas7bdat","16.2GB","Sas7bdat","C:\OPTUM
Data\Zip5\zip5_r2018q1.sas7bdat","11Jul2018:20:07:01"
)
(data sas.query file;
set optum.zip5_m2007q1
(keep = patid, Proc_Cd);
if Proc_Cd = '94621');
proc print data= data.query file
%END;
%MEND LOOP;
%LOOP;
Topic: SAS Library
Difficulty: path(File name and Location) is changing after each run but only some of the details are changing but not the full path (as given in below example). we have also highlighted those field in Bold
I want to write only one code where I can cover all kind of change which happening in file name and location, is it possible?
%let path='C:\Data\variationstring\empcat**A**\person**34**_**1212**\persondata_empcatA_**34**';
libname test "&path";
proc import datafile="&path\Accounts_**34**.xls"
out=mydata
sheet="thefile";
getnames=no;
run;
When another user run that program then above path will be changed:
%let path='C:\Data\variationstring\empcat**A**\person**49**_**1684**\persondata_empcatA_**49**';
libname test "&path";
proc import datafile="&path\Accouns_**49**.xls"
out=mydata
sheet="thefile";
getnames=no;
run;
Can anyone help me for this, please?
Thanks
Try to put it in a macro like this:
%macro import (macro_var1,macro_var2,macro_var3);
%let path="C:\Data\variationstring\empcat**A**\person**¯o_var1.**_**¯o_var2.**\persondata_empcatA_**¯o_var3.**";
libname test "&path";
proc import datafile="&path\Accouns_**49**.xls"
out=mydata
sheet="thefile";
getnames=no;
run;
%mend;
%import (34, 1212, 34);
%imoprt (49, 1684, 49);
etc.
when defining path, don't forget to put it in double quotes (") insted of single (')
In addition to Grigory's excellent suggestion, it looks to me like you're doing something that would be well served utilizing a data-driven programming approach.
If you have, say, an excel spreadsheet with all of your personnel records - let's say the first number is store and the second is employeeID - and you want to run one report per employeeID, then you write the macro like Grigory suggested; but you call the report from the first dataset.
So here:
proc import file="C:\Data\employeeID.xlsx"
out=employees dbms=xlsx
replace;
run;
%macro get_employee(store=, employeeID=, empCat=);
%let path="C:\Data\variationstring\empcat&empcat.\person&store._&employeeID.\persondata_empcat&empcat._&store.";
libname test "&path";
proc import datafile="&path\Accouns_&store..xls"
out=mydata
sheet="thefile";
getnames=no;
run;
%mend get_employee;
proc sql; *this generates macro calls, look at output to see what the macro variable contains;
select cats('%get_employee(employeeid=',employeeID,',store=',store,',empcat=',empcat,')')
into :get_emp_list separated by ' '
from employees;
quit;
&get_emp_list.; *This actually runs all those macro calls;
You can read my paper, Writing Code With Your Data for more details, or find other similar papers online.
I have just started learning SAS, and I'm using the following code to read xlsx files:
proc import out = data_lib.dataset_1
datafile = 'C:\data_folder\data_file_1.xlsx'
dbms = xlsx replace;
sheet = 'Sheet1';
getnames = yes;
run;
This has been working fine for me, but I'd like to supply the code with a list of filenames to read and a list of dataset names to create, so that the code need only appear once. I have looked at several instructional web pages about using macros, but I've been unable to translate that information into working code. Any help would be greatly appreciated. I'm using SAS 9.4, 64 bit.
I'd offer a modified version of kl78's suggestion, avoiding macros. Again, assuming you have the file names in a SAS data set, use a data step to read the list of file names and use call execute to run your proc import code for each file name.
data _null_;
set t_list;
call execute (
"proc import out = " || datasetname || "
datafile = '"|| filename ||"'
dbms = xlsx replace;
sheet = 'Sheet1';
getnames = yes;
run;");
run;
So, suppose you have your filenames and datanames in a table called t_list with variablename datasetname and filename, you could try something like this:
%macro readexcels;
data _null_;
set t_list (nobs=nobs);
call symputx(cat("libname_",_n_), datasetname);
call symputx(cat("filename_",_n_), filename);
if _n_=1 then
call symputx("nobs", nobs);
run;
%do i=1 %to &nobs;
proc import out = &&libname_&i;
datafile = "&&filename_&i"
dbms = xlsx replace;
sheet = 'Sheet1';
getnames = yes;
run;
%end;
%mend;
%readexcels;
In the datastep you read every entry of your table with datasetname and listname and create macrovariables with a numeric suffix. You only need to create a macrovariable for the number of entries once, so i did it when n = 1, you could also do this at eof.
Then you have a do loop, and with every loop you read the specific excel and write it in the specific dataset.
You need to write it like &&libname&i, because at first this resolves to &libname_1, and after this resolves to the variablevalue...
I'm working with a rather large several dataset that are provided to me as a CSV files. When I attempt to import one of the files the data will come in fine but, the number of variables in the file is too large for SAS, so it stops reading the variable names and starts assigning them sequential numbers. In order to maintain the variable names off of the data set I read in the file with the data row starting on 1 so it did not read the first row as variable names -
proc import file="X:\xxx\xxx\xxx\Extract\Live\Live.xlsx" out=raw_names dbms=xlsx replace;
SHEET="live";
GETNAMES=no;
DATAROW=1;
run;
I then run a macro to start breaking down the dataset and rename the variables based on the first observations in each variable -
%macro raw_sas_datasets(lib,output,start,end);
data raw_names2;
raw_names;
if _n_ ne 1 then delete;
keep A -- E &start. -- &end.;
run;
proc transpose data=raw_names2 out=raw_names2;
var A -- &end.;
run;
data raw_names2;
set raw_names2;
col1=compress(col1);
run;
data raw_values;
set raw;
keep A -- E &start. -- &end.;
run;
%macro rename(old,new);
data raw_values;
set raw_values;
rename &old.=&new.;
run;
%mend rename;
data _null_;
set raw_names2;
call execute('%rename('||_name_||","||col1||")");
run;
%macro freq(var);
proc freq data=raw_values noprint;
tables &var. / out=&var.;
run;
%mend freq;
data raw_names3;
set raw_names2;
if _n_ < 6 then delete;
run;
data _null_;
set raw_names3;
call execute('%freq('||col1||")");
run;
proc sort data=raw_values;
by StudySubjectID;
run;
data &lib..&output.;
set raw_values;
run;
%mend raw_sas_datasets;
The problem I'm running into is that the variable names are now all set properly and the data is lined up correctly, but the labels are still the original SAS assigned sequential numbers. Is there any way to set all of the labels equal to the variable names?
If you just want to remove the variable labels (at which point they default to the variable name), that's easy. From the SAS Documentation:
proc datasets lib=&lib.;
modify &output.;
attrib _all_ label=' ';
run;
I suspect you have a simpler solution than the above, though.
The actual renaming step needs to be done differently. Right now it's rewriting the entire dataset over and over again - for a lot of variables that is a terrible idea. Get your rename statements all into one datastep, or into a PROC DATASETS, or something else. Look up 'list processing SAS' for details on how to do that; on this site or on google you will find lots of solutions.
You likely can get SAS to read in the whole first line. The number of variables isn't the problem; it is probably the length of the line. There's another question that I'll find if I can on this site from a few months ago that deals with this exact problem.
My preferred option is not to use PROC IMPORT for CSVs anyway; I would suggest writing a metadata table that stores the variable names and the length/types for the variables, then using that to write import code. A little more work at first, but only has to be done once per study and you guarantee PROC IMPORT isn't making silly decisions for you.
In the library sashelp is a table vcolumn. vcolumn contains all the names of your variables for each library by table. You could write a macro that puts all your variable names into macro variables and then from there set the label.
Here's some code that I put together (not very pretty) but it does what you're looking for:
data test.label_var;
x=1;
y=1;
label x = 'xx';
label y = 'yy';
run;
proc sql noprint;
select count(*) into: cnt
from sashelp.vcolumn
where memname = 'LABEL_VAR';quit;
%let cnt = &cnt;
proc sql noprint;
select name into: name1 - :name&cnt
from sashelp.vcolumn
where memname = 'LABEL_VAR';quit;
%macro test;
%do i = 1 %to &cnt;
proc datasets library=test nolist;
modify label_var;
label &&name&i=&&name&i;
quit;
%end;
%mend test;
%test;
My goal is to create a SAS stored process is to return data for a single dataset and to filter the columns in that dataset based on a multi-value input parameter passed into the stored process.
Is there a simple way to do this?
Is there a way to do this at all?
Here's what I have so far. I'm using a macro to dynamically generate the KEEP statement to define which columns to return. I've defined macro variables at the top to mimic what gets passed into the stored process when called through SAS BI Web Services, so unfortunately those have to remain as they are. That's why I've tried to use the VVALUEX method to turn the column name strings into variable names.
Note - I'm new to SAS
libname fetchlib meta library="lib01" metaserver="123.12.123.123"
password="password" port=1234
repname="myRepo" user="myUserName";
/* This data represents input parameters to stored process and
* is removed in the actual stored process*/
%let inccol0=3;
%let inccol='STREET';
%let inccol1='STREET';
%let inccol2='ADDRESS';
%let inccol3='POSTAL';
%let inccol_count=3;
%macro keepInputColumns;
%if &INCCOL_COUNT = 1 %then
&inccol;
%else
%do k=1 %to (&INCCOL_COUNT);
var&k = VVALUEX(&&inccol&k);
%end;
KEEP
%do k=1 %to (&INCCOL_COUNT);
var&k
%end;
;
%mend;
data test1;
SET fetchlib.Table1;
%keepInputColumns;
run;
/*I switch this output to _WEBOUT in the actual stored process*/
proc json out='C:\Logs\Log1.txt';
options firstobs=1 obs=10;
export test1 /nosastags;
run;
There are some problems with this. The ouput uses var1, var2 and var3 as the column names and not the actual column names. It also doesn't filter by any columns when I change the output to _webout and run it using BI Web Services.
OK, I think I have some understanding of what you're doing here.
You can use KEEP and RENAME in conjunction to get your variable names back.
KEEP
%do k=1 %to (&INCCOL_COUNT);
var&k
%end;
;
This has an equivalent
RENAME
%do k=1 %to (&INCCOL_COUNT);
var&k = &&inccol&k.
%end;
;
and now, as long as the user doesn't separately keep the original variables, you're okay. (If they do, then you will get a conflict and an error).
If this way doesn't work for your needs, and I don't have a solution for the _webout as I don't have a server to play with, you might consider trying this in a slightly different way.
proc format;
value agef
11-13 = '11-13'
14-16 = '14-16';
quit;
ods output report=mydata(drop=_BREAK_);
proc report data=sashelp.class nowd;
format age agef.;
columns name age;
run;
ods output close;
The first part is just a proc format to show that this grabs the formatted value not the underlying value. (I assume that's desired, as if it's not this is a LOT easier.)
Now you have the data in a dataset a bit more conveniently, I think, and can put it out to JSON however you want. In your example you'd do something like
ods output report=work.mydata(drop=_BREAK_);
proc report data=fetchlib.Table1 nowd;
columns
%do k=1 %to (&INCCOL_COUNT);
&&inccol&k.;
%end;
;
run;
ods output close;
And then you can send that dataset to JSON or whatever. It's actually possible that you might be able to go more directly than that even, but I don't know almost anything about PROC JSON.
Reading more about JSON, you may actually have an easier way to do this.
On the export line, you have the various format options. So, assuming we have a dataset that is just a subset of the original:
proc json out='C:\Logs\Log1.txt';
options firstobs=1 obs=10;
export fetchlib.Table1
(
%do k=1 %to (&INCCOL_COUNT);
&&inccol&k.;
%end;
)
/ nosastags FMTCHARACTER FMTDATETIME FMTNUMERIC ;
run;
This method doesn't allow for the variable order to be changed; if you need that, you can use an intermediate dataset:
data intermediate/view=intermediate;
set fetchlib.Table1;
retain
%do k=1 %to (&INCCOL_COUNT);
&&inccol&k.;
%end;
;
keep
%do k=1 %to (&INCCOL_COUNT);
&&inccol&k.;
%end;
;
run;
and then write that out. I'm just guessing that you can use a view in this context.
It turns out that the simplest way to implement this was to change the way that the columns (aka SAS variables) were passed into the stored process. Although Joe's answer was helpful, I ended up solving the problem by passing in the columns to the keep statement as a space-separated column list, which greatly simplified the SAS code because I didn't have to deal with a dynamic list of columns.
libname fetchlib meta library="lib01" metaserver="123.12.123.123"
password="password" port=1234
repname="myRepo" user="myUserName";"&repository" user="&user";
proc json out=_webout;
export fetchlib.&tablename(keep=&columns) /nosastags;
run;
Where &columns gets set to something like this:
Column1 Column2 Column3